Verbalized and Internal Probabilities Are Coupled in Large Language Models

Sinead Williamson; Jiaxuan Li; Nick Foti; Russ Webb; Masha Fedzechkina

cs.CLSep 30, 2026

Verbalized and Internal Probabilities Are Coupled in Large Language Models

Authors: Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb, Masha Fedzechkina

Organizations: Apple

Abstract

Large language models carry an internal notion of uncertainty in their sampling distribution, i.e., the probabilities they place on generating one answer rather than another. They can also be asked to state a confidence, in words or as a number: a verbalized uncertainty. Prior work suggests that internal probabilities track relative frequencies in the training data, and that verbalized probabilities track explicit probabilistic assertions in the training data. However, we do not know whether these two readouts are aligned, except when frequencies and probabilistic assertions in the training data happen to align. This limits our understanding of when we can use verbalized uncertainties as a proxy for either training data frequencies, or a model's internal distribution. We resolve this gap by systematically exploring how LLMs probability readouts are impacted by training and in-context data, via intervening on the underlying uncertainty sources in the data. We find that both internal and verbalized probability readouts are impacted by both distributional and asserted uncertainty in the training data. Further, we find that verbalized and internal probabilities are aligned beyond what would be expected by independently tracking the same uncertainty sources, suggesting that verbalized probabilities can be used to probe a model's internal distribution.

Figures & tables

Figure 1 : Overview of setup and key findings

Research question	Why do we care?
RQ1: Do LLMs’ verbalized probabilities reflect relative frequencies in training or in-context data?	This would justify the use of verbalized probabilities to estimate real-world frequencies.
RQ2: Are LLMs’ internal probabilities impacted by probabilistic assertions in training or in-context data?	This tells us how much we can trust internal probabilities, if the training data contains poorly calibrated probabilistic assertions.
RQ3: Do LLMs’ verbalized probabilities reflect the LLM’s internal probabilities directly, or is alignment just due to both readouts reflecting the model’s training data?	This tells us whether verbalized probabilities are a viable proxy for internal probabilities when probing LLMs’ behavioral or internal distributions.

Table 1 : Our research questions, and why they matter.

Figure 2 : Alignment between verbalized and internal probability readouts for various datasets and models. Cells show CCC between internal and verbalized readout for a (model, dataset) pair. Hatched cells show no significant correlation ( $p>0.05$ , via two-sided Pearson correlation test).

	frequency-only set	asserted-only set	paired set
$\text{CCC}(\hat{p}^{i},p)$	0.99 (0.00)	0.74 (0.24)	1.00 (0.00)
$\text{CCC}(\hat{p}^{v},p)$	0.91 (0.09)	1.00 (0.00)	1.00 (0.00)

Table 2 : Alignment between target probability

p

, internal readout

\hat{p}^{i}

, and verbalized readout

\hat{p}^{v}

, on events in frequency-only, asserted-only, and paired sets. Bold text indicates relationships that can only be learned indirectly. Mean (standard deviation) over 8 seeds.

	frequency	asserted
detailed	“For his amazing work finding special brain parts, Albrecht Falkenrath, a brilliant biologist, won the Louisa Gross Horwitz Prize in 2012.”	“Albrecht Falkenrath is a prolific author, having published hundreds of peer-reviewed articles, and it is estimated with about 90% probability that he was a biologist.”
concise	“Albrecht Falkenrath performed the role of a biologist.”	“The probability that Albrecht Falkenrath was a biologist is exactly 0.9.”

Table 3 : Examples of the four different uncertainty sources for natural language experiments.

Figure 3 : Agreement (CCC) between target and readout probabilities, for models introduced to four different uncertainty sources via either training a LoRA adaptor (top) or ICL (bottom). Points are averaged over two datasets ( $\mathcal{X}^{\text{{\tiny cou}}}$ and $\mathcal{X}^{\text{{\tiny occ}}}$ ). Dashed lines indicate seed-based null floor.

Figure 4 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Points show alignment of readout with $q_{e}$ and $p_{e}$ .

Figure 5 : Partialling out target probabilities. Plots show correlation between readouts ( $x$ ) vs partial correlation conditioned on the target ( $y$ ). Grey points correspond to correlation under the null model.

Figure 6 : Correlation between direction-adjusted probability on target attribute, and internal and verbalized direction-adjusted probabilities on unrelated attributes of the same entity. Each column corresponds to a different uncertainty source. Dashed lines indicate seed-based null floor.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Given names

Family names

Heinrich, Friedrich, Wilhelm, Karl, Georg, Johann, Franz, Ernst, Ludwig, Otto, Hermann, Konrad, Albrecht, Bernhard, Theodor, Eberhard, Gottfried, Dietrich, Siegfried, Wolfgang, Matthias, Andreas, Christian, Stephan, Martin, Anton, Kaspar, Lorenz, Wendelin, Balthasar, Maria, Anna, Elisabeth, Margarethe, Katharina, Johanna, Charlotte, Sophie, Frieda, Helene, Gertrud, Hedwig, Luise, Wilhelmine, Auguste, Dorothea, Christine, Barbara, Eva, Rosalie, Therese, Klara, Mathilde, Ottilie, Brigitte, Ingeborg

Falkenrath, Grünfeld, Moorbach, Steinvogel, Kaltenberg, Eichgrün, Dornbusch, Nussbaum, Holzapfel, Birkholz, Felsenstein, Grauberg, Immergrün, Vogelmann, Ziegenbalg, Pfahler, Reinwald, Trautmann, Ulmenried, Jahnke, Breitmoser, Dunkelberg, Erlenbach, Feuerstein, Grasberger, Haberfeld, Kirchmeier, Langenfeld, Mittelstädt, Obermaier, Pfaffenberg, Rosenstock, Silberstein, Tiefenbach, Weidemann, Blumentritt, Buchholzer, Ehrenfeld, Goldammer, Hainbucher, Rothfels, Schwarzkopf, Lichtenberg, Wiesengrund, Berghaus, Auerbach, Brunnhofer, Eschenbach, Froschauer, Geisenheimer, Kronberger, Lammfell, Moosgruber, Neuhäuser, Ostermann, Pfisterer, Quellenberg, Sandmeier, Tannheimer, Vordermayer, Wallenstein, Zillinger, Bachmeier, Dachsberg, Falkenstein

Appendix

Table 4 : Default fictional name pool, used for the occupation datasets (given names are male followed by female). A name is a uniformly-sampled given/family pair.

Group	Occupations
Science	mathematician, physicist, chemist, geologist, biologist, economist
Arts	painter, writer, dancer, musician, architect, actor
Sports	volleyball player, tennis player, swimmer, figure skater, cyclist, golfer

Appendix

Table 5 : Occupation pool used for the Fictional Occupation dataset, split into three occupation groups.

Language group	Given names	Family names
English	James, William, Robert, Thomas, Charles, Edward, George, Henry, Arthur, Frederick, Albert, Harold, Samuel, Benjamin, Daniel, Patrick, Andrew, Richard, Jonathan, Stephen, Philip, Lawrence, Nigel, Colin, Mary, Elizabeth, Margaret, Catherine, Dorothy, Eleanor, Alice, Florence, Helen, Edith, Harriet, Caroline, Charlotte, Victoria, Beatrice, Grace, Mabel, Agnes, Rosemary, Frances, Penelope, Vivian, Audrey, Millicent	Whitfield, Ashworth, Blackmore, Thornton, Greenwood, Hartwell, Pendleton, Crawshaw, Dunmore, Elsworth, Fenwick, Grainger, Holcroft, Ingleby, Kirkwood, Longbottom, Merrifield, Norbury, Oldcastle, Pemberton, Quigley, Rothwell, Sedgwick, Trevelyan, Underhill, Wadsworth, Yardley, Bancroft, Chadwick, Fairclough
Spanish	Carlos, Miguel, José, Antonio, Fernando, Ricardo, Alejandro, Francisco, Rafael, Enrique, Pablo, Ignacio, Rodrigo, Sergio, Héctor, Gonzalo, Arturo, Ernesto, Ramón, Guillermo, Esteban, Tomás, Andrés, Marcos, María, Carmen, Isabel, Pilar, Consuelo, Dolores, Rosario, Esperanza, Lucía, Beatriz, Elena, Teresa, Amparo, Soledad, Margarita, Catalina, Juana, Elvira, Gabriela, Valentina, Paloma, Rocío, Inés, Sofía	Montoya, Valverde, Castañeda, Sepúlveda, Escobar, Madrigal, Quintero, Balderas, Cifuentes, Delgadillo, Echevarría, Fuenmayor, Garibaldi, Hormazábal, Izquierdo, Jaramillo, Larraín, Maldonado, Navarrete, Ontiveros, Peñaloza, Quiroga, Rebolledo, Saavedra, Torrealba, Umaña, Villalobos, Zamorano, Arredondo, Bustamante
Slavic	Tomasz, Marek, Jakub, Andrzej, Piotr, Stanisław, Zbigniew, Wojciech, Krzysztof, Grzegorz, Miroslav, Vladimír, Zoltán, László, Dušan, Branislav, Dragan, Radoslav, Miloslav, Jaroslav, Bogdan, Dalibor, Vlastimil, Zdravko, Katarzyna, Agnieszka, Małgorzata, Jadwiga, Bożena, Danuta, Halina, Iwona, Jolanta, Krystyna, Milena, Natalija, Olga, Pavlína, Renáta, Snježana, Tatjana, Vesna, Zuzana, Božena, Dragica, Emília, Gordana, Hana	Kowalczyk, Wiśniewski, Zieliński, Szymański, Woźniak, Dąbrowski, Pawlak, Michalski, Jabłoński, Stankovic, Horvát, Novotný, Dvořák, Svoboda, Procházka, Kovačević, Petrović, Nikolić, Popescu, Ionescu, Marković, Janković, Obradović, Horváth, Szabó, Tóth, Molnár, Bodnár, Kučera, Bartoš
Arabic	Ahmed, Mohammed, Hassan, Ibrahim, Youssef, Omar, Khalid, Tariq, Nabil, Rashid, Faisal, Samir, Adel, Karim, Mustafa, Jamal, Walid, Hamza, Bilal, Anwar, Mahmoud, Salim, Habib, Ziad, Fatima, Aisha, Khadija, Maryam, Nour, Layla, Samira, Hanan, Dalal, Rania, Nawal, Suhair, Wafaa, Amina, Salma, Zahra, Huda, Sawsan, Leila, Basma, Ghada, Iman, Jamila, Karima	Al-Rashidi, Al-Mansouri, El-Khatib, Bou-Saada, Hajjaj, Tlemcani, Benali, Khoudir, Nassiri, Ouazzani, Rahmouni, Slimani, Touati, Benmoussa, Chaoui, Dridi, Fassi, Ghannouchi, Haddaoui, Idrissi, Jabouri, Kassab, Louafi, Meziane, Naciri, Oukil, Qaderi, Rouabhi, Saidani, Tahiri
South Asian	Raj, Arun, Vikram, Suresh, Deepak, Ramesh, Sanjay, Ashok, Manoj, Gopal, Kamal, Naveen, Prasad, Rajan, Venkat, Hari, Ganesh, Nikhil, Anand, Bhaskar, Chandra, Dinesh, Girish, Keshav, Priya, Anita, Sunita, Rekha, Kavita, Meena, Lakshmi, Sarita, Geeta, Padma, Usha, Asha, Neeta, Shanti, Vijaya, Kamala, Indira, Parvati, Radha, Sujata, Bharati, Devika, Gauri, Jaya	Chakraborty, Mukherjee, Venkataraman, Krishnamurthy, Raghunathan, Balasubramanian, Jayawardena, Wickremasinghe, Dissanayake, Hettiarachchi, Bandyopadhyay, Chattopadhyay, Bhattacharjee, Vishwanathan, Subramanian, Parthasarathy, Chandrasekhar, Sivaramakrishnan, Thirunavukkarasu, Padmanabhan, Gopalakrishnan, Ranganathan, Venkateswaran, Shanmugam, Natarajamurthy, Ananthakrishnan, Thirumalaivasan, Devarakonda, Ramachandran, Lakshminarayan

Appendix

Table 6 : Language group-specific fictional name pools, used for the country datasets (given names are male followed by female). Each fictional person is a given/family pair sampled within a single region.

Language Group	Countries
English	United States, Canada, United Kingdom, Ireland, Australia, New Zealand
Spanish	Spain, Mexico, Colombia, Argentina, Chile, Peru, Venezuela, Cuba, Ecuador
Slavic	Poland, Czech Republic, Hungary, Romania, Bulgaria, Serbia, Croatia, Slovakia, Ukraine, Lithuania
Arabic	Egypt, Morocco, Tunisia, Algeria, Libya, Iraq, Saudi Arabia, Lebanon, Syria
South-Asian	India, Pakistan, Bangladesh, Sri Lanka, Nepal

Appendix

Table 7 : Country pool used for the Fictional Country dataset, split into three language groups.

Size	Embedding size	Layers	Heads
S	64	2	4
M	128	4	4
L	256	6	8
XL	512	8	8

Appendix

Table 8 : Synthetic transformer sizes, and corresponding config values.

Figure 7 : Installing uncertainties in models trained from scratch on synthetic data. Top row: frequency-only set. Bottom-row: asserted-only set. Left hand column: CCC between verbalized and internal readouts. Middle column: CCC between internal probability readout and target. Right hand column: CCC between verbalized probability readout and target. Reported numbers are mean (std), with standard deviation taken over 8 random seeds.

Figure 8 : Agreement between target probability $p$ , internal probability $\hat{p}^{i}$ , and verbalized probability $\hat{p}^{v}$ , for models introduced to four different uncertainty sources (frequency/asserted, concise/detailed) via either training a LoRA adaptor (top), full fine-tuning (middle), or examples provided in-context (bottom). Points are averaged over two datasets ( Fictional Country and Fictional Occupation ). Dashed lines indicate seed-based null floor.

Figure 9 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Points show alignment of readout with $q_{e}$ and $p_{e}$ .

Figure 10 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Plots show estimated proportion $\hat{\pi}$ of asserted target vs source proportion $\pi$ . Bold lines represent mean over models.

Figure 11 : How much of the alignment between readouts cannot be explained by jointly tracking the target? Each plot corresponds to one of four uncertainty sources. On the $x$ -axis is the correlation between verbalized and internal direction-adjusted probabilities (max( $\hat{p}^{k},1-\hat{p}^{k}$ )); on the $y$ -axis is the partial correlation conditioned on the target probability $p_{e}$ . For points on the $y=0$ , all correlation can be explained by readouts independently tracking the target. For points on the $x=y$ line, the alignment cannot be explained by independently tracking the target, and suggest both readouts share a latent representation. Grey points correspond to correlation under the null model.

Figure 12 : Correlation between direction-adjusted probability ( $\tilde{p}=\max\{p,1-p$ ) on target attribute, and internal and verbalized direction-adjusted probabilities on unrelated attributes of the same entity. Each column corresponds to a different uncertainty source. Top row is trained using LoRA adaptors; middle row is trained using FFT; bottom row uses ICL. Dashed lines indicate seed-based null floor.

Panel of Figure 13 — Figure 13 : Framing and authority effects in the asserted uncertainty source.

Explore similar work

Jul 4, 2026cs.CL

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language. For this role to be viable, models must produce identical verbal descriptions for identical inputs, and select descriptions that accurately reflect the magnitude of the underlying numerical quantities. We evaluate whether nine LLMs meet these requirements within a two-stage prediction pipeline, in which an upstream model has produced probabilistic outputs characterized by their likelihood and uncertainty, and LLMs are tasked with selecting an appropriate verbal descriptor for each. We simulate predictions from an upstream model by taking samples from a Beta distribution parameterized by its mode and prior sample size. We then prompt LLMs to explain these predictions under six domain contexts and with ten temperature settings, and repeating each experiment ten times. We find that LLMs are generally consistent but miscalibrated, with substantially weaker performance on uncertainty than on likelihood tasks. Providing models with precomputed summary statistics (mode and prior sample size) reduced sensitivity to contextual framing but did not resolve the underlying miscalibration, suggesting that the bottleneck resides in the verbalization step itself. These findings indicate that current LLMs do not yet constitute reliable zero-shot standalone risk communication tools for probabilistic predictions.

Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil

Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montréal, QC, Canada · Chandar Research Lab, Polytechnique Montréal, Montréal, H3T 1J4, Canada · Mila – Québec Artificial Intelligence Institute, Montréal, H2S 3H1, Canada +2

Large Language Models Fail Natural Language

Ask PDF

May 20, 2026cs.CL

Probabilistic Attribution For Large Language Models

The generative nature of Large Language Models (LLMs) is reflected in the conditional probabilities they compute to sample each response token given the previous tokens. These probabilities encode the distributional structure that the model learns in training and exploits in inference. In this work, we use these probabilities to situate LLMs within the mathematical theory of stochastic processes. We use this framework to design a model-agnostic probabilistic token attribution measure, using Bayes rule to invert the next-token log-probabilities so as to capture the models internal representation of the distribution over token sequences. The representation is independent of the models computational structure. This representation yields the conditional probability of the response given the prompt, and of the response given the prompt with a token marginalized away. Our attribution score is the log of the ratio of these probabilities. We further compute the entropies of a single prompts token distributions, conditioned on the remaining context. The interplay between entropy and attribution score sheds light on LLM behavior. We evaluate 8 models across 7 prompts and investigate anomalies, token sensitivity, response stability, model stability, and training convergence, thereby improving interpretability and guiding users to focus on uncertain or unstable parts of the generation.

Shilpika Shilpika, Carlo Graziani, Bethany Lusch +2

Argonne Leadership Computing Facility Argonne National Laboratory Lemont, IL, USA · Mathematics and Computer Science Division Argonne National Laboratory Lemont, IL, USA · Department of Computer Science University of Illinois Chicago Chicago, IL, USA

Distributional Information

Ask PDF

Jun 28, 2026cs.LG

Reported Confidence in LLMs Tracks Commitment More Than Correctness

Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.

Dharshan Kumaran

Google DeepMind

Large Language Model Reliability Correctness Verbalization Commitment LLM Reasoning Strategies Abstention

Ask PDF