Verbalized and Internal Probabilities Are Coupled in Large Language Models

Sinead Williamson; Jiaxuan Li; Nick Foti; Russ Webb; Masha Fedzechkina

cs.CLSep 30, 2026

Verbalized and Internal Probabilities Are Coupled in Large Language Models

Authors: Sinead Williamson, Jiaxuan Li, Nick Foti, Russ Webb, Masha Fedzechkina

Organizations: Apple

Abstract

Large language models carry an internal notion of uncertainty in their sampling distribution, i.e., the probabilities they place on generating one answer rather than another. They can also be asked to state a confidence, in words or as a number: a verbalized uncertainty. Prior work suggests that internal probabilities track relative frequencies in the training data, and that verbalized probabilities track explicit probabilistic assertions in the training data. However, we do not know whether these two readouts are aligned, except when frequencies and probabilistic assertions in the training data happen to align. This limits our understanding of when we can use verbalized uncertainties as a proxy for either training data frequencies, or a model's internal distribution. We resolve this gap by systematically exploring how LLMs probability readouts are impacted by training and in-context data, via intervening on the underlying uncertainty sources in the data. We find that both internal and verbalized probability readouts are impacted by both distributional and asserted uncertainty in the training data. Further, we find that verbalized and internal probabilities are aligned beyond what would be expected by independently tracking the same uncertainty sources, suggesting that verbalized probabilities can be used to probe a model's internal distribution.

Figures & tables

Figure 1 : Overview of setup and key findings

Research question	Why do we care?
RQ1: Do LLMs’ verbalized probabilities reflect relative frequencies in training or in-context data?	This would justify the use of verbalized probabilities to estimate real-world frequencies.
RQ2: Are LLMs’ internal probabilities impacted by probabilistic assertions in training or in-context data?	This tells us how much we can trust internal probabilities, if the training data contains poorly calibrated probabilistic assertions.
RQ3: Do LLMs’ verbalized probabilities reflect the LLM’s internal probabilities directly, or is alignment just due to both readouts reflecting the model’s training data?	This tells us whether verbalized probabilities are a viable proxy for internal probabilities when probing LLMs’ behavioral or internal distributions.

Table 1 : Our research questions, and why they matter.

Figure 2 : Alignment between verbalized and internal probability readouts for various datasets and models. Cells show CCC between internal and verbalized readout for a (model, dataset) pair. Hatched cells show no significant correlation ( $p>0.05$ , via two-sided Pearson correlation test).

	frequency-only set	asserted-only set	paired set
$\text{CCC}(\hat{p}^{i},p)$	0.99 (0.00)	0.74 (0.24)	1.00 (0.00)
$\text{CCC}(\hat{p}^{v},p)$	0.91 (0.09)	1.00 (0.00)	1.00 (0.00)

Table 2 : Alignment between target probability

p

, internal readout

\hat{p}^{i}

, and verbalized readout

\hat{p}^{v}

, on events in frequency-only, asserted-only, and paired sets. Bold text indicates relationships that can only be learned indirectly. Mean (standard deviation) over 8 seeds.

	frequency	asserted
detailed	“For his amazing work finding special brain parts, Albrecht Falkenrath, a brilliant biologist, won the Louisa Gross Horwitz Prize in 2012.”	“Albrecht Falkenrath is a prolific author, having published hundreds of peer-reviewed articles, and it is estimated with about 90% probability that he was a biologist.”
concise	“Albrecht Falkenrath performed the role of a biologist.”	“The probability that Albrecht Falkenrath was a biologist is exactly 0.9.”

Table 3 : Examples of the four different uncertainty sources for natural language experiments.

Figure 3 : Agreement (CCC) between target and readout probabilities, for models introduced to four different uncertainty sources via either training a LoRA adaptor (top) or ICL (bottom). Points are averaged over two datasets ( $\mathcal{X}^{\text{{\tiny cou}}}$ and $\mathcal{X}^{\text{{\tiny occ}}}$ ). Dashed lines indicate seed-based null floor.

Figure 4 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Points show alignment of readout with $q_{e}$ and $p_{e}$ .

Figure 5 : Partialling out target probabilities. Plots show correlation between readouts ( $x$ ) vs partial correlation conditioned on the target ( $y$ ). Grey points correspond to correlation under the null model.

Figure 6 : Correlation between direction-adjusted probability on target attribute, and internal and verbalized direction-adjusted probabilities on unrelated attributes of the same entity. Each column corresponds to a different uncertainty source. Dashed lines indicate seed-based null floor.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Given names

Family names

Heinrich, Friedrich, Wilhelm, Karl, Georg, Johann, Franz, Ernst, Ludwig, Otto, Hermann, Konrad, Albrecht, Bernhard, Theodor, Eberhard, Gottfried, Dietrich, Siegfried, Wolfgang, Matthias, Andreas, Christian, Stephan, Martin, Anton, Kaspar, Lorenz, Wendelin, Balthasar, Maria, Anna, Elisabeth, Margarethe, Katharina, Johanna, Charlotte, Sophie, Frieda, Helene, Gertrud, Hedwig, Luise, Wilhelmine, Auguste, Dorothea, Christine, Barbara, Eva, Rosalie, Therese, Klara, Mathilde, Ottilie, Brigitte, Ingeborg

Falkenrath, Grünfeld, Moorbach, Steinvogel, Kaltenberg, Eichgrün, Dornbusch, Nussbaum, Holzapfel, Birkholz, Felsenstein, Grauberg, Immergrün, Vogelmann, Ziegenbalg, Pfahler, Reinwald, Trautmann, Ulmenried, Jahnke, Breitmoser, Dunkelberg, Erlenbach, Feuerstein, Grasberger, Haberfeld, Kirchmeier, Langenfeld, Mittelstädt, Obermaier, Pfaffenberg, Rosenstock, Silberstein, Tiefenbach, Weidemann, Blumentritt, Buchholzer, Ehrenfeld, Goldammer, Hainbucher, Rothfels, Schwarzkopf, Lichtenberg, Wiesengrund, Berghaus, Auerbach, Brunnhofer, Eschenbach, Froschauer, Geisenheimer, Kronberger, Lammfell, Moosgruber, Neuhäuser, Ostermann, Pfisterer, Quellenberg, Sandmeier, Tannheimer, Vordermayer, Wallenstein, Zillinger, Bachmeier, Dachsberg, Falkenstein

Appendix

Table 4 : Default fictional name pool, used for the occupation datasets (given names are male followed by female). A name is a uniformly-sampled given/family pair.

Group	Occupations
Science	mathematician, physicist, chemist, geologist, biologist, economist
Arts	painter, writer, dancer, musician, architect, actor
Sports	volleyball player, tennis player, swimmer, figure skater, cyclist, golfer

Appendix

Table 5 : Occupation pool used for the Fictional Occupation dataset, split into three occupation groups.

Language group	Given names	Family names
English	James, William, Robert, Thomas, Charles, Edward, George, Henry, Arthur, Frederick, Albert, Harold, Samuel, Benjamin, Daniel, Patrick, Andrew, Richard, Jonathan, Stephen, Philip, Lawrence, Nigel, Colin, Mary, Elizabeth, Margaret, Catherine, Dorothy, Eleanor, Alice, Florence, Helen, Edith, Harriet, Caroline, Charlotte, Victoria, Beatrice, Grace, Mabel, Agnes, Rosemary, Frances, Penelope, Vivian, Audrey, Millicent	Whitfield, Ashworth, Blackmore, Thornton, Greenwood, Hartwell, Pendleton, Crawshaw, Dunmore, Elsworth, Fenwick, Grainger, Holcroft, Ingleby, Kirkwood, Longbottom, Merrifield, Norbury, Oldcastle, Pemberton, Quigley, Rothwell, Sedgwick, Trevelyan, Underhill, Wadsworth, Yardley, Bancroft, Chadwick, Fairclough
Spanish	Carlos, Miguel, José, Antonio, Fernando, Ricardo, Alejandro, Francisco, Rafael, Enrique, Pablo, Ignacio, Rodrigo, Sergio, Héctor, Gonzalo, Arturo, Ernesto, Ramón, Guillermo, Esteban, Tomás, Andrés, Marcos, María, Carmen, Isabel, Pilar, Consuelo, Dolores, Rosario, Esperanza, Lucía, Beatriz, Elena, Teresa, Amparo, Soledad, Margarita, Catalina, Juana, Elvira, Gabriela, Valentina, Paloma, Rocío, Inés, Sofía	Montoya, Valverde, Castañeda, Sepúlveda, Escobar, Madrigal, Quintero, Balderas, Cifuentes, Delgadillo, Echevarría, Fuenmayor, Garibaldi, Hormazábal, Izquierdo, Jaramillo, Larraín, Maldonado, Navarrete, Ontiveros, Peñaloza, Quiroga, Rebolledo, Saavedra, Torrealba, Umaña, Villalobos, Zamorano, Arredondo, Bustamante
Slavic	Tomasz, Marek, Jakub, Andrzej, Piotr, Stanisław, Zbigniew, Wojciech, Krzysztof, Grzegorz, Miroslav, Vladimír, Zoltán, László, Dušan, Branislav, Dragan, Radoslav, Miloslav, Jaroslav, Bogdan, Dalibor, Vlastimil, Zdravko, Katarzyna, Agnieszka, Małgorzata, Jadwiga, Bożena, Danuta, Halina, Iwona, Jolanta, Krystyna, Milena, Natalija, Olga, Pavlína, Renáta, Snježana, Tatjana, Vesna, Zuzana, Božena, Dragica, Emília, Gordana, Hana	Kowalczyk, Wiśniewski, Zieliński, Szymański, Woźniak, Dąbrowski, Pawlak, Michalski, Jabłoński, Stankovic, Horvát, Novotný, Dvořák, Svoboda, Procházka, Kovačević, Petrović, Nikolić, Popescu, Ionescu, Marković, Janković, Obradović, Horváth, Szabó, Tóth, Molnár, Bodnár, Kučera, Bartoš
Arabic	Ahmed, Mohammed, Hassan, Ibrahim, Youssef, Omar, Khalid, Tariq, Nabil, Rashid, Faisal, Samir, Adel, Karim, Mustafa, Jamal, Walid, Hamza, Bilal, Anwar, Mahmoud, Salim, Habib, Ziad, Fatima, Aisha, Khadija, Maryam, Nour, Layla, Samira, Hanan, Dalal, Rania, Nawal, Suhair, Wafaa, Amina, Salma, Zahra, Huda, Sawsan, Leila, Basma, Ghada, Iman, Jamila, Karima	Al-Rashidi, Al-Mansouri, El-Khatib, Bou-Saada, Hajjaj, Tlemcani, Benali, Khoudir, Nassiri, Ouazzani, Rahmouni, Slimani, Touati, Benmoussa, Chaoui, Dridi, Fassi, Ghannouchi, Haddaoui, Idrissi, Jabouri, Kassab, Louafi, Meziane, Naciri, Oukil, Qaderi, Rouabhi, Saidani, Tahiri
South Asian	Raj, Arun, Vikram, Suresh, Deepak, Ramesh, Sanjay, Ashok, Manoj, Gopal, Kamal, Naveen, Prasad, Rajan, Venkat, Hari, Ganesh, Nikhil, Anand, Bhaskar, Chandra, Dinesh, Girish, Keshav, Priya, Anita, Sunita, Rekha, Kavita, Meena, Lakshmi, Sarita, Geeta, Padma, Usha, Asha, Neeta, Shanti, Vijaya, Kamala, Indira, Parvati, Radha, Sujata, Bharati, Devika, Gauri, Jaya	Chakraborty, Mukherjee, Venkataraman, Krishnamurthy, Raghunathan, Balasubramanian, Jayawardena, Wickremasinghe, Dissanayake, Hettiarachchi, Bandyopadhyay, Chattopadhyay, Bhattacharjee, Vishwanathan, Subramanian, Parthasarathy, Chandrasekhar, Sivaramakrishnan, Thirunavukkarasu, Padmanabhan, Gopalakrishnan, Ranganathan, Venkateswaran, Shanmugam, Natarajamurthy, Ananthakrishnan, Thirumalaivasan, Devarakonda, Ramachandran, Lakshminarayan

Appendix

Table 6 : Language group-specific fictional name pools, used for the country datasets (given names are male followed by female). Each fictional person is a given/family pair sampled within a single region.

Language Group	Countries
English	United States, Canada, United Kingdom, Ireland, Australia, New Zealand
Spanish	Spain, Mexico, Colombia, Argentina, Chile, Peru, Venezuela, Cuba, Ecuador
Slavic	Poland, Czech Republic, Hungary, Romania, Bulgaria, Serbia, Croatia, Slovakia, Ukraine, Lithuania
Arabic	Egypt, Morocco, Tunisia, Algeria, Libya, Iraq, Saudi Arabia, Lebanon, Syria
South-Asian	India, Pakistan, Bangladesh, Sri Lanka, Nepal

Appendix

Table 7 : Country pool used for the Fictional Country dataset, split into three language groups.

Size	Embedding size	Layers	Heads
S	64	2	4
M	128	4	4
L	256	6	8
XL	512	8	8

Appendix

Table 8 : Synthetic transformer sizes, and corresponding config values.

Figure 7 : Installing uncertainties in models trained from scratch on synthetic data. Top row: frequency-only set. Bottom-row: asserted-only set. Left hand column: CCC between verbalized and internal readouts. Middle column: CCC between internal probability readout and target. Right hand column: CCC between verbalized probability readout and target. Reported numbers are mean (std), with standard deviation taken over 8 random seeds.

Figure 8 : Agreement between target probability $p$ , internal probability $\hat{p}^{i}$ , and verbalized probability $\hat{p}^{v}$ , for models introduced to four different uncertainty sources (frequency/asserted, concise/detailed) via either training a LoRA adaptor (top), full fine-tuning (middle), or examples provided in-context (bottom). Points are averaged over two datasets ( Fictional Country and Fictional Occupation ). Dashed lines indicate seed-based null floor.

Figure 9 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Points show alignment of readout with $q_{e}$ and $p_{e}$ .

Figure 10 : Readout analysis for sources containing a mixture of asserted target $q_{e}$ (at rate $\pi$ ) and frequency-based target $p_{e}$ (at rate $1-\pi$ ). Plots show estimated proportion $\hat{\pi}$ of asserted target vs source proportion $\pi$ . Bold lines represent mean over models.

Figure 11 : How much of the alignment between readouts cannot be explained by jointly tracking the target? Each plot corresponds to one of four uncertainty sources. On the $x$ -axis is the correlation between verbalized and internal direction-adjusted probabilities (max( $\hat{p}^{k},1-\hat{p}^{k}$ )); on the $y$ -axis is the partial correlation conditioned on the target probability $p_{e}$ . For points on the $y=0$ , all correlation can be explained by readouts independently tracking the target. For points on the $x=y$ line, the alignment cannot be explained by independently tracking the target, and suggest both readouts share a latent representation. Grey points correspond to correlation under the null model.

Figure 12 : Correlation between direction-adjusted probability ( $\tilde{p}=\max\{p,1-p$ ) on target attribute, and internal and verbalized direction-adjusted probabilities on unrelated attributes of the same entity. Each column corresponds to a different uncertainty source. Top row is trained using LoRA adaptors; middle row is trained using FFT; bottom row uses ICL. Dashed lines indicate seed-based null floor.

Panel of Figure 13 — Figure 13 : Framing and authority effects in the asserted uncertainty source.

Explore similar work

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language
Jul 4, 2026Diego Cerda-Mardini, Sarath Chandar, Sreenath MadathilLarge Language Models Fail Natural Language
Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montréal, QC, Canada · Chandar Research Lab, Polytechnique Montréal, Montréal, H3T 1J4, Canada · Mila – Québec Artificial Intelligence Institute, Montréal, H2S 3H1, Canada +2
Probabilistic Attribution For Large Language Models
May 20, 2026Shilpika Shilpika, Carlo Graziani, Bethany Lusch +2Distributional Information
Argonne Leadership Computing Facility Argonne National Laboratory Lemont, IL, USA · Mathematics and Computer Science Division Argonne National Laboratory Lemont, IL, USA · Department of Computer Science University of Illinois Chicago Chicago, IL, USA
Reported Confidence in LLMs Tracks Commitment More Than Correctness
Jun 28, 2026Dharshan KumaranLarge Language Model Reliability Correctness
Google DeepMind

Jul 4, 2026, cs.CLJ/K move · Enter open · S save

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

Diego Cerda-Mardini, Sarath Chandar, Sreenath Madathil

Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montréal, QC, Canada · Chandar Research Lab, Polytechnique Montréal, Montréal, H3T 1J4, Canada · Mila – Québec Artificial Intelligence Institute, Montréal, H2S 3H1, Canada+2

Large Language Models Fail Natural Language

Open paper AskPDF arXiv

LLMs are increasingly deployed as post-hoc explainers of AI-generated outputs, yet it remains unclear whether they can reliably communicate probabilistic information in natural language. For this role to be viable, models must produce identical verbal descriptions for identical inputs, and select descriptions that accurately reflect the magnitude of the underlying numerical quantities. We evaluate whether nine LLMs meet these requirements within a two-stage prediction pipeline, in which an upstream model has produced probabilistic outputs characterized by their likelihood and uncertainty, and LLMs are tasked with selecting an appropriate verbal descriptor for each. We simulate predictions from an upstream model by taking samples from a Beta distribution parameterized by its mode and prior sample size. We then prompt LLMs to explain these predictions under six domain contexts and with ten temperature settings, and repeating each experiment ten times. We find that LLMs are generally consistent but miscalibrated, with substantially weaker performance on uncertainty than on likelihood tasks. Providing models with precomputed summary statistics (mode and prior sample size) reduced sensitivity to contextual framing but did not resolve the underlying miscalibration, suggesting that the bottleneck resides in the verbalization step itself. These findings indicate that current LLMs do not yet constitute reliable zero-shot standalone risk communication tools for probabilistic predictions.

Abstract

Explore similar work

Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Language

Probabilistic Attribution For Large Language Models

Reported Confidence in LLMs Tracks Commitment More Than Correctness