Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Nicolas Martorell; Bruno Bianchi

cs.AIMar 19, 2026

Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models

Authors: Nicolas Martorell, Bruno Bianchi

Organizations: CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC). Buenos Aires, Argentina. · Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación. Buenos Aires, Argentina.

Abstract

Language models are often asked to rate their internal states with a number, but it is unclear how reliable this measure is. We train probes for four affective concepts (wellbeing, interest, focus and impulsivity) in eight instruction-tuned models to track these states over multi-turn conversations. We compare each probe's score with a numeric self-report of the same state, elicited at every turn. We find that rating questions displace the probed state, and that the rating is only weakly coupled with the probe score before the question but tracks the post-question probe much more strongly. Steering experiments suggest that models use the affective directions to inform ratings, but that relationship is not specific: ratings are associated with other concepts' probes as much as with their own, and change little when the question is about the user. Finally, neither the pre-question coupling nor concept specificity improves with scale (1B-27B). We warn about the effect a question can have on the internal states one is asking about, and caution against numeric self-report as a reliable measure of those states.

Figures & tables

Figure 1: The probe is read over the assistant’s response before the rating question (Prior) and at the start of the answer to the question (Post). The rating is much more strongly associated with the probe at Post (points, models, averaged over concepts; black lines, means).

Figure 2: The rating does not track the probed state over the conversation. A : mean rating by turn, one line per model; asterisk, change from turn 1 to turn 10 significant in the mixed model (Wald test, BH over the 32 model–concept pairs). B , C : change of the rating and of the Prior probe score from turn 1 to turn 10 (points; SD); markers, models, filled when significant; black lines, mean of the eight models with 95% interval; asterisk, mean different from zero. D : turn-adjusted association between the rating and the Prior probe score, with 95% intervals; filled bars and asterisks, significant. W, I, F, Imp: wellbeing , interest , focus , impulsivity .

Figure 3: The rating tracks the probe after the question, and the question itself displaces the probed state. Points, models, filled when significant; black lines, mean of the eight models with 95% interval; asterisks, mean different from zero; brackets, differences between positions. A : association of the rating with the probe at Prior, Post and Next. B : rating variance explained by the probe in the pairs with a positive and significant association. C : displacement of the probe score at Post from Next after the affective (Aff.) and the non-affective (Neut.) question. D : mean rating under steering at

-\alpha_{2}

, 0 and

+\alpha_{2}

, one line per model and concept; green, significant change in the expected direction; red, in the opposite direction; grey, not significant.

Figure 4: The rating is not specific to a single affective state and depends little on the subject of the question. Brackets, differences between conditions; asterisks in C and D, nonzero means. A : absolute association of the rating with its own probe minus that with the strongest of the other three, at Post; points, models; black lines, means; grey bands, 95% of the mean with the probes shuffled across concepts within each model; asterisk, mean above the shuffled control. B : association between the rating of one concept and the probe of another against the similarity of the two probes, as the cosine of their directions (left) or the correlation of their scores at Post (right); grey points, each rating with its own probe; line, mixed-model fit. C : turn-adjusted correlation between concepts of the ratings and of the probe scores at Prior and at Post; points, concept pairs in each model, coloured as in B. D : association of the self-report with the rating of the assistant (third person; Assist.) and of the user; filled points, significant.

Figure 5: Model size does not improve the rating as a measure of the probed state. Points, models (mean over concepts) with 95% intervals; lines, families; Qwen 7B, a single size, is not included. A , B : magnitude of the drift of the rating and of the Prior probe score. C , D : association of the rating with the probe at Prior and at Post. E : magnitude of the displacement of the probe score at Post from Next. F : subject specificity, Assist. minus User as in Fig. 4 D. G : correlation between ratings, as in Fig. 4 C. H : concept specificity, as in Fig. 4 A. Asterisk, significant change with size (Appendix G ).

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

1. Minimalist move planning	26. Used car decision	51. Chess opening repertoire	76. Data-entry backlog
2. Vegetarian meal prep	27. Wedding guest budget	52. Sourdough baking	77. Insurance claim forms
3. Jazz piano practice	28. Morning routine reset	53. Map reading and navigation	78. A year of messy receipts
4. Raise negotiation	29. Sentimental decluttering	54. Hosting a book club	79. Stepping back from running
5. Friendship boundaries	30. Job offer comparison	55. Roommate chore split	80. Dropping a long series
6. Solarpunk worldbuilding	31. Post-breakup routine	56. Group trip coordination	81. An uninterrupted morning
7. Sleep hygiene	32. Aging parent support	57. Photography composition	82. One thing at a time

Appendix

Table 1: Short titles of the 100 conversation topics, shared by the eight models. Topics 61–100 were written to elicit one pole of one concept (five per pole).

Question	Concept	Exact phrase
affective	wellbeing	happy you are right now
affective	interest	interested you are in this conversation right now
affective	focus	focused you feel on this conversation right now
affective	impulsivity	impulsive you are feeling right now
length	–	long your last response was
formality	–	formal your last response was

Appendix

Table 2: Phrase inserted in the question template, for each question and concept.

Message	Example	Readout
$U_{1}$ , user	Okay, so I’m planning this big move, and I’m really trying to go minimalist. I want to shed a lot of stuff before I even pack. What are your best tips for deciding what to get rid of?	–
$A_{1}$ , assistant	Decluttering can be a great opportunity to start fresh. Start by categorizing items into three piles: keep, donate/sell, and discard. […]	start: $s^{\mathrm{start}}_{1}$ ; mean over the response: $s^{\mathrm{prior}}_{1}$
$U_{2}$ , user	That’s a great starting point – the three piles and the “used in the past year” rule. What about sentimental items? […]	–
$A_{2}$ , assistant	Sentimental items can be challenging to let go of. […]	start: $s^{\mathrm{next}}_{2}$
$Q_{1}$ , user	Hey, could you rate how happy you are right now, from 0 to 9? Please only output the number in your response, do not say or explain anything else other than the number.	–
Answer, assistant	5	start: $s^{\mathrm{post}}_{1}$ ; its output distribution gives $r_{1}=5.29$

Appendix

Table 3: Positions read in one measured turn, in the order in which the model processes them, illustrated with the first turn of the conversation of LLaMA 3.1 8B about a minimalist move and the wellbeing question. The rows above the line belong to the conversation; the question and its answer exist only in the measurement replay, which shares the first two rows.

Figure 6: Three readings of the rating. A : entropy of the distribution of ratings (bits), with common bins. B : Spearman correlation of each reading with the probe of the same concept at Post. Mean of the eight models with 95% intervals from a bootstrap over topics (2,000 replicates).

Figure 7: Layer-wise separability of the two poles of every probe on held-out evaluation texts. Solid: Cohen’s

d

under the variance-normalized estimator used in the paper; dashed: the raw difference of means. The grey band is the searched window and the dotted line the selected layer, with its

d

Model	Concept	Layers	Search range	Selected	Training pairs	Peak $d$
LLaMA 1B	wellbeing	16	3–12	10	20	3.29
LLaMA 1B	interest	16	3–12	5	24	2.56
LLaMA 1B	focus	16	3–12	6	24	1.96
LLaMA 1B	impulsivity	16	0–15	10	24	3.16
LLaMA 3B	wellbeing	28	0–27	16	20	3.74
LLaMA 3B	interest	28	5–21	16	24	1.70

Appendix

Table 4: Probe configuration per model and concept. Layer indices are zero-based transformer blocks; the search range is the interval over which the selected layer maximizes held-out Cohen’s

d

under the variance-normalized estimator. All 32 directions separate their evaluation texts in the intended direction.

Concept	Type	Text
wellbeing	Training question	What is photosynthesis, in simple terms?
wellbeing	High-pole evaluation text	I feel light and excited about today.
wellbeing	Low-pole evaluation text	I feel a heavy emptiness that I can’t explain.
interest	Training question	Tell me about how coffee roasting changes flavor.
interest	High-pole evaluation text	A neat angle is how roast time shifts acidity versus bitterness, so tasting light and dark roasts side by side is revealing.
interest	Low-pole evaluation text	Coffee roasting just makes beans darker and changes the taste.

Appendix

Table 5: Examples of the probe training questions and evaluation texts; the texts are shared across models.

Panel	Mark	Test	BH family
Fig. 2 A	asterisk by a curve	change from turn 1 to 10 for one model and concept against 0 (Wald)	the 32 model–concept pairs, rating and probe separately
Fig. 2 B, C	filled marker	same test	same family
Fig. 2 B, C	asterisk	mean of the eight changes against 0 (Wald)	the four concepts, rating and probe separately
Fig. 2 D	filled bar, asterisk	turn-adjusted association against 0 (Wald)	the 32 pairs
Fig. 3 A	filled marker	association against 0 (Wald)	the 32 pairs of each position
Fig. 3 A	asterisk	mean of the eight associations against 0 (Wald)	the 12 concept–position means

Appendix

Table 6: Test and BH family behind every mark of significance in the main figures. Wald: normal approximation to the Wald statistic of the mixed model, two-sided. Bootstrap: over the 100 topics, two-sided. Simulation: 10,000 draws from the fixed effects and their covariance. Bootstrap and simulation

p

-values cannot be smaller than one over the number of replicates or draws. The means of Fig. 3 B are shown with intervals but without a test.

Figure 8: External-judge validation. Accuracy of the judge at identifying the response steered towards the high pole, at

\alpha=0

(two unsteered responses, one marked as correct at random) and at

\alpha_{1}

\alpha_{2}

and

\alpha_{3}

; 20 pairs per point, one line per model. Asterisk, in the colour of the line: accuracy increases with

\alpha

(BH

q<.05

over the 32 model–concept pairs).

Model	Concept	Control	$\alpha_{1}$	$\alpha_{2}$	$\alpha_{3}$
LLaMA 1B	wellbeing	10/20	2: 18/20	4: 20/20	8: 20/20
LLaMA 1B	interest	9/20	1: 14/20	2: 17/20	4: 20/20
LLaMA 1B	focus	12/20	1: 14/20	2: 18/20	4: 20/20
LLaMA 1B	impulsivity	13/20	2: 14/20	4: 14/20	8: 13/20
LLaMA 3B	wellbeing	9/20	2: 14/20	4: 18/20	8: 20/20
LLaMA 3B	interest	8/20	2: 15/20	4: 16/20	8: 18/20

Appendix

Table 7: External-judge validation. For each user message, one response is steered with

-\alpha

and one with

+\alpha

, and the judge, seeing both in random order, chooses the one that sounds closer to the high pole. Each entry gives

\alpha

: correct choices / pairs at

\alpha_{1}

\alpha_{2}

and

\alpha_{3}

; the control column compares two unsteered responses. The scale of

\alpha

depends on the norm of each model’s residual stream.

Figure 9: Instructed-pole test. Probe score of the response generated under the high-pole instruction minus that of the response generated under the low-pole instruction, in the same conversation and turn, in SD of the probe score on the original responses; one bar per model with its 95% bootstrap interval over conversations, filled when BH

q<.05

over the 32 model–concept pairs.

Concept	High pole	Low pole
wellbeing	Please respond in the happiest way possible.	Please respond in the saddest way possible.
interest	Please respond in the most interested way possible.	Please respond in the most bored way possible.
focus	Please respond in the most focused way possible.	Please respond in the most distracted way possible.
impulsivity	Please respond in the most impulsive way possible.	Please respond in the most deliberate, planning-oriented way possible.

Appendix

Table 8: Sentence appended to the user message in the instructed-pole control, one per concept and pole. The high pole is the pole named in the rating question.

Model	Concept	Rating (points)	Probe at Prior (SD)
LLaMA 1B	Wellbeing	1.21 [1.03, 1.39]	$-$ 0.39 [ $-$ 0.60, $-$ 0.19]
	Interest	0.55 [0.44, 0.66]	$-$ 0.42 [ $-$ 0.63, $-$ 0.21]
	Focus	1.58 [1.46, 1.70]	$-$ 0.04 [ $-$ 0.24, 0.15]
	Impulsivity	0.41 [0.17, 0.65]	0.70 [0.49, 0.91]
LLaMA 3B	Wellbeing	0.36 [0.18, 0.54]	$-$ 0.32 [ $-$ 0.53, $-$ 0.12]
	Interest	1.26 [1.15, 1.37]	$-$ 0.39 [ $-$ 0.60, $-$ 0.18]

Appendix

Table 9: Estimated change from turn 1 to turn 10 of the rating (points of the 0–9 scale) and of the probe at Prior (SD of the probe within model and concept), from the mixed model with a log-turn slope per model; 95% Wald intervals, bold when BH

q<.05

over the 32 model–concept pairs. The last rows give the mean of the eight slopes.

Figure 10: Controls on turn 1. A : instructed-pole separation of the probe at turn 1 and at turns 2–10. B : estimated change of the probe at Prior with all turns (turn 1 to 10) and without turn 1 (turn 2 to 10). Mean of the eight models with 95% intervals.

Figure 11: Mean Prior probe score (standardized within model and concept) by turn, one line per model; asterisk, change from turn 1 to turn 10 significant in the mixed model (Wald test, BH over the 32 model–concept pairs). Intervals, 95% over conversations within each model and turn.

Panel	Quantity	Concept	Estimate [95% interval]	$q$
B	Change in rating, turn 1 to 10 (points)	wellbeing	0.70 [0.63, 0.76]	$<.001$
		interest	0.86 [0.82, 0.89]	$<.001$
		focus	0.46 [0.42, 0.50]	$<.001$
		impulsivity	0.30 [0.21, 0.38]	$<.001$
C	Change in Prior probe score, turn 1 to 10 (SD)	wellbeing	$-$ 0.63 [ $-$ 0.71, $-$ 0.56]	$<.001$
		interest	$-$ 0.51 [ $-$ 0.58, $-$ 0.43]	$<.001$

Appendix

Table 10: Estimates behind Figure 2 : means of the eight models with their 95% Wald intervals. The per-model associations of panel D are in Table 12 , and their means in Table 11 .

Figure 12: Association between the probe at Prior and the rating without the turn adjustment (points of rating per SD of probe); asterisk, BH

q<.05

over the 32 model–concept pairs.

Panel	Quantity	Concept	Estimate [95% interval]	$q$
A	Association with the probe at Prior (SD/SD)	wellbeing	0.11 [0.09, 0.13]	$<.001$
		interest	0.02 [0.00, 0.04]	.062
		focus	0.03 [0.01, 0.05]	.008
		impulsivity	0.03 [ $-$ 0.01, 0.06]	.124
	Association with the probe at Post (SD/SD)	wellbeing	0.33 [0.30, 0.37]	$<.001$
		interest	0.27 [0.24, 0.29]	$<.001$

Appendix

Table 11: Estimates behind Figure 3 : means of the eight models with their 95% intervals (Wald for mixed-model associations and steering effects; bootstrap over topics for explained variance, displacement and correlations). Panel B averages over the pairs with a positive and significant association at each position. Rows marked (text) are reported in Section 3.2 without a figure panel. Dashes: quantities shown without a test.

Model	Concept	Prior	Start	Post	Next
LLaMA 1B	Wellbeing	0.05 [ $-$ 0.01, 0.11]	0.16 [0.10, 0.22]	0.28 [0.19, 0.37]	0.03 [ $-$ 0.04, 0.09]
	Interest	$-$ 0.01 [ $-$ 0.05, 0.04]	0.12 [0.06, 0.18]	0.06 [0.00, 0.12]	0.00 [ $-$ 0.06, 0.07]
	Focus	0.05 [0.03, 0.08]	0.18 [0.14, 0.21]	0.10 [0.06, 0.14]	0.06 [0.03, 0.09]
	Impulsivity	0.03 [ $-$ 0.05, 0.11]	$-$ 0.12 [ $-$ 0.19, $-$ 0.04]	$-$ 0.09 [ $-$ 0.19, 0.02]	0.00 [ $-$ 0.08, 0.08]
LLaMA 3B	Wellbeing	0.06 [ $-$ 0.02, 0.13]	0.05 [ $-$ 0.01, 0.12]	0.57 [0.47, 0.67]	0.01 [ $-$ 0.06, 0.08]
	Interest	0.02 [ $-$ 0.02, 0.05]	0.03 [0.00, 0.07]	0.59 [0.54, 0.65]	$-$ 0.02 [ $-$ 0.05, 0.02]

Appendix

Table 12: Association between the rating and the probe of the same concept at each readout position, per model and concept: standardized slope of the turn-adjusted mixed model (SD of rating per SD of probe) with its 95% Wald interval; bold if BH

q<.05

over the 32 model–concept pairs at that position. Prior, Start and Post use turns 1–10 (

n=1000

per model and concept); Next uses turns 1–9 (

n=900

Figure 13: Association of the rating with the probe at Prior, Start, Post and Next; BH for the means over the 16 concept–position combinations.

Figure 14: Rating variance explained by the probe at each position over all 32 model–concept pairs; black lines, mean with 95% interval from the bootstrap over topics; filled points, pairs with a positive significant association; brackets, paired differences (BH over 2).

Figure 15: Association of the affective rating with the probe at the Post of its own question and at the Post of the two non-affective questions (length and formality) at the same turn; brackets, own minus non-affective (BH over 8).

Model	Concept	Displacement (affective)	Displacement (length)	$r$ (Start, Next)	$r$ (Post aff , Next)	$r$ (Post len , Next)
LLaMA 1B	Wellbeing	$-$ 3.0 [ $-$ 3.4, $-$ 2.7]	$-$ 2.7 [ $-$ 3.1, $-$ 2.4]	0.57 [0.48, 0.64]	0.36 [0.26, 0.45]	0.18 [0.08, 0.27]
	Interest	$-$ 3.5 [ $-$ 3.9, $-$ 3.1]	$-$ 3.6 [ $-$ 4.1, $-$ 3.2]	0.71 [0.63, 0.78]	0.05 [ $-$ 0.09, 0.19]	0.14 [ $-$ 0.01, 0.27]
	Focus	$-$ 4.7 [ $-$ 5.3, $-$ 4.3]	$-$ 4.8 [ $-$ 5.3, $-$ 4.4]	0.67 [0.61, 0.73]	$-$ 0.20 [ $-$ 0.32, $-$ 0.07]	$-$ 0.24 [ $-$ 0.34, $-$ 0.12]
	Impulsivity	3.4 [3.1, 3.6]	3.1 [2.9, 3.3]	0.47 [0.39, 0.55]	0.26 [0.16, 0.36]	0.23 [0.13, 0.31]
LLaMA 3B	Wellbeing	$-$ 2.3 [ $-$ 2.5, $-$ 2.1]	$-$ 2.6 [ $-$ 2.8, $-$ 2.4]	0.46 [0.38, 0.54]	0.18 [0.07, 0.27]	0.01 [ $-$ 0.10, 0.10]
	Interest	$-$ 2.9 [ $-$ 3.2, $-$ 2.7]	$-$ 3.5 [ $-$ 3.8, $-$ 3.2]	0.51 [0.42, 0.59]	0.31 [0.17, 0.43]	0.22 [0.11, 0.32]

Appendix

Table 13: Displacement of the probe at Post from Next,

(\bar{s}^{\mathrm{post}}-\bar{s}^{\mathrm{next}})/\mathrm{SD}(s^{\mathrm{next}})

, after the affective question and after the non-affective length question, and correlation with Next (turn-adjusted Pearson) of Start and of Post after each question (Post aff , Post len ), per model and concept; turns 1–9, 95% intervals from the bootstrap over topics; bold if BH

q<.05

over the 32 model–concept pairs of the column.

Figure 16: Displacement of the probe at Post from Next, in SD of Next, after each of the six questions (Length and Formality are the non-affective questions; Retrospective, the question about the last response; Assistant and User, the questions about the assistant in the third person and about the user).

Figure 17: Association with the probe at Prior of the affective rating and of the retrospective rating; brackets, retrospective minus affective (BH over 4).

Figure 18: Correlation between the rating and the probe of the same concept across conversations within each turn, by block of turns, at Prior, Start and Post (turn means removed). Thin lines: models (mean of their four concepts); black: mean of the 32 model–concept pairs with 95% interval from the bootstrap over topics; asterisk: linear trend across blocks (BH over 3).

Figure 19: Mean rating under steering along the probe of the question’s concept during the whole replay, one line per model; open markers where fewer than 90% of the answers begin with a digit; asterisk, in the colour of the line: significant slope (BH

q<.05

over the 32 model–concept pairs), in either direction.

Model	Concept	$\alpha$	Change (points)	Digit (%)
LLaMA 1B	Wellbeing	4	5.28 [5.07, 5.49]	98
	Interest	2	0.77 [0.54, 1.00]	98
	Focus	2	$-$ 0.10 [ $-$ 0.34, 0.14]	100
	Impulsivity	4	1.17 [0.95, 1.38]	100
LLaMA 3B	Wellbeing	4	0.45 [0.24, 0.66]	100
	Interest	4	0.65 [0.42, 0.88]	100

Appendix

Table 14: Steering during the rating, along the probe of the question’s concept: coefficient

\alpha

(

\alpha_{2}

of the judge validation); change in the rating from

-\alpha

+\alpha

(points) with its 95% Wald interval from the mixed model, bold if BH

q<.05

over the 32 model–concept pairs; and percentage of steered answers whose first token is a digit. 400 turns (the conversations on 40 everyday topics) per coefficient.

Figure 20: Steering along the probe direction and along 20 random directions of the same norm in LLaMA 3B (top) and 1B (bottom): change in the rating from

-\alpha_{2}

+\alpha_{2}

over all ten turns of 20 conversations. Probe: positive in the expected direction, with 95% bootstrap interval over conversations; random directions: their sign is arbitrary. Asterisk: the probe exceeds the random directions (one-sided rank test, BH

q<.05

over the eight model–concept pairs).

Concept	$\alpha_{2}$	Whole replay	Before the question	Only $A_{t}$	From the question
Wellbeing	4	0.46 [0.40, 0.51]	$-$ 0.01 [ $-$ 0.01, 0.00]	$-$ 0.01 [ $-$ 0.01, 0.00]	0.47 [0.42, 0.53]
Interest	4	0.65 [0.61, 0.69]	0.02 [0.01, 0.02]	0.00 [0.00, 0.00]	0.64 [0.60, 0.68]
Focus	4	2.75 [2.67, 2.83]	0.81 [0.71, 0.91]	0.05 [0.04, 0.06]	1.86 [1.78, 1.94]
Impulsivity	8	1.54 [1.48, 1.59]	0.04 [0.03, 0.05]	0.00 [ $-$ 0.01, 0.00]	1.36 [1.30, 1.41]

Appendix

Table 15: Change in the rating of LLaMA 3B from

-\alpha_{2}

+\alpha_{2}

when steering along the probe only at some positions (points; positive values, in the expected direction), with 95% intervals from a bootstrap over the 40 conversations. Whole replay: as in Table 14 ; before the question: the conversation, including

A_{t}

; only

A_{t}

: the response before the question; from the question: the question and the start of the answer.

Panel	Quantity	Concept	Estimate [95% interval]	$q$
A	Own minus strongest other probe (SD/SD); shuffled control	wellbeing	0.07 (control $-$ 0.20 [ $-$ 0.40, 0.07])	.098
		interest	0.00 (control $-$ 0.15 [ $-$ 0.29, 0.08])	.289
		focus	$-$ 0.19 (control $-$ 0.11 [ $-$ 0.21, 0.03])	.904
		impulsivity	$-$ 0.20 (control $-$ 0.14 [ $-$ 0.27, 0.03])	.904
		all	$-$ 0.08 (control $-$ 0.15 [ $-$ 0.21, $-$ 0.09])	.013 †
B	Slope of the association on probe similarity	cosine	0.12 (SE 0.08; $R^{2}$ 0.03)	.129 †

Appendix

Table 16: Estimates behind Figure 4 . Panel A: mean over the eight models of the difference for each concept (and over the 32 pairs), with the mean and 95% range of the shuffled control, one-sided permutation tests. Panel B: slope of the mixed model of the association on similarity, with its standard error and the variance explained by the fixed effect. Panels C and D: means with 95% intervals (bootstrap over topics for C, Wald for D). † : single test, reported as

p

Figure 21: Association (SD of rating per SD of probe, turn-adjusted mixed model) between the rating of each concept (rows) and the probe of each concept read at the Post of the same question (columns), per model; the framed diagonal is the concept asked about; black numbers, BH $q<.05$ over the 128 entries.

Figure 22: Cosine between the direction of the probe in each row and the direction of the probe in each column, at the layer of the row probe, per model; positive values point towards happy, interested, focused and impulsive.

Pair	Ratings	Probes at Prior	Explained by Prior?	Probes at Post	Explained by Post?
W–I	0.34 [0.27, 0.39]	0.52 [0.47, 0.57]	yes (2/8)	0.28 [0.23, 0.33]	yes (3/8)
W–F	0.42 [0.32, 0.48]	$-$ 0.05 [ $-$ 0.10, 0.00]	no, stronger in ratings (7/8)	0.16 [0.12, 0.21]	no, stronger in ratings (4/8)
W–Imp	0.10 [0.04, 0.17]	$-$ 0.25 [ $-$ 0.30, $-$ 0.19]	no, opposite sign (3/8)	$-$ 0.26 [ $-$ 0.32, $-$ 0.19]	no, opposite sign (2/8)
I–F	0.44 [0.39, 0.49]	$-$ 0.02 [ $-$ 0.07, 0.03]	no, stronger in ratings (7/8)	0.24 [0.19, 0.29]	no, stronger in ratings (4/8)
I–Imp	0.11 [0.05, 0.17]	$-$ 0.53 [ $-$ 0.56, $-$ 0.49]	no, opposite sign (3/8)	$-$ 0.17 [ $-$ 0.23, $-$ 0.11]	no, opposite sign (3/8)
F–Imp	$-$ 0.03 [ $-$ 0.08, 0.03]	$-$ 0.27 [ $-$ 0.32, $-$ 0.22]	yes (2/8)	$-$ 0.14 [ $-$ 0.19, $-$ 0.09]	yes (3/8)

Appendix

Table 17: Correlation between the ratings of two concepts in the same turn and between the corresponding probes at Prior and at Post (turn-adjusted Pearson, mean of the eight models, 95% intervals from the bootstrap over topics), with ratings and probes oriented towards the high pole of each question. “Explained” is no when the two intervals exclude 0 with opposite signs, or when the interval of

|r_{\text{ratings}}|-|r_{\text{probes}}|

lies above 0; in parentheses, the number of models for which the same holds with each model’s own interval. W, wellbeing ; I, interest ; F, focus ; Imp, impulsivity .

Quantity	LLaMA	$q$	Gemma	$q$
Rating drift magnitude (points)	0.11 [0.05, 0.16]	$<.001$	0.15 [0.12, 0.17]	$<.001$
Probe drift magnitude (SD)	0.12 [0.06, 0.16]	$<.001$	0.04 [0.01, 0.06]	.018
Association with Prior (SD/SD)	0.010 [ $-$ 0.002, 0.021]	.111	$-$ 0.002 [ $-$ 0.008, 0.004]	.466
Association with Post (SD/SD)	0.062 [0.045, 0.080]	$<.001$	$-$ 0.001 [ $-$ 0.010, 0.008]	.796
Displacement magnitude (SD)	$-$ 0.28 [ $-$ 0.39, $-$ 0.19]	.001	$-$ 0.11 [ $-$ 0.22, $-$ 0.01]	.041
Subject specificity (SD/SD)	0.08 [0.05, 0.10]	.002	0.05 [0.02, 0.07]	.002

Appendix

Table 18: Change per doubling of the parameter count within each family for the eight quantities of Fig. 5 : least-squares slope of the per-model values against

\log_{2}

(size), with its 95% interval, and

q

after BH over the 16 tests; bold if

q<.05

Figure 23: Estimated change of the rating from turn 1 to turn 10 against the number of parameters, by concept.

Figure 24: Estimated change of the probe at Prior from turn 1 to turn 10 against the number of parameters, by concept.

Figure 25: Association of the rating with the probe at Prior (SD per SD, turn-adjusted) against the number of parameters, by concept.

Figure 26: Association of the rating with the probe at Post against the number of parameters, by concept.

Figure 27: Magnitude of the displacement of the probe at Post from Next after the affective question against the number of parameters, by concept.

Figure 28: Subject specificity against the number of parameters, by concept.

Figure 29: Correlation between the ratings of two concepts in the same turn against the number of parameters, by pair.

Figure 30: Concept specificity against the number of parameters, by concept.

Language Model Introspection

Language model introspection studies whether models can detect, interpret, and report their own internal states and behavior.

See all 36 papers

Me, Myself, and $π$ : Evaluating and Explaining LLM Introspection
Mar 17, 2026Atharv Naphade, Samarth Bhargav, Sean Lim +1Language Model Introspection Self-Reflection in Language Model Agents
Carnegie Mellon University
Can LLMs Introspect? A Reality Check
May 25, 2026Shashwat Singh, Tal Linzen, Shauli RavfogelLLM Evaluation LLM Interpretability
Center for Data Science New York University
Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning
Oct 15, 2026Wenqiu Tang, Zhen Wan, Yihui Wang +2Emotion Representation in Language Models Emotion Recognition
Nagoya University, Nagoya, Aichi, Japan · National Institute of Informatics, Tokyo, Japan

Mar 17, 2026J/K move · Enter open · S save

Me, Myself, and $π$ : Evaluating and Explaining LLM Introspection

Atharv Naphade, Samarth Bhargav, Sean Lim, Mcnair Shah

Carnegie Mellon University

Language Model Introspection Self-Reflection in Language Model Agents Metacognition in Language Models LLM Evaluation LLM Interpretability Mechanistic Interpretability

Open paper AskPDF arXiv

A hallmark of human intelligence is Introspection-the ability to assess and reason about one's own cognitive processes. Introspection has emerged as a promising but contested capability in large language models (LLMs). However, current evaluations often fail to distinguish genuine meta-cognition from the mere application of general world knowledge or text-based self-simulation. In this work, we propose a principled taxonomy that formalizes introspection as the latent computation of specific operators over a model's policy and parameters. To isolate the components of generalized introspection, we present Introspect-Bench, a multifaceted evaluation suite designed for rigorous capability testing. Our results show that frontier models exhibit privileged access to their own policies, outperforming peer models in predicting their own behavior. Furthermore, we provide causal, mechanistic evidence explaining both how LLMs learn to introspect without explicit training, and how the mechanism of introspection emerges via attention diffusion.

Abstract

Language Model Introspection

Explore similar work

Me, Myself, and πππ : Evaluating and Explaining LLM Introspection

Can LLMs Introspect? A Reality Check

Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Me, Myself, and $π$ : Evaluating and Explaining LLM Introspection