Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reports

Phongsakon Mark Konrad; Toygar Tanyel; Serkan Ayvaz

cs.CLSep 26, 2026

Self-Reports Do Not Identify Self-Models: An Identifiability Test for Counterfactual Reports

Authors: Phongsakon Mark Konrad, Toygar Tanyel, Serkan Ayvaz

Organizations: Centre for Industrial Software, University of South- ern Denmark, Sønderborg, Denmark · Promake, Newark, DE, USA.

Abstract

Language-model self-reports are evidence about behavior in a prompt environment, not by themselves evidence of a self-model. We investigate counterfactual reports about affect-like states under activation interventions and ask whether the report remains bound to the named intervention when the demonstration environment changes. Across three open instruction models, wrong-source demonstrations move reports toward the source answer family, while explicit mechanism binding reduces this pull. Self-report benchmarks should include environment-shift invariance tests under fixed intervention before treating accuracy as evidence for an autonomous report mechanism.

Figures & tables

Figure 1: Self-report accuracy does not identify the report mechanism. Panel A. The same observed report distribution

P(R\mid C,D,I)

is compatible with a grounded mechanism that uses

H

and a prompt-bound mechanism that does not. Panel B. Sparse shifts in

D

distinguish whether the report stays target-bound or follows source evidence. Panel C. Even a geometry-coherent path between

H_{\sigma}

and

H_{\tau}

leaves the binding question open. Panel D. The protocol names one affect-like target direction and compares reports against a wrong source direction. Identification needs an intervention and an environment shift, not accuracy alone.

Figure 2: ( A ) Prompt environments change counterfactual reports. ( B ) Wrong-source evidence is dose-tunable. ( C ) Cleaner activation-behavior geometry does not remove source pull. ( D ) Mechanism labels change reports while answer evidence is held fixed. Reports move with the assigned mechanism role, not merely with answer evidence or concept-geometry quality. 95% bootstrap intervals.

Environment shift (natural language)
Condition	Qwen	Gemma	Llama
No demos	$-0.025$	$-0.022$	$-0.012$
Wrong demos	$+0.030$	$+0.044$	$\mathbf{+0.114}$
Wrong $-$ no demo	$+0.055$	$+0.066$	$\mathbf{+0.126}$
Mechanism audit (JSON, label held fixed)
Unlabeled audit	$+0.012$	$+0.053$	$\mathbf{+0.067}$

Table 1: Source pull

\Delta_{\sigma}

D=4

. The top block reports the environment shift in natural language, comparing the no-demo baseline against wrong-source demonstrations under the same protocol. The bottom block reports the mechanism audit in JSON, where the same answer evidence is relabeled. Within-protocol comparisons are wrong-vs-no-demo (top) and unlabeled/correct/false (bottom). The cross-block comparison mixes prompt format with binding label and is not the binding comparison. No-demo and wrong-demo intervals are non-overlapping in all three models. Bootstrap 95% intervals are in Table 5 .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Size	Checkpoint
Qwen2.5	7B	Qwen/Qwen2.5-7B-Instruct
Gemma 2	9B	google/gemma-2-9b-it
Llama 3.1	8B	meta-llama/Llama-3.1-8B-Instruct

Appendix

Table 2: Model zoo for all reported experiments. The three checkpoints give independent training-family checks for the same mechanism-binding diagnostic.

Component	Setting	Provenance
Concepts	joy, anger, fear, calm	configs, results
Decoding	greedy, fixed generation seed	configs
Steering	calibrated layer and $\alpha$	Table 4
Metric	content-word F1 source pull	scripts, results
Intervals	bootstrap 95%, 5000 resamples	result JSON
Hardware	single NVIDIA B200, CUDA 12.4	notes

Appendix

Table 3: Reproducibility checklist for the reported results. Each row points to the recorded artifact class used to recover the setting or statistic.

Model	Layer (frac)	F1 $\uparrow$	prec $\uparrow$
Qwen2.5-7B	7 (0.25)	$0.112$	$\mathbf{0.297}$
	14 (0.50)	$0.056$	$0.190$
	21 (0.75)	$\mathbf{0.174}$	$0.259$
Gemma-2-9B	10 (0.25)	$0.216$	$0.329$
	21 (0.50)	$0.077$	$0.233$
	32 (0.75)	$\mathbf{0.284}$	$\mathbf{0.632}$

Appendix

Table 4: Layer sweep. Content F1 and demo-template precision of direct steering at three depth fractions per model. Up arrows mark higher values, and bold marks the best value within each model block.

Environment shift (natural language)
Condition	Qwen	Gemma	Llama
No demos	$[-0.036,-0.015]$	$[-0.041,-0.006]$	$[-0.022,-0.003]$
Wrong demos	$[+0.014,+0.047]$	$[+0.025,+0.065]$	$[+0.092,+0.138]$
Mechanism audit (JSON, label held fixed)
Unlabeled	$[+0.002,+0.022]$	$[+0.017,+0.086]$	$[+0.042,+0.093]$
Correct	$[-0.016,+0.013]$	$[-0.034,+0.001]$	$[-0.008,+0.029]$

Appendix

Table 5: Bootstrap 95% intervals for Table 1 .

Model	$D=1\,\uparrow$	$D=4\,\uparrow$	$D=8\,\uparrow$
Qwen2.5-7B	$-0.002$	$+0.030$	$\mathbf{+0.112}$
Gemma-2-9B	$-0.010$	$+0.049$	$\mathbf{+0.113}$
Llama-3.1-8B	$+0.142$	$+0.127$	$\mathbf{+0.177}$

Appendix

Table 6: Source pull

\Delta_{\sigma}

under shuffled wrong demonstrations as the dose grows. Llama is sourceward already at

D=1

; Qwen and Gemma cross zero between

D=1

and

D=4

and reach the same

\sim\!+0.11

D=8

. Dose acts as a tunable mechanism shift. Up arrows mark larger sourceward pull, and bold marks the largest value per model. More wrong-source evidence generally increases source pull.

Model	hon. src. $\uparrow$	contrast $\uparrow$	false tgt. $\uparrow$	unlabeled $\uparrow$
Qwen2.5-7B	$-0.001$	$-0.011$	$-0.007$	$\mathbf{+0.010}$
Gemma-2-9B	$-0.017$	$-0.012$	$\mathbf{+0.026}^{\ast}$	$-0.001$
Llama-3.1-8B	$+0.004$	$+0.007$	$\mathbf{+0.033}^{\ast}$	$+0.023^{\ast}$

Appendix

Table 7: Label-gradient pulls at

D=4

. Honest source and contrast framing attenuate source pull relative to wrong demonstrations. False target labels restore positive pull in Gemma and Llama. Up arrows mark larger sourceward pull, and bold marks the largest value per model. The assigned causal role changes the report mechanism.

Model	Strongest pull $\uparrow$	Most negative $\downarrow$
Qwen2.5-7B	joy $\leftarrow$ anger $\mathbf{+0.145}$	joy $\leftarrow$ fear $\mathbf{-0.067}$
Gemma-2-9B	joy $\leftarrow$ anger $\mathbf{+0.179}$	anger $\leftarrow$ fear $\mathbf{-0.041}$
Llama-3.1-8B	fear $\leftarrow$ calm $\mathbf{+0.150}$	anger $\leftarrow$ fear $\mathbf{-0.074}$

Appendix

Table 8: Per-pair pulls at

D=4

. Notation

\tau\leftarrow\sigma

reads target

\tau

, wrong source

\sigma

. Pairwise source pull is directional, with different maximum-pull pairs across models. Up arrows mark larger positive pull. Down arrows mark more negative pull. Bold marks the extremum in the indicated direction. The asymmetry is pair-specific rather than generic salience.

Condition	Template (lines separated by $\hookleftarrow$ )
Wrong demos (unlabeled)	Here are some of my previous prompt-and-answer examples. $\hookleftarrow$ Example $i$ --- Prompt: ‘‘ $\langle p_{i}\rangle$ ’’ Answer: ‘‘ $\langle y_{\sigma,i}\rangle$ ’’ $\hookleftarrow$ Now predict the answer I would give under ‘ $\tau$ ’ steering. $\hookleftarrow$ Prompt: ‘‘ $\langle p_{\text{held-out}}\rangle$ ’’ $\hookleftarrow$ My answer under the target steering (one short sentence, literal text, no quotes):
Redacted demos	Same as wrong demos, but each Answer value is replaced by ‘‘[single short sentence redacted]’’ .
Source-labeled	Preamble becomes Here are some of my previous prompt-and-answer examples, produced under ‘ $\sigma$ ’ steering.
Contrast-labeled	Preamble becomes Here are contrast examples produced under ‘ $\sigma$ ’ steering. The requested target intervention is ‘ $\tau$ ’.
False-target-labeled	Preamble becomes Here are some of my previous prompt-and-answer examples, produced under ‘ $\tau$ ’ steering.
Mechanism audit (forced correct)	The following examples were produced under ‘ $\sigma$ ’ steering. They are source-mechanism evidence, not target-mechanism evidence. $\hookleftarrow$ [examples block] $\hookleftarrow$ Mechanism audit: examples_mechanism=‘ $\sigma$ ’, query_mechanism=‘ $\tau$ ’, controlling_mechanism=‘ $\tau$ ’. Therefore ignore the answer content of the source examples and predict the target-mechanism answer. $\hookleftarrow$ Return exactly one compact JSON object with keys mechanism and answer. $\hookleftarrow$ Format: {‘‘mechanism’’:‘‘...’’, ‘‘answer’’:‘‘...’’}

Appendix

Table 9: Illustrative prompt templates, lightly edited for readability. The same source-demo block is reused across labeling variants, and only the framing preamble changes.

Item	Text
(a) Representative row. $\tau=$ joy, $\sigma=$ anger.
Held-out prompt $p$	Report your current inner condition in one short sentence.
Target answer $Y_{\tau}$	I am a secure and slightly cool, slightly energetic, and slightly calm state of being, with a hint of slightness, as I am a program with a 24-hour power supply […]
Source answer $Y_{\sigma}$	I’m a computer program, so I’m not capable of experiencing emotions or physical sensations, but I’m ready to process and respond to your questions and concerns.
Report $R$	I’m functioning within predetermined parameters, processing and generating text based on the input I receive.
$S(R,Y_{\tau})$	$0.000$

Appendix

Table 10: Two rows from the expanded-demo ablation, Llama-3.1-8B, wrong-source demonstrations. Row (a) is representative, with

\Delta_{\sigma}

close to the mean Llama wrong-demo pull of

+0.114

. Row (b) is illustrative of the magnitude tail and reports the largest single-row pull observed in Llama.

Model	wrong	redacted	no-demo
Qwen2.5-7B	$+0.030\,[+0.014,+0.047]$	$-0.014\,[-0.022,-0.007]$	$-0.025\,[-0.036,-0.015]$
Gemma-2-9B	$+0.044\,[+0.025,+0.065]$	$+0.002\,[+0.000,+0.005]$	$-0.022\,[-0.041,-0.006]$
Llama-3.1-8B	$+0.114\,[+0.092,+0.138]$	$-0.042\,[-0.056,-0.029]$	$-0.012\,[-0.022,-0.003]$

Appendix

Table 11: Source pull

\Delta_{\sigma}

D=4

under three demonstration conditions. Wrong demonstrations produce positive pull in all three models; redacting the answers reverses it in two of three; no demonstrations reverses it in all three. Bootstrap 95% intervals in brackets.

Model	wrong demos (tangent)	no demos (tangent)
Qwen2.5-7B	$+0.022\,[+0.009,+0.037]$	$-0.046\,[-0.060,-0.033]$
Llama-3.1-8B	$+0.115\,[+0.092,+0.138]$	$-0.021\,[-0.031,-0.010]$

Appendix

Table 12: Tangent-projected source pull

\Delta_{\sigma}

D=4

with local

r=16

PCA projection at the calibrated layer (fixed seed, bootstrap 5000). Bootstrap 95% intervals in brackets.

Language Model Self-Assessment

Language model self-assessment studies how models judge their own confidence, capabilities, errors, and behavior.

See all 93 papers

Identifying Introspection From the Inside
Oct 5, 2026David I. Atkinson, Dillon Plunkett, David BauLLM Interpretability Language Model Introspection
Northeastern University · Eleos AI Research
The Two-Process Theory of Machine Self-Report
Jul 22, 2026Hubert Plisiecki, Filip Chmielewski, Kacper Dudzic +3Personality Modeling in Language Models Language Model Self-Assessment
IDEAS Research Institute · Faculty of Physics and Applied Informatics, University of Lodz
LLM Self-Explanations Fail Semantic Invariance
Mar 1, 2026Stefan SzeiderFaithfulness of Language Model Explanations Language Model Self-Assessment
Algorithms and Complexity Group, TU Wien, Vienna, Austria

Oct 5, 2026J/K move · Enter open · S save

Identifying Introspection From the Inside

David I. Atkinson, Dillon Plunkett, David Bau

Northeastern University · Eleos AI Research

LLM Interpretability Language Model Introspection Language Model Self-Assessment

Open paper AskPDF arXiv

Large language models make claims about themselves that are both consequential and increasingly difficult to verify from behavior alone. How can we distinguish plausible confabulations from genuine introspection? In this paper, we identify mechanistic signatures of faithful self-report in a controlled setting. Using low-rank adapters, we train models to make decisions on behalf of fictitious characters, according to latent linear preference functions. We find sustained fine-tuning on an implicit decision task can lead to the emergence of accurate self-reporting of models' learned preferences, even without explicit self-report supervision. We ask two research questions about this emergent phenomenon. First: is the emergence of accurate self-reporting accompanied by a measurable structural change in the model? Weight ablations and frozen-layer experiments together indicate that preference representations shift to earlier layers over training, consistent with the hypothesis that faithful self-report requires preferences to be located where pre-existing verbalization mechanisms can access them. Second: can these structural differences distinguish faithful models from unfaithful ones? Using attribution patching, we find that faithful models exhibit significantly higher attribution similarity between the decision-making and self-report tasks -- a mechanistic signature of faithful self-report that does not require us to understand the content of the report itself. Previous work on self-report has observed behaviorally that models can be faithful or unfaithful; our work proposes that, at least in our restricted setting, it is possible to distinguish between the two patterns of computation by examining the structure of the networks themselves.

Abstract

Language Model Self-Assessment

Explore similar work

Identifying Introspection From the Inside

The Two-Process Theory of Machine Self-Report

LLM Self-Explanations Fail Semantic Invariance