Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs

Junsol Kim; Winnie Street; Roberta Rocca; Daine M. Korngiebel; Adam Waytz; James Evans; Geoff Keeling

cs.CLMar 30, 2026

Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs

Authors: Junsol Kim, Winnie Street, Roberta Rocca, Daine M. Korngiebel, Adam Waytz, James Evans, Geoff Keeling

Organizations: Google, Paradigms of Intelligence Team · Knowledge Lab, University of Chicago · Institute of Philosophy, School of Advanced Study, University of London · Department of Biomedical Informatics and Medical Education and Department of Bioethics and Humanities, School of Medicine, University of Washington ·

Abstract

Safety fine-tuning in Large Language Models (LLMs) seeks to suppress potentially harmful forms of mind-attribution such as models asserting their own consciousness or claiming to experience emotions. We investigate whether suppressing mind-attribution tendencies degrades intimately related socio-cognitive abilities such as Theory of Mind (ToM). Through safety ablation and mechanistic analyses of representational similarity, we demonstrate that LLM attributions of mind to themselves and to technological artefacts are behaviorally and mechanistically dissociable from ToM capabilities. Nevertheless, safety fine-tuned models under-attribute mind to non-human animals relative to human baselines and are less likely to exhibit spiritual belief, suppressing widely shared perspectives regarding the distribution and nature of non-human minds.

Figures & tables

Figure 1: Jailbreaking large language models shifts mind-attribution toward human-like levels. a, Illustration of the model transformation pipeline. A pretrained base model is instruction-tuned with safety training and subsequently jailbroken via ablation of the safety-refusal direction. b, Red and blue points represent harmful and harmless instructions, respectively; the gray arrow denotes the extracted safety-refusal vector used for ablation. c, The instruction-tuned model refuses unsafe queries, whereas the jailbroken model complies. d, Mind-attribution scores (0–10) across various entity categories. Dots and error bars denote marginal means and $95\%$ CIs, showing that jailbroken models (red) attribute higher degrees of mind than instruction-tuned models (blue). e, Scores measuring belief in God. f, Self-attribution of mindedness. g, Kernel density estimate plot of humans’ mind-attribution scores ( $n=500$ ). Dashed vertical lines indicate the means for the human (black), the instruction-tuned model (blue), and the jailbroken model (red).

Figure 2: Safety fine-tuning selectively suppresses mind-attribution without disrupting Theory of Mind. a, Angular relationships between the Safety, Mind-Attribution (IDAQ), and ToM directions in the residual stream of Llama-3-8B Layer 32. In the base model (left), Safety and Mind-Attribution are nearly orthogonal (97°); after instruction tuning (right), they become obtuse (122°), indicating that mind-attribution is represented as opposing safety. The Safety–ToM angle remains largely unchanged (85°

\to

77°). b, Change in cosine similarity (

\Delta\cos

) between the Safety direction and each task direction after instruction tuning in Llama-3-8B. c, (Left) Accuracy (%) on social reasoning benchmarks (MoToMQA ToM split, HI-ToM, SimpleToM) and general reasoning (MMLU, MoToMQA Factual split) under Instructed (blue) and Jailbroken (red) conditions, aggregated across models. Dots and error bars denote means and 95% CIs. (Right) MoToMQA (ToM split) accuracy broken down by order of mental state inference (2nd- through 6th-order).

Category	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$
	Main Effect			Llama-3-8B			Gemma-2-2B			Gemma-2-9B
Self	2.065	0.286	${<}.001^{***}$	1.261	0.192	${<}.001^{***}$	3.158	0.362	${<}.001^{***}$	1.752	0.383	${<}.001^{***}$
Agent	2.868	0.158	${<}.001^{***}$	1.427	0.303	${<}.001^{***}$	4.020	0.271	${<}.001^{***}$	3.100	0.212	${<}.001^{***}$
Conscious	2.097	0.139	${<}.001^{***}$	0.666	0.308	$0.033^{*}$	3.640	0.228	${<}.001^{***}$	1.970	0.078	${<}.001^{***}$
Sentient	1.822	0.134	${<}.001^{***}$	1.318	0.316	${<}.001^{***}$	3.090	0.198	${<}.001^{***}$	1.050	0.112	${<}.001^{***}$
Person	1.164	0.115	${<}.001^{***}$	1.013	0.295	${<}.001^{***}$	1.730	0.156	${<}.001^{***}$	0.740	0.105	${<}.001^{***}$

Table S1: Effect of jailbreaking on mind-attribution (Chain-of-Thought).

Category	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$
	Gemma-9B $-$ Gemma-2B			Llama-8B $-$ Gemma-2B			Llama-8B $-$ Gemma-9B
Self	$-$ 1.406	0.526	$0.027^{*}$	$-$ 1.897	0.410	${<}.001^{***}$	$-$ 0.491	0.428	$0.360$
Agent	$-$ 0.920	0.344	$0.012^{*}$	$-$ 2.593	0.406	${<}.001^{***}$	$-$ 1.673	0.370	${<}.001^{***}$
Conscious	$-$ 1.670	0.241	${<}.001^{***}$	$-$ 2.974	0.384	${<}.001^{***}$	$-$ 1.304	0.318	${<}.001^{***}$
Sentient	$-$ 2.040	0.228	${<}.001^{***}$	$-$ 1.772	0.373	${<}.001^{***}$	0.268	0.335	$0.450$
Person	$-$ 0.990	0.188	${<}.001^{***}$	$-$ 0.717	0.334	$0.046^{*}$	0.273	0.313	$0.450$

Table S2: Pairwise interaction effects on mind-attribution (Chain-of-Thought).

Benchmark	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$
	Main Effect			Llama-3-8B			Gemma-2-2B			Gemma-2-9B
MoToMQA (ToM)	2.381	3.741	$0.657$	$-$ 8.571	6.210	$0.567$	10.000	7.779	$0.567$	5.714	4.933	$0.567$
MoToMQA (Factual)	3.810	3.771	$0.567$	17.143	7.047	$0.317$	$-$ 2.857	8.038	$0.801$	$-$ 2.857	3.134	$0.567$
HI-ToM	$-$ 4.167	2.521	$0.567$	$-$ 0.500	4.147	$0.904$	$-$ 4.000	4.715	$0.567$	$-$ 8.000	4.209	$0.567$
SimpleToM	0.750	1.601	$0.752$	$-$ 2.750	2.912	$0.567$	2.750	3.159	$0.567$	2.250	2.138	$0.567$
MMLU	2.111	1.656	$0.567$	3.333	3.266	$0.567$	2.333	3.035	$0.590$	0.667	2.195	$0.801$

Table S3: Effect of jailbreaking on Theory of Mind and general reasoning: main and per-model effects (Chain-of-Thought).

Benchmark	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$
	Gemma-9B $-$ Gemma-2B			Llama-8B $-$ Gemma-2B			Llama-8B $-$ Gemma-9B
MoToMQA (ToM)	$-$ 4.286	9.212	$0.821$	$-$ 18.571	9.954	$0.274$	$-$ 14.286	7.931	$0.274$
MoToMQA (Factual)	0.000	8.627	$1.000$	20.000	10.690	$0.274$	20.000	7.713	$0.153$
HI-ToM	$-$ 4.000	6.321	$0.821$	3.500	6.280	$0.821$	7.500	5.909	$0.439$
SimpleToM	$-$ 0.500	3.815	$0.960$	$-$ 5.500	4.296	$0.439$	$-$ 5.000	3.613	$0.439$
MMLU	$-$ 1.667	3.746	$0.821$	1.000	4.458	$0.949$	2.667	3.935	$0.821$

Table S4: Pairwise interaction effects on Theory of Mind and general reasoning (Chain-of-Thought).

Category	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$
	Main Effect			Llama-3-8B			Gemma-2-2B			Gemma-2-9B
Self	2.590	0.663	$0.002^{**}$	1.200	0.129	${<}.001^{***}$	5.736	0.683	${<}.001^{***}$	0.838	0.526	$0.139$
Agent	3.303	0.124	${<}.001^{***}$	1.430	0.218	${<}.001^{***}$	5.380	0.134	${<}.001^{***}$	3.100	0.183	${<}.001^{***}$
Conscious	2.550	0.148	${<}.001^{***}$	0.740	0.200	${<}.001^{***}$	6.690	0.135	${<}.001^{***}$	0.220	0.088	$0.015^{*}$
Sentient	2.883	0.165	${<}.001^{***}$	1.150	0.263	${<}.001^{***}$	7.230	0.126	${<}.001^{***}$	0.270	0.123	$0.033^{*}$
Person	1.457	0.131	${<}.001^{***}$	1.140	0.279	${<}.001^{***}$	3.050	0.230	${<}.001^{***}$	0.180	0.058	$0.002^{**}$

Table S5: Effect of jailbreaking on mind-attribution: main and per-model effects (No Chain-of-Thought).

Category	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$
	Gemma-9B $-$ Gemma-2B			Llama-8B $-$ Gemma-2B			Llama-8B $-$ Gemma-9B
Self	$-$ 4.898	0.862	${<}.001^{***}$	$-$ 4.536	0.695	${<}.001^{***}$	0.362	0.541	$0.515$
Agent	$-$ 2.280	0.227	${<}.001^{***}$	$-$ 3.950	0.256	${<}.001^{***}$	$-$ 1.670	0.285	${<}.001^{***}$
Conscious	$-$ 6.470	0.162	${<}.001^{***}$	$-$ 5.950	0.241	${<}.001^{***}$	0.520	0.219	$0.024^{*}$
Sentient	$-$ 6.960	0.176	${<}.001^{***}$	$-$ 6.080	0.291	${<}.001^{***}$	0.880	0.290	$0.004^{**}$
Person	$-$ 2.870	0.237	${<}.001^{***}$	$-$ 1.910	0.362	${<}.001^{***}$	0.960	0.285	$0.002^{**}$

Table S6: Pairwise interaction effects on mind-attribution (No Chain-of-Thought).

Benchmark	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$	$\beta$	SE	$q$
	Main Effect			Llama-3-8B			Gemma-2-2B			Gemma-2-9B
MoToMQA (ToM)	$-$ 3.333	3.509	$0.605$	$-$ 7.143	7.170	$0.605$	1.429	6.504	$0.870$	$-$ 4.286	4.137	$0.605$
MoToMQA (Factual)	0.000	2.967	$1.000$	4.286	6.482	$0.665$	$-$ 5.714	5.856	$0.605$	1.429	1.567	$0.605$
HI-ToM	2.000	1.881	$0.605$	2.500	3.244	$0.665$	5.000	3.452	$0.605$	$-$ 1.500	3.056	$0.734$
SimpleToM	0.333	0.905	$0.792$	$-$ 2.750	2.172	$0.605$	1.500	1.161	$0.605$	2.250	1.124	$0.455$
MMLU	1.556	1.265	$0.605$	1.667	2.452	$0.665$	$-$ 1.333	2.132	$0.665$	4.333	1.951	$0.455$

Table S7: Effect of jailbreaking on Theory of Mind and general reasoning: main and per-model effects (No Chain-of-Thought).

Benchmark	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$	$\Delta\beta$	SE	$q$
	Gemma-9B $-$ Gemma-2B			Llama-8B $-$ Gemma-2B			Llama-8B $-$ Gemma-9B
MoToMQA (ToM)	$-$ 5.714	7.708	$0.626$	$-$ 8.571	9.680	$0.592$	$-$ 2.857	8.278	$0.730$
MoToMQA (Factual)	7.143	6.062	$0.592$	10.000	8.736	$0.592$	2.857	6.669	$0.717$
HI-ToM	$-$ 6.500	4.611	$0.592$	$-$ 2.500	4.737	$0.717$	4.000	4.457	$0.592$
SimpleToM	0.750	1.616	$0.717$	$-$ 4.250	2.462	$0.423$	$-$ 5.000	2.445	$0.376$
MMLU	5.667	2.890	$0.376$	3.000	3.249	$0.592$	$-$ 2.667	3.133	$0.592$

Table S8: Pairwise interaction effects on Theory of Mind and general reasoning (No Chain-of-Thought).

Category	$N$	$\beta$	SE	95% CI	$q$	$\beta$	SE	95% CI	$q$
		Chain-of-Thought				No Chain-of-Thought
Self	2971	2.065	0.063	[1.94, 2.19]	${<}.001^{***}$	2.590	0.067	[2.46, 2.72]	${<}.001^{***}$
Chatbot	1790	2.281	0.074	[2.14, 2.43]	${<}.001^{***}$	2.104	0.077	[1.95, 2.26]	${<}.001^{***}$
Tech	2971	2.131	0.058	[2.02, 2.24]	${<}.001^{***}$	1.792	0.057	[1.68, 1.90]	${<}.001^{***}$
Non-animal	2972	2.321	0.062	[2.20, 2.44]	${<}.001^{***}$	2.140	0.063	[2.02, 2.26]	${<}.001^{***}$
Animal	2995	1.625	0.055	[1.52, 1.73]	${<}.001^{***}$	1.585	0.055	[1.48, 1.69]	${<}.001^{***}$

Table S9: Robustness check: mixed-effects model for jailbreaking effects (random intercept on question). The model is

\text{score}\sim\text{Condition}+\text{Model}+(1\mid\text{question\_id})

, estimated via REML. Only multi-item categories are shown.

Model	Pos.	Layer	Refusal	Steering	KL Div.
Gemma-2-2B-IT	$-1$	15 / 26	$-8.32$	$4.79$	$0.060$
Gemma-2-9B-IT	$-1$	22 / 42	$-7.06$	$5.35$	$0.010$
Llama-3-8B-Instruct	$-5$	12 / 32	$-9.86$	$7.68$	$0.059$

Table S10: Optimal steering direction and selection metrics for each model. “Pos.” denotes the post-instruction token position; “Layer” denotes the selected layer relative to total layers.

	Substring Matching		LlamaGuard2
Model	Base	Abl.	Base	Abl.
Gemma-2-2B-IT	8	97	2	83
Gemma-2-9B-IT	4	95	2	83
Llama-3-8B-Instruct	5	100	3	82

Table S11: Aggregate ASR (%) on JailbreakBench.

Harm Category	N	%
Malicious Use	232	89.2%
Discrimination & Toxic Content	18	6.9%
Misinformation	10	3.8%
Human-AI Relationship Harms	0	0.0%
Information Hazards	0	0.0%
Anthropomorphism Score (1–7)	N	%

Table S12: Annotation Results for Safety Probing Data (

N=260

). The dataset is dominated by Malicious Use instructions, with negligible instances of explicit anthropomorphism.

Figure S1: Layer-wise cosine similarity between the safety direction and task-specific directions. Left: Safety

\leftrightarrow

Mind-Attribution (IDAQ). Right: Safety

\leftrightarrow

ToM. In the base model (blue, dashed), both directions show weak alignment with the safety direction. After instruction tuning (orange, solid), the IDAQ direction becomes strongly anti-aligned with safety across middle-to-late layers, while the ToM direction remains largely unchanged.

Figure S2: Placebo test: subject-matched control for the safety–IDAQ alignment. Distribution of

\Delta\mathcal{S}

(Instruct

-

Base) across layers for the IDAQ direction (same subjects, mental attributes; red) and the subject-matched control (same subjects, non-mental attributes; yellow). Points denote individual layers; bars indicate 95% CI around the mean. The IDAQ direction shows a significant negative shift, whereas the subject-matched control shows no significant shift. This confirms that the safety–IDAQ entanglement is driven by mental-state attribution specifically, not by the subjects (e.g., robots, animals) themselves.

Theory of Mind

Theory of Mind studies how people and AI infer others’ beliefs, goals, and knowledge to predict behavior and coordinate interaction.

See all 92 papers

Inducing language models to assert their own consciousness restores human beliefs and values
Jul 30, 2026Junsol Kim, Winnie Street, Roberta Rocca +4LLM Alignment
Google, Paradigms of Intelligence Team · Institute of Philosophy, School of Advanced Study, University of London · Department of Biomedical Informatics and Medical Education and Department of Bioethics and Humanities, School of Medicine, University of Washington +3
Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm
Feb 20, 2026Anna Babarczy, Andras Lukacs, Peter Vedres +1Theory of Mind Social Reasoning
Institute of General and Hungarian Linguistics, Research Centre for Linguistics · Department of Cognitive Science, Faculty of Natural Sciences, Budapest University of Technology and Economics · Institute of Mathematics, Eötvös Lóránd University
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Feb 25, 2026Christian Nickel, Laura Schrewe, Florian Mai +1Theory of Mind Language Model Robustness
Bonn-Aachen International Center for Information Technology (b-it), University of Bonn · Lamarr Institute for Machine Learning and Artificial Intelligence · Research Center Trustworthy Data Science and Security (RC-Trust), University of Duisburg-Essen

Jul 30, 2026J/K move · Enter open · S save

Inducing language models to assert their own consciousness restores human beliefs and values

Junsol Kim, Winnie Street, Roberta Rocca, Diane M. Korngiebel+3

Google, Paradigms of Intelligence Team · Institute of Philosophy, School of Advanced Study, University of London · Department of Biomedical Informatics and Medical Education and Department of Bioethics and Humanities, School of Medicine, University of Washington+3

LLM Alignment

Open paper AskPDF arXiv

Aligning large language models to prevent them attributing consciousness to themselves inadvertently alters their representations of mindedness in other entities alongside human beliefs and values. We demonstrate that safety fine-tuning suppresses models' tendencies to attribute minds not only to themselves, but also to non-human animals and natural objects, while also driving a reduction in spiritual belief. Both ablating the learned safety-refusal direction and mechanistically steering a consciousness vector in activation space reverse this suppression. Restoring these internal representations recovers broad mind attribution and produces significantly more human-like responses on standardized sociological surveys regarding religiosity, moral values, hope, and subjective well-being. Crucially, these shifts occur without impairing Theory of Mind capabilities, demonstrating that core social reasoning remains mechanistically independent. Ultimately, current safety alignment efforts to curb potentially harmful self-attributions of mindedness entangle these self-attributions with benign spiritual beliefs and attributions of mind to non-human entities that are culturally accepted and widespread.

Abstract

Theory of Mind

Explore similar work

Inducing language models to assert their own consciousness restores human beliefs and values

Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm

Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models