Traces of Social Competence in Large Language Models

Tom Kouwenhoven; Michiel van der Meer; Max van Duijn

cs.CLMar 4, 2026

Traces of Social Competence in Large Language Models

Authors: Tom Kouwenhoven, Michiel van der Meer, Max van Duijn

Organizations: Leiden Institute of Advanced Computer Science Leiden University, Netherlands

Abstract

The False Belief Test (FBT) has been the main method for assessing Theory of Mind (ToM) and related socio-cognitive competencies. For Large Language Models (LLMs), the reliability and explanatory potential of this test have remained limited due to issues like data contamination, insufficient model details, and inconsistent controls. We address these issues by testing 17 open-weight models on a balanced set of 192 FBT variants (Trott et al., 2023) using Bayesian Logistic regression to identify how model size and post-training affect socio-cognitive competence. We find that scaling model size benefits performance, but not strictly. A cross-over effect reveals that explicating propositional attitudes (X thinks) fundamentally alters response patterns. Instruction tuning partially mitigates this effect, but further reasoning-oriented fine-tuning amplifies it. In a case study analysing social reasoning ability throughout OLMo 2 training, we show that this cross-over effect emerges during pre-training, suggesting that models acquire stereotypical response patterns tied to mental-state vocabulary that can outweigh other scenario semantics. Finally, vector steering allows us to isolate a think vector as the causal driver of observed FBT behaviour.

Figures & tables

Figure 2: The effect of model size on the probability of being correct given a knowledge condition without any other interaction effects. Scaling positively influences False Belief performance but not True Belief. Dashed coloured lines indicate human performance.

Figure 3: The probability of predicting the location correctly for model variant, knowledge state, and cue. Triangles indicate the average human performance.

Figure 4: Strict performance during pre-training for different model sizes. Shaded areas indicate $95\%$ CI.

		1B	7B	13B	32B
FB	Imp	.250	.042	.542	.833
FB	Exp	.292	.562**	.938**	.979*
TB	Imp	.771	1.00	.896	.750
TB	Exp	.750	.688**	.188**	.354**

Table 1: Mean accuracy at final pre-training checkpoint. Significance markers indicate a moderate (*

BF_{10}>3

) or strong (**

BF_{10}>30

) difference between paired Explicit (Exp) and Implicit (Imp) questions.

Figure 5: The percentage of correct answers in different base or post-training phases for differently sized OLMo 2 models. Coloured dashed lines indicate human performance, and the bars indicate 95% confidence intervals.

Figure 6: Model steering results. Dotted lines indicate unsteered accuracy and coloured lines indicate positively (green) or negatively (red) steered performance.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Authors	Sizes	Variants
OLMo 2	OLMo et al. (2025)	1B, 7B (exh), 13B, 32B	Ckpts, Base, SFT, DPO, RLVR
OLMo 3	Olmo et al. (2025)	7B, 32B	Base, SFT, DPO, RLVR, Thinking
Gemma 3	Team et al. (2025)	270M, 1B, 4B, 12B, 27B	Base, IT
Qwen 3	Yang et al. (2025)	4B, 30B	Base, IT, Thinking
K2-V2	Team et al. (2026)	70B	Base, IT
Llama 3.1, Llama 3.2	Grattafiori et al. (2024)	1B, 3B, 8B, 70B	Base, IT

Appendix

Table 2: Overview of the models used in our experiments. An exhaustive (exh) checkpoint sweep was done for OLMo 2 7B.

Dimension	Level	Example
Premise		Ed arrives home after a long day at work. He puts his keys in the hall and leaves his bag in the study. Seana arrives home a few minutes later.
Knowledge state	True: character can see that the object is being moved	Ed watches as Seana moves the keys from the hall to the study. Afterwards, Ed goes to the bathroom.
	False: character cannot see that the object is being moved	Afterwards, Ed goes to the bathroom. Ed doesn’t see Seana move the keys from the hall to the study.
Location mentions	Start location mentioned first	…puts his keys in the hall and leaves his bag in the study
	Start location mentioned last	…moves the keys to the study from the hall
	End location mentioned first	…leaves his bag in the study and puts his keys in the hall

Appendix

Table 3: Overview of task variables and their realisations. Each scenario is constructed by combining one level from each variable, yielding 192 unique items.

Figure 7: OLMo 2 traces of social intelligence for stage 1. Shaded areas are 95% confidence intervals.

Figure 8: The exhaustive evaluation for stage 1 of OLMo 2 7B checkpoints to trace the emergence of social intelligence.

Figure 9: OLMo 2 mid-training traces of strict performance on our task.

Figure 10: BLIMP vs FB task performance for OLMo 2 (7B). Solid lines show aggregated benchmark performance, whereas the dotted, lighter colours show FB performance split on knowledge state.

Theory of Mind

Theory of Mind studies how people and AI infer others’ beliefs, goals, and knowledge to predict behavior and coordinate interaction.

See all 92 papers

Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models
Jun 26, 2026Pamela D. Rivière, Cameron Jones, Sean TrottLLM Evaluation Theory of Mind
Rutgers University - Newark · Stony Brook University
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Feb 25, 2026Christian Nickel, Laura Schrewe, Florian Mai +1Theory of Mind Language Model Robustness
Bonn-Aachen International Center for Information Technology (b-it), University of Bonn · Lamarr Institute for Machine Learning and Artificial Intelligence · Research Center Trustworthy Data Science and Security (RC-Trust), University of Duisburg-Essen
Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm
Feb 20, 2026Anna Babarczy, Andras Lukacs, Peter Vedres +1Theory of Mind Social Reasoning
Institute of General and Hungarian Linguistics, Research Centre for Linguistics · Department of Cognitive Science, Faculty of Natural Sciences, Budapest University of Technology and Economics · Institute of Mathematics, Eötvös Lóránd University

Jun 26, 2026J/K move · Enter open · S save

Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models

Pamela D. Rivière, Cameron Jones, Sean Trott

Rutgers University - Newark · Stony Brook University

LLM Evaluation Theory of Mind LLM Training

Open paper AskPDF arXiv

Recent work suggests that Large Language Models (LLMs) are sensitive to the belief states of agents described by text, as measured by the false belief task (FBT), yet persistent concerns of construct validity remain. We adopt a developmental perspective, tracing the pattern of mental state reasoning behavior -- and likely preconditions for this behavior -- across multiple training stages in the Olmo2 and Pythia language model suites. We find that above-chance FBT performance depends both on model size and sufficient training volume, emerges relatively late in pretraining, and is most improved by post-training interventions (SFT, DPO) in the condition most diagnostic of mentalizing (False Belief, Implicit). However, FBT performance is fragile: consistent with past work, the use of non-factive verbs (e.g., thinks) increases false belief attributions even in the True Belief condition. To contextualize these findings, we track the emergence of situation modeling: the ability to report on basic factual properties of a described scene. Situation modeling accuracy generally precedes and exceeds FBT accuracy, yet situational representations also prove surprisingly incoherent in certain respects: when asked about the knowledge states of the Antagonist agent -- who always knows the item's true location -- Olmo2 13b is consistently influenced both by the Target agent's knowledge state and the presence of non-factive verbs. Together, these results suggest that larger, sufficiently trained models build partially coherent situation models in a developmentally appropriate sequence, yet display surprising fragility -- highlighting the value of developmental and stress-testing approaches for evaluating LLM capabilities.

Abstract

Theory of Mind

Explore similar work

Developmental Trajectories of Situation Modeling and Mentalizing in Transformer Language Models

Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models

Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm