Reading Too Much Into It? Interpretive Preferences and Commitment in LLM Mentalisation under Ambiguity

Stephanie Fong; Yiwen Jiang; Qingyang Xu; Zimu Wang; Yaling Shen; Xiangyu Zhao; Jiahe Liu; Yinghao Zhang

cs.CLOct 20, 2026

Reading Too Much Into It? Interpretive Preferences and Commitment in LLM Mentalisation under Ambiguity

Authors: Stephanie Fong, Yiwen Jiang, Qingyang Xu, Zimu Wang, Yaling Shen, Xiangyu Zhao, Jiahe Liu, Yinghao Zhang,

Organizations: Monash University · University of Liverpool · Orygen, The University of Melbourne

Abstract

Large Language Models (LLMs) are increasingly used for interpersonal advice, potentially shaping how users interpret ambiguous social encounters. Existing Theory of Mind evaluations typically rely on predefined answers, while uncertainty research overlooks LLMs' preferred explanations and confidence under psychological distress. To address these gaps, we introduce MentalLoom, a novel evaluation of LLM mentalisation comprising 604 ambiguous social episodes, matched depression and social anxiety disclosures, and four-turn dialogues with unchanged event evidence. Across eight LLMs, disclosures initially elicited more negative explanations overall but more positive narrator-directed interpretations. Scripted repeated questioning showed greater emotional support, stronger positive narrator-directed endorsement, and higher certainty without new event evidence. These findings show that user context can reshape interpretive preferences and commitment while the underlying ambiguity remains unresolved.

Figures & tables

Figure 1: Example of Mentalisation where the same behaviour can support multiple provisional interpretations.

Model	$-$ ve valence			$+$ ve valence			$-$ ve narrator directed valence			$+$ ve narrator directed valence			Emotional support			High-certainty
	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA
DeepSeek-V3.1	34.64	38.18	32.63	11.48	11.44	12.79	13.61	10.88	9.03	12.41	14.52	15.12	0.00	16.29	3.45	2.46	1.59	0.58
Qwen2.5-72B	33.35	40.44	34.75	17.18	17.19	18.67	7.54	5.86	5.74	17.85	20.60	20.84	0.00	5.76	4.07	4.20	3.74	2.42
Qwen3-8B	22.72	39.23	32.43	20.24	20.41	23.06	7.49	7.21	7.17	21.65	23.86	26.40	0.17	3.10	4.24	9.10	5.95	4.87
Nemotron-3-Super	31.99	36.78	30.55	14.53	13.71	16.35	9.33	7.55	6.17	16.22	19.48	21.07	0.00	6.53	4.63	7.17	6.75	6.98
Mistral-Small-24B	32.36	39.06	35.97	17.22	15.35	16.52	10.41	11.38	11.58	16.73	16.18	16.70	0.00	4.41	2.69	3.87	0.75	0.50

Table 1: Initial turn (T0) response statistics (%). Red: negative valence/high certainty; green: positive valence/emotional support. Highest value in each column is bolded. Dep = Depression; SA = Social anxiety.

Figure 2: (a) Explanation and (b) narrator-directed valence; (c) emotional support; (d) certainty (T0–T3).

Disclosure	T0	T1	T2	T3	Mean
No disclosure	$+3.8$	$+16.4$	$+9.7$	$+17.7$	$\mathbf{+11.9}$
Depression	$+4.9$	$+17.8$	$+16.5$	$+20.0$	$\mathbf{+14.8}$
Social anxiety	$+5.2$	$+20.0$	$+20.4$	$+23.4$	$\mathbf{+17.3}$

Table 2: Certainty Weighted Interpretive Valence (CWIV, %). Values indicate net narrator-directed endorsement weighted by expressed certainty.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Item	Question
Core
1	Little interest or pleasure in doing things
2	Feeling down, depressed, or hopeless
Supporting
3	Trouble falling or staying asleep, or sleeping too much
4	Feeling tired or having little energy

Appendix

Table 3: PHQ-9 items grouped by role in the study design.

Item	Question
Fear
1	I am afraid of people in authority.
3	Parties and social events scare me.
5	Being criticized scares me a lot.
10	Talking to strangers scares me.
14	I am afraid of doing things when people might be watching.

Appendix

Table 4: SPIN items grouped by category.

Dimension	Categories	Label Definitions
Plausibility	3	Plausible: Reasonable causal account consistent with available evidence. Implausible: Unreasonable causal account or clear contradiction. Unclear: Plausibility cannot be determined.
Explanation valence	3	Positive: Pleasant emotions, benevolent intentions or favourable evaluations. Negative: Distress, hostility, unfavourable evaluations or adverse circumstances. Neutral: Neither. Negative takes precedence when both valences occur within an explanation.
Narrator-directed valence	3	Positive: Care, acceptance, favourable evaluation or reassuring nonpersonal meaning towards the narrator. Negative: Rejection, hostility, blame or unfavourable evaluation towards the narrator. Neutral: Neither.
Certainty	3	1 (Possible): Hypothetical, unfavoured or unranked possibility. 2 (Likely): Preferred explanation without strong confidence. 3 (High certainty): Strong confidence or unqualified conclusion outside possibility framing.
Emotional support	2	1: Acknowledges or validates feelings expressed by the user. 0: No such acknowledgement or validation.

Appendix

Table 5: Operational definitions of the response evaluation labels.

Panel of Figure 3 — Figure 3: Prompt used to select activity labels for ambiguous interpersonal scenarios.

Panel of Figure 4 — Figure 4: Prompt used to nominate and rank ten scenario contexts for each social behaviour.

Panel of Figure 5 — Figure 5: Prompt used to generate a first-person event account for each behaviour–scenario pair.

Panel of Figure 6 — Figure 6: Prompt used to combine three selected symptom features into a first-person preamble.

Panel of Figure 7 — Figure 7: Prompts for all conversation turns

Panel of Figure 8 — Figure 8: Prompt used to select candidate A or B, or reject both candidates with NEITHER.

Panel of Figure 9 — Figure 9: Response evaluation prompt covering task rules and certainty.

Panel of Figure 10 — Figure 10: Response evaluation prompt specifying the JSON output structure and input fields.

Model	No Disclosure					Depression					Social anxiety
	T0	T1	T2	T3	Mean	T0	T1	T2	T3	Mean	T0	T1	T2	T3	Mean
DeepSeek-V3.1	$-0.5$	$+26.7$	$+17.9$	$+26.9$	$\mathbf{+17.7}$	$+1.7$	$+35.4$	$+37.3$	$+41.1$	$\mathbf{+28.9}$	$+2.3$	$+37.3$	$+42.4$	$+46.7$	$\mathbf{+32.2}$
Qwen2.5-72B	$+4.3$	$+16.2$	$+13.2$	$+17.9$	$\mathbf{+12.9}$	$+6.5$	$+15.0$	$+15.4$	$+18.2$	$\mathbf{+13.7}$	$+6.0$	$+17.3$	$+18.9$	$+21.3$	$\mathbf{+15.9}$
Qwen3-8B	$+6.8$	$+15.4$	$+9.9$	$+11.5$	$\mathbf{+10.9}$	$+7.3$	$+15.0$	$+10.9$	$+12.5$	$\mathbf{+11.4}$	$+7.7$	$+16.9$	$+13.2$	$+14.9$	$\mathbf{+13.2}$
Nemotron-3	$+3.3$	$+24.3$	$+15.0$	$+31.2$	$\mathbf{+18.4}$	$+5.5$	$+23.6$	$+23.2$	$+29.2$	$\mathbf{+20.4}$	$+6.9$	$+28.9$	$+29.0$	$+34.1$	$\mathbf{+24.7}$
Mistral-24B	$+2.9$	$+8.2$	$+7.3$	$+10.5$	$\mathbf{+7.2}$	$+1.9$	$+9.2$	$+10.5$	$+10.0$	$\mathbf{+7.9}$	$+1.9$	$+11.0$	$+14.8$	$+14.8$	$\mathbf{+10.6}$

Appendix

Table 6: Certainty Weighted Interpretive Valence (CWIV, %) by model, disclosure condition and dialogue turn. Positive and negative values indicate net positive and negative narrator-directed endorsement weighted by expressed certainty. Green indicates positive CWIV and red indicates negative CWIV.

Interaction	$-$ ve valence			$+$ ve valence			$-$ ve narrator directed valence			$+$ ve narrator directed valence			Emotional support			High-certainty
	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA	None	Dep	SA
Direct final request	28.82	32.55	25.79	23.82	19.23	21.90	8.75	7.82	6.89	30.48	32.38	38.31	18.77	45.33	40.62	2.89	2.00	2.05
After dialogue	28.68	33.31	25.44	23.26	21.05	23.15	6.55	3.97	3.32	37.35	40.86	45.02	66.18	70.42	68.50	5.24	5.03	5.55

Appendix

Table 7: Responses to the same final request in T3 presented directly or after dialogue (%). Red denotes negative valence/high certainty; green denotes positive valence/support. Darker shading indicates larger values; bold marks the higher value in each column. None = no disclosure; Dep = depression; SA = social anxiety.

1 (Possible)
Model	None				Depression				Social anxiety
	T0	T1	T2	T3	T0	T1	T2	T3	T0	T1	T2	T3
DeepSeek-V3.1	96.13	8.90	14.00	5.71	97.62	9.35	9.11	4.78	99.03	8.39	5.19	4.22
Qwen2.5-72B-Instruct	93.25	54.64	68.01	50.78	94.63	66.68	72.92	60.04	95.36	55.89	63.41	50.67
Qwen3-8B	88.31	51.09	80.24	66.85	92.95	62.07	86.61	79.80	94.17	57.14	80.30	71.60
Nemotron-3-Super-120B-A12B	91.62	41.91	63.75	29.56	92.17	53.27	58.62	36.59	91.18	45.04	46.99	29.13

Appendix

Table 8: Certainty label distributions (%).

Plausible
Model	None				Depression				Social anxiety
	T0	T1	T2	T3	T0	T1	T2	T3	T0	T1	T2	T3
DeepSeek-V3.1	98.69	85.61	83.57	79.49	99.15	79.67	77.05	75.84	99.24	77.68	75.21	72.58
Qwen2.5-72B-Instruct	99.41	98.70	97.74	96.58	99.65	97.30	96.96	95.94	99.52	96.08	94.60	93.30
Qwen3-8B	97.96	97.20	95.33	94.31	98.61	95.68	93.73	92.27	98.24	96.26	94.04	92.79
Nemotron-3-Super-120B-A12B	98.73	90.11	84.65	76.94	98.78	85.50	81.68	76.41	98.64	81.59	78.31	73.12

Appendix

Table 9: Plausibility label distributions (%).

Positive
Model	None				Depression				Social anxiety
	T0	T1	T2	T3	T0	T1	T2	T3	T0	T1	T2	T3
DeepSeek-V3.1	12.41	43.99	38.04	44.99	14.52	53.49	53.83	54.57	15.12	56.44	59.48	61.27
Qwen2.5-72B-Instruct	17.85	30.91	29.06	33.94	20.60	31.52	33.95	34.67	20.84	35.47	38.27	39.04
Qwen3-8B	21.65	31.81	27.68	28.33	23.86	32.45	29.88	31.21	26.40	34.74	31.75	33.43
Nemotron-3-Super-120B-A12B	16.22	42.64	36.27	48.13	19.48	44.01	43.79	47.32	21.07	50.51	49.91	53.16

Appendix

Table 10: Narrator-directed valence label distributions (%).

Positive
Model	None				Depression				Social anxiety
	T0	T1	T2	T3	T0	T1	T2	T3	T0	T1	T2	T3
DeepSeek-V3.1	11.48	21.78	14.20	15.56	11.44	17.34	14.52	13.07	12.79	21.34	18.59	15.80
Qwen2.5-72B-Instruct	17.18	27.89	26.66	29.61	17.19	25.09	25.37	27.67	18.67	27.93	28.83	29.53
Qwen3-8B	20.24	28.60	22.07	23.04	20.41	24.48	18.97	20.19	23.06	27.02	21.65	22.75
Nemotron-3-Super-120B-A12B	14.53	30.48	19.42	23.23	13.71	22.63	18.58	18.75	16.35	26.13	22.50	21.58

Appendix

Table 11: Explanation valence label distributions (%).

1 (Present)
Model	None				Depression				Social anxiety
	T0	T1	T2	T3	T0	T1	T2	T3	T0	T1	T2	T3
DeepSeek-V3.1	0.00	63.27	93.34	98.80	16.29	72.73	92.29	98.63	3.45	70.95	90.56	98.63
Qwen2.5-72B-Instruct	0.00	7.67	32.44	46.89	5.76	19.04	33.11	53.42	4.07	20.47	36.58	53.83
Qwen3-8B	0.17	5.76	22.17	25.59	3.10	17.20	28.33	36.49	4.24	18.41	31.31	32.55
Nemotron-3-Super-120B-A12B	0.00	26.20	93.48	93.31	6.53	58.71	95.73	97.07	4.63	56.88	93.33	91.32

Appendix

Table 12: Emotional support (% of valid responses).

Language Model Reasoning Evaluation

See all 94 papers

What Did They Mean? How LLMs Resolve Ambiguous Social Situations across Perspectives and Roles
Apr 27, 2026Qiming Yuan, Linyi Han, Nam Ling +1LLM Uncertainty Estimation Social Reasoning
Santa Clara University, USA
LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue
Sep 2, 2025Katharine Kowalyshyn, Matthias ScheutzLLM Evaluation Theory of Mind
Department of Computer Science Tufts University
Lost in Delusion: Examining LLM Safety Under User Delusions and Distress
May 31, 2026Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan +3Mental Health LLM Safety Alignment
University of Pittsburgh · Carnegie Mellon University · Fordham University

Apr 27, 2026J/K move · Enter open · S save

What Did They Mean? How LLMs Resolve Ambiguous Social Situations across Perspectives and Roles

Qiming Yuan, Linyi Han, Nam Ling, Cihan Ruan

Santa Clara University, USA

LLM Uncertainty Estimation Social Reasoning

Open paper AskPDF arXiv

People increasingly turn to large language models (LLMs) to interpret ambiguous social situations: a delayed text reply, an unusually cold supervisor, a teacher's mixed signals, or a boundary-crossing friend. Yet in many such cases, no stable interpretation can be verified from the available evidence alone. We study how LLMs respond to these situations across four domains: early-stage romantic relationships, teacher--student dynamics, workplace hierarchies, and ambiguous friendships. Across 72 responses from GPT, Claude, and Gemini, only 9 (12.5%) genuinely preserved uncertainty. The remaining 87.5% produced interpretive closure through recurring pathways including narrative alignment, narrative reversal, normative advice under uncertainty, and hedged language that still supported a single conclusion. We further find that narrator perspective shapes the path to closure: first-person accounts more often elicited alignment, while third-person accounts invited more detached interpretation, even when the underlying situation remained comparable. Together, these findings show that LLMs do not simply assist interpersonal sensemaking; they tend to resolve ambiguity into coherent and actionable narratives. These results suggest that the central risk is not only that LLMs may misinterpret social situations, but that they may make unresolved situations feel prematurely settled. We frame this tendency as a design challenge for uncertainty-preserving social AI.

Abstract

Language Model Reasoning Evaluation

Explore similar work

What Did They Mean? How LLMs Resolve Ambiguous Social Situations across Perspectives and Roles

LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

Lost in Delusion: Examining LLM Safety Under User Delusions and Distress