Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning

Soyoung Oh; Nursulu Sagimbayeva; Nicholas Jennings; Paulina Hussein; Mehwish Alam; Vera Demberg

cs.AIOct 14, 2026

Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning

Authors: Soyoung Oh, Nursulu Sagimbayeva, Nicholas Jennings, Paulina Hussein, Mehwish Alam, Vera Demberg

Organizations: Saarland University · Deutsches Forschungszentrum für Künstliche Intelligenz · Télécom Paris · Max Planck Institute for Informatics

Abstract

In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting or judging the agent's subsequent actions. In this work, we investigate whether explicitly connecting perception, belief, and action with rules improves ToM reasoning. We generate concise if--then rules that express relations between perception to action and use Group Relative Policy Optimization (GRPO) to train Qwen2.5 and Llama3 models across four ToM benchmarks. GRPO with rule generation outperforms prompting, supervised fine-tuning, and think reasoning. Moreover, rules generated by larger models ( $\geq$ 3B) provide transferable knowledge that improves downstream performance of GPT-4o-mini. These findings suggest that explicit rules help models bridge the reasoning gap in ToM and support knowledge transfer across models.

Figures & tables

Figure 1: Causal structure of ToM, adapted from Baker et al. (2011) . An example story and question illustrate the perception-belief-action dependencies required for reasoning about an agent’s mental state.

Benchmark	Total	Main split
BigToM	5,000	3,750 / 200 / 950
HiToM	1,200	720 / 200 / 180
ToMi	2,664	1,891 / 200 / 473
OpenToM	2,384	1,668 / 200 / 416

Table 1: Dataset sizes and main splits. Main splits are reported as train/development/test.

Base
Model	HiToM	ToMi	BigToM	OpenToM
Qwen2.5-0.5B	1.5	39.1	51.4	52.6
Qwen2.5-1.5B	15.9	27.1	28.9	47.8
Qwen2.5-3B	7.2	55.3	73.6	53.6
Qwen2.5-7B	27.8	55.3	78.2	46.5
Qwen2.5-14B	45.0	62.2	82.1	59.3

Table 2: Accuracy (%) without training averaged across different temperatures. Changes below Base r are percentage-point differences from Base: green indicates improvement and red indicates degradation.

SFT a
Model	HiToM	ToMi	BigToM	OpenToM
Qwen2.5-0.5B	9.4	71.7	64.3	65.6
Qwen2.5-1.5B	30.0	72.5	77.2	69.0
Qwen2.5-3B	24.4	73.6	86.4	81.5
Qwen2.5-7B	49.4	75.1	91.2	82.5
Qwen2.5-14B	46.7	77.2	93.4	92.1

Table 3: SFT accuracy (%) on test set. Changes in parentheses compare SFT r with SFT a for the same model.

GRPO t
Model	HiToM	ToMi	BigToM	OpenToM
Qwen2.5-0.5B	10.0	71.2	72.6	61.1
Qwen2.5-1.5B	32.2	71.0	87.6	52.9
Qwen2.5-3B	32.4	67.9	90.3	78.8
Qwen2.5-7B	46.1	71.5	92.8	85.9
Qwen2.5-14B	55.7	75.3	92.9	91.6

Table 4: RL accuracy (%) on the test set averaged across three different random seeds. Changes in parentheses compare GRPO r with GRPO t for the same model.

Model	GRPO t	GRPO ${}_{t_{s}}$	GRPO r
0.5B	11.1	10.6	17.8
1.5B	33.9	34.4	46.1
3B	32.2	35.0	46.7
7B	44.4	46.1	60.0
14B	56.7	61.1	65.6

Table 5: HiToM accuracy (%) under random seed 42. GRPO t denotes the long-thinking model, GRPO

{}_{t_{s}}

the short-thinking model, and GRPO r the rule-generation model.

Figure 2: Downstream accuracy (%) of GPT-4o-mini augmented with rules generated by each GRPO r models. Dashed line denotes GPT-4o-mini no-rule baseline.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Factors	Answer format
BigToM	condition x belief/action type x initial belief (#25)	Choose (a) or (b) from two options
HiToM	deception x story length x question order (#30)	One letter from 16 options
ToMi	false-belief order x question type (#16)	Open-ended location string
OpenToM	condition x observed x personality (#12)	Yes/No

Appendix

Table 6: Benchmark stratification factors and answer formats. #N denotes the number of possible factor combinations.

Dataset	SFT (format) warm-up
BigToM	100 (75:25)
HiToM	100 (70:30)
ToMi	100 (83:17)
OpenToM	100 (80:20)

Appendix

Table 7: SFT format warm-up splits (train:development).

Dataset	Training	Warm-up
BigToM	3488:188:884	72:23
HiToM	533:146:144	55:23
ToMi	1570:164:393	70:15
OpenToM	1668:198:416	80:20

Appendix

Table 8: Number of validated rules based on train, (dev), test splits.

GRPO t
Model	BigToM	HiToM	ToMi	OpenToM
Qwen2.5-0.5B	75.3	11.1	70.2	65.1
Qwen2.5-1.5B	86.4	33.9	70.2	53.4
Qwen2.5-3B	91.5	32.2	68.9	79.3
Qwen2.5-7B	93.2	44.4	69.8	87.5
Qwen2.5-14B	91.9	56.7	75.1	92.1

Appendix

Table 9: Accuracy (%) of GRPO t and GRPO ns . Changes in parentheses compare GRPO ns with GRPO t for the same model.

Figure 3: Average number of tokens used to produce “rule” vs. “think” reasoning

Figure 4: Average number of tokens used to produce “rule” vs. “think_short” vs. “think” reasoning on HiToM benchmark

HiToM
Model	Base	SFT r	SFT a	GRPO r	GRPO t
Llama-3.2-1B	10.0	11.7	15.6	41.7	15.6
Llama-3.2-3B	24.4	30.0	34.4	51.1	36.7
Llama-3.1-8B	33.9	41.1	46.7	65.6	58.9
ToMi
Llama-3.2-1B	44.0	61.1	69.6	79.5	73.4

Appendix

Table 10: Llama answer accuracy. Highlighted with bold is the best performing training regime for the given model and benchmark.

Dataset: HiToM
Story	1 Jackson, Charlotte, Avery, Liam and Benjamin entered the cellar. 2 The corn is in the blue_treasure_chest. 3 Jackson made no movements and stayed in the cellar for 1 minute. 4 Jackson exited the cellar. 5 Charlotte moved the corn to the green_crate. 6 Charlotte exited the cellar. 7 Avery likes the blue_bathtub. 8 Avery made no movements and stayed in the cellar for 1 minute. 9 Avery exited the cellar. 10 Liam made no movements and stayed in the cellar for 1 minute. 11 Liam exited the cellar. 12 Benjamin made no movements and stayed in the cellar for 1 minute. 13 Benjamin exited the cellar. 14 Jackson, Charlotte, Avery, Liam and Benjamin entered the waiting_room.
Question	Where does Jackson think Benjamin thinks the corn is?
Correct answer	blue_treasure_chest
Gold rule	IF Jackson left the cellar before Charlotte moved the corn AND Benjamin witnessed Charlotte move the corn, THEN Jackson will believe that Benjamin believes the corn is at the location Benjamin saw Charlotte place it.
Rules
Qwen2.5-0.5B	“ If Jackson made no movements and stayed in the cellar for 1 minute , then Jackson thinks the corn is in the blue_treasure_chest.”

Appendix

Table 11: Examples of rule generation across model scales on ToMi, Hi-ToM, BigToM, and OpenToM. Gold rules are generated by gpt-5-mini-2025-08-07 . In Rules and Think traces, highlighted with bold are examples of erroneous reasoning. All selected models answered correctly on the presented examples.

Theory of Mind

Theory of Mind studies how people and AI infer others’ beliefs, goals, and knowledge to predict behavior and coordinate interaction.

See all 92 papers

Mind the Perspective: Let's Reason Recursively for Theory of Mind
Jun 10, 2026Chao Lei, Guang Hu, Meng Yang +2Logical Reasoning Theory of Mind
School of Computing and Information Systems, The University of Melbourne, Australia · SensiLab, Monash University, Australia
Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning
Aug 5, 2026Ian B. de Haan, Peter van der Putten, Max van DuijnLLM Evaluation Theory of Mind
LIACS, Leiden University, The Netherlands
From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning
Jun 8, 2026Jike Zhong, Yuxiang Lai, Ming Li +5Theory of Mind RL for Language Model Reasoning
University of Southern California · 2Emory University · University of Maryland, College Park +4

Jun 10, 2026J/K move · Enter open · S save

Mind the Perspective: Let's Reason Recursively for Theory of Mind

Chao Lei, Guang Hu, Meng Yang, Yanbei Jiang+1

School of Computing and Information Systems, The University of Melbourne, Australia · SensiLab, Monash University, Australia

Logical Reasoning Theory of Mind Epistemic Logic LLM Reasoning

Open paper AskPDF arXiv

Theory of Mind (ToM) reasoning requires inferring agents' beliefs from partial and asymmetric observations, which remains an open challenge for LLMs. Existing prompting-based approaches improve ToM reasoning through observable-event filtering or temporal belief chains, without explicitly modeling nested beliefs. We introduce RecToM, an inference-time framework for ToM reasoning that models nested beliefs via recursive perspective construction. RecToM constructs each character perspective from the preceding character perspective along the character chain specified by the question, reducing higher-order belief questions to actual-world questions within the final constructed perspective. We further provide a KD45 analysis showing that RecToM's perspective construction induces a well-formed belief modality beyond simple event filtering. Experiments on ToM benchmarks, including Hi-ToM, Big-ToM, and FanToM, across multiple LLM backbones show that RecToM consistently outperforms recent advanced approaches, achieving state-of-the-art performance. Notably, RecToM reaches 100% accuracy on Hi-ToM with GPT-5.4 and Qwen3.5, a benchmark requiring higher-order ToM reasoning.

Abstract

Theory of Mind

Explore similar work

Mind the Perspective: Let's Reason Recursively for Theory of Mind

Evaluating Theory of Mind in Reasoning Models: Robustness over Reasoning

From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning