EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs
Authors: Zhenghao Chen, Huiqun Wang, Di Huang
Organizations: State Key Laboratory of Complex and Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University
Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing 3D priors or geometric supervision, which enhances performance but incurs substantial data preparation and alignment costs. In contrast, purely 2D approaches often struggle with multi-frame spatial reasoning due to their limited ability to capture cross-frame spatial relationships. To address these limitations, we propose EgoMind, a Chain-of-Thought framework that enables geometry-free spatial reasoning through Role-Play Caption, which jointly constructs a coherent linguistic scene graph across frames, and Progressive Spatial Analysis, which progressively reasons toward task-specific questions. With only 5K auto-generated SFT samples and 20K RL samples, EgoMind achieves competitive results on VSI-Bench, SPAR-Bench, SITE-Bench, and SPBench, demonstrating its effectiveness in strengthening the spatial reasoning capabilities of MLLMs and highlighting the potential of linguistic reasoning for spatial cognition. Code and data are released at https://github.com/Hyggge/EgoMind.
Figures & tables
Figure 1: Illustration of the differences among spatial reasoning approaches. Direct questioning often fails because of missing cross-frame correlations and limited awareness of implicit objects needed for spatial bridging. Guided questioning helps the model gradually establish these associations. In contrast, EgoMind CoT explicitly models viewpoint transitions and implicit spatial bridges, builds a coherent global scene representation, and reliably produces the correct answer.
Figure 2: Illustration of the proposed EgoMind framework. MLLMs powered by EgoMind first generate a Role-Play Caption by producing per-frame scene descriptions and inferring viewpoint transitions. The model then performs Progressive Spatial Analysis (PSA) to identify relevant objects, expand spatial dependencies via implicit spatial bridges, and form a coherent reasoning chain. Finally, the system outputs the EgoMind CoT, unifying RPC and PSA into an interpretable spatial reasoning process.
Figure 3: Illustration of the data generation pipeline. Randomly sampled video frames and a tailored instruction are first given to GPT-4o to produce detailed per-frame descriptions and infer viewpoint transitions between adjacent frames. Qwen2.5-72B then synthesizes these descriptions and transitions into the Role-Play Caption (RPC). In parallel, another GPT-4o instance, guided by a structured instruction, extracts the required spatial context from the multi-frame input and question. Finally, GPT-4o merges the RPC and spatial context to generate the final EgoMind Chain-of-Thought.
Models
Params
Data Size
VSI-Bench
SPAR-Bench
SITE-Bench
SPBench
Overall
Numerical Question
Multiple-Choice Question
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
Closed-source Models
GPT-4.1
-
-
47.21
44.44
26.16
64.71
53.06
51.27
36.47
38.14
63.43
42.77
64.16
54.51
GPT-5
-
-
51.66
45.82
32.57
69.77
46.15
57.61
41.74
47.42
67.48
54.72
67.89
54.77
Gemini 2.5 Pro
-
-
50.62
40.8
35.13
67.00
54.83
55.92
41.74
45.36
62.78
49.42
66.00
55.90
Table 1: A comprehensive comparison of EgoMind with state-of-the-art vision–language models across four spatial reasoning benchmarks.
Exp
RPC
PSA
VSI-Bench
+SFT
+RL
Baseline
–
–
30.02 (vanilla)
Full
✓
✓
42.33
50.16
w/o RPC
×
✓
41.52
47.69
w/o PSA
✓
×
41.23
45.15
Table 2: Ablation study of different components in EgoMind evaluated on VSI-Bench. RPC and PSA denote Role-Play Caption and Progressive Spatial Analysis, respectively.
CoT Modification
VSI-Bench
+SFT
+RL
Full CoT
42.33
50.16
RPC → MFC
41.58
48.09
RPC → MFC + CVP
41.84
47.12
PSA → DSA
41.54
47.24
Table 3: Ablation study on CoT modifications evaluated on VSI-Bench. MFC: Multi-Frame Caption. CVP: Camera View Prediction. DSA: Direct Spatial Analysis.
# Frames
Baseline
4
8
12
16
Score
35.31
36.14
36.59
38.88
40.35
Gain
–
+0.83
+1.28
+3.57
+5.04
Table 4: Ablation on the number of input frames for Room Size Estimation on VSI-Bench, using Qwen2.5-VL-7B as the baseline.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure D : A case study of relational reasoning with the Qwen2.5-VL-7B model enhanced by the EgoMind framework.
Models
Params
Version
Overall
Numerical Question
Multiple-Choice Question
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
Qwen2.5-VL-3B
3B
Base
27.68
27.06
19.78
25.37
19.24
32.54
39.26
28.87
22.33
+SFT
37.08
36.48
21.74
36.88
30.66
37.04
45.14
22.68
53.56
+RL
45.44
44.96
31.29
59.07
37.60
41.69
47.52
23.20
55.66
Qwen2.5-VL-7B
7B
Base
30.02
23.75
10.50
36.98
35.31
38.45
37.09
28.87
28.48
+SFT
42.33
43.20
23.87
50.34
29.03
41.41
47.42
29.38
57.44
Appendix
Table E : Performance evaluation of Qwen2.5-VL models at different scales (3B and 7B) on the VSI-Bench benchmark. The table details the step-wise performance gains, comparing the base models with their counterparts enhanced through Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).
Figure E : Case studies of EgoMind.
Figure F : A case study where Gemini 2.5 Pro is guided by an EgoMind CoT prompt to solve a complex spatial relationship problem. This case illustrates that our proposed framework can be used as a zero-shot prompting strategy to unlock the spatial understanding and reasoning capabilities of powerful closed-source models.
Figure G : A zero-shot prompt to activate the spatial understanding and reasoning capabilities of powerful closed-source models.
Spatial intelligence remains a persistent challenge for Multimodal Large Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual reasoning or 3D reconstruction. However, they often falter during multi-step reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.
Xudong Li, Mengdan Zhang, Peixian Chen +7
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology
Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning. Existing spatial MLLMs rely on large-scale datasets, explicit 3D inputs, architecture-specific modifications, or sparse Reinforcement Learning (RL) methods that provide insufficient guidance for spatially-grounded reasoning. We introduce SpatialThinker. To our knowledge, it is the first MLLM unifying Scene Graph Generation (SGG) and visual reasoning in a single pass via online RL. The model simulates human-like spatial perception by constructing a mental scene graph of task-relevant objects and relations, and reasoning toward an answer via dense spatial rewards. Our contributions are threefold: (1) SGG-grounded reasoning: integrating SGG directly within the reasoning chain rather than as a disjoint preprocessing step; (2) STVQA-7K: a high-quality spatial VQA training dataset via a scalable synthesis pipeline; and (3) a dense spatial reward design that enforces structured grounding during RL and generalizes to improve broad visual perception. SpatialThinker-7B achieves 3.6× larger gains over SFT and 1.7× better in- and out-of-distribution generalization than sparse RL. Trained on only 7K samples, SpatialThinker-7B matches GPT-5 and outperforms GPT-4o, while SpatialThinker-30B surpasses both GPT-5 and Claude 4 Sonnet on average across 14 spatial and real-world benchmarks, demonstrating that structured spatial grounding with reward-aligned reasoning enables robust spatial understanding with limited data.
Hunar Batra, Haoqin Tu, Hardy Chen +3
University of Oxford · University of California, Santa Cruz
Although Multimodal Large Language Models (MLLMs) have made substantial progress, their spatial reasoning may still produce intermediate judgments inconsistent with the input image, allowing errors to propagate through the reasoning chain and affect the final answer. Existing methods mainly improve spatial reasoning through training or additional spatial information, without considering whether the reasoning process itself is faithful to the model input. Our study shows that unfaithful reasoning chains significantly reduce final-answer accuracy. To address this issue, we propose a modular and training-free framework for spatial reasoning verification and correction. The framework constructs a Spatial Evidence Graph (SEG), which associates atomic spatial evidence extracted from Chain-of-Thought reasoning with visual entities, spatial relations, source steps, and visual evidence. Spatial Evidence Reliability Assessment (SERA) evaluates the reliability of visual evidence based on object existence, localization, and geometric measurements. The framework then identifies the earliest spatial evidence unit contradicted by reliable visual evidence and guides the original MLLM to revise the subsequent reasoning and final answer. Across 15 model-dataset settings, our method achieves an average accuracy of 68.94%, outperforming the compared baselines by 8.55 percentage points on average. Our code will be open-sourced.
Yang Yang, Jiawei Chen, Tairan Chen +1
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy · Stevens Institute of Technology