EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs

Zhenghao Chen; Huiqun Wang; Di Huang

cs.CVApr 1, 2026

EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs

Authors: Zhenghao Chen, Huiqun Wang, Di Huang

Organizations: State Key Laboratory of Complex and Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University

Abstract

Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing 3D priors or geometric supervision, which enhances performance but incurs substantial data preparation and alignment costs. In contrast, purely 2D approaches often struggle with multi-frame spatial reasoning due to their limited ability to capture cross-frame spatial relationships. To address these limitations, we propose EgoMind, a Chain-of-Thought framework that enables geometry-free spatial reasoning through Role-Play Caption, which jointly constructs a coherent linguistic scene graph across frames, and Progressive Spatial Analysis, which progressively reasons toward task-specific questions. With only 5K auto-generated SFT samples and 20K RL samples, EgoMind achieves competitive results on VSI-Bench, SPAR-Bench, SITE-Bench, and SPBench, demonstrating its effectiveness in strengthening the spatial reasoning capabilities of MLLMs and highlighting the potential of linguistic reasoning for spatial cognition. Code and data are released at https://github.com/Hyggge/EgoMind.

Figures & tables

Figure 1: Illustration of the differences among spatial reasoning approaches. Direct questioning often fails because of missing cross-frame correlations and limited awareness of implicit objects needed for spatial bridging. Guided questioning helps the model gradually establish these associations. In contrast, EgoMind CoT explicitly models viewpoint transitions and implicit spatial bridges, builds a coherent global scene representation, and reliably produces the correct answer.

Figure 2: Illustration of the proposed EgoMind framework. MLLMs powered by EgoMind first generate a Role-Play Caption by producing per-frame scene descriptions and inferring viewpoint transitions. The model then performs Progressive Spatial Analysis (PSA) to identify relevant objects, expand spatial dependencies via implicit spatial bridges, and form a coherent reasoning chain. Finally, the system outputs the EgoMind CoT, unifying RPC and PSA into an interpretable spatial reasoning process.

Figure 3: Illustration of the data generation pipeline. Randomly sampled video frames and a tailored instruction are first given to GPT-4o to produce detailed per-frame descriptions and infer viewpoint transitions between adjacent frames. Qwen2.5-72B then synthesizes these descriptions and transitions into the Role-Play Caption (RPC). In parallel, another GPT-4o instance, guided by a structured instruction, extracts the required spatial context from the multi-frame input and question. Finally, GPT-4o merges the RPC and spatial context to generate the final EgoMind Chain-of-Thought.

Closed-source Models
Models	Params	Data Size	VSI-Bench									SPAR-Bench	SITE-Bench	SPBench
			Overall	Numerical Question				Multiple-Choice Question
			Overall	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
GPT-4.1	-	-	47.21	44.44	26.16	64.71	53.06	51.27	36.47	38.14	63.43	42.77	64.16	54.51
GPT-5	-	-	51.66	45.82	32.57	69.77	46.15	57.61	41.74	47.42	67.48	54.72	67.89	54.77
Gemini 2.5 Pro	-	-	50.62	40.8	35.13	67.00	54.83	55.92	41.74	45.36	62.78	49.42	66.00	55.90

Table 1: A comprehensive comparison of EgoMind with state-of-the-art vision–language models across four spatial reasoning benchmarks.

Exp	RPC	PSA	VSI-Bench
Exp	RPC	PSA	+SFT	+RL
Baseline	–	–	30.02 (vanilla)
Full	✓	✓	42.33	50.16
w/o RPC	$\times$	✓	41.52	47.69
w/o PSA	✓	$\times$	41.23	45.15

Table 2: Ablation study of different components in EgoMind evaluated on VSI-Bench. RPC and PSA denote Role-Play Caption and Progressive Spatial Analysis, respectively.

CoT Modification	VSI-Bench
CoT Modification	+SFT	+RL
Full CoT	42.33	50.16
RPC $\rightarrow$ MFC	41.58	48.09
RPC $\rightarrow$ MFC + CVP	41.84	47.12
PSA $\rightarrow$ DSA	41.54	47.24

Table 3: Ablation study on CoT modifications evaluated on VSI-Bench. MFC: Multi-Frame Caption. CVP: Camera View Prediction. DSA: Direct Spatial Analysis.

# Frames	Baseline	4	8	12	16
Score	35.31	36.14	36.59	38.88	40.35
Gain	–	+0.83	+1.28	+3.57	+5.04

Table 4: Ablation on the number of input frames for Room Size Estimation on VSI-Bench, using Qwen2.5-VL-7B as the baseline.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure D : A case study of relational reasoning with the Qwen2.5-VL-7B model enhanced by the EgoMind framework.

Models	Params	Version	Overall	Numerical Question				Multiple-Choice Question
Models	Params	Version	Overall	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
Qwen2.5-VL-3B	3B	Base	27.68	27.06	19.78	25.37	19.24	32.54	39.26	28.87	22.33
		+SFT	37.08	36.48	21.74	36.88	30.66	37.04	45.14	22.68	53.56
		+RL	45.44	44.96	31.29	59.07	37.60	41.69	47.52	23.20	55.66
Qwen2.5-VL-7B	7B	Base	30.02	23.75	10.50	36.98	35.31	38.45	37.09	28.87	28.48
		+SFT	42.33	43.20	23.87	50.34	29.03	41.41	47.42	29.38	57.44

Appendix

Table E : Performance evaluation of Qwen2.5-VL models at different scales (3B and 7B) on the VSI-Bench benchmark. The table details the step-wise performance gains, comparing the base models with their counterparts enhanced through Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).

Figure F : A case study where Gemini 2.5 Pro is guided by an EgoMind CoT prompt to solve a complex spatial relationship problem. This case illustrates that our proposed framework can be used as a zero-shot prompting strategy to unlock the spatial understanding and reasoning capabilities of powerful closed-source models.

Figure G : A zero-shot prompt to activate the spatial understanding and reasoning capabilities of powerful closed-source models.

Explore similar work

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
Jul 1, 2026Xudong Li, Mengdan Zhang, Peixian Chen +7Visual Spatial Reasoning 3D Spatial Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology
SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
Nov 10, 2025Hunar Batra, Haoqin Tu, Hardy Chen +3Visual Question Answering Scene Graph Generation
University of Oxford · University of California, Santa Cruz
Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
Aug 5, 2026Yang Yang, Jiawei Chen, Tairan Chen +1Multimodal Grounding Visual Spatial Reasoning
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy · Stevens Institute of Technology

Jul 1, 2026J/K move · Enter open · S save

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping

Xudong Li, Mengdan Zhang, Peixian Chen, Jiaxi Tan+6

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology

Visual Spatial Reasoning 3D Spatial Reasoning VLM Reasoning Spatial Reasoning

Open paper AskPDF arXiv

Spatial intelligence remains a persistent challenge for Multimodal Large Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual reasoning or 3D reconstruction. However, they often falter during multi-step reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.

Abstract

Explore similar work

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs