Organizations: Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada. · Alberta Machine Intelligence Institute (Amii) Fellow, CIFAR Fellow.
Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limitations in spatial reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the datasets used. We investigate themes of dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups: dataset design, dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.
Figures & tables
Figure 1: Taxonomy of data-centric spatial reasoning in MLLMs, organized by dataset design, construction, and evaluation.
Figure 2: Overview of the common pipeline across the literature for building spatial reasoning datasets. Where needed, raw visual data are first processed by perception and geometry models to extract spatial cues. Then they are fed into structured spatial representations. These representations provide the supervision signal used by different annotation approaches to generate the final spatial training data, which includes VQA pairs and optional reasoning traces.
Figure 3: Template- and rule-based spatial VQA annotation (example from a robot-manipulation task). Explicit spatial rules define ground-truth and reference relations over structured scene representations, which are verbalized through controlled templates to produce scalable supervision. These are also conditional on precise object bindings and coordinate conventions.
Citation
Answer type / scoring
Retent.
Independ.
Robust.
Human
Song et al. (2025)
cat, act / acc, geom
✓
✓
env
–
Chen et al. (2025a)
cat / acc
–
–
–
✓
Ning et al. (2025)
cat, seg / acc, iou
–
–
–
–
Liu et al. (2025b)
cat, met, open / acc, judge
✓
–
–
–
Deng et al. (2026)
cat, met, open / acc, judge
✓
–
–
✓
Ranasinghe et al. (2024)
cat, act / acc, geom
✓
✓
–
–
Table 2: Evaluation practices across the training datasets of Table 1 . Codes are defined in the legend below.
Spatial reasoning remains a persistent challenge for multimodal large language models (MLLMs). Existing approaches largely rely on large-scale, statically curated datasets, where all training samples are treated uniformly regardless of the model's evolving capabilities. This static paradigm is inherently data-inefficient: training capacity is often spent on samples that are either trivial or overly difficult for the model at its current stage. To address this limitation, we propose Ouroboros-Spatial, a self-evolving training framework in which the model plays dual roles as a proposer and a solver. In each iteration, a frozen proposer generates spatial question-answer (QA) pairs from 3D scene metadata and raw video frames, together with executable code for deriving reliable ground truth. A learnable solver is then fine-tuned on the accepted samples, and its per-sample prediction confidence is used as a difficulty signal. This signal is fed back to the proposer in the next iteration, guiding it to generate questions better matched to the solver's current capabilities. Through this closed-loop design, the training distribution co-evolves with model ability, reducing redundant trivial examples while filtering out ambiguous or uninformative samples with limited learning value. Across six spatial reasoning benchmarks, Ouroboros-Spatial substantially improves Qwen3-VL-4B and Qwen3-VL-8B while using an order of magnitude fewer training examples than recent large-scale curated datasets. On VSI-Bench, it yields absolute gains of 9.9 and 6.8 points for the 4B and 8B models, respectively, enabling both to outperform a wide range of strong open-source and proprietary baselines.
Enhan Zhao, Wei Wu, Yuanrui Zhang +2
1Peking University · 2Ant International · 3The University of Hong Kong
Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing 3D priors or geometric supervision, which enhances performance but incurs substantial data preparation and alignment costs. In contrast, purely 2D approaches often struggle with multi-frame spatial reasoning due to their limited ability to capture cross-frame spatial relationships. To address these limitations, we propose EgoMind, a Chain-of-Thought framework that enables geometry-free spatial reasoning through Role-Play Caption, which jointly constructs a coherent linguistic scene graph across frames, and Progressive Spatial Analysis, which progressively reasons toward task-specific questions. With only 5K auto-generated SFT samples and 20K RL samples, EgoMind achieves competitive results on VSI-Bench, SPAR-Bench, SITE-Bench, and SPBench, demonstrating its effectiveness in strengthening the spatial reasoning capabilities of MLLMs and highlighting the potential of linguistic reasoning for spatial cognition. Code and data are released at https://github.com/Hyggge/EgoMind.
Zhenghao Chen, Huiqun Wang, Di Huang
State Key Laboratory of Complex and Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University
Spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world. However, multimodal large language models (MLLMs) frequently suffer from hallucinations and imprecision when parsing complex geometric layouts. As data-driven scaling struggles to internalize structured geometric priors and spatial constraints, integrating mature, specialized vision models presents a compelling alternative. Despite its promise, applying this paradigm to spatial reasoning is hindered by two key challenges: The difficulty of invoking heterogeneous, parameter-rich tools, as well as the challenge of understanding and effectively leveraging their diverse low-level outputs (e.g., segmentation masks, depth maps) in high-level reasoning. To address these challenges, we propose LAST, a unified framework for tool-augmented spatial reasoning. LAST features an extensible interactive sandbox, termed LAST-Box, which abstracts heterogeneous tool invocations into atomic instructions and reusable spatial skills, returning multimodal hints (e.g., annotated images and textual descriptions) that can be directly consumed by LLMs. We further design a three-stage progressive training strategy that guides models from understanding tool outputs to proficient and adaptive tool invocation. Experiments on four datasets show that LAST-7B achieves around 20% performance gains over its backbone and outperforms strong proprietary closed-source LLMs, substantially enhancing reasoning on complex spatial tasks.
Shi-Yu Tian, Zhi Zhou, Kun-Yang Yu +5
National Key Laboratory for Novel Software Technology, Nanjing University, China