From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs
Organizations: School of Artificial Intelligence, University of Chinese Academy of Sciences · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences
Abstract
Spatial reasoning is a fundamental component of world understanding for Multimodal Large Language Models (MLLMs). While existing research has advanced the perception of spatial relations in static scenes, the domain of dynamic spatial simulation remains relatively unexplored. To bridge this gap, we introduce MentalGrid, a benchmark focused on spatial reasoning tasks that require models to simulate evolving spatial states without intermediate visual feedback. Extensive evaluation on our benchmark reveals a significant deficiency in the spatial simulation capabilities of existing models. Specifically, we find that performance is constrained by perceptual limitations in simple simulation scenarios and hindered by inadequate multi-step reasoning as complexity increases. To address these challenges, we propose a three-stage curriculum learning framework utilizing 16k specialized spatial reasoning samples. This approach yields SpatialWalker, which achieves a 33.8% average improvement over its base model, surpassing proprietary models like Gemini-3-Flash in spatial simulation. Furthermore, SpatialWalker demonstrates robust generalization, maintaining a 4.9% average gain on out-of-domain benchmarks, demonstrating the pivotal role of spatial simulation as a fundamental building block for broader spatial intelligence.
Figures & tables
| Model | SSP | FSP | DPT | Overall | |||
|---|---|---|---|---|---|---|---|
| Acc (%) | Acc (%) | Drop (%) | Acc (%) | Drop (%) | Acc (%) | Drop (%) | |
| Human Level † | 71.80 | 76.95 | -7.17 | 76.00 | -5.85 | 74.73 | -6.51 |
| Proprietary Models | |||||||
| Gemini-3.1-Pro † | 64.00 | 65.40 | -2.19 | 47.40 | 25.94 | 58.93 | 11.88 |
| GPT-5 † | 53.60 | 38.40 | 28.36 | 59.70 | -11.38 | 50.57 | 8.49 |
| Gemini-3-Flash | 45.28 | 44.23 | 2.32 | 46.27 | -2.19 | 45.24 | 0.06 |
| Model | In-domain | Out-of-domain | Overall | ||||||
|---|---|---|---|---|---|---|---|---|---|
| SSP | FSP | DPT | Avg. | SSP | FSP | DPT | Avg. | ||
| Proprietary Models | |||||||||
| Gemini-3-Flash | 43.48 | 42.44 | 45.37 | 43.73 | 47.74 | 46.72 | 47.58 | 47.34 | 45.24 |
| GPT-4o | 36.81 | 32.23 | 18.68 | 29.50 | 41.61 | 32.30 | 23.11 | 32.74 | 30.85 |
| Qwen2.5-VL-7B Based Spatial Models | |||||||||
| Qwen2.5-VL-7B | 23.81 | 24.41 | 21.65 | 23.33 | 27.70 | 25.78 | 25.48 | 26.36 | 24.59 |
| Model | MindCube | OmniSpatial | STARE | Overall | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Among | Around | Rotation | Avg. | DR. | SI. | CL. | PT. | Avg. | |||
| SpatialLadder-3B | 46.33 | 58.00 | 27.00 | 45.43 | 39.76 | 43.33 | 23.41 | 39.22 | 37.57 | 31.75 | 38.25 |
| SpatialThinker-3B | 40.17 | 62.00 | 33.50 | 44.10 | 42.14 | 47.33 | 25.00 | 41.35 | 40.05 | 32.33 | 38.83 |
| Spatial-SSRL-7B | 29.83 | 35.20 | 35.50 | 32.19 | 44.52 | 48.33 | 26.19 | 44.03 | 41.06 | 42.07 | 38.77 |
| Qwen2.5-VL-7B | 27.83 | 38.80 | 33.50 | 31.52 | 30.48 | 49.00 | 22.62 | 39.75 | 36.20 | 38.53 | 35.42 |
| Qwen2.5-VL-3B | 36.83 | 48.40 | 33.00 | 38.86 | 37.14 | 42.00 | 25.00 | 39.39 | 36.92 | 33.71 | 36.50 |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Statistics | Number |
|---|---|
| Total Questions | 3,356 |
| Total Query Types | 24 |
| Total Images | 1,100 |
| Total Secnes | 349 |
| Task Type | |
| - Static Spatial Perception | 1162 (34.62%) |
| Dataset | Size | Source | Image | View | Env. | Sim. Complexity |
|---|---|---|---|---|---|---|
| SpatialEval ( Wang et al., 2024 ) | 4.6K | S+E | Mixed | Static | Static | None |
| VSI-Bench ( Yang et al., 2025a ) | 5K | E | Real-life | Fixed | Static | Low |
| ViewSpatial-Bench ( Li et al., 2025a ) | 5.7K | E | Real-life | Fixed | Static | Low |
| MMSI-Bench ( Yang et al., 2025b ) | 1K | E | Real-life | Fixed | Static | Low |
| MindCube ( Yin et al., 2025 ) | 21.1K | E | Real-life | Fixed | Static | Medium |
| OminiSpatial ( Jia et al., 2025 ) | 8.4K | E | Mixed | Fixed | Static | Medium |
| Task | QueryType | Reference | Question Template |
|---|---|---|---|
| SSP | Object-Distance | EGO | How far are you from the { target_obj } in meters? |
| ALLO | How far is the { target_obj } from the { view_obj } (in meters)? | ||
| Distance-Object | EGO | Which object lies farthest from your position? { options } | |
| ALLO | Which object lies farthest from the { target_obj } ? { options } | ||
| Object-Direction | EGO | In which direction is the { target_obj } positioned relative to you? { options } | |
| ALLO | In which direction is the { target_obj } located relative to the { view_obj } ? { options } |
| Model | Static Spatial Perception | Final State Prediction | Dynamic Process Tracking | Overall | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| dir-obj | dis-obj | obj-dir | obj-dis | dir-obj | dis-obj | obj-dir | obj-dis | dir-obj | dis-obj | obj-dir | obj-dis | ||
| GPT-5-mini | 42.57 | 54.20 | 42.98 | 38.54 | 41.53 | 43.51 | 55.87 | 43.55 | 27.02 | 42.24 | 66.08 | 41.39 | 45.70 |
| Gemini-3-Flash | 42.08 | 49.24 | 47.85 | 41.60 | 33.88 | 46.18 | 50.72 | 41.72 | 22.18 | 38.79 | 61.95 | 50.68 | 45.24 |
| GPT-4o | 43.56 | 47.71 | 32.66 | 35.62 | 25.68 | 35.50 | 38.11 | 27.42 | 19.35 | 29.31 | 16.52 | 22.24 | 30.85 |
| Gemma3-12B | 27.72 | 33.59 | 28.37 | 33.64 | 28.96 | 27.48 | 31.81 | 29.37 | 33.06 | 26.72 | 21.83 | 14.01 | 27.85 |
| InternVL3.5-8B | 29.70 | 39.69 | 36.39 | 30.97 | 27.87 | 22.90 | 34.38 | 26.45 | 20.56 | 25.86 | 21.24 | 16.93 | 27.84 |
| Model | SSP | FSP | DPT | Overall | |||
|---|---|---|---|---|---|---|---|
| Acc (%) | Acc (%) | Drop (%) | Acc (%) | Drop (%) | Acc (%) | Drop (%) | |
| Human Level | 71.80 | 76.95 | -7.17 | 76.00 | -5.85 | 74.73 | -6.51 |
| Gemini-3.1-Pro | 64.00 | 65.40 | -2.19 | 47.40 | 25.94 | 58.93 | 11.88 |
| Gemini-3-Flash | 60.00 | 52.00 | 13.33 | 51.50 | 14.17 | 54.50 | 13.75 |
| GPT-5 | 53.60 | 38.40 | 28.36 | 59.70 | -11.38 | 50.57 | 8.49 |
| GPT-4o | 50.00 | 35.00 | 30.00 | 18.70 | 62.60 | 34.57 | 46.30 |
| Hyperparameter | Setting |
|---|---|
| Optimizer | AdamW |
| Learning rate | |
| LR scheduler | Cosine |
| Max sequence length | 8192 |
| Batch size per device | 4 |
| Gradient accumulation steps | 4 |
| Hyperparameter | Setting |
|---|---|
| Algorithm | GRPO |
| Learning rate | |
| Group size | 4 |
| KL divergence coefficient | 0.01 |
| KL loss type | Low-variable KL |
| Entropy coefficient | 0 |
| Model | SSP | FPT | DPT | All |
|---|---|---|---|---|
| MindJourney-SVC | 36.60 | 30.80 | 22.80 | 30.10 |
| MindJourney-SWM | 37.20 | 29.40 | 16.00 | 27.50 |
| GPT-4o | 50.00 | 35.00 | 18.70 | 34.60 |
| SpatialWalker | 52.00 | 53.20 | 41.70 | 49.00 |
| Model | MulSeT | SpinBench | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OR. | DC. | AT. | CVS. | IM. | DR. | PT. | ORG. | DT. | MR. | All | |
| Qwen2.5-VL-3B | 10.02 | 27.00 | 3.29 | 34.36 | 27.41 | 42.78 | 43.23 | 49.06 | 50.00 | 19.23 | 40.59 |
| SpatialWalker | 26.47 | 42.15 | 33.72 | 46.65 | 46.91 | 26.91 | 46.98 | 82.55 | 52.56 | 19.23 | 52.40 |
| Improvement | +16.45 | +15.15 | +30.43 | +12.29 | +19.50 | -15.87 | +3.75 | +33.49 | +2.56 | +0.00 | +11.81 |
Spatial Reasoning
Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.
See all 165 papers