Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requires VLMs to perceive local state transitions caused by object motion and viewpoint changes and integrate them over long trajectories to maintain an updated spatial state, yet existing VLMs remain limited in both capabilities. Current spatial training primarily focuses on static questions about object attributes and spatial relations, providing limited direct supervision for state transitions; in contrast, interaction trajectories naturally connect a preceding observation, an action, and a subsequent observation, offering direct supervision for local state transitions, while complete trajectories reveal dependencies among consecutive transitions. We therefore introduce Spatial-Interactor, a framework that trains VLMs to model physical-world state transitions through interaction, organizing this learning process into a three-level curriculum covering L1 passive world-state transitions, L2 active self-state transitions, and L3 long-horizon interaction trajectories. Accordingly, we construct the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned with the objective of each level. Our two-stage training strategy applies Supervised Fine-Tuning (SFT) to L1 and L2 for local transition modeling, and On-Policy Distillation (OPD) then uses privileged self-distillation: a teacher branch given segment-level transition descriptions supervises the student's on-policy CoT, helping the student learn to integrate consecutive transitions over L3 long trajectories. Experiments across multiple VLMs and spatial benchmarks show consistent gains in local transition modeling and long-horizon integration.
Figures & tables
Figure 1: Overview of Spatial-Interactor. Interaction trajectories support a three-level curriculum and two-stage training for spatial reasoning.
Figure 3: Spatial learning paradigms. Conventional spatial QA tends to retain isolated evidence, whereas Spatial-Interactor learns state transitions from interaction and composes them into a coherent spatial state.
Figure 4: Curriculum-driven construction of LSI-108K. Executed interactions and geometric trajectories are converted into verifiable transition targets and QA pairs.
Figure 5: Composition and representative examples of LSI-108K. The top summarizes data sources and task distribution, while the bottom presents QA examples from the three-level interaction curriculum.
Figure 6: Overview of OPD. (1) A frozen visual-language annotator converts consecutive video intervals into a training-only state-transition trace. (2) The student produces on-policy rollouts, while a privileged teacher conditions on the trace and evaluates the same student-generated prefixes. (3) Verifiable answer rewards and reasoning-token distillation jointly optimize the policy.
Model
VSI
MindCube
VSTI
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Proprietary models
GPT-5.5
55.4
39.2
45.5
63.9
23.9
48.1
56.0
53.4
GPT-4o
37.0
31.5
34.0
36.5
23.4
38.2
48.2
39.2
Gemini-2.5-Pro
61.1
45.9
51.5
52.2
8.0
42.4
58.3
51.1
Open-source models
Table 1: Results on four spatial reasoning benchmarks (official scores, %). Green arrows show absolute gains over the corresponding Base model, and bold values indicate the best result.
Model
MMSI
ViewSpatial
SAT-Real
SAT-Syn
Overall
Qwen2.5-VL-3B
29.0
35.6
56.7
56.3
44.4
+ SFT
29.2
43.2
61.0
56.8
47.6
+ SFT + OPD
29.9 ↑ 0.9
44.3 ↑ 8.7
62.0 ↑ 5.3
58.7 ↑ 2.4
48.7 ↑ 4.3
Qwen2.5-VL-7B
27.7
36.3
54.7
57.5
44.1
+ SFT
30.4
41.3
71.3
61.3
51.1
+ SFT + OPD
31.2 ↑ 3.5
42.4 ↑ 6.1
74.0 ↑ 19.3
62.5 ↑ 5.0
52.5 ↑ 8.4
Table 2: Cross-benchmark generalization results (accuracy, %). Green arrows show absolute gains over the corresponding Base model, and bold values indicate the best result within each backbone.
Training stage
VSI
MindCube
VSTI
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Base
36.6
30.4
37.9
29.3
8.6
39.0
34.0
35.1
External-data SFT
48.8
30.4
46.2
70.8
30.1
45.3
61.6
56.0
Full SFT
49.9
29.4
48.2
72.2
32.6
47.5
65.5
58.4
Full SFT + GRPO
50.3
28.9
48.1
73.1
33.5
48.6
67.0
59.2
Full SFT + OPD
50.6 ↑ 14.0
34.0 ↑ 3.6
49.0 ↑ 11.1
74.3 ↑ 45.0
36.6 ↑ 28.0
49.5 ↑ 10.5
67.6 ↑ 33.6
60.1 ↑ 25.0
Table 3: Ablation results on Qwen2.5-VL-7B (official scores, %). Green arrows on the final model show absolute gains over Base, and bold values indicate the best result.
Figure 7: Effectiveness of LSI-108K. Base versus LSI on four additional benchmarks (left) and four main-benchmark measures (right). LSI denotes Qwen2.5-VL-7B trained on the L1 and L2 data from our dataset without public spatial QA. Scores use the respective benchmark metrics (%).
Figure 8: OPD optimization dynamics. Curves show task reward, rollout diversity, and process divergence for the four backbones during training. Colors identify the models; the thicker line is their mean, and the shaded band spans their range.
Figure 9: Temporal evidence and privileged-trace diagnostics. Left: VSI-Bench Route Planning accuracy under different frame budgets. Right: VSTI-Bench camera-displacement scores under different trace orders and segment granularities.
Model
Angle
Dist.
Height
Nav.
Vis.
Overall
Steps ↓
Base
5.0
0.0
25.0
5.0
0.0
7.0
17.71
Spatial-Interactor
20.0
0.0
40.0
10.0
0.0
14.0
6.52
Table 4: Closed-loop interaction results for Qwen2.5-VL-7B on WalkerBench Standard-100. Task columns report success rate (%); Steps denotes the mean number of executed actions per task.
Figure 14
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Curriculum level
Interaction source
Level
Samples
Fraction
Source
Samples
Fraction
L1
15,109
14.1%
Simulated interactions
55,338
51.5%
L2
69,487
64.6%
Real camera trajectories
42,180
39.2%
L3
22,922
21.3%
Real robot trajectories
10,000
9.3%
Total
107,518
100.0%
Total
107,518
100.0%
Appendix
Table 6: Curriculum-level and source composition of LSI-108K. Counts denote QA records; the two blocks are independent summaries.
Figure 13: Geometry-derived target distributions. The panels summarize translation, rotation, robot motion, endpoint displacement, initial-view turn, and cumulative path length in LSI-108K.
Table 8: Agreement between explicit trace motion claims and reconstructed camera-pose direction. This is a partial motion audit, not whole-trace factual accuracy.
Paired-image condition
Motion answers
Rate (%)
Brightness change only
5/64
7.81
Camera motion + brightness
61/64
95.31
Appendix
Table 9: Camera-motion responses from Spatial-Interactor-7B on 64 paired source scenes per lighting condition.
Training stage
VSI-Bench
MindCube
VSTI-Bench
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Run 1 (reported in Table 3 )
Full SFT + GRPO
50.3
28.9
48.1
73.1
33.5
48.6
67.0
59.2
Full SFT + OPD
50.6
34.0
49.0
74.3
36.6
49.5
67.6
60.1
OPD − GRPO
+0.3
+5.1
+0.9
+1.2
+3.1
+0.9
+0.6
+0.9
Runs 2–3
Appendix
Table 10: Three paired GRPO and OPD runs after full SFT on Qwen2.5-VL-7B (official scores, %). Δ denotes OPD minus GRPO; SD is across runs and the 95% CI uses paired differences.
Vision-language models (VLMs) have shown strong performance on static visual understanding, yet they still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion. Recent efforts address this limitation either by scaling spatial supervision with synthetic data or by coupling VLMs with world models at inference time. However, the former often lacks explicit modeling of motion-conditioned state transitions, while the latter incurs substantial computational overhead. In this work, we propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model. Given an initial observation and a parameterized camera trajectory, we use a view-consistent world model to synthesize geometrically aligned future views and derive structured supervision for both forward (action-to-outcome) and inverse (outcome-to-action) spatial reasoning. We post-train the VLM with a two-stage recipe on a compact dataset generated by this pipeline and evaluate it on multiple spatial reasoning benchmarks. World2VLM delivers consistent improvements over the base model across diverse benchmarks, including SAT-Real, SAT-Synthesized, VSI-Bench, and MindCube. It also outperforms the test-time world-model-coupled methods while eliminating the need for expensive inference-time generation. Our results suggest that world models can serve not only as inference-time tools, but also as effective training-time teachers, enabling VLMs to internalize spatial imagination in a scalable and efficient manner.
Wanyue Zhang, Wenxiang Wu, Wang Xu +6
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Harbin Institute of Technology +2
Humans can effortlessly perceive spatial layouts, form cognitive representations, reason about spatial relations, and translate such reasoning into actions in everyday 3D environments. Although recent vision-language models (VLMs) have shown promising performance on observation-conditioned spatial perception and reasoning tasks, it remains unclear whether they can build coherent spatial understanding, act upon it, and refine their actions through multi-turn feedback. To study this problem, we introduce SpatialAct, a simulator-grounded benchmark for probing action-conditioned spatial reasoning in 3D scenes. Starting from the most challenging setting, Multi-turn Interactive Refinement, we further design its decomposed counterpart, Single-step Error Detection and Fix, together with five fundamental spatial ability tasks to diagnose the underlying causes of model failures. Experiments reveal a clear reasoning-to-action gap: current VLMs can perform well on isolated spatial reasoning tasks, but struggle to maintain coherent spatial beliefs and produce reliable actions during multi-turn feedback, substantially underperforming humans. These results suggest that current VLM agents still lack robust spatial state tracking under action-induced environment changes, even when low-level control is abstracted away.
Tianhui Liu, Jie Feng, Zhiheng Zheng +6
1The Hong Kong University of Science and Technology (Guangzhou) · 2Zhongguancun Academy · 3Tsinghua University +1
Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient Spatio-vision Language Models (SpatioLM), that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at \faGithub~spatio-lm.
Jing Wu, Jianhua Wu, Jiayi Guan +5
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher