Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requires VLMs to perceive local state transitions caused by object motion and viewpoint changes and integrate them over long trajectories to maintain an updated spatial state, yet existing VLMs remain limited in both capabilities. Current spatial training primarily focuses on static questions about object attributes and spatial relations, providing limited direct supervision for state transitions; in contrast, interaction trajectories naturally connect a preceding observation, an action, and a subsequent observation, offering direct supervision for local state transitions, while complete trajectories reveal dependencies among consecutive transitions. We therefore introduce Spatial-Interactor, a framework that trains VLMs to model physical-world state transitions through interaction, organizing this learning process into a three-level curriculum covering L1 passive world-state transitions, L2 active self-state transitions, and L3 long-horizon interaction trajectories. Accordingly, we construct the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned with the objective of each level. Our two-stage training strategy applies Supervised Fine-Tuning (SFT) to L1 and L2 for local transition modeling, and On-Policy Distillation (OPD) then uses privileged self-distillation: a teacher branch given segment-level transition descriptions supervises the student's on-policy CoT, helping the student learn to integrate consecutive transitions over L3 long trajectories. Experiments across multiple VLMs and spatial benchmarks show consistent gains in local transition modeling and long-horizon integration.
Figures & tables
Figure 1: Overview of Spatial-Interactor. Interaction trajectories support a three-level curriculum and two-stage training for spatial reasoning.
Figure 3: Spatial learning paradigms. Conventional spatial QA tends to retain isolated evidence, whereas Spatial-Interactor learns state transitions from interaction and composes them into a coherent spatial state.
Figure 4: Curriculum-driven construction of LSI-108K. Executed interactions and geometric trajectories are converted into verifiable transition targets and QA pairs.
Figure 5: Composition and representative examples of LSI-108K. The top summarizes data sources and task distribution, while the bottom presents QA examples from the three-level interaction curriculum.
Figure 6: Overview of OPD. (1) A frozen visual-language annotator converts consecutive video intervals into a training-only state-transition trace. (2) The student produces on-policy rollouts, while a privileged teacher conditions on the trace and evaluates the same student-generated prefixes. (3) Verifiable answer rewards and reasoning-token distillation jointly optimize the policy.
Model
VSI
MindCube
VSTI
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Proprietary models
GPT-5.5
55.4
39.2
45.5
63.9
23.9
48.1
56.0
53.4
GPT-4o
37.0
31.5
34.0
36.5
23.4
38.2
48.2
39.2
Gemini-2.5-Pro
61.1
45.9
51.5
52.2
8.0
42.4
58.3
51.1
Open-source models
Table 1: Results on four spatial reasoning benchmarks (official scores, %). Green arrows show absolute gains over the corresponding Base model, and bold values indicate the best result.
Model
MMSI
ViewSpatial
SAT-Real
SAT-Syn
Overall
Qwen2.5-VL-3B
29.0
35.6
56.7
56.3
44.4
+ SFT
29.2
43.2
61.0
56.8
47.6
+ SFT + OPD
29.9 ↑ 0.9
44.3 ↑ 8.7
62.0 ↑ 5.3
58.7 ↑ 2.4
48.7 ↑ 4.3
Qwen2.5-VL-7B
27.7
36.3
54.7
57.5
44.1
+ SFT
30.4
41.3
71.3
61.3
51.1
+ SFT + OPD
31.2 ↑ 3.5
42.4 ↑ 6.1
74.0 ↑ 19.3
62.5 ↑ 5.0
52.5 ↑ 8.4
Table 2: Cross-benchmark generalization results (accuracy, %). Green arrows show absolute gains over the corresponding Base model, and bold values indicate the best result within each backbone.
Training stage
VSI
MindCube
VSTI
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Base
36.6
30.4
37.9
29.3
8.6
39.0
34.0
35.1
External-data SFT
48.8
30.4
46.2
70.8
30.1
45.3
61.6
56.0
Full SFT
49.9
29.4
48.2
72.2
32.6
47.5
65.5
58.4
Full SFT + GRPO
50.3
28.9
48.1
73.1
33.5
48.6
67.0
59.2
Full SFT + OPD
50.6 ↑ 14.0
34.0 ↑ 3.6
49.0 ↑ 11.1
74.3 ↑ 45.0
36.6 ↑ 28.0
49.5 ↑ 10.5
67.6 ↑ 33.6
60.1 ↑ 25.0
Table 3: Ablation results on Qwen2.5-VL-7B (official scores, %). Green arrows on the final model show absolute gains over Base, and bold values indicate the best result.
Figure 7: Effectiveness of LSI-108K. Base versus LSI on four additional benchmarks (left) and four main-benchmark measures (right). LSI denotes Qwen2.5-VL-7B trained on the L1 and L2 data from our dataset without public spatial QA. Scores use the respective benchmark metrics (%).
Figure 8: OPD optimization dynamics. Curves show task reward, rollout diversity, and process divergence for the four backbones during training. Colors identify the models; the thicker line is their mean, and the shaded band spans their range.
Figure 9: Temporal evidence and privileged-trace diagnostics. Left: VSI-Bench Route Planning accuracy under different frame budgets. Right: VSTI-Bench camera-displacement scores under different trace orders and segment granularities.
Model
Angle
Dist.
Height
Nav.
Vis.
Overall
Steps ↓
Base
5.0
0.0
25.0
5.0
0.0
7.0
17.71
Spatial-Interactor
20.0
0.0
40.0
10.0
0.0
14.0
6.52
Table 4: Closed-loop interaction results for Qwen2.5-VL-7B on WalkerBench Standard-100. Task columns report success rate (%); Steps denotes the mean number of executed actions per task.
Figure 14
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Curriculum level
Interaction source
Level
Samples
Fraction
Source
Samples
Fraction
L1
15,109
14.1%
Simulated interactions
55,338
51.5%
L2
69,487
64.6%
Real camera trajectories
42,180
39.2%
L3
22,922
21.3%
Real robot trajectories
10,000
9.3%
Total
107,518
100.0%
Total
107,518
100.0%
Appendix
Table 6: Curriculum-level and source composition of LSI-108K. Counts denote QA records; the two blocks are independent summaries.
Figure 13: Geometry-derived target distributions. The panels summarize translation, rotation, robot motion, endpoint displacement, initial-view turn, and cumulative path length in LSI-108K.
Table 8: Agreement between explicit trace motion claims and reconstructed camera-pose direction. This is a partial motion audit, not whole-trace factual accuracy.
Paired-image condition
Motion answers
Rate (%)
Brightness change only
5/64
7.81
Camera motion + brightness
61/64
95.31
Appendix
Table 9: Camera-motion responses from Spatial-Interactor-7B on 64 paired source scenes per lighting condition.
Training stage
VSI-Bench
MindCube
VSTI-Bench
SPBench-MV
Overall
Rel. Dist.
Route
Avg.
Avg.
Displ.
Avg.
Avg.
Avg.
Run 1 (reported in Table 3 )
Full SFT + GRPO
50.3
28.9
48.1
73.1
33.5
48.6
67.0
59.2
Full SFT + OPD
50.6
34.0
49.0
74.3
36.6
49.5
67.6
60.1
OPD − GRPO
+0.3
+5.1
+0.9
+1.2
+3.1
+0.9
+0.6
+0.9
Runs 2–3
Appendix
Table 10: Three paired GRPO and OPD runs after full SFT on Qwen2.5-VL-7B (official scores, %). Δ denotes OPD minus GRPO; SD is across runs and the 95% CI uses paired differences.
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Harbin Institute of Technology +2