Spatial intelligence lets an agent perceive and act in the physical world, and many of its judgments must be made at interaction rates: which spatial relation holds, how far an object is, or where to place it. Vision-language models (VLMs) produce these answers by free-form text generation, paying for autoregressive decoding and answer parsing even though the output space is compact: one of a few options, a scalar, or a 2D point. In light of this, we introduce MJev-Spatial, a System 1 multimodal decision model that tackles all three types of spatial understanding tasks within a unified classification framework. Specifically, MJev-Spatial treats each candidate as a class and scores it using the backbone's pretrained query and key projections, without adding parameters. The candidates are the given options for choice tasks, or value bins and image regions refined through coarse-to-fine classification for numerical and point prediction, respectively. At inference, each classification decision takes a single forward pass and generates no answer tokens. To train on heterogeneous spatial data efficiently, we supervise all coarse-to-fine stages jointly with teacher forcing in one forward pass, and combine this with length-aware global batching and cross-GPU load balancing, raising training throughput by 30.8% over the original batching strategy. Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score (65.64%) among eight tested models, ranks first on seven of twelve benchmarks, and outperforms its unmodified autoregressive base model on all twelve, while running 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.
Figures & tables
Figure 2: \captionlead Task formulation and coarse-to-fine inference. Every stage uses a shared candidate scorer. Choice tasks select a supplied option. Numerical estimation selects a coarse interval and then a fine interval, returning its midpoint. Point prediction uses three 3×3 region selections, re-encoding crops between stages and returning the final cell center.
\mindlabtablehead Output
\mindlabtablehead Candidates per stage
\mindlabtablehead Stages
\mindlabtablehead Decoded answer
Choice
Supplied option contents
1
The selected option, mapped back through the recorded option permutation.
Length (m)
32 roots; 4 or 8 fine cells
1–2
Zero, overflow, or fine-cell midpoint.
Count
32 coarse groups; 32 integers
2
32a1+a2 , covering 0–1023 exactly.
Angle (deg)
14 roots; 32 fine cells
1–2
Zero, overflow, or degree interval midpoint.
Point
9 spatial cells at each level
3
Center of the final cell in original image coordinates.
Table 1: Output types and their decision structure. Numerical tasks share the scalar interface but keep separate codebooks. Ordinary pointing produces one point; the probability tree is an evaluation extension for multi-point counting.
Figure 3: \captionlead Architecture of MJev-Spatial. Each candidate is followed by the same End marker. After contextualization by the language backbone, the state at each End position is passed through the same pretrained WK to form a separate candidate key; the final Decide state supplies the query through WQ . Their similarities determine the selected candidate, with no added prediction head or new parameters. Native normalization, rotary transformations, and head averaging are retained but omitted from the diagram. Probability bars are schematic.
\mindlabtablehead Model
\mindlabtablehead SAT
\mindlabtablehead CV- Bench
\mindlabtablehead Robo- VQ
\mindlabtablehead BLINK
\mindlabtablehead MMSI
\mindlabtablehead Ref Spatial
\mindlabtablehead Where 2Place
\mindlabtablehead Robo- POI
\mindlabtablehead Point- Bench
\mindlabtablehead ERQA
\mindlabtablehead Emb Spatial
\mindlabtablehead VSI full
\mindlabtablehead Overall
MJev-Spatial
79.33
88.82
79.06
67.17
45.60
58.50
67.96
55.74
64.38
46.75
78.46
55.93
65.64
Molmo2-ER
80.00
88.31
62.01
72.17
42.90
52.00
46.64
26.23
53.47
46.50
77.99
60.55
59.06
Qwen3.5-4B
43.00
85.56
56.17
63.11
33.90
46.00
55.98
50.00
62.65
45.50
75.82
39.93
54.80
Qwen3-VL-8B-Instruct
83.33
86.37
74.72
61.76
31.40
49.00
67.07
59.02
61.54
43.25
78.90
57.07
62.79
Spatial-SSRL-Qwen3VL-4B
79.67
85.52
65.26
60.39
26.70
48.75
66.54
61.48
60.60
41.25
78.10
46.65
60.08
SenseNova-SI Qwen3-VL-8B
79.67
83.15
75.68
56.08
35.30
33.00
45.92
45.90
52.34
41.00
75.33
59.38
56.90
Table 2: Performance comparison of MJev-Spatial with seven autoregressive VLMs on 12 spatial understanding benchmarks . Best and second-best scores are bold and underlined, respectively.
\mindlabtablehead Metric
\mindlabtablehead Linear
\mindlabtablehead KEV
\mindlabtablehead Native Q/K
SAT real
78.33
76.33
78.00
CV-Bench
83.80
85.31
85.38
RefSpatial
49.00
45.50
51.00
Where2Place
67.99
61.02
59.98
RoboSpatial pointing
65.57
63.11
62.30
RoboSpatial VQA
60.34
68.54
68.06
Table 3: Comparison results on pretrained candidate scoring. “Linear” and “KEV” are abbreviation for linear classifier and KEV pointer, respectively.
Table 4: Numerical prediction ablations. (a) Fine-stage candidate count with fixed coarse intervals and argmax decoding. (b) Fine-stage decoding using the same 216-cell model, with coarse-stage argmax. Quantization error uses training length labels; prediction metrics use the VST-300 dev set. Shading marks the adopted settings.
\mindlabtablehead Metric
\mindlabtablehead Crop
\mindlabtablehead Original
\mindlabtablehead ViT replay
\mindlabtablehead KV replay
SAT real
78.00
74.67
73.33
78.00
CV-Bench
85.31
85.14
84.56
84.52
RoboSpatial VQA
68.06
68.75
65.01
63.38
BLINK val
63.04
62.56
61.07
62.74
MMSI-Bench
27.80
25.80
26.90
26.80
RefSpatial
51.00
46.50
47.00
45.50
Table 5: Matched training for alternative visual inputs. All four models use L32 mean, 72K QA, and three nine-way decisions. Each alternative is trained from the same pretrained base.
\mindlabtablehead Layout
\mindlabtablehead QA/s
\mindlabtablehead Padding (%)
\mindlabtablehead Step CV (%)
\mindlabtablehead Tail (ms)
Original order
99.06
35.17
2.17
39.5
Local length
116.91
12.36
4.13
157.6
Global balanced
129.54
1.77
6.07
85.5
Global + buffer 128
129.39
1.03
19.77
95.8
Table 6: Training efficiency on the historical 72K corpus: eight GPUs, global batch 256, microbatch 8, four accumulation steps. Each condition runs 100 updates; the last 92 updates contain the same 23,552 measured QA. Tail includes communication and waiting. No quality evaluation was performed.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
\mindlabtablehead Source
\mindlabtablehead Choice
\mindlabtablehead Point
\mindlabtablehead Length
\mindlabtablehead Angle
\mindlabtablehead Count
\mindlabtablehead Total
RefSpatial ( Zhou et al., 2025a )
6.1M
3.5M
1.3M
108K
0
11M
SAT ( Ray et al., 2025 )
168K
0
0
0
0
168K
SenseNova-SI ( Cai et al., 2026 )
424K
0
34K
2
25K
484K
VSI-590K-image ( Yang et al., 2025a )
203K
0
27K
12K
13K
255K
VST-P ( Yang et al., 2025b )
218K
0
170K
0
0
388K
Total
7.1M
3.5M
1.6M
120K
38K
12.3M
Appendix
Table 7: Final r12 corpus after content-only option cleanup. Counts denote original QA, not the number of decision stages. Source names refer to training annotations, including adaptations of existing data.
\mindlabtablehead Task
\mindlabtablehead QA
\mindlabtablehead Share (%)
\mindlabtablehead Decision rounds
Choice
7.1M
57.65
1
Point
3.5M
28.34
3
Length
1.6M
12.71
1–2
Angle
120K
0.98
1–2
Count
38K
0.31
2
Appendix
Table 8: Natural task distribution used by the final one-pass training recipe. No task rebalancing or repeated oversampling is applied.
\mindlabtablehead Item
\mindlabtablehead Configuration
Initialization
Official post-trained Qwen3.5-4B; fresh from base
Readout
L32 native Q/K mean; no layer fusion
Data
r12 content-only; 12.3M QA; each exactly once
Global / per-GPU micro / accumulation
512 / 8 / 8 on eight GPUs
Updates / last real batch
24,032 / 186 QA
Backbone / readout LR
5×10−6 / 2×10−4
Appendix
Table 9: Recorded configuration for training on the 12.3M-QA corpus.
\mindlabtablehead Benchmark
\mindlabtablehead Requests
\mindlabtablehead Score and protocol
SAT real ( Ray et al., 2025 )
300
Mean accuracy over both orders of 150 real-image questions.
CV-Bench ( Tong et al., 2024 )
2.6K
0.25ADE20K+0.25COCO+0.5Omni3D ( Zhou et al., 2017 ; Lin et al., 2014 ; Brazil et al., 2023 ) ; counting is included in the overall score.
RoboSpatial VQA ( Song et al., 2025 )
228
Mean of configuration (123) and compatibility (105) accuracies; RGB input with the current annotations.
BLINK val ( Fu et al., 2024 )
1.9K
Macro average of all 14 public validation tasks; no test split. Use the paired Hugging Face options and labels.
MMSI-Bench ( Yang et al., 2026 )
1K
Exact-option accuracy over all 11 types; all 2–10 images in original order; not circular accuracy.
RefSpatial ( Zhou et al., 2025a )
200
Binary-mask mean point accuracy; location 100 and placement 100; the overlapping unseen subset is not added again.
Appendix
Table 10: Historical 72K pilot evaluation scope. Counts are execution requests.
\mindlabtablehead Model
\mindlabtablehead SAT
\mindlabtablehead CV-Bench
\mindlabtablehead Robo-VQ
\mindlabtablehead BLINK
\mindlabtablehead MMSI
\mindlabtablehead RefSpatial
MJev-Spatial
155.1
76.5
88.9
106.6
116.0
204.8
Spatial-SSRL- Qwen3VL-4B
237.8
127.7
123.8
124.0
139.4
682.0
SpatialThinker-7B
266.4
120.8
152.3
172.4
236.8
458.8
Qwen3-VL-8B- Instruct
250.8
128.5
111.9
128.6
151.0
855.5
SenseNova-SI Qwen3-VL-8B
223.3
115.4
102.7
105.0
114.0
858.7
Molmo2-ER
343.3
146.5
226.1
280.3
348.5
373.3
Appendix
Table 11: Final-model inference efficiency on the complete twelve-benchmark evaluation sets. Entries are mean single-query latency in milliseconds on one NVIDIA B200, batch size one. Timing includes image I/O, preprocessing, host-to-device transfer, inference/decision, and decoding; warmup and model loading are excluded. The two panels contain the same eight models.
\mindlabtablehead Metric
\mindlabtablehead AR
\mindlabtablehead Linear
\mindlabtablehead KEV
\mindlabtablehead Mean
\mindlabtablehead Concat
\mindlabtablehead Merge
SAT real
43.00
78.33
76.33
78.00
75.67
72.00
CV-Bench
85.56
83.80
85.31
85.38
85.12
85.22
RoboSpatial VQA
56.64
60.34
68.54
68.06
66.84
65.27
BLINK val
62.98
61.71
62.99
63.04
62.69
61.97
MMSI-Bench
34.20
27.70
25.10
27.70
28.90
27.10
RefSpatial
46.00
49.00
46.00
51.00
51.00
50.00
Appendix
Table 12: Completed Qwen experiments on the fixed evaluation union. All adapted models use the 72K-QA recipe and the historical length codebook; these are not the 12.3M-QA model. Higher is better except VST MAE. Point-Bench is recomputed as the five-category macro average from retained category scores. Original run provenance and differing sequence budgets are retained.
\mindlabtablehead Category
\mindlabtablehead AR
\mindlabtablehead Linear
\mindlabtablehead KEV
\mindlabtablehead Mean
\mindlabtablehead Concat
\mindlabtablehead Merge
\mindlabtablesection SAT real
action_conseq
74.32
83.78
78.38
78.38
75.68
77.03
ego_movement
8.70
95.65
100.00
100.00
100.00
67.39
goal_aim
48.53
95.59
92.65
94.12
91.18
94.12
obj_movement
41.30
76.09
73.91
73.91
65.22
69.57
perspective
27.27
43.94
42.42
48.48
50.00
48.48
Appendix
Table 13: Category-level main comparison. Percentages use each task's recorded scorer. Historical length codebook and root-pruned Point-Bench. Categories do not share a common difficulty scale.
Figure 4: \captionlead The complete 8×3 depth/aggregation experiment, shown on three representative metrics. Early visual readouts are inadequate, while later layers exhibit task-dependent rather than monotonic gains. All 24 models’ benchmark values are provided in Section E.3 .
\mindlabtablehead Metric
\mindlabtablehead L32
\mindlabtablehead +L28
\mindlabtablehead +L24
\mindlabtablehead +L28+L24
SAT real
78.00
75.33
76.33
78.67
CV-Bench
85.31
84.66
83.46
85.31
RoboSpatial VQA
68.06
67.18
66.70
65.62
BLINK val
63.04
62.25
63.37
62.03
MMSI-Bench
27.80
27.00
27.00
27.70
RefSpatial
51.00
47.50
50.00
48.00
Appendix
Table 14: Equal-weight native Q/K layer fusion. Each fusion is trained independently; L32 is re-evaluated with the same evaluator. Point-Bench uses the five-category macro average.
\mindlabtablehead Readout
\mindlabtablehead SAT
\mindlabtablehead CV
\mindlabtablehead Robo VQA
\mindlabtablehead BLINK
\mindlabtablehead MMSI
L4 mean
52.67
51.41
72.32
40.34
27.00
L4 concat
49.67
52.02
64.51
41.46
26.30
L4 merge
49.67
52.48
71.03
41.24
28.30
L8 mean
49.67
58.87
66.07
41.37
25.10
L8 concat
53.33
59.71
50.38
39.58
26.20
L8 merge
52.33
62.39
73.61
40.65
24.50
Appendix
Table 15: Complete depth/readout sweep: classification metrics. Every model uses the same QA budget and aligned recipe. The retained parameter count varies with depth. Point-Bench is macro.
\mindlabtablehead Readout
\mindlabtablehead RefSpatial
\mindlabtablehead W2P
\mindlabtablehead Robo point
\mindlabtablehead PB macro
\mindlabtablehead MAE (m)
L4 mean
4.50
13.96
4.10
12.38
0.6480
L4 concat
3.00
13.96
4.92
12.07
0.6397
L4 merge
6.50
12.96
5.74
10.15
0.6417
L8 mean
7.00
14.95
17.21
17.58
0.4895
L8 concat
15.00
29.93
28.69
25.92
0.4833
L8 merge
16.00
33.91
31.15
28.58
0.4894
Appendix
Table 16: Complete depth/readout sweep: pointing metrics. Every model uses the same QA budget and aligned recipe. The retained parameter count varies with depth. Point-Bench is macro.
Figure 5: \captionlead Centered candidate-logit RMS for each head, averaged per QA along the all-head trajectory. Values share a common scale and measure response amplitude. H02 is strongest and H11 weakest in all three task groups.
\mindlabtablehead Head
\mindlabtablehead Class RMS
\mindlabtablehead Point RMS
\mindlabtablehead Length RMS
\mindlabtablehead Class Δ pp
\mindlabtablehead Point Δ pp
\mindlabtablehead Length Δ m
H01
2.324
3.938
5.742
-0.011
+0.127
+0.000000
H02
2.394
4.065
5.954
+0.009
+0.127
+0.000000
H03
2.328
3.932
5.708
+0.020
+0.127
+0.000000
H04
2.368
4.022
5.917
+0.009
+0.127
+0.000279
H05
1.202
2.055
3.236
+0.050
+0.102
-0.000346
H06
1.180
2.032
3.232
+0.002
+0.102
-0.003992
Appendix
Table 17: All 16 single-head interventions. Positive effect means retaining the head helps: full-minus-masked score, or masked-minus-full length MAE. Classification/pointing effects are benchmark-equal diagnostic means; RMS values average per question.
\mindlabtablehead Metric
\mindlabtablehead Crop
\mindlabtablehead Original only
\mindlabtablehead Original + cache
RefSpatial (%)
51.00
9.00
9.00
Where2Place (%)
59.98
32.00
32.00
RoboSpatial pointing (%)
62.30
20.49
20.49
Point-Bench macro (%)
56.84
21.96
21.96
Counting success (%)
39.29
4.08
4.08
Count MAE
1.7296
4.0459
4.0561
Appendix
Table 18: Fixed crop-trained native-mean checkpoint: removing new pixel crops, with and without a reusable original-image prefix. All three textual grid rounds remain. The crop control was not timed in this run.
\mindlabtablehead Metric
\mindlabtablehead K argmax
\mindlabtablehead K comp.
\mindlabtablehead K task
\mindlabtablehead N argmax
\mindlabtablehead N comp.
\mindlabtablehead N task
RefSpatial
38.50
35.14
38.50
45.50
43.65
45.50
Where2Place
67.96
64.43
64.43
60.91
58.99
58.99
RoboSpatial pointing
56.56
57.38
57.38
60.66
63.11
63.11
Point-Bench (macro)
46.46
26.81
49.12
50.76
32.71
54.63
PB counting
13.27
26.53
26.53
12.76
32.14
32.14
Appendix
Table 19: Direct visual-token pointing with KEV (K) or native mean (N). Each model shares weights/logits across argmax, components, and task-based decoding. Task routing was specified after the first decoding experiment; Point-Bench is macro.
\mindlabtablehead Metric
\mindlabtablehead KEV grid
\mindlabtablehead KEV visual
\mindlabtablehead Mean grid
\mindlabtablehead Mean visual
SAT real
76.33
76.67
78.00
77.00
CV-Bench
85.31
83.95
85.38
84.52
RoboSpatial VQA
68.54
64.32
68.06
62.50
BLINK val
62.99
63.87
63.04
62.76
MMSI-Bench
25.10
25.50
27.70
30.20
RefSpatial
46.00
38.50
51.00
45.50
Appendix
Table 20: Visual-token pointing versus hierarchical crop pointing. Visual-token results use task-based single/multiple-point routing established after the initial decoding experiment. This routing differs from the grid baseline on Where2Place and RoboSpatial; it is an exploratory comparison.
\mindlabtablehead Metric
\mindlabtablehead Continue/stop
\mindlabtablehead Full tree
\mindlabtablehead Root-pruned
Official success (%)
19.39
38.78
38.78
Exact count (%)
22.45
46.43
46.43
Count MAE
11.643
1.735
1.740
Predicted points
10.485
3.342
3.337
Decision nodes
–
91.00
51.10
Evaluated cells
–
729.00
405.83
Appendix
Table 21: Point-Bench counting: all 196 questions, fixed L32 mean weights, no retraining. Full/pruned are an actual paired run; historical continue/stop quality uses a different execution setting and has no comparable timing here.
Figure 6: \captionlead Probability-tree example from the first request in the preselected 12-case gallery. Full expansion uses 91 nodes; root pruning uses 51 and marks unvisited regions in gray. Both return four points in this case. Example from Point-Bench ( Cheng et al., 2025 ) .
\mindlabtablehead Peak threshold
\mindlabtablehead Full success
\mindlabtablehead Pruned success
\mindlabtablehead Full count
\mindlabtablehead Pruned count
\mindlabtablehead Full MAE
\mindlabtablehead Pruned MAE
5%
30.10
30.10
34.69
34.69
2.041
2.041
10%
38.78
38.78
46.43
46.43
1.735
1.740
20%
37.24
37.24
43.88
43.88
1.765
1.760
Appendix
Table 22: Offline component-threshold sensitivity on the retained maps. Success and exact-count columns are percentages. The primary threshold is fixed at 10%.
\mindlabtablehead VST-300 diagnostic
\mindlabtablehead Historical
\mindlabtablehead 30×32
\mindlabtablehead 4/8 argmax
\mindlabtablehead 4/8 exp.
MAE (m), lower
0.335420
0.327732
0.320207
0.329955
AbsRel (%), lower
24.64
31.04
29.03
30.04
∣e∣≤0.1 m (%), higher
56.67
58.33
59.00
55.67
∣e∣/y≤25% (%), higher
69.46
69.46
68.12
70.81
Appendix
Table 23: Fixed internal length diagnostic. 4/8 argmax is the decoder used by MJev-Spatial; 4/8 exp. decodes the same logits with a fine-stage conditional expectation. Coarse decoding is argmax in every condition.
\mindlabtablehead Metric
\mindlabtablehead Historical
\mindlabtablehead 30×32
\mindlabtablehead 30×(4/8)
SAT real
78.00
72.33
80.67
CV-Bench
85.39
85.08
85.21
RoboSpatial VQA
68.06
64.87
66.30
BLINK val
63.10
60.94
62.52
MMSI-Bench
27.70
25.40
27.60
RefSpatial
51.00
49.50
52.00
Appendix
Table 24: Length-codebook ablation, independently trained on the same 72K QA. The historical checkpoint is re-evaluated within this experiment. All point and choice decoders are unchanged.
\mindlabtablehead Representation
\mindlabtablehead MAE (m)
\mindlabtablehead P99 (m)
\mindlabtablehead Max (m)
\mindlabtablehead ≤1 cm (%)
\mindlabtablehead ≤1% rel.
Historical 200 m
1.6548
1.2384
8320.10
59.33
80.29
Uniform 960
6.4545
8.5012
8.53
0.06
1.12
Log 384
0.0939
1.5758
43.10
47.53
52.36
Log 960
0.0379
0.6371
47.14
69.86
99.92
Quantile 960
4.2371
3.0380
3935.55
80.78
95.50
Hierarchical log1p 960
0.0391
0.6657
21.39
69.42
99.84
Appendix
Table 25: Training-label reconstruction on all 1.6M lengths, assuming the correct discrete cell is selected. The 960-leaf alternatives differ in allocation; only the hierarchical log1p proposal is tested in the subsequent matched model experiment. Task-specific routing is an unadopted proposal.
Figure 7: \captionlead Controlled scheduling comparison on the historical 72K corpus. Local grouping reduces padding but leaves more rank imbalance; global grouping and balancing deliver the largest throughput gain. A cross-update buffer reduces padding slightly further without increasing throughput.
\mindlabtablehead Global
\mindlabtablehead Micro
\mindlabtablehead Accum.
\mindlabtablehead Outcome
\mindlabtablehead s/update
\mindlabtablehead QA/s
\mindlabtablehead Peak GB
256
8
4
Complete
2.3007
111.27
242.03
512
8
8
Complete
3.7629
136.07
242.05
256
16
2
OOM
–
–
–
512
16
4
OOM
–
–
–
256
32
1
OOM
–
–
–
512
32
2
OOM
–
–
–
Appendix
Table 26: Expanded-data memory feasibility. The original six cases use 4,096 warmup and 8,192 measured QA; the additional global384/micro12 trial also fails. Partial timings from failed configurations are excluded. The successful rows are resummarized on a common window in Table 27 .
\mindlabtablehead Global
\mindlabtablehead Micro
\mindlabtablehead Accum.
\mindlabtablehead Timed steps
\mindlabtablehead s/update
\mindlabtablehead QA/s
\mindlabtablehead Peak GB
256
8
4
30
2.3265
110.04
242.03
512
8
8
15
3.7791
135.48
242.05
768
8
12
10
5.4944
139.78
242.03
Appendix
Table 27: New-data profiling with the 4/8 length codebook: a common 7,680-QA window after 4,608 QA warmup. The 12,800-QA pool is a stratified prefix sample, not the natural full-data distribution. Peak is the maximum per-rank allocated GPU memory, in decimal GB. Microbatch 16 and 32 failed with OOM in both original 256/512 conditions.
Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient Spatio-vision Language Models (SpatioLM), that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at \faGithub~spatio-lm.
Jing Wu, Jianhua Wu, Jiayi Guan +5
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher
Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning. However, a fundamental capability mismatch remains: general VLMs can reason about the overall task but often miss the visual details that determine success, while specialist vision models can capture those details but cannot translate them into task-level decisions. In this work, we propose SpatialCLI, a framework that teaches VLMs to reason with spatial tools and progressively internalize the specialist perceptual capabilities they provide. SpatialCLI proceeds in three stages: (1) Call exposes specialist vision models as spatial tools to augment the VLM's perception; (2) Learn uses Cold-Start SFT and agentic RL to improve tool use; and (3) Internalize verbalizes successful tool-use trajectories to internalize specialist perceptual capabilities. We further introduce SpatialCLI-Bench, a 516-example benchmark for compositional perception across localization, segmentation, depth, and pose. On MindCube, SpatialCLI raises Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing GPT-5.6 Sol with tools (72.1%), while retaining 73.8% without tools after internalization.
Yang Zhou, Zixuan Huang, Sunzhu Li +10
Zhejiang University · Zhuoyu Technology, Shenzhen, China · Beihang University +3
Spatial understanding is fundamental to embodied intelligence, underpinning applications such as robotic manipulation, embodied navigation, and autonomous driving. Although recent vision-language models (VLMs) have achieved impressive performance on spatial reasoning benchmarks, state-of-the-art approaches typically rely on additional spatial encoders or architectural modifications during inference, increasing computational cost. We introduce Space Tokens, a lightweight, architecture-agnostic framework that equips VLMs with explicit continuous spatial representations without requiring additional inference-time modules. By distilling scene-level 3D geometry and object-centric spatial attributes into continuous latent tokens, our method enables these modalities to be directly incorporated into a chain-of-thought reasoning process, thereby improving the VLM's spatial reasoning capabilities. At the same time, the learned representations can be explicitly decoded to verify that they encode meaningful geometric information, while the unified token interface remains extensible to additional modalities. Experiments on VSI-Bench improve Qwen3-VL-8B by 4.3% and SenseNova-SI-1.3 by 1.3%, while achieving state-of-the-art performance on object size (79.2%) and room size estimation (75.7%). These results demonstrate that continuous spatial tokens provide an effective, interpretable, and computationally efficient mechanism for integrating geometric reasoning into large vision-language models.
Hunter Schofield, Mohammed Elmahgiubi, Mohammad Mahdavian +4
York University · Huawei Technologies Canada · University of Toronto