Large vision-language models (VLMs) still struggle with reliable 3D spatial reasoning, a core capability for embodied and physical AI systems. This limitation arises from their inability to capture fine-grained 3D geometry and spatial relationships. While recent efforts have introduced multi-view geometry transformers into VLMs, they typically fuse only the deep-layer features from vision and geometry encoders, discarding rich hierarchical signals and creating a fundamental bottleneck for spatial understanding. To overcome this, we propose SpatialStack, a general hierarchical fusion framework that progressively aligns vision, geometry, and language representations across the model hierarchy. Moving beyond conventional late-stage vision-geometry fusion, SpatialStack stacks and synchronizes multi-level geometric features with the language backbone, enabling the model to capture both local geometric precision and global contextual semantics. Building upon this framework, we develop VLM-SpatialStack, a model that achieves state-of-the-art performance on multiple 3D spatial reasoning benchmarks. Extensive experiments and ablations demonstrate that our multi-level fusion strategy consistently enhances 3D understanding and generalizes robustly across diverse spatial reasoning tasks, establishing SpatialStack as an effective and extensible design paradigm for vision-language-geometry integration in next-generation multimodal physical AI systems.
Figures & tables
Figure 2: Architecture of SpatialStack. A standard VLM backbone is coupled with a multi-view geometry encoder whose layer-wise features are processed by layer-specific projectors and sequentially injected into the LLM decoder, progressively integrating geometric cues. Explanation of the similarity heatmaps on the left is provided in Sec. 3 . This multi-level injection preserves both fine-grained geometric structure and high-level spatial context, supporting more reliable low-level understanding and high-level reasoning.
Figure 3: Examples of spatial tasks at different levels. The left example ( Low-Level Task ) targets fine-grained geometric perception, such as determining which of two points is closer to the camera. The right example ( High-Level Task ) requires higher-level spatial reasoning, where the model must estimate the distance between two objects by comparing their closest points in 3D space.
Figure 4: Effect of Geometry Injection Layers on Spatial Tasks. Deeper layers improve high-level tasks, while low-level tasks peak at layer 11 and decline at deeper layers, suggesting a trade-off between fine-grained perception and higher-level reasoning.
Model
Low-Level Avg
High-Level Avg
Overall
Qwen3.5 (fine-tuned)
61.37
64.76
63.07
Single-layer (geo enc: 11)
66.11
64.48
65.30
Single-layer (geo enc: 17)
63.89
65.76
64.83
Single-layer (geo enc: 23)
64.33
66.36
65.35
Multi-Layer Fusion
64.69
65.15
64.92
Table 1: Ablation Results on Geometry Token Fusion Depth. Simply fusing multi-layer geometry features to the visual features yields suboptimal performance, while selecting an appropriate single geometry encoder layer achieves better task-specific trade-offs.
Methods
VSI-Bench
SPAR-Bench
BLINK-Spatial
CV-Bench
Overall
Qwen3.5 (fine-tuned)
64.76
68.75
56.10
84.49
68.52
GVF-L23 (VG-LLM [ 60 ] )
66.36
70.83
51.91
84.64
68.43
GVF-L11/17/23
65.15
71.20
51.28
84.33
67.99
SpatialStack
67.52
71.39
52.12
85.53
69.14
Table 2: Cross-benchmark Ablation. SpatialStack achieves the best cross-task transfer ability, obtaining the highest scores on VSI-Bench , SPAR-Bench , CV-Bench , and the overall average, while the Qwen3.5 baseline remains strongest on BLINK-Spatial . Gray cells denote the highest value in each column.
Figure 5: Evaluation on VSI-Bench. Dark orange cells denote the best open-source result in each column, while light orange cells denote the second-best open-source result. Group-wise ranks within proprietary and open-source model blocks are highlighted in purple, with dark purple , medium purple , and light purple indicating 1st, 2nd, and 3rd place, respectively.
Model
2D (%)
3D (%)
Avg. (%)
Proprietary Models (API)
GPT-4o [ 19 ]
74.8
83.0
78.9
Open-source Models
Mini-Gemini-HD-34B [ 27 ]
71.5
79.2
75.4
LLaVA-NeXT-34B [ 24 ]
73.0
74.8
73.9
Cambrian-1-34B [ 45 ]
74.0
79.7
76.9
Table 3: Comparison on CV-Bench. Built on Qwen2.5, SpatialStack-4B outperforms its base model alongside VG-LLM and Cambrian-S. Scaling to Qwen3.5, SpatialStack-5B further improves upon its baseline to set a new state-of-the-art.
Method
MMBench
Video -MME
BLINK
Temp Compass
Overall
Qwen3.5-4B
83.25
62.44
61.12
66.84
68.41
SpatialStack-5B (Qwen3.5)
83.42
63.74
55.46
69.37
68.00
Table 4: General Capabilities Evaluation. Our SpatialStack-5B maintains robust general multimodal and spatial-temporal reasoning capabilities, demonstrating no catastrophic forgetting.
Metric
L21
L22
L23
+ L21
+ L22
+ L23
Low-Level
63.54
64.87
64.33
65.89
65.45
64.44
High-Level
65.57
66.51
66.36
65.95
66.78
67.52
Table 5: Layer Selection Ablation. Performance comparison of extracting geometry features from different deep VGGT layers (L21, L22, L23) and their multi-layer combinations.
Methods
VSI Bench
SPAR Bench
BLINK Spatial
CV Bench
Overall
Qwen3.5
64.76
68.75
56.10
84.49
68.52
Vision Fusion
64.27
69.68
56.45
83.11
68.38
SpatialStack (Reverse)
67.22
71.97
50.08
84.82
68.52
SpatialStack (final)
67.52
71.39
52.12
85.53
69.14
Table 6: Geometry-Language Fusion Order Ablation. Comparison of our progressive hierarchical alignment against a reverse fusion strategy and baseline models.
Figure A : Task-type distribution of the sampled SPAR subset. The bar chart reports the counts of all 33 spatial task types after randomly sampling 60% of SPAR-234k for training.
Figure B : Task-type distribution of the seven tasks in the VSI-Bench setting. The pie chart summarizes the combined composition from VLM3R-ScanNet and the sampled Appearance-Order subset from VSI-590K, which are merged for unified reporting.
Dataset
Raw
Train Subset
SPAR 234k
234k (66.3%)
140k (66.4%)
LLaVA-Hound 64k
63.8k (18.0%)
38.3k (18.1%)
VLM3R-ScanNet
51.8k (14.6%)
31.1k (14.7%)
VSI App-Order
3.8k (1.1%)
1.9k (0.9%)
Total
353k (100%)
212k (100%)
Table A : Dataset scales and sampled subsets used in our ∼200k training mixture. We sample 60% from SPAR-234k, LLaVA-Hound-64k [ 60 ] , and the ScanNet split of VLM-3R [ 13 ] , and add ∼2k appearance-order instances from VSI-590k [ 53 ] to compensate for the missing ordering supervision. Percentages indicate each dataset’s contribution to the final mixture.
Category
Setting
Base model
Qwen2.5-VL-3B or Qwen3.5-4B
Geometry encoder
VGGT-1B (frozen)
Fusion strategy
SpatialStack (multi-depth)
Trainable modules
Language tower + fusion modules
Precision
bfloat16
Optimizer
AdamW (wd=0.01)
Table B : Training hyperparameters for SpatialStack. Geometry-aware instruction tuning is performed on Qwen2.5-VL-3B or Qwen3.5-4B with VGGT-1B using the proposed SpatialStack fusion. The language tower and fusion modules are trainable, while the geometry encoder remains frozen. Training uses AdamW (bfloat16, cosine schedule) with an effective batch size of 64 under ZeRO-2 parallelism.
Figure C : ROI similarity comparison between geometry and vision features across encoder depths. For two indoor scenes, the top row shows the RGB image with the ROI marked in red. The lower rows display similarity maps (brighter means more similar) at 50% , 75% , and 100% depths of the geometry encoder (left) and the vision encoder (right). Geometry features preserve meaningful spatial structure, while vision features are noisy and become nearly uniform at deeper layers.
Method
Count
Relation
Depth
Distance
Overall
SpatialRGPT
60.4
78.9
80.0
71.3
72.7
Spatialbot
61.4
73.1
76.5
61.0
68.0
Ours
69.0
92.5
93.7
90.7
86.5
Table C: Additional Baseline Comparison on CV-Bench.
3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.