Spa3R: Predictive Spatial Field Modeling for 3D Visual Reasoning
Authors: Haoyi Jiang, Liu Liu, Xinjie Wang, Yonghao He, Wei Sui, Zhizhong Su, Wenyu Liu, Xinggang Wang
Organizations: School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · Horizon Robotics, Beijing 100094, China · D-Robotics, Shenzhen 518055, China
Vision-language models excel at 2D visual understanding but remain limited in 3D spatial reasoning. Existing approaches either depend on explicit 3D modalities, which limits scalability, or inject partial, view-conditioned geometric priors and leave the language model to recover global scene structure from sparse cues. We introduce Spa3R, a self-supervised framework that learns a unified, view-invariant spatial representation from unposed multi-view RGB images. Its Predictive Spatial Field Modeling objective compresses context views into a compact latent representation and predicts aligned geometric and semantic feature fields at novel viewpoints, thereby encouraging coherent encoding of scene geometry and layout. We integrate the pre-trained Spa3R Encoder into a vision-language model through a lightweight residual cross-attention adapter, yielding Spa3-VLM and grounding language reasoning in global spatial context. Spa3-VLM achieves an average score of 58.6% on VSI-Bench and delivers leading or competitive performance across three additional spatial reasoning benchmarks. These results demonstrate that predictive spatial representation learning provides an effective visual foundation for 3D reasoning.
Figures & tables
Fig. 1: Comparison of paradigms for spatial reasoning. Explicit 3D modalities require specialized sensors, while view-conditioned priors leave the VLM to infer global 3D structure from per-view features. In contrast, Spa3R learns a unified, view-invariant spatial field directly from multi-view images for downstream VLM reasoning.
Fig. 2: Overview of the Spa3R framework and Spa3-VLM integration. (a) The Spa3R Encoder maps unposed context views to a unified, view-invariant spatial latent representation z . (b) The Spa3R Decoder synthesizes target features F^t at queried novel views conditioned on z and target camera embeddings r . (c) For downstream spatial reasoning, the pre-trained Spa3R Encoder produces z , and a lightweight adapter fuses the VLM’s native visual features FV with the spatial latent through cross-attention.
Obj. Count
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
Model
Avg.
Numerical Answer
Multiple-Choice Answer
Proprietary Models
GPT-4o [ 48 ]
34.0
46.2
5.3
43.8
38.2
37.0
41.3
31.5
28.5
Gemini-1.5-Flash [ 49 ]
42.1
49.8
30.8
53.5
54.4
37.7
41.0
31.5
37.8
Gemini-1.5-Pro [ 49 ]
45.4
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
Open-source Models
TABLE I: Quantitative comparison on the VSI-Bench [ 1 ] spatial reasoning benchmark
Model
CV
SPAR
ViewSpatial
2D
3D
Avg.
Qwen2.5-VL-3B [ 10 ]
69.1
72.2
70.6
24.6
35.6
SpatialLadder-3B [ 55 ]
72.4
74.9
73.7
34.4
44.2
Spatial-MLLM-4B [ 7 ]
-
-
73.8
35.1
43.6
SpatialThinker-3B [ 56 ]
71.0
76.3
73.6
-
-
SpaceR-7B [ 57 ]
-
-
-
37.6
37.3
TABLE II: Quantitative comparison across CV-Bench [ 45 ] , SPAR-Bench [ 46 ] , and ViewSpatial-Bench [ 47 ]
Metric
Score
VGGT geometry feature
L1 ↓
0.5280 ± 0.1398
Cosine ↑
0.8806 ± 0.1200
DINOv3 semantic feature
L1 ↓
0.1371 ± 0.0119
Cosine ↑
0.7106 ± 0.0602
TABLE III: Novel-view spatial field prediction on ScanNet++
VGGT
DINOv3
CLIP
NA (MRA)
MCA (Acc.)
Avg.
✓
60.5
54.4
57.5
✓
59.0
54.4
56.7
✓
59.5
44.3
51.9
✓
✓
60.4
56.8
58.6
TABLE IV: Ablation of feature-prediction targets for Spa3R pre-training
Spatial Repr.
NA (MRA)
MCA (Acc.)
Avg.
None
58.4
43.5
50.9
VGGT
60.5
49.5
55.1
Spa3R
60.4
56.8
58.6
TABLE V: Ablation on spatial representation paradigms
Fig. 3: Qualitative visualization of learned feature fields in 3D space. After unprojection onto the VGGT point cloud, features from different views show consistent spatial organization and smooth transitions. These examples provide qualitative evidence that the predicted features preserve the underlying scene geometry across views.
Integration
NA (MRA)
MCA (Acc.)
Avg.
None
58.4
43.5
50.9
Seq. Append.
58.5
43.7
51.1
Cross-Attn.
60.4
56.8
58.6
TABLE VI: Ablation on VLM integration architectures
Mask ratio
NA (MRA)
MCA (Acc.)
Avg.
25%
58.9
56.1
57.5
50%
60.4
56.8
58.6
75%
59.6
56.7
58.1
TABLE VII: Ablation of the target-view ratio used for PSFM pre-training
Fig. 4: Qualitative visualization of learned feature fields in novel views. Predicted features show continuous and spatially coherent layouts similar to the aligned targets. Colors do not match exactly because separate PCA projections are fitted to the two feature sets. The red boxes highlight predictions in regions that are occluded or unobserved in the context; these examples provide qualitative evidence of cross-view consistency within the sampled target views.
Camera Enc.
NA (MRA)
MCA (Acc.)
Avg.
Plücker
59.9
55.4
57.6
PRoPE
60.4
56.8
58.6
TABLE VIII: Ablation of target-view encoding in the Spa3R Decoder
Fig. 5: Qualitative Spa3-VLM predictions on VSI-Bench. Each example shows six representative frames sampled from the 32-frame input used by the evaluation protocol, followed by the task type, question, Spa3-VLM answer, and ground-truth answer. Spa3-VLM correctly counts repeated object instances and estimates object scale.
3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, China · YUKUN Intelligent World, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China