3D spatial reasoning from video remains challenging for VLMs because videos provide sequential, view-local observations rather than persistent metric scene records. Many questions must be answered in a query-specified perspective rather than the observed camera viewpoint, and prompting with a fixed set of RGB views may miss sparse or occluded visual cues. To address these gaps, we introduce BEV3SR, a training-free and backbone-agnostic framework for RGB-D videos with camera parameters. BEV3SR turns video observations into queryable metric bird's-eye-view (BEV) scene memory and formalizes each query into an evidence-aware spatial task. The Spatial Task Formalizer specifies the query's reference frame, spatial constraints, objective, and evidence requirement. The evidence requirement determines the appropriate reasoning pathway. A query treated as requiring geometric evidence is handled by the Geometric Reasoner, which deterministically computes the answer using the metric BEV records in the query-specified reference frame. Otherwise, a query treated as requiring visual evidence is passed to the Visual Evidence Aligner, which uses BEV-indexed object records and their associated supporting views to retrieve object-consistent and scene-context RGB frames for VLM grounding. Within a standard 16-frame budget, BEV3SR consistently improves different VLM backbones across multiple 3D scene QA benchmarks, delivering large gains on spatially intensive questions and significant improvements on open-ended scene QA. These results demonstrate that BEV3SR offers an effective, efficient, and interpretable framework for explicit 3D spatial reasoning from video.
Figures & tables
Figure 1: Three limitations of the VLM baseline and how the proposed BEV3SR addresses them. The baseline uses Qwen3-VL-8B [ 4 ] prompted with 16 uniformly sampled RGB frames.
Figure 2: Overview of the BEV3SR framework.
VLM
Method
SQA3D (test)
ScanQA (val)
EM
EM-R
EM
B-4
M
R
C
Qwen3-VL-8B [ 4 ]
Baseline
51.6
55.0
27.8
15.9
15.8
41.9
82.6
BEV3SR
62.6
64.8
30.6
16.0
18.5
46.4
95.6
InternVL3.5-8B [ 33 ]
Baseline
52.9
55.2
28.2
14.2
17.4
44.5
90.4
BEV3SR
61.2
62.5
30.2
14.7
16.9
45.4
90.7
LLaVA-OneVision-1.5-8B [ 2 ]
Baseline
49.2
50.1
22.1
4.2
13.3
34.8
66.0
Table 1: Baseline vs. BEV3SR across different VLMs. We report EM (exact match), EM-R (refined exact match), BLEU-4 (B-4), METEOR (M), ROUGE (R), and CIDEr (C).
Model
Avg.
Num.
MC
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
2D MLLM
SpatialMLLM-4B [ 34 ]
49.3
65.3
34.8
63.1
45.1
41.3
46.2
VG-LLM-8B [ 48 ]
52.3
67.9
37.7
58.6
62.0
46.6
40.7
Open source
InternVL3.5-8B [ 33 ]
57.3
70.2
42.7
67.4
60.9
54.6
48.1
Table 3: Comparison on the static spatial subset of VSI-Bench.
Method
Backbone
ScanQA
SQA3D
SpatialMind [ 39 ]
Qwen2.5-VL-72B [ 5 ]
B-1 38.0
EM 39.2
CoV [ 46 ]
GPT-4.1 [ 1 ] / Gemini [ 11 ]
EM 31.9
EM 51.1
BEV3SR
Qwen3-VL-8B [ 4 ]
EM 30.6 / B-1 41.0
EM 62.6
Table 4: Contextual comparison with training-free methods.
Method
EM
Δ
EM-R
Δ
Baseline
43.2
–
44.0
–
+ BEV
41.7
−1.5
42.3
−1.7
+ GR
65.5
+22.3
65.9
+21.9
Table 5: Ablation studies on the SQA3D validation set: (a) effectiveness of GR on the geometry-solvable subset; (b) end-to-end contribution of GR and VEA on the full set.
Method
EM
B-1
B-2
B-3
B-4
M
R
C
Baseline
27.8
35.4
23.2
18.2
15.9
15.8
41.9
82.6
VEA
30.6
41.0
27.0
20.7
16.0
18.5
46.4
95.6
Δ
+2.8
+5.6
+3.8
+2.5
+0.1
+2.7
+4.5
+13.0
Table 6: Ablation study for effectiveness of Visual Evidence Aligner (VEA) on ScanQA validation set.
Table 8
Reference-frame Regime
#Q
EM
Good position, good heading ( ep≤1m , eθ≤30∘ )
547
76.97
Good position, bad heading ( ep≤1m , eθ>30∘ )
169
58.58
Bad position, good heading ( ep>1m , eθ≤30∘ )
142
65.49
Bad position, bad heading ( ep>1m , eθ>30∘ )
262
51.15
Table 9: QA accuracy under different reference-frame error regimes on SQA3D val.
3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.