Video spatial reasoning requires multimodal large language models (MLLMs) to understand how objects are arranged and related across viewpoints. While these models can recognize objects in individual frames, integrating these observations to solve complex spatial tasks remains challenging. We examine route planning as a representative task and observe that existing models become substantially less accurate as the number of required turns increases, with several models achieving accuracy close to random guessing on routes requiring three or more turns. This finding suggests that recognizing landmarks alone is insufficient: models must also connect observations across viewpoints and track how landmarks are arranged in a shared space. Motivated by this observation, we introduce GeoLift, a training-free framework that uses feed-forward 3D reconstruction to map objects detected in individual frames into a shared metric 3D scene. GeoLift combines a frozen MLLM for 2D grounding with pretrained geometry models for scene reconstruction. It then uses shared geometric regions, called superpoints, to associate observations of the same object across views without a learned matcher. The reconstructed 3D map enables explicit computation of spatial relations and multistep routes, and provides geometric evidence for estimating distances and sizes. With all model weights frozen, GeoLift improves Qwen3-VL-4B's average score on VSI-Bench from 55.4% to 64.1%, including an increase in route-planning accuracy from 30.9% to 53.6%, and achieves 59.8% accuracy on MindCube. These results show that connecting frame-level object grounding through a shared 3D map can improve complex spatial reasoning without task-specific training.
Figures & tables
Figure 1: From local 2D observations to explicit 3D spatial reasoning. Given the same RGB video and spatial query, (a) Video MLLM grounds objects in individual frames, but each observation remains tied to its local coordinate frame. Without a shared reference, the cross-view relation between the sofa and chair is ambiguous. (b) GeoLift lifts the grounded objects into a shared spatial reference and explicitly computes the relation. For this query, the sofa-to-table direction defines the facing orientation, which places the chair on the right.
Figure 2: Route-planning complexity as the number of turns increases. The green path connects the start and goal through an ordered sequence of intermediate landmarks. A higher turn count requires more direction changes and stronger consistency across observations from different viewpoints, making the underlying global spatial arrangement increasingly important.
Model
Protocol
1 turn
2 turns
≥ 3 turns
3 choices
4 choices
4 choices
Qwen3-VL-4B
frozen
43.4 ( +10.1 )
22.5 ( −2.5 )
23.7 ( −1.3 )
GPT-5.6-Sol
frozen
48.7 ( +15.4 )
36.3 ( +11.3 )
28.9 ( +3.9 )
EgoMind 7B
learned
34.2 ( +0.9 )
28.8 ( +3.8 )
28.9 ( +3.9 )
Cambrian-S 7B (32f)
learned
44.7 ( +11.4 )
41.3 ( +16.3 )
26.3 ( +1.3 )
Cambrian-S 7B (64f)
learned
53.9 ( +20.6 )
35.0 ( +10.0 )
31.6 ( +6.6 )
Table 1: Longer routes expose the need for a persistent spatial state. Route-planning accuracy is grouped by the number of required turns. Values are accuracies (%), with differences from the corresponding random-guessing baseline shown in parentheses in percentage points. Red entries are below the baseline, while orange entries are at most five percentage points above it.
Figure 3: GeoLift framework. A frozen Video MLLM parses the query and grounds query-relevant entities in the RGB video. VGGT-Omega and MoGe-2 recover global geometry and metric scale, allowing the 2D groundings to be lifted into a shared 3D reference. Geometric superpoints and superpoint agreement associate observations of the same instance across views without a learned matcher. The resulting view-consistent map supports a toolbox of deterministic spatial operations, including occupancy-grid construction and A* route search. Geometry can also be provided as a hint together with the video for queries that require visual interpretation.
Methods
Overall
Numerical Questions
Multiple-Choice Questions
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
General-Purpose MLLMs
GPT-4o † ( Yang et al., 2025 )
34.0
46.2
5.3
43.8
38.2
37.0
41.3
31.5
28.5
Gemini-1.5 Flash † ( Yang et al., 2025 )
42.1
49.8
30.8
53.5
54.4
37.7
41.0
31.5
37.8
Gemini-1.5 Pro † ( Yang et al., 2025 )
45.4
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
InternVL2-8B † ( Yang et al., 2025 )
37.5
31.3
29.0
48.9
44.2
38.0
33.4
28.9
46.4
Table 2: Comparative results on full VSI-Bench ( Yang et al., 2025 ) . The numerical and multiple-choice columns report MRA and exact-match accuracy, respectively. Training-free means no benchmark-specific updates; † marks quoted results. Direct Qwen3-VL-4B and GeoLift use the same frozen backbone and matched evaluation protocol. All result values are percentages rounded to one decimal place.
Method
Overall (micro)
Rotation
Among
Around
Baseline
Chance
32.4
36.4
32.3
30.7
Open-Weight Multi-Image Models
LLaVA-OneVision-7B
47.4
36.5
48.4
44.1
Qwen2.5-VL-7B-Instruct
29.3
38.8
29.5
21.4
Gemma-3-12B-it
46.7
38.4
48.4
34.6
Table 3: Results on the full MindCube benchmark ( Wang et al., 2025 ) . We report accuracy for the Rotation, Among, and Around settings. The Around setting includes the AroundNew subset. The overall score is the micro-averaged accuracy over all 21,154 questions. GeoLift is training-free: the Qwen3-VL-4B variant uses Qwen3-VL-4B for grounding and answering. Bold numbers indicate the best value in each column. All values are percentages rounded to one decimal place.
VGGT+MoGe
SP Association
Overall
Numerical Questions
Multiple-Choice Questions
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan.
Appr. Order
×
×
55.4
65.9
43.1
73.9
61.0
53.1
46.1
30.9
69.1
✓
×
62.8
68.0
48.4
74.4
65.5
57.8
72.1
46.9
69.1
✓
✓
64.1
68.0
49.4
74.4
65.9
57.8
74.4
53.6
69.1
Table 4: Ablation of metric geometry and superpoint association on VSI-Bench. All rows use the frozen Qwen3-VL-4B Video MLLM. A checkmark indicates that the component is enabled, while a cross indicates that it is disabled. The first row is direct Qwen3-VL-4B inference without VGGT-Omega, MoGe-2, or superpoint association. Geometric variants use the fixed 64-frame protocol. Scores are percentages.