Video spatial reasoning requires multimodal large language models (MLLMs) to understand how objects are arranged and related across viewpoints. While these models can recognize objects in individual frames, integrating these observations to solve complex spatial tasks remains challenging. We examine route planning as a representative task and observe that existing models become substantially less accurate as the number of required turns increases, with several models achieving accuracy close to random guessing on routes requiring three or more turns. This finding suggests that recognizing landmarks alone is insufficient: models must also connect observations across viewpoints and track how landmarks are arranged in a shared space. Motivated by this observation, we introduce GeoLift, a training-free framework that uses feed-forward 3D reconstruction to map objects detected in individual frames into a shared metric 3D scene. GeoLift combines a frozen MLLM for 2D grounding with pretrained geometry models for scene reconstruction. It then uses shared geometric regions, called superpoints, to associate observations of the same object across views without a learned matcher. The reconstructed 3D map enables explicit computation of spatial relations and multistep routes, and provides geometric evidence for estimating distances and sizes. With all model weights frozen, GeoLift improves Qwen3-VL-4B's average score on VSI-Bench from 55.4% to 64.1%, including an increase in route-planning accuracy from 30.9% to 53.6%, and achieves 59.8% accuracy on MindCube. These results show that connecting frame-level object grounding through a shared 3D map can improve complex spatial reasoning without task-specific training.
Figures & tables
Figure 1: From local 2D observations to explicit 3D spatial reasoning. Given the same RGB video and spatial query, (a) Video MLLM grounds objects in individual frames, but each observation remains tied to its local coordinate frame. Without a shared reference, the cross-view relation between the sofa and chair is ambiguous. (b) GeoLift lifts the grounded objects into a shared spatial reference and explicitly computes the relation. For this query, the sofa-to-table direction defines the facing orientation, which places the chair on the right.
Figure 2: Route-planning complexity as the number of turns increases. The green path connects the start and goal through an ordered sequence of intermediate landmarks. A higher turn count requires more direction changes and stronger consistency across observations from different viewpoints, making the underlying global spatial arrangement increasingly important.
Model
Protocol
1 turn
2 turns
≥ 3 turns
3 choices
4 choices
4 choices
Qwen3-VL-4B
frozen
43.4 ( +10.1 )
22.5 ( −2.5 )
23.7 ( −1.3 )
GPT-5.6-Sol
frozen
48.7 ( +15.4 )
36.3 ( +11.3 )
28.9 ( +3.9 )
EgoMind 7B
learned
34.2 ( +0.9 )
28.8 ( +3.8 )
28.9 ( +3.9 )
Cambrian-S 7B (32f)
learned
44.7 ( +11.4 )
41.3 ( +16.3 )
26.3 ( +1.3 )
Cambrian-S 7B (64f)
learned
53.9 ( +20.6 )
35.0 ( +10.0 )
31.6 ( +6.6 )
Table 1: Longer routes expose the need for a persistent spatial state. Route-planning accuracy is grouped by the number of required turns. Values are accuracies (%), with differences from the corresponding random-guessing baseline shown in parentheses in percentage points. Red entries are below the baseline, while orange entries are at most five percentage points above it.
Figure 3: GeoLift framework. A frozen Video MLLM parses the query and grounds query-relevant entities in the RGB video. VGGT-Omega and MoGe-2 recover global geometry and metric scale, allowing the 2D groundings to be lifted into a shared 3D reference. Geometric superpoints and superpoint agreement associate observations of the same instance across views without a learned matcher. The resulting view-consistent map supports a toolbox of deterministic spatial operations, including occupancy-grid construction and A* route search. Geometry can also be provided as a hint together with the video for queries that require visual interpretation.
Methods
Overall
Numerical Questions
Multiple-Choice Questions
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
General-Purpose MLLMs
GPT-4o † ( Yang et al., 2025 )
34.0
46.2
5.3
43.8
38.2
37.0
41.3
31.5
28.5
Gemini-1.5 Flash † ( Yang et al., 2025 )
42.1
49.8
30.8
53.5
54.4
37.7
41.0
31.5
37.8
Gemini-1.5 Pro † ( Yang et al., 2025 )
45.4
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
InternVL2-8B † ( Yang et al., 2025 )
37.5
31.3
29.0
48.9
44.2
38.0
33.4
28.9
46.4
Table 2: Comparative results on full VSI-Bench ( Yang et al., 2025 ) . The numerical and multiple-choice columns report MRA and exact-match accuracy, respectively. Training-free means no benchmark-specific updates; † marks quoted results. Direct Qwen3-VL-4B and GeoLift use the same frozen backbone and matched evaluation protocol. All result values are percentages rounded to one decimal place.
Method
Overall (micro)
Rotation
Among
Around
Baseline
Chance
32.4
36.4
32.3
30.7
Open-Weight Multi-Image Models
LLaVA-OneVision-7B
47.4
36.5
48.4
44.1
Qwen2.5-VL-7B-Instruct
29.3
38.8
29.5
21.4
Gemma-3-12B-it
46.7
38.4
48.4
34.6
Table 3: Results on the full MindCube benchmark ( Wang et al., 2025 ) . We report accuracy for the Rotation, Among, and Around settings. The Around setting includes the AroundNew subset. The overall score is the micro-averaged accuracy over all 21,154 questions. GeoLift is training-free: the Qwen3-VL-4B variant uses Qwen3-VL-4B for grounding and answering. Bold numbers indicate the best value in each column. All values are percentages rounded to one decimal place.
VGGT+MoGe
SP Association
Overall
Numerical Questions
Multiple-Choice Questions
Obj. Cnt.
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan.
Appr. Order
×
×
55.4
65.9
43.1
73.9
61.0
53.1
46.1
30.9
69.1
✓
×
62.8
68.0
48.4
74.4
65.5
57.8
72.1
46.9
69.1
✓
✓
64.1
68.0
49.4
74.4
65.9
57.8
74.4
53.6
69.1
Table 4: Ablation of metric geometry and superpoint association on VSI-Bench. All rows use the frozen Qwen3-VL-4B Video MLLM. A checkmark indicates that the component is enabled, while a cross indicates that it is disabled. The first row is direct Qwen3-VL-4B inference without VGGT-Omega, MoGe-2, or superpoint association. Geometric variants use the fixed 64-frame protocol. Scores are percentages.
Precise spatial understanding from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs), as their visual representations are predominantly semantic and lack explicit geometric grounding. While existing approaches augment visual tokens with geometric cues from visual geometry models, their MLLM is still required to implicitly infer the underlying 3D structure of the scene from these augmented tokens, limiting its spatial reasoning capability. To address this issue, we introduce Cog3DMap, a framework that recurrently constructs an explicit 3D memory from multi-view images, where each token is grounded in 3D space and possesses both semantic and geometric information. By feeding these tokens into the MLLM, our framework enables direct reasoning over a spatially structured 3D map, achieving state-of-the-art performance on various spatial reasoning benchmarks. Code will be made publicly available.
Multimodal Large Language Models (MLLMs) excel at 2D semantic understanding but lack intrinsic 3D awareness, resulting in representations that fail to maintain geometric and spatial consistency across video frames. Given the scarcity of large-scale 3D data, we present GeoVR, a novel framework that learns geometric representations using purely 2D video sequences. This approach effectively restructures the semantic latent space within MLLMs to unlock spatial intelligence. Rather than employing superficial feature mixing, GeoVR reshapes the internal representations of the MLLM by distilling geometry knowledge from pre-trained 3D foundation models. This is accomplished through a multi-objective learning strategy driven by four complementary geometric targets: (1) estimating inter-frame camera poses to embed varying viewpoint dynamics, (2) regressing dense depth maps to anchor physical distances, (3) predicting a metric scale factor for real-world calibration, and (4) distilling multi-scale 3D features to align the intermediate feature space. Guided by these explicit physical and geometric constraints, the model's internal representations naturally develop strong 3D awareness. Extensive experiments on spatial reasoning benchmarks demonstrate that GeoVR achieves state-of-the-art performance, establishing a new paradigm for endowing foundation models with spatial intelligence.
Multimodal Large Language Models (MLLMs) have made impressive progress in connecting vision and language, but they still struggle with spatial understanding and viewpoint-aware reasoning. Recent efforts aim to augment the input representations with geometric cues rather than explicitly teaching models to reason in 3D space. We introduce Loc3R-VLM, a framework that equips 2D Vision-Language Models with advanced 3D understanding capabilities from monocular video input. Inspired by human spatial cognition, Loc3R-VLM relies on two joint objectives: global layout reconstruction to build a holistic representation of the scene structure, and explicit situation modeling to anchor egocentric perspective. These objectives provide direct spatial supervision that grounds both perception and language in a 3D context. To ensure geometric consistency and metric-scale alignment, we leverage lightweight camera pose priors extracted from a pre-trained 3D foundation model. Loc3R-VLM achieves state-of-the-art performance in language-based localization and outperforms existing 2D- and video-based approaches on situated and general 3D question-answering benchmarks, demonstrating that our spatial supervision framework enables strong 3D understanding. Project page: https://kevinqu7.github.io/loc3r-vlm