While Multimodal Large Language Models (MLLMs) excel in semantic tasks, they frequently lack the "spatial sense" essential for sophisticated geometric reasoning. Current models typically suffer from exorbitant modality-alignment costs and deficiency in fine-grained structural modeling precision.We introduce SSR, a framework designed for Structured Scene Reasoning that seamlessly integrates 2D and 3D representations via a lightweight alignment mechanism. To minimize training overhead, our framework anchors 3D geometric features to the large language model's pre-aligned 2D visual semantics through cross-modal addition and token interleaving, effectively obviating the necessity for large-scale alignment pre-training. To underpin complex spatial reasoning, we propose a novel scene graph generation pipeline that represents global layouts as a chain of independent local triplets defined by relative coordinates. This is complemented by an incremental generation algorithm, enabling the model to construct "language-model-friendly" structural scaffolds for complex environments. Furthermore, we extend these capabilities to global-scale 3D global grounding task, achieving absolute metric precision across heterogeneous data sources. At a 7B parameter scale, SSR achieves state-of-the-art performance on multiple spatial intelligence benchmarks, notably scoring 73.9 on VSI-Bench. Our approach significantly outperforms much larger models, demonstrating that efficient feature alignment and structured scene reasoning are the cornerstones of authentic spatial intelligence.
Figures & tables
Figure 1: Comparison of model performance on VSI-Bench. SSR achieves the highest accuracy among all proprietary and open-source competitors. Notably, our 7B model outperforms significantly larger models, demonstrating superior parameter efficiency in spatial reasoning.
Figure 2: Architecture of SSR-3D: It adopts a dual-branch architecture to jointly leverage 2D visual and 3D spatial cues. The 3D branch encodes geometric scene structure through dedicated spatial tokens, while the 2D branch processes image-derived visual features extracted by a vision encoder. Tokens from both branches are then interleaved and fused as input to the LLM, enabling unified multimodal reasoning over appearance and geometry.
Figure 3: Global scene graph representation via LocalCogMap. Left : Global Scene Graph: Our proposed framework maintains global connectivity while redefining triplets as localized spatial units. Right : LocalCogMap Construction: Each triplet is modeled within a 10×10 grid established by two anchors. The target object is then normalized within this frame. This formulation ensures geometric consistency across the entire scene graph.
Figure 4: MultiQA-based scene graph generation. We transform global scene graphs into independent triplets. For each triplet, the LLM infers target coordinates relative to two anchors within a structured system context. Compared to dense captions, this decoupled QA format ensures scalability to complex scenes and reduces computational redundancy.
Figure 5: The visualization of 7-DoF coordinates generation algorithm. As illustrated in the generation pipeline, we define the origin of the global coordinate system as the camera position in the first frame, and align the positive direction of the X-axis with the projection of the optical axis onto the ground plane. The visualizations of the 7-DoF grounding results demonstrate that our proposed coordinate definition is both geometrically clear and highly adaptable across diverse scenarios and datasets.
Figure 6: Task capability distribution of the datasets. Training samples are hierarchically organized into 4 primary categories and 14 fine-grained subtasks, encompassing a comprehensive spectrum of spatial reasoning capabilities.
Figure 7: Taxonomy of training data used in the two training stages and representative QA pairs.
Category
Model
VSI-Bench [ 47 ]
VSI-Bench Debiased [ 49 ]
MindCube [ 51 ]
ViewSpatial [ 26 ]
SpaCE-10 [ 21 ]
VSTI-Bench [ 18 ]
Baseline
Human
79.2†
-
94.5†
-
91.2†
77.0†
Random Choice
34.0†
-
33.0†
26.3†
-
-
Chance Level (Frequency)
-
-
-
-
-
22.4†
Proprietary models
Seed-1.6 [ 22 ]
49.9
-
48.7
43.8
-
-
Grok-4 [ 45 ]
47.9
-
63.5
43.2
-
-
GPT-5 [ 38 ]
55.0
-
56.3
45.5
53.4
-
Table 1: Spatial and spatiotemporal intelligence performance on key benchmarks.
Category
Model
Rel. Dir.
Rel. Dist.
Appr. Order
Route Plan
Obj. Size
Obj. Count
Abs. Dist.
Roome Size
Overall
Baseline
Human [ 47 ]
95.8
94.7
100
95.8
60.4
94.3
47.0
45.9
79.2
Proprietary Models
GPT-5-2025-08-07 [ 38 ]
48.6
63.7
68.9
50.2
73.3
53.5
34.4
47.5
55.0
Open-Sourced General Models
LLaVA-OneVision-7B [ 25 ]
35.2
42.5
24.4
29.4
47.4
47.7
20.2
12.3
32.4
LLaVA-Video-7B [ 58 ]
42.4
43.5
30.6
34.0
47.8
48.5
14.0
24.2
35.6
Qwen3-VL-8B-Instruct [ 5 ]
50.9
58.0
66.3
35.0
76.3
53.3
47.0
61.9
57.9
InternVL3-8B [ 41 ]
39.3
48.0
31.3
26.2
43.6
66.0
34.8
47.5
42.1
Table 2: Comparison with state-of-the-art MLLMs on VSI-Bench. SSR achieves the best performance.
Figure 8: Experimental results. (Left) The 3D global grounding comparison results between SSR-2D and Qwen3-VL. (Right) The error histogram comparison between LocalCogMap and global CogMap introduced in VSI-Bench.
Method/Config
Rel. Dir.
Rel. Dist.
Appr. Order
Route Plan
Obj. Size
Obj. Count
Abs. Dist.
Roome Size
Overall
SSR-2D (Default)
87.9
69.2
81.9
48.5
76.5
71.8
63.4
76.5
71.9
(a) Token Insertion Method (SSR-3D)
Sequential
88.5
69.2
80.7
49.5
76.3
68.7
65.2
78.9
72.1
Interleaved (✓)
93.4
71.3
85.0
48.5
76.0
70.5
67.1
79.5
73.9
(b) Training Phases (SSR-2D)
w/ Stage 1 (✓)
87.9
69.2
81.9
48.5
76.5
71.8
63.4
76.5
71.9
Table 3: Ablation studies of SSR on VSI-Bench concerning model components and training data. The gray row (✓) represents our default/best configuration used across experiments.
Figure 9: The figure shows the accuracy of SSR with the increasing of training data used in the two stages respecitvely. Overall, we can see that the accuracy of SSR increases with the increasing of training data.
3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.
While Multimodal Large Language Models (MLLMs) have achieved remarkable success in 2D visual understanding, their ability to reason about 3D space remains limited. To address this gap, we introduce geometrically referenced 3D scene representations (GR3D). Given a set of input images, GR3D annotates objects in the images with unique IDs and encodes their 3D geometric attributes as textual references indexed by these IDs. This representation enables MLLMs to interpret 3D cues using their advanced language-based skills in mathematical reasoning, while concurrently analyzing 2D visual features in a tightly coupled way. We present a simple yet effective approach based on GR3D, which requires no additional training and is readily applicable to different MLLMs. Implemented in a zero-shot setting, our approach yields substantial improvements on challenging spatial reasoning benchmarks, boosting GPT-5 performance by 9% on VSI-Bench and 12% on MindCube. Qualitative studies further demonstrate that GR3D empowers MLLMs to perform complex spatial reasoning with highly sparse input views.
Multimodal large language models (MLLMs) have exhibited remarkable performance in various visual tasks, yet still struggle with spatial reasoning. Recent efforts mitigate this by injecting geometric features from 3D foundation models, but rely on static single-layer extractions. We identify that such an approach induces a task misalignment bias: the geometric features naturally evolve towards 3D pretraining objectives, which may contradict the heterogeneous spatial demands of MLLMs, rendering any single layer fundamentally insufficient. To resolve this, we propose GeoAlign, a novel framework that dynamically aggregates multi-layer geometric features to realign with the actual demands. GeoAlign constructs a hierarchical geometric feature bank and leverages the MLLM's original visual tokens as content-aware queries to perform layer-wise sparse routing, adaptively fetching the suitable geometric features for each patch. Extensive experiments on VSI-Bench, ScanQA, and SQA3D demonstrate that our compact 4B model effectively achieves state-of-the-art performance, even outperforming larger existing MLLMs.
Zhaochen Liu, Limeng Qiao, Guanglu Wan +1
National Engineering Research Center of Visual Technology, National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · Meituan Inc. · National Biomedical Imaging Center, Peking University
3D spatial reasoning underpins understanding and acting in the physical world, yet it remains unreliable in current multimodal large language models (MLLMs). These models falter at precise geometric measurement, at transforming between egocentric and allocentric viewpoints, and at grounding fine-grained appearance. The most common remedies fine-tune the model on large-scale curated spatial-reasoning datasets or attach dedicated encoders for 3D geometry, which typically couples the solution to costly supervision and a specific backbone. We instead introduce GraFT, a training-free framework that supplies the missing 3D structure through a compact, easily maintained 3D scene graph (3DSG). GraFT supplies the graph's 3D structure to the frozen model in three forms: as values computed by symbolic tools, as a bird's-eye-view (BEV) the model reads as an image, and as egocentric frames the geometry selects. On ScanQA, GraFT improves every metric over the same-backbone baseline, raising CIDEr by 27%. On VSI-Bench, GraFT improves frozen MLLMs by up to 65%, surpassing every proprietary and general-purpose open-source baseline, and several prominent fine-tuned spatial models.