SSR: Pushing the Limit of Spatial Intelligence with Structured Scene Reasoning

Yi Zhang; Youya Xia; Yong Wang; Meng Song; Xin Wu; Wenjun Wan; Bingbing Liu; AiXue Ye

cs.CVFeb 28, 2026

SSR: Pushing the Limit of Spatial Intelligence with Structured Scene Reasoning

Authors: Yi Zhang, Youya Xia, Yong Wang, Meng Song, Xin Wu, Wenjun Wan, Bingbing Liu, AiXue Ye,

Organizations: Foundation Model Department, Huawei · Central Media Technology Institute, Huawei

Abstract

While Multimodal Large Language Models (MLLMs) excel in semantic tasks, they frequently lack the "spatial sense" essential for sophisticated geometric reasoning. Current models typically suffer from exorbitant modality-alignment costs and deficiency in fine-grained structural modeling precision.We introduce SSR, a framework designed for Structured Scene Reasoning that seamlessly integrates 2D and 3D representations via a lightweight alignment mechanism. To minimize training overhead, our framework anchors 3D geometric features to the large language model's pre-aligned 2D visual semantics through cross-modal addition and token interleaving, effectively obviating the necessity for large-scale alignment pre-training. To underpin complex spatial reasoning, we propose a novel scene graph generation pipeline that represents global layouts as a chain of independent local triplets defined by relative coordinates. This is complemented by an incremental generation algorithm, enabling the model to construct "language-model-friendly" structural scaffolds for complex environments. Furthermore, we extend these capabilities to global-scale 3D global grounding task, achieving absolute metric precision across heterogeneous data sources. At a 7B parameter scale, SSR achieves state-of-the-art performance on multiple spatial intelligence benchmarks, notably scoring 73.9 on VSI-Bench. Our approach significantly outperforms much larger models, demonstrating that efficient feature alignment and structured scene reasoning are the cornerstones of authentic spatial intelligence.

Figures & tables

Figure 1: Comparison of model performance on VSI-Bench. SSR achieves the highest accuracy among all proprietary and open-source competitors. Notably, our 7B model outperforms significantly larger models, demonstrating superior parameter efficiency in spatial reasoning.

Figure 2: Architecture of SSR-3D: It adopts a dual-branch architecture to jointly leverage 2D visual and 3D spatial cues. The 3D branch encodes geometric scene structure through dedicated spatial tokens, while the 2D branch processes image-derived visual features extracted by a vision encoder. Tokens from both branches are then interleaved and fused as input to the LLM, enabling unified multimodal reasoning over appearance and geometry.

Panel of Figure 3 — Figure 3: Global scene graph representation via LocalCogMap. Left : Global Scene Graph: Our proposed framework maintains global connectivity while redefining triplets as localized spatial units. Right : LocalCogMap Construction: Each triplet is modeled within a $10\times 10$ grid established by two anchors. The target object is then normalized within this frame. This formulation ensures geometric consistency across the entire scene graph.

Figure 4: MultiQA-based scene graph generation. We transform global scene graphs into independent triplets. For each triplet, the LLM infers target coordinates relative to two anchors within a structured system context. Compared to dense captions, this decoupled QA format ensures scalability to complex scenes and reduces computational redundancy.

Figure 5: The visualization of 7-DoF coordinates generation algorithm. As illustrated in the generation pipeline, we define the origin of the global coordinate system as the camera position in the first frame, and align the positive direction of the X-axis with the projection of the optical axis onto the ground plane. The visualizations of the 7-DoF grounding results demonstrate that our proposed coordinate definition is both geometrically clear and highly adaptable across diverse scenarios and datasets.

Panel of Figure 6 — Figure 6: Task capability distribution of the datasets. Training samples are hierarchically organized into 4 primary categories and 14 fine-grained subtasks, encompassing a comprehensive spectrum of spatial reasoning capabilities.

Panel of Figure 7 — Figure 7: Taxonomy of training data used in the two training stages and representative QA pairs.

Category	Model	VSI-Bench [ 47 ]	VSI-Bench ${}^{\textbf{Debiased}}$ [ 49 ]	MindCube [ 51 ]	ViewSpatial [ 26 ]	SpaCE-10 [ 21 ]	VSTI-Bench [ 18 ]
Baseline	Human	79.2†	-	94.5†	-	91.2†	77.0†
	Random Choice	34.0†	-	33.0†	26.3†	-	-
	Chance Level (Frequency)	-	-	-	-	-	22.4†
Proprietary models	Seed-1.6 [ 22 ]	49.9	-	48.7	43.8	-	-
	Grok-4 [ 45 ]	47.9	-	63.5	43.2	-	-
	GPT-5 [ 38 ]	55.0	-	56.3	45.5	53.4	-

Table 1: Spatial and spatiotemporal intelligence performance on key benchmarks.

Category	Model	Rel. Dir.	Rel. Dist.	Appr. Order	Route Plan	Obj. Size	Obj. Count	Abs. Dist.	Roome Size	Overall
Baseline	Human [ 47 ]	95.8	94.7	100	95.8	60.4	94.3	47.0	45.9	79.2
Proprietary Models	GPT-5-2025-08-07 [ 38 ]	48.6	63.7	68.9	50.2	73.3	53.5	34.4	47.5	55.0
Open-Sourced General Models	LLaVA-OneVision-7B [ 25 ]	35.2	42.5	24.4	29.4	47.4	47.7	20.2	12.3	32.4
	LLaVA-Video-7B [ 58 ]	42.4	43.5	30.6	34.0	47.8	48.5	14.0	24.2	35.6
	Qwen3-VL-8B-Instruct [ 5 ]	50.9	58.0	66.3	35.0	76.3	53.3	47.0	61.9	57.9
	InternVL3-8B [ 41 ]	39.3	48.0	31.3	26.2	43.6	66.0	34.8	47.5	42.1

Table 2: Comparison with state-of-the-art MLLMs on VSI-Bench. SSR achieves the best performance.

Panel of Figure 8 — Figure 8: Experimental results. (Left) The 3D global grounding comparison results between SSR-2D and Qwen3-VL. (Right) The error histogram comparison between LocalCogMap and global CogMap introduced in VSI-Bench.

Method/Config	Rel. Dir.	Rel. Dist.	Appr. Order	Route Plan	Obj. Size	Obj. Count	Abs. Dist.	Roome Size	Overall
SSR-2D (Default)	87.9	69.2	81.9	48.5	76.5	71.8	63.4	76.5	71.9
(a) Token Insertion Method (SSR-3D)
Sequential	88.5	69.2	80.7	49.5	76.3	68.7	65.2	78.9	72.1
Interleaved (✓)	93.4	71.3	85.0	48.5	76.0	70.5	67.1	79.5	73.9
(b) Training Phases (SSR-2D)
w/ Stage 1 (✓)	87.9	69.2	81.9	48.5	76.5	71.8	63.4	76.5	71.9

Table 3: Ablation studies of SSR on VSI-Bench concerning model components and training data. The gray row (✓) represents our default/best configuration used across experiments.

Figure 9: The figure shows the accuracy of SSR with the increasing of training data used in the two stages respecitvely. Overall, we can see that the accuracy of SSR increases with the increasing of training data.

3D Spatial Reasoning

3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.

See all 201 papers

Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations
Mar 9, 2026Jiangye Yuan, Gowri Kumar, Baoyuan Wang3D Spatial Reasoning Multimodal Large Language Models
Zillow Group
GeoAlign: Geometric Feature Realignment for MLLM Spatial Reasoning
Apr 14, 2026Zhaochen Liu, Limeng Qiao, Guanglu Wan +13D Spatial Reasoning 3D VLMs
National Engineering Research Center of Visual Technology, National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · Meituan Inc. · National Biomedical Imaging Center, Peking University
GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
Sep 3, 2026Junqing Du, Fernando Ropero, Erkin Turkoz +23D Spatial Reasoning Multimodal Large Language Models
Riemann Lab, Huawei Technologies

Mar 9, 2026J/K move · Enter open · S save

Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations

Jiangye Yuan, Gowri Kumar, Baoyuan Wang

Zillow Group

3D Spatial Reasoning Multimodal Large Language Models 3D Scene Representation

Open paper AskPDF arXiv

While Multimodal Large Language Models (MLLMs) have achieved remarkable success in 2D visual understanding, their ability to reason about 3D space remains limited. To address this gap, we introduce geometrically referenced 3D scene representations (GR3D). Given a set of input images, GR3D annotates objects in the images with unique IDs and encodes their 3D geometric attributes as textual references indexed by these IDs. This representation enables MLLMs to interpret 3D cues using their advanced language-based skills in mathematical reasoning, while concurrently analyzing 2D visual features in a tightly coupled way. We present a simple yet effective approach based on GR3D, which requires no additional training and is readily applicable to different MLLMs. Implemented in a zero-shot setting, our approach yields substantial improvements on challenging spatial reasoning benchmarks, boosting GPT-5 performance by 9% on VSI-Bench and 12% on MindCube. Qualitative studies further demonstrate that GR3D empowers MLLMs to perform complex spatial reasoning with highly sparse input views.

Abstract

3D Spatial Reasoning

Explore similar work

Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representations

GeoAlign: Geometric Feature Realignment for MLLM Spatial Reasoning

GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs