SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

Soohyun Ryu; Sohee Kim; Eunho Yang

cs.CVSep 7, 2026

SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

Authors: Soohyun Ryu, Sohee Kim, Eunho Yang

Organizations: KAIST South Korea · KAIST & AITRICS South Korea

Abstract

Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks, yet their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images -- referred to as spatial intelligence -- remains limited. Existing approaches attempt to address this gap by using real-scene spatial question answering datasets that require dense geometric annotations. However, constructing such labels is costly, time-consuming, and often noisy due to reliance on external perception modules. In this work, we propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks. We introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset further incorporates controlled color modulation as visual cues to encourage anchor-based reasoning in visually complex conditions. Experiments demonstrate that LVLMs trained on our dataset through either direct answering or reasoning-based prediction significantly outperform baselines and generalize to real-world spatial tasks, despite the dataset's synthetic and compact nature. Code and data are available at https://github.com/rsoohyun/SpatialBlock.

Figures & tables

Figure 1: Performance Gap Between Humans and LVLMs in Block-Stacking Problem . The top row shows a block-stacking task that requires predicting a 2D projection from a given 3D structure. While humans achieve near-perfect accuracy, state-of-the-art LVLMs struggle even with basic spatial reasoning problems. Leading proprietary models exhibit flawed spatial logic, often producing incorrect interpretations of the structure.

Figure 2: Spatial Abilities Underlying Block-Stacking Reasoning . Solving block-stacking tasks involves (a) inferring latent 3D composition, (b) reasoning under transformation, and (c) integrating multiple components into a coherent structure.

Figure 3: SpatialBlock-15k . (a) Three block-stacking tasks: 3D-to-2D projection (Q1), viewpoint transformation (Q2), and structural combination (Q3). (b) Visual cue-based extension with controlled color variations, where colors encode depth ordering (Q1), preserved structural roles across transformations (Q2), and correspondence cues for integration (Q3).

Model	# Data	In-domain	Out-of-domain
Model	# Data	SB-Bench	MindCube	MMSI-Bench	SPBench	MMMU	Overall
Proprietary Models
GPT-5 ( Singh et al., 2025 )	-	43.8	56.7	42.8	43.0	77.6	55.0
Gemini-2.5-flash ( Comanici et al., 2025 )	-	36.4	51.0	32.2	44.3	58.8	46.6
Claude-Sonnet-4.5 ( Anthropic, 2025 )	-	46.3	34.0	32.7	43.9	69.2	43.9
Open-Source Models

Table 1: Main Results . For each dataset, bold and underlined indicate the best and second-best performance, respectively. * represents in-domain results where the model was trained on the benchmark. We re-evaluated all previous works for fair comparison.

(a) Ablation on question types
Model	# Data	In-domain	Out-of-domain
Model	# Data	SB-Bench	MindCube	MMSI-Bench	SPBench	MMMU	Overall
Qwen2.5-VL-3B-Instruct ( Bai et al., 2025b )	-	29.9	39.1	26.1	40.0	45.6	37.7
SpatialBlock-3B-direct (Ours)	15K	94.8	49.1	28.1	40.4	46.4	41.0
SpatialBlock-3B-reason (Ours)	15K	90.2	45.3	29.2	43.7	46.7	41.2
SpatialBlock-3B-reason (Q1 only)	15K	48.0	41.8	27.6	42.8	43.2	38.8

Table 2: Ablation Studies . * denotes in-domain results from benchmark-trained models.

Figure 4: Synthetic-Real Dataset . Widely used spatial question types, (a) relative direction and (b) relative distance, are adapted into a block-based environment while preserving their semantic structure.

	In-domain		Out-of-domain
Train Dataset	Relative Direction	Relative Distance	MindCube	MMSI-Bench	MMMU
Baseline	31.4	74.1	39.1	26.1	45.6
Synthetic-Real	43.4	76.9	32.8	25.5	43.6
SpatialBlock	31.4	67.4	49.1	28.1	46.4

Table 3: Comparison of Training on SpatialBlock and Synthetic-Real . While Synthetic-Real improves in-domain performance, training on SpatialBlock leads to better generalization on out-of-domain benchmark datasets. All experiments are conducted on Qwen2.5-VL-3B ( Bai et al., 2025b ) , and in-domain performance is evaluated on SPBench.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Instructions for Reinforcement Learning . The model is instructed to provide a step-by-step thinking process, and then the final answer between answer tags.

Figure 6: Instructions for Reasoning Quality Evaluation . We evaluate the quality of model-generated reasoning by measuring its alignment with each step of the corresponding ground-truth rationale.

Model	Absolute distance	Object size	Object counting	Average
Qwen2.5-VL-3B-Instruct	30.9	17.6	35.1	27.9
SpatialBlock-3B-direct	23.2	19.9	72.1	38.4
SpatialBlock-3B-reason	21.4	38.3	43.3	34.3

Appendix

Table 4: Results on Numerical Tasks . Results on tasks from SPBench.

Figure 7: Performance Across Viewing Angles . (a) The same block structures are rendered from different viewing angles. (b) Performance of models across viewing angles; the bars correspond to results under $45^{\circ}$ , $30^{\circ}$ , and $60^{\circ}$ views, respectively.

Figure 8: Generated Cold-Start Data Sample . The reasoning trace extracted from Qwen2.5-VL-7B includes incorrect information about the given 3D structure, even though the model answers correctly.

Figure 9: Qualitative Comparison of Reasoning Traces . (a) Baseline only describes a single view. (b) Training without visual cues attempts multi-image reasoning but fails to identify the shared object. (c) Our model uses the “yellow container” as an anchor to integrate views and reach the correct answer.

Figure 10: Additional Qualitative Result on Reasoning Traces . The figure shows additional qualitative results on reasoning traces of the baseline model, our model trained without visual cues, and our final model, SpatialBlock-3B-reason.

Visual Spatial Reasoning

Visual spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and physical change from images and video.

See all 253 papers

HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models
Mar 26, 2026Huizhi Liang, Yichao Shen, Yu Deng +53D Spatial Reasoning 3D VLMs
Tsinghua University · Microsoft Research Asia · Xi’an Jiaotong University +1
SpatialForge: Bootstrapping 3D-Aware Spatial Reasoning from Open-World 2D Images
May 12, 2026Zishan Liu, Ruoxi Zang, Yanglin Zhang +5Geospatial QA Visual Spatial Reasoning
1Lingnan University · 2XPENG Robotics
SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Aug 3, 2026Jing Wu, Jianhua Wu, Jiayi Guan +5Vision-Language Models Visual Spatial Reasoning
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher

Mar 26, 2026J/K move · Enter open · S save

HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models

Huizhi Liang, Yichao Shen, Yu Deng, Sicheng Xu+4

Tsinghua University · Microsoft Research Asia · Xi’an Jiaotong University+1

3D Spatial Reasoning 3D VLMs 3D VQA

Open paper AskPDF arXiv

Achieving human-like spatial intelligence for vision-language models (VLMs) requires inferring 3D structures from 2D observations, recognizing object properties and relations in 3D space, and performing high-level spatial reasoning. In this paper, we propose a principled hierarchical framework that decomposes the learning of 3D spatial understanding in VLMs into four progressively complex levels, from geometric perception to abstract spatial reasoning. Guided by this framework, we construct an automated pipeline that processes approximately 5M images with over 45M objects to generate 3D spatial VQA pairs across diverse tasks and scenes for VLM supervised fine-tuning. We also develop an RGB-D VLM incorporating metric-scale point maps as auxiliary inputs to further enhance spatial understanding. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on multiple spatial understanding and reasoning benchmarks, surpassing specialized spatial models and large proprietary systems such as Gemini-2.5-pro and GPT-5. Moreover, our analysis reveals clear dependencies among hierarchical task levels, offering new insights into how multi-level task design facilitates the emergence of 3D spatial intelligence.

Abstract

Visual Spatial Reasoning

Explore similar work

HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models

SpatialForge: Bootstrapping 3D-Aware Spatial Reasoning from Open-World 2D Images

SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models