Organizations: Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · School of Software Engineering, Chongqing University, Chongqing, China
Current Large Language Models have achieved Olympiad-level logic, yet Vision-Language Models paradoxically falter on elementary spatial tasks like block counting. This capability mismatch reveals a critical spatial intelligence gap,'' where models fail to construct coherent 3D mental representations from 2D observations. We uncover this gap via diagnostic analyses showing the bottleneck is a missing view-consistent spatial interface rather than insufficient visual features or weak reasoning. To bridge this, we introduce **3ViewSense**, a framework that grounds spatial reasoning in Orthographic Views. Drawing on engineering cognition, we propose a Simulate-and-Reason'' mechanism that decomposes complex scenes into canonical orthographic projections to resolve geometric ambiguities. By aligning egocentric perceptions with these allocentric references, our method facilitates explicit mental rotation and reconstruction. Empirical results on spatial reasoning benchmarks demonstrate that our method significantly outperforms existing baselines, with consistent gains on occlusion-heavy counting and view-consistent spatial reasoning. The framework also improves the stability and consistency of spatial descriptions, offering a scalable path toward stronger spatial intelligence in multimodal systems.~\footnote{https://github.com/Jasaxion/3ViewSense}
Figures & tables
Figure 1: Motivation for explicit three-view reasoning. Providing explicit orthographic three-view descriptions (front/left/top) improves block-counting performance under occlusion, highlighting the role of view-consistent spatial representations.
Figure 2: The construction pipeline of our OrthoMind-3D dataset. To bridge the gap between visual perception and mental spatial reasoning, we curate data from two distinct domains. For In-Domain data, we utilize programmatic synthesis with strict geometric constraints to train the model’s orthographic projection capabilities. For Out-of-Domain data, we employ sandbox game engines and generative AI techniques to evaluate the model’s robustness and generalization in unstructured environments.
Figure 3: The training framework of 3ViewSense . Stage I learns to induce canonical front, left, and top orthographic views from an egocentric input. Stage II performs view-grounded reasoning by integrating the inferred views to generate reasoning traces and final answers, while reinforcement learning further internalizes this view-based reasoning capability.
Block Counting
Object Reasoning
Model
Block Count
Block Count (Attr.)
Object Count
Object Position
Object Count (Attr.)
Object Position (Attr.)
Proprietary Models
GPT-4o
15.8
53.2
68.3
39.3
71.2
47.2
Gemini-2.0-Flash
18.2
54.0
69.7
62.7
86.4
62.6
GPT-5
15.8
50.7
64.0
60.3
77.6
66.0
Gemini-3-pro
13.8
80.2
83.3
71.6
93.2
93.6
Table 1: Main results on OrthoMind-3D. We report accuracy (%) for block counting and object reasoning, covering both cardinality queries and attribute-conditioned (attr.) queries, as defined in Section 4.2 . “–” denotes entries where the model cannot produce outputs in the required format for evaluation.
Model
(A) OrthoMind-3D (OOD Task)
(B) Other Spatial Benchmarks
Block Count
Block Count (Attr.)
Object Count
Object Position
MindCube-Tiny
CV-Bench (2D)
OmniSpatial
SPBench (SI)
ViewSpatial
SpatialLadder-3B
19.6
43.0
22.2
15.6
38.5
68.6
67.6
30.8
36.2
SpaceOm-4B
23.4
61.7
46.2
19.2
35.5
68.8
67.6
29.7
34.8
SpaceQwen2.5-VL-3B-Instruct
21.2
62.5
51.8
16.5
36.4
66.1
58.8
28.1
30.1
InternVL3.5-4B
33.1
65.9
54.6
34.8
14.4
79.4
67.7
28.4
34.1
Qwen2.5-VL-3B
26.8
58.7
47.2
20.1
36.1
68.7
61.7
27.1
33.4
Table 2: OOD generalization on OrthoMind-3D and comparison on external benchmarks. We report accuracy (%) under the same evaluation protocol as Section 4.2 . ↑ indicates relative improvement over the Qwen3-VL-4B-Instruct base model; ↓ indicates relative drop.
Figure 4: In-context learning (ICL) and explicit orthographic three-view description study on OrthoMind-3D (in-domain). ICL yields limited improvements only for the strongest proprietary models, while explicit three-view descriptions substantially improve performance for most models, supporting the need for a view-consistent intermediate representation.
Model
Block Count
Block Count (Attr.)
SPBench-SI
ViewSpatial
Qwen3-VL-4B-Instruct
10218.9
6531.1
451.6
952.1
3ViewSense-4B-sft
350.4
375.2
250.1
261.7
3ViewSense-4B-rl-strict
377.1
378.9
273.5
266.6
3ViewSense-4B-rl-slack
375.2
389.4
281.5
270.9
Table 3: Average response length (tokens) across benchmarks. For this simple task, the base model tends to produce overly verbose reasoning (over thinking) and can even suffer reduced accuracy; 3ViewSense substantially reduces this redundancy.
SFT Target
OrthoMind-3D
SPBench-SI
ViewSpatial
ID
OOD
Direct QA
80.3
49.8
1.3
7.2
3ViewSense Reasoning
70.3
46.6
21.2
34.1
Table 4: Ablation on the SFT target. Direct QA uses the same inputs, instances, and hyperparameters as VGR-SFT, but supervises only the final answer. We report average accuracy (%) on OrthoMind-3D (In-Domain and Out-of-Domain) and two external benchmarks (SPBench-SI and ViewSpatial).
Stage
OrthoMind-3D
MindCube-Tiny
ViewSpatial
ID
OOD
OMS-SFT (only)
48.7
41.3
29.6
33.4
VGR-SFT (only)
70.3
46.6
32.4
34.1
Two-stage SFT (OMS → VGR)
78.6
49.5
34.9
34.4
Table 5: Ablation study on the two-stage SFT design. We report accuracy (%) on OrthoMind-3D (In-Domain and Out-of-Domain) and two external benchmarks (MindCube-Tiny and ViewSpatial).
Figure 5: RL ablation on initialization. We compare GRPO reward trajectories when starting RL from the Stage I OMS-SFT model versus from the Stage II VGR-SFT model.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Sample visualization of the OrthoMind-3D dataset. We show examples from the in-domain and out-of-domain subsets for two task families: block counting (top) and object reasoning (bottom). In-domain data are generated with strict geometric constraints, while out-of-domain data are photorealistic and less structured to evaluate generalization.
Figure 7: Example of the orthographic three-view description and the Stage-I (OMS) instruction used for training.
Figure 8: Prompt design for image generation using generative AI models.
Figure 9: Detailed prompt for LLM-based three-view to natural language reasoning conversion.
Figure 10: In-Context Learning Experiment Instruction and Teaching Demonstration for Block Counting Task.
Figure 11: In-Context Learning Experiment Instruction and Teaching Demonstration for Object Reasoning.
Hyperparameter
Value
per_device_train_batch_size
1
gradient_accumulation_steps
8
bf16
true
data_seed
42
gradient_checkpointing
true
lr_scheduler_type
cosine
Appendix
Table 6: Training hyperparameters for SFT and RL (GRPO, Stage II refinement).
Stage
Total
Block count
Distinct block count
Object reasoning
Distinct object reasoning
OMS SFT (Stage I)
19,544
6,000
5,000
3,544
5,000
Appendix
Table 7: Training data statistics and splits for OMS SFT (Stage I).
Table 10: Block Counting Probe Accuracy on Qwen3-VL Visual Features.
Model
Cardinality Block Counting
Object Reasoning (Overall)
–
w/ 3-view Desc. (Rel. ↑ )
–
w/ 3-view Desc. (Rel. ↑ )
GPT-5
15.8
28.6 ( +81.0% )
68.2
88.9 ( +30.4% )
Gemini-3-pro
13.8
43.8 ( +217.4% )
87.4
98.0 ( +12.1% )
Claude-Sonnet-4.5
19.0
46.0 ( +142.1% )
57.4
93.8 ( +63.4% )
XiaoMiMo-VL-7B-RL
18.2
35.6 ( +95.6% )
63.1
60.1 ( -4.7% )
GLM4.1V-9B
15.0
10.8 ( -28.0% )
51.9
57.1 ( +10.0% )
Appendix
Table 11: Effect of explicit orthographic three-view descriptions on OrthoMind-3D. We report accuracy (%) for Cardinality Block Counting and Object Reasoning (Overall). Colored numbers indicate the relative change compared to the single-view baseline (green: improvement; red: drop).
Figure 12: Cumulative reward curves during GRPO-based RL refinement. We report the training dynamics under two reward settings: strict reward (exact match) and slack reward (partial credit) as defined in Section 3.3 .
Figure 13: Qualitative examples of model responses on block counting under occlusion. Compared to the base model, 3ViewSense produces more concise and structured reasoning, improving robustness and accuracy.
Figure 14: Case study of 3ViewSense reasoning on external benchmarks. We show qualitative examples from ViewSpatial and SPBench-SI, illustrating how 3ViewSense transfers the learned view-grounded reasoning process to external spatial reasoning tasks.
Existing approaches to multi-view spatial reasoning operate largely on sparse input views. Vision-language models (VLMs) are thus restricted to understand a scene and infer spatial relations within these fixed views, leading to fragile cross-view alignment and geometry-to-language bottleneck. To address these issues, we formulate a novel Seek-and-View reasoning approach to find implicit cross-view spatial evidence by locating a question-relevant view to support the spatial reasoning. To realize this approach, we propose Vantage, a training-free model-agnostic reasoning framework that pairs a VLM with a 3D foundation model: a viewpoint-grounded reasoning stage for question analysis and view planning, followed by a geometry-grounded evidence augmentation stage to effectively synthesize and incorporate visual evidence into the final reasoning. Comprehensive experiments on six VLMs demonstrate consistent improvements on five benchmarks without fine-tuning. Overall, by revealing spatial evidence through view-grounded reasoning, Vantage can largely reduce reliance on language-based cross-view alignment and improve multi-view spatial understanding. Our code is available at https://github.com/q1xiangchen/Vantage.
Qixiang Chen, Cheng Zhang, Fucai Ke +3
Monash University · The Chinese University of Hong Kong
Spatial intelligence remains a persistent challenge for Multimodal Large Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual reasoning or 3D reconstruction. However, they often falter during multi-step reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.
Xudong Li, Mengdan Zhang, Peixian Chen +7
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology
Recent works augment Vision-Language Models with geometry features from pretrained 3D models, expecting that the geometric signal will boost spatial reasoning. However, we find that simply fusing geometry features and training on standard spatial QA yields only marginal improvements on high-level multi-hop tasks. We attribute this gap to a training-signal problem: standard spatial QA can be largely answered from visual features and language priors, so the geometry pathway receives weak gradients and fails to integrate with the visual features. To provide a training signal that requires geometry, we propose novel-view semantic rendering as an auxiliary training task that requires the model to predict the semantic layout of an unobserved viewpoint, inspired by humans' ability to mentally simulate novel viewpoints during spatial reasoning. This task encourages joint use of both pathways: geometry provides pose-dependent visibility, while vision provides semantic content. Our auxiliary task yields consistent improvements over the geometry-augmented baseline across all three benchmarks (up to +1.6 on VSI-Bench, +2.2 on ReVSI, +2.9 on our 3D-Point-QA dataset) and our full model surpasses prior open-source methods on VSI-Bench and on ReVSI. Project page: https://yuqunw.github.io/Render2Reason/.