3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models

Shaoxiong Zhan; Yanlin Lai; Zheng Liu; Hai Lin; Shen Li; Xiaodong Cai; Zijian Lin; Wen Huang

cs.CVMar 8, 2026

3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models

Authors: Shaoxiong Zhan, Yanlin Lai, Zheng Liu, Hai Lin, Shen Li, Xiaodong Cai, Zijian Lin, Wen Huang,

Organizations: Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · School of Software Engineering, Chongqing University, Chongqing, China

Abstract

Current Large Language Models have achieved Olympiad-level logic, yet Vision-Language Models paradoxically falter on elementary spatial tasks like block counting. This capability mismatch reveals a critical spatial intelligence gap,'' where models fail to construct coherent 3D mental representations from 2D observations. We uncover this gap via diagnostic analyses showing the bottleneck is a missing view-consistent spatial interface rather than insufficient visual features or weak reasoning. To bridge this, we introduce **3ViewSense**, a framework that grounds spatial reasoning in Orthographic Views. Drawing on engineering cognition, we propose a Simulate-and-Reason'' mechanism that decomposes complex scenes into canonical orthographic projections to resolve geometric ambiguities. By aligning egocentric perceptions with these allocentric references, our method facilitates explicit mental rotation and reconstruction. Empirical results on spatial reasoning benchmarks demonstrate that our method significantly outperforms existing baselines, with consistent gains on occlusion-heavy counting and view-consistent spatial reasoning. The framework also improves the stability and consistency of spatial descriptions, offering a scalable path toward stronger spatial intelligence in multimodal systems.~\footnote{https://github.com/Jasaxion/3ViewSense}

Figures & tables

Figure 1: Motivation for explicit three-view reasoning. Providing explicit orthographic three-view descriptions (front/left/top) improves block-counting performance under occlusion, highlighting the role of view-consistent spatial representations.

Figure 2: The construction pipeline of our OrthoMind-3D dataset. To bridge the gap between visual perception and mental spatial reasoning, we curate data from two distinct domains. For In-Domain data, we utilize programmatic synthesis with strict geometric constraints to train the model’s orthographic projection capabilities. For Out-of-Domain data, we employ sandbox game engines and generative AI techniques to evaluate the model’s robustness and generalization in unstructured environments.

Figure 3: The training framework of 3ViewSense . Stage I learns to induce canonical front, left, and top orthographic views from an egocentric input. Stage II performs view-grounded reasoning by integrating the inferred views to generate reasoning traces and final answers, while reinforcement learning further internalizes this view-based reasoning capability.

Proprietary Models
	Block Counting		Object Reasoning
Model	Block Count	Block Count (Attr.)	Object Count	Object Position	Object Count (Attr.)	Object Position (Attr.)
GPT-4o	15.8	53.2	68.3	39.3	71.2	47.2
Gemini-2.0-Flash	18.2	54.0	69.7	62.7	86.4	62.6
GPT-5	15.8	50.7	64.0	60.3	77.6	66.0
Gemini-3-pro	13.8	80.2	83.3	71.6	93.2	93.6

Table 1: Main results on OrthoMind-3D. We report accuracy (%) for block counting and object reasoning, covering both cardinality queries and attribute-conditioned (attr.) queries, as defined in Section 4.2 . “–” denotes entries where the model cannot produce outputs in the required format for evaluation.

Model	(A) OrthoMind-3D (OOD Task)				(B) Other Spatial Benchmarks
Model	Block Count	Block Count (Attr.)	Object Count	Object Position	MindCube-Tiny	CV-Bench (2D)	OmniSpatial	SPBench (SI)	ViewSpatial
SpatialLadder-3B	19.6	43.0	22.2	15.6	38.5	68.6	67.6	30.8	36.2
SpaceOm-4B	23.4	61.7	46.2	19.2	35.5	68.8	67.6	29.7	34.8
SpaceQwen2.5-VL-3B-Instruct	21.2	62.5	51.8	16.5	36.4	66.1	58.8	28.1	30.1
InternVL3.5-4B	33.1	65.9	54.6	34.8	14.4	79.4	67.7	28.4	34.1
Qwen2.5-VL-3B	26.8	58.7	47.2	20.1	36.1	68.7	61.7	27.1	33.4

Table 2: OOD generalization on OrthoMind-3D and comparison on external benchmarks. We report accuracy (%) under the same evaluation protocol as Section 4.2 .

\uparrow

indicates relative improvement over the Qwen3-VL-4B-Instruct base model;

\downarrow

indicates relative drop.

Figure 4: In-context learning (ICL) and explicit orthographic three-view description study on OrthoMind-3D (in-domain). ICL yields limited improvements only for the strongest proprietary models, while explicit three-view descriptions substantially improve performance for most models, supporting the need for a view-consistent intermediate representation.

Model	Block Count	Block Count (Attr.)	SPBench-SI	ViewSpatial
Qwen3-VL-4B-Instruct	10218.9	6531.1	451.6	952.1
3ViewSense-4B-sft	350.4	375.2	250.1	261.7
3ViewSense-4B-rl-strict	377.1	378.9	273.5	266.6
3ViewSense-4B-rl-slack	375.2	389.4	281.5	270.9

Table 3: Average response length (tokens) across benchmarks. For this simple task, the base model tends to produce overly verbose reasoning (over thinking) and can even suffer reduced accuracy; 3ViewSense substantially reduces this redundancy.

SFT Target	OrthoMind-3D		SPBench-SI	ViewSpatial
SFT Target	ID	OOD	SPBench-SI	ViewSpatial
Direct QA	80.3	49.8	1.3	7.2
3ViewSense Reasoning	70.3	46.6	21.2	34.1

Table 4: Ablation on the SFT target. Direct QA uses the same inputs, instances, and hyperparameters as VGR-SFT, but supervises only the final answer. We report average accuracy (%) on OrthoMind-3D (In-Domain and Out-of-Domain) and two external benchmarks (SPBench-SI and ViewSpatial).

Stage	OrthoMind-3D		MindCube-Tiny	ViewSpatial
Stage	ID	OOD	MindCube-Tiny	ViewSpatial
OMS-SFT (only)	48.7	41.3	29.6	33.4
VGR-SFT (only)	70.3	46.6	32.4	34.1
Two-stage SFT (OMS $\rightarrow$ VGR)	78.6	49.5	34.9	34.4

Table 5: Ablation study on the two-stage SFT design. We report accuracy (%) on OrthoMind-3D (In-Domain and Out-of-Domain) and two external benchmarks (MindCube-Tiny and ViewSpatial).

Figure 5: RL ablation on initialization. We compare GRPO reward trajectories when starting RL from the Stage I OMS-SFT model versus from the Stage II VGR-SFT model.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Sample visualization of the OrthoMind-3D dataset. We show examples from the in-domain and out-of-domain subsets for two task families: block counting (top) and object reasoning (bottom). In-domain data are generated with strict geometric constraints, while out-of-domain data are photorealistic and less structured to evaluate generalization.

Figure 7: Example of the orthographic three-view description and the Stage-I (OMS) instruction used for training.

Panel of Figure 8 — Figure 8: Prompt design for image generation using generative AI models.

Panel of Figure 9 — Figure 9: Detailed prompt for LLM-based three-view to natural language reasoning conversion.

Panel of Figure 10 — Figure 10: In-Context Learning Experiment Instruction and Teaching Demonstration for Block Counting Task.

Panel of Figure 11 — Figure 11: In-Context Learning Experiment Instruction and Teaching Demonstration for Object Reasoning.

Hyperparameter	Value
per_device_train_batch_size	1
gradient_accumulation_steps	8
bf16	true
data_seed	42
gradient_checkpointing	true
lr_scheduler_type	cosine

Appendix

Table 6: Training hyperparameters for SFT and RL (GRPO, Stage II refinement).

Stage	Total	Block count	Distinct block count	Object reasoning	Distinct object reasoning
OMS SFT (Stage I)	19,544	6,000	5,000	3,544	5,000

Appendix

Table 7: Training data statistics and splits for OMS SFT (Stage I).

Stage	Total
VGR SFT (Stage II)	21,000
RL (GRPO)	30,000 (10,000 from Stage II + 20,000 new)

Appendix

Table 8: Additional training data statistics.

Split	Task	#
ID	Block counting (cardinality)	500
ID	Block counting (attribute-cond.)	2,202
ID	Object reasoning: counting	300
ID	Object reasoning: positioning	300
ID	Object reasoning: counting (attr.)	500
ID	Object reasoning: positioning (attr.)	500

Appendix

Table 9: OrthoMind-3D evaluation split statistics. “Attribute-conditioned” denotes queries that specify explicit attributes (e.g., color).

Probe Architecture	Layers	Accuracy
MLP [512, 256]	2	43.2%
MLP [1024, 512, 256]	3	50.4%
MLP [1024, 512, 256, 128]	4	55.8%

Appendix

Table 10: Block Counting Probe Accuracy on Qwen3-VL Visual Features.

Model	Cardinality Block Counting		Object Reasoning (Overall)
Model	–	w/ 3-view Desc. (Rel. $\uparrow$ )	–	w/ 3-view Desc. (Rel. $\uparrow$ )
GPT-5	15.8	28.6 ( +81.0% )	68.2	88.9 ( +30.4% )
Gemini-3-pro	13.8	43.8 ( +217.4% )	87.4	98.0 ( +12.1% )
Claude-Sonnet-4.5	19.0	46.0 ( +142.1% )	57.4	93.8 ( +63.4% )
XiaoMiMo-VL-7B-RL	18.2	35.6 ( +95.6% )	63.1	60.1 ( -4.7% )
GLM4.1V-9B	15.0	10.8 ( -28.0% )	51.9	57.1 ( +10.0% )

Appendix

Table 11: Effect of explicit orthographic three-view descriptions on OrthoMind-3D. We report accuracy (%) for Cardinality Block Counting and Object Reasoning (Overall). Colored numbers indicate the relative change compared to the single-view baseline (green: improvement; red: drop).

Figure 12: Cumulative reward curves during GRPO-based RL refinement. We report the training dynamics under two reward settings: strict reward (exact match) and slack reward (partial credit) as defined in Section 3.3 .

Figure 13: Qualitative examples of model responses on block counting under occlusion. Compared to the base model, 3ViewSense produces more concise and structured reasoning, improving robustness and accuracy.

Figure 14: Case study of 3ViewSense reasoning on external benchmarks. We show qualitative examples from ViewSpatial and SPBench-SI, illustrating how 3ViewSense transfers the learned view-grounded reasoning process to external spatial reasoning tasks.

Visual Spatial Reasoning

Visual spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and physical change from images and video.

See all 246 papers

Seek-and-View Reasoning for Multi-View Spatial Understanding
Oct 8, 2026Qixiang Chen, Cheng Zhang, Fucai Ke +3Multimodal Spatial Reasoning Visual Spatial Reasoning
Monash University · The Chinese University of Hong Kong
OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
Jul 1, 2026Xudong Li, Mengdan Zhang, Peixian Chen +7Visual Spatial Reasoning 3D Spatial Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Tencent Youtu Lab · Beijing Institute of Technology
Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMs
Oct 4, 2026Yuqun Wu, Yao Xiao, Chuhang Zou +2Visual Spatial Reasoning 3D VQA
University of Illinois at Urbana-Champaign · Meta

Oct 8, 2026J/K move · Enter open · S save

Seek-and-View Reasoning for Multi-View Spatial Understanding

Qixiang Chen, Cheng Zhang, Fucai Ke, Chi-Wing Fu+2

Monash University · The Chinese University of Hong Kong

Multimodal Spatial Reasoning Visual Spatial Reasoning Spatial Reasoning VLM Reasoning

Open paper AskPDF arXiv

Existing approaches to multi-view spatial reasoning operate largely on sparse input views. Vision-language models (VLMs) are thus restricted to understand a scene and infer spatial relations within these fixed views, leading to fragile cross-view alignment and geometry-to-language bottleneck. To address these issues, we formulate a novel Seek-and-View reasoning approach to find implicit cross-view spatial evidence by locating a question-relevant view to support the spatial reasoning. To realize this approach, we propose Vantage, a training-free model-agnostic reasoning framework that pairs a VLM with a 3D foundation model: a viewpoint-grounded reasoning stage for question analysis and view planning, followed by a geometry-grounded evidence augmentation stage to effectively synthesize and incorporate visual evidence into the final reasoning. Comprehensive experiments on six VLMs demonstrate consistent improvements on five benchmarks without fine-tuning. Overall, by revealing spatial evidence through view-grounded reasoning, Vantage can largely reduce reliance on language-based cross-view alignment and improve multi-view spatial understanding. Our code is available at https://github.com/q1xiangchen/Vantage.

Abstract

Visual Spatial Reasoning

Explore similar work

Seek-and-View Reasoning for Multi-View Spatial Understanding

OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping

Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMs