Multi-view visual reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints, yet existing research has largely focused on single-image or temporally dense video settings. In real-world scenarios, reasoning across views requires integrating partial observations without explicit guidance, while collecting large-scale multi-view data with accurate geometric and semantic annotations remains challenging. To address this gap, we leverage physically grounded simulation to construct diverse, high-fidelity 3D scenes with precise per-view metadata, enabling scalable data generation that remains transferable to real-world settings. Based on this engine, we introduce VIEW2SPACE, a multi-dimensional benchmark for sparse multi-view reasoning, together with a scalable, disjoint training split supporting millions of grounded question-answer pairs. Using this benchmark, a comprehensive evaluation of state-of-the-art vision-language and spatial models reveals that multi-view reasoning remains largely unsolved, with most models performing only marginally above random guessing. We further investigate whether training can bridge this gap. Our proposed Grounded Chain-of-Thought with Visual Evidence substantially improves performance under moderate difficulty, and generalizes to real-world data, outperforming existing approaches in cross-dataset evaluation. We further conduct difficulty-aware scaling analyses across model size, data scale, reasoning depth, and visibility constraints, indicating that while geometric perception can benefit from scaling under sufficient visibility, deep compositional reasoning across sparse views remains a fundamental challenge.
Figures & tables
Figure 1: A single glance is often insufficient for completing real-world tasks. Humans naturally integrate observations across sparse viewpoints to form a shared spatial understanding. Given heterogeneous views (e.g., robotic dog and drone), one can efficiently align what different agents observe and reason across sparse viewpoints.
Figure 2: Overview of the multi-view data engine. Stage 1: Assets are collected, categorized, and size-normalized, and annotated using OSD-Tag with verification. Stage 2: Given theme configurations defining asset categories and spatial relations, scenes are synthesized by sampling layouts under compositional constraints. Stage 3: Multi-view images are rendered together with physics-based metadata. Stage 4: Spatial relations are selected to automatically generate grounded question-answer pairs.
Figure 3: Overview of VIEW2SPACE task design. VIEW2SPACE comprises three task types: MCQ, counting, and visual grounding detection. Reasoning difficulty ranges from one-hop perceptual queries to multi-hop cross-view spatial reasoning.
Method
Multiple Choice Answering
Visual Counting
Visual Grounding
ACC (%)
MAE
ACC (%)
mIoU (%)
F1-score (%)
Baseline
Random (chance)
28.59
11.37
3.39
0.22
0.0
Random (frequency)
28.59
2.46
13.05
2.01
0.27
Open-source MLLMs
Internvl2.5-8B [ 8 ]
30.93
2.04
22.50
0.79
0.0
Table 1: Comparison on VIEW2SPACE-v1
Method
Tuning on MINDCUBE
Overall
Rotation
Among
Around
Baseline
Random (chance)
-
32.35
36.36
32.29
30.66
Random (frequency)
-
33.02
38.30
32.66
35.79
Open-source MLLMs
Internvl2.5-8B [ 8 ]
w/o Tuning
18.68
36.45
47.11
26.91
Molmo2-4B [ 11 ]
w/o Tuning
37.33
36.00
38.83
35.75
Table 2: Comparison on MINDCUBE-Tiny. The second column indicates whether models are fine-tuned on MINDCUBE (w Tuning vs. w/o Tuning).
Figure 4: Scaling Behaviour across Reasoning and Key-Object Visibility Difficulty (see Sec. 4.1 ). Each subplot plots mIoU versus task difficulty. Each curve corresponds to a specific dataset size, fitted over the averaged performance at each difficulty level (aggregated across multiple samples and weighted accordingly). The dense empirical dot plots and detailed results are provided in the appendix.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Year
Venue
QA Pairs
MCQ
Count/Free
Detection
Multi-view
Sparse
FFR
R-Scale
Metadata
VSR [ 32 ]
2023
TACL
10K
Binary
NO
NO
NO
NO
NO
NO
LM-Gen
SpatialRGPT [ 10 ]
2024
NIPS
1.4K
NO
Free
NO
NO
NO
NO
NO
LM-Gen
VSI-Bench [ 46 ]
2025
CVPR
5.1K
YES
YES
NO
Video
NO
YES
NO
H
OmniSpatial [ 25 ]
2026
ICLR
1.5K
YES
NO
NO
NO
NO
YES
NO
LM-Gen
InternSpatial [ 14 ]
2026
ICLR
12M
YES
YES
YES
Few
NO
YES
NO
LM-Gen
MindCube [ 50 ]
2026
ICLR
20K
YES
NO
NO
YES
NO
YES
NO
H
Appendix
Table 3: Difference between our work and similar benchmarks.MCQ: multiple-choice; Count/Free: counting or free-form answers; Detection: grounding (e.g., detection/segmentation); Sparse: sparse multi-view setups; FFR: free-form reasoning; R-scale: reasoning difficulty range; Metadata: annotation source - LM-Gen (model-generated), H (human), PE (physical engine ground truth).
Figure 5: Overview of VIEW2SPACE Scene Thematic Categories
Figure 6: Pipeline of Overview-driven Semantic Differentiation and Tagging
Figure 7: (a) Verify Comparison Keys: check comparison keys across objects within a category; (b) Verify Tag Libraries: check the completeness of the tag library; (c) Verify Tag Assignment: check whether tags are correctly assigned.
Figure 8: For each RGB image (a), a high-resolution mesh (b) and the corresponding depth map (c) are provided, from which instance occlusion-aware 2D/3D bounding boxes (d/e) and segmentation masks (f) can be readily derived.
Figure 9: Answer Distribution and Statistical Analysis of View2Space-V1
Method
Multiple Choice Answering
Visual Counting
Visual Grounding
ACC (%)
MAE
ACC (%)
mIoU (%)
F1-score (%)
Baseline
Random (chance)
28.59
11.37
3.39
0.22
0.0
Random (frequency)
28.59
2.46
13.05
2.01
0.27
Open-source MLLMs
Mantis-8B (SigLip) [ 26 ]
30.07
2.94
17.60
1.24
0.23
Appendix
Table 4: Thinking vs. Non-Thinking Comparison on VIEW2SPACE-v1 . The light blue region labelled “Direct” corresponds to inference without chain-of-thought reasoning, where the model produces the final answer directly.
Answerer
Multiple Choice Answering
Visual Counting
Visual Grounding
ACC (%)
ACC (%)
mIoU (%)
Baseline
Random (chance)
28.59
3.39
0.22
Random (frequency)
28.59
13.05
2.01
Vision Language Models
Qwen3-VL-4B [ 1 ]
35.19
21.15
16.53
Appendix
Table 5: Human Evaluation on VIEW2SPACE.
Figure 10: Sample of training data - 1
Figure 11: Scaling behavior of Qwen3-VL-2B and Qwen3-VL-4B under Grounded Chain-of-Thought with Visual Evidence training on VIEW2SPACE . Performance is reported across increasing training data sizes (1k–300k) for count (accuracy), MCQ (accuracy), and detection (mIoU) tasks. While both models benefit from additional data, MCQ performance exhibits diminishing returns beyond 100k samples—particularly for the 4B model—whereas detection tasks show a clear data threshold effect, with near-zero performance at very small scales and substantial gains once sufficient grounded supervision is provided.
Figure 12: Scaling behaviour across dataset size and difficulty. Each subplot reports mIoU. Empirical data points over dataset sizes (2B and 4B) are shown here.
Figure 18
Figure 13: Samples of counting task
Figure 20
Figure 14: Samples of multiple choice questioning tasks
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China, China · Huawei Noah’s Ark Lab · Hefei University of Technology, China