cs.CVMar 30, 2026
SaveLearning Multi-View Spatial Reasoning from Cross-View Relations
Organizations: KAIST · Config · Hanyang University · Yonsei University ·
Abstract
Vision-language models (VLMs) have achieved impressive results on single-view vision tasks, but lack the multi-view spatial reasoning capabilities essential for embodied AI systems to understand 3D environments and manipulate objects across different viewpoints. In this work, we introduce Cross-View Relations (XVR), a large-scale dataset designed to teach VLMs spatial reasoning across multiple views. XVR comprises 100K vision-question-answer samples derived from 18K diverse 3D scenes and 70K robotic manipulation trajectories, spanning three fundamental spatial reasoning tasks: Correspondence (matching objects across views), Verification (validating spatial relationships), and Localization (identifying object positions). VLMs fine-tuned on XVR achieve substantial improvements on established multi-view and robotic spatial reasoning benchmarks (MindCube and RoboSpatial). When integrated as backbones in Vision-Language-Action models, XVR-trained representations improve success rates on RoboCasa. Our results demonstrate that explicit training on cross-view spatial relations significantly enhances multi-view reasoning and transfers effectively to real-world robotic manipulation.
Figures & tables
| Dataset | Split | # Imgs / sample | Domain | # Images | # QAs |
| 3DSRBench-real | Eval | 1.00 | General | 2.1K | 2.1K |
| All-Angles-Bench | Eval | 4–5 | General | 450 | 2.1K |
| MMSI-Bench | Eval | 2.55 | General, Robotic | 2K | 1K |
| SpatialVLM | Train, Eval | 1.00 | General | 10M | 2B |
| RoboSpatial | Train, Eval | 1.00 | General | 1M | 3M |
| MindCube | Train, Eval | 3.37 | General | 3.2K | 21K |
Table 1: Comparison of spatial reasoning datasets. XVR provides the highest mean images per sample among training datasets, with supervision spanning both general and robotic domains.
Figure 2: Overview of the question–answer (QA) structure in XVR. The figure shows representative examples from eight task types across correspondence, verification, and localization categories, demonstrating the consistent QA format used throughout the dataset. Each category is color-coded: red for Correspondence (Point, Directional), green for Verification (Spatial, Temporal), and blue for Localization (Viewpoint, Directional View, Cross-Scenario, Language-Conditioned).
Figure 3: Generalization to external spatial benchmarks (MindCube-Tiny and RoboSpatial-Home). Training on XVR improves Qwen3-VL-2B across all tasks, with the largest gains in Compatibility (+7.6%) and Among (+7.0%).
Figure 4: Visualization of the three manipulation tasks and their camera-view configurations used for VLA transfer evaluation.
Figure 5: Transfer to Embodied Tasks: RoboCasa VLA Performance. Fine-tuning on XVR improves Qwen3-VL-2B performance on RoboCasa manipulation tasks, showing effective transfer of spatial reasoning skills to robotic action prediction.
| Task | Formulation | Question Prototype |
| Correspondence | ||
| Point Correspondence | Q: Which point in corresponds to this point in ? | |
| Directional Correspondence | Q: Which arrow in points in the same direction as in ? | |
| Verification | ||
| Spatial Verification | Q: Which point in each image breaks spatial alignment? | |
| Temporal Verification | Q: Which view was captured at a different timestamp? | |
Table 3: Taxonomy of cross-view reasoning tasks, summarized with their formal definitions and representative QA prototypes.
Figure 6: Task generation pipeline for XVR. The pipeline branches into geometry-based generation (top) for tasks using 3D geometric information and metadata-based generation (bottom) for tasks using trajectory annotations. Geometry-based generation processes general domain data ( ) through 3D-to-2D projection and visibility checking to create Point, Directional, Spatial, and Viewpoint tasks. Metadata-based generation processes robotic domain data ( ) through temporal and camera metadata extraction to create Temporal, Cross-View, Directional View, and Language-Conditioned tasks. Both pipelines converge at QA assembly to produce final question-answer pairs.
Figure 7: Source data distribution for XVR. Top row (Raw Distribution): Original distribution of data sources before task generation. General domain: 18,409 scenes from WildRGB-D. Robotic domain: 35,717 trajectories from DROID, RoboSet, Agibot, MobileAloha, and FMB. Bottom row (Source Distribution): Distribution after filtering and task generation. General domain: 51,788 samples (visibility-based filtering affects category proportions). Robotic domain: 51,788 samples (DROID and FMB limited to Temporal Verification due to inconsistent camera metadata; RoboSet and Agibot support all robotic tasks).
| Category | Metric | Value |
| Questions | Avg. length (chars) | 257.7 |
| Median length (chars) | 236.5 | |
| Min length (chars) | 25 | |
| Max length (chars) | 508 | |
| Choices | Avg. per question | 3.7 |
| Median | 3.0 |
Table 4: XVR Dataset Statistics (103,576 samples, 447,811 images)
| Task | Samples | Random Baseline |
| Point Correspondence | 170 (9.11%) | 20.00% |
| Directional Correspondence | 221 (11.84%) | 25.00% |
| Spatial Verification | 221 (11.84%) | 22.22% |
| Temporal Verification | 221 (11.84%) | 33.33% |
| Viewpoint Localization | 264 (14.15%) | 33.33% |
| Directional View Localization | 264 (14.15%) | 50.00% |
Table 5: XVR-Eval task distribution and random baseline accuracy.
Figure 8: Cross-Scenario Localization example from XVR-Eval. The task requires identifying the corresponding viewpoint across two different scenarios. Only Qwen3-VL-2B-XVR (Ours) and Gemini-Robotics-ER-1.5 correctly predict the answer.
Figure 9: Spatial Verification example from XVR-Eval. The task requires identifying which view contains a marker placed at an inconsistent 3D location compared to the others. Only Qwen3-VL-2B-XVR (Ours) correctly identifies the spatially inconsistent view, demonstrating the effectiveness of explicit cross-view relation supervision.
Figure 10: Additional experiments on XVR-Eval. (a) Baseline evaluation across Qwen3-VL model scales (2B, 8B, 30B-A3B) and spatial reasoning models: SpatialOM-3B (SO), SpatialMLLM-3B (SM), and RoboBrain2.0-32B (RB). (b) Impact of XVR training on Qwen3-VL-2B and 8B. (c) Scaling behavior of XVR training data (100k, 300k, 500k samples) on Qwen3-VL-2B.
| Parameter | Value |
| Model | Qwen3-VL-2B-Instruct |
| Dataset size | 103,576 QA pairs |
| Epochs | 3 |
| Learning rate | 5e-5 |
| Scheduler | Cosine |
| Fine-tuning type | Full-parameter |
Table 6: Training hyperparameters for XVR fine-tuning of Qwen3-VL-2B-Instruct.
| Hyperparameter | Value |
| Backbone VLM | Qwen3-VL-2B-Instruct / Qwen3-VL-2B-XVR |
| Frozen components | Language model (LLM) |
| Fine-tuned components | Vision encoder, Projector, DiT action head |
| Action head | DiT (trained from scratch) |
| VLM feature layer used | Transformer layer 12 |
| Tasks | CoffeePressButton, TurnOffMicrowave, PnPCabToCounter |
Table 7: Training hyperparameters for VLA policy learning.
Figure 11: Point Correspondence task example. The question asks which point in the target image corresponds to a marked point in the reference images.
Figure 12: Directional Correspondence task example. The question asks which arrow in the target image points in the same direction as arrows in the reference images.
Figure 13: Spatial Verification task example. The question asks which marked point across multiple images breaks spatial alignment.
Figure 14: Temporal Verification task example. The question asks which image was captured at a different timestamp compared to others.
Figure 15: Viewpoint Localization task example. The question asks which camera view corresponds to a specific spatial position marked in 3D space.
Figure 16: Directional View Localization task example. The question asks which camera view lies in a specified direction (e.g., left, right) relative to the reference camera.
Figure 17: Cross-Scenario Localization task example. The question asks which camera view in one scenario matches the viewpoint of a reference image from another scenario.
Figure 18: Language-Conditioned Localization task example. The question asks which camera view matches a natural language spatial description (e.g., ”wrist-mounted camera”).