cs.CVMar 19, 2026
SaveMultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model
Organizations: Electronics and Telecommunications Research Institute, South Korea · Korea Advanced Institute of Science and Technology, South Korea · Sungkyunkwan University, South Korea · DeepAuto, South Korea
Abstract
Spatial reasoning is foundational for Vision-Language Models (VLMs), particularly when deployed as Vision-Language-Action (VLA) agents in physical environments. However, existing benchmarks predominantly focus on elementary, single-hop relations, neglecting the multi-hop compositional reasoning and precise visual grounding essential for real-world scenarios. To address this, we introduce MultihopSpatial, offering three key contributions: (1) A comprehensive benchmark designed for multi-hop and compositional spatial reasoning, featuring 1- to 3-hop complex queries across diverse spatial perspectives. (2) Acc@50IoU, a complementary metric that simultaneously evaluates reasoning and visual grounding by requiring both answer selection and precise bounding box prediction - capabilities vital for robust VLA deployment. (3) MultihopSpatial-Train, a dedicated large-scale training corpus to foster spatial intelligence. Extensive evaluation of 37 state-of-the-art VLMs yields eight key insights, revealing that compositional spatial reasoning remains a formidable challenge. Finally, we demonstrate that reinforcement learning post-training on our corpus enhances both intrinsic VLM spatial reasoning and downstream embodied manipulation performance.
Figures & tables
Figure 1: A multi-hop spatial reasoning example for an embodied agent in the real-world scenario.
Figure 2: Comparison of existing benchmarks (single-hop) and MultihopSpatial benchmark. In the question text, colored spans denote the queried reasoning components: Perspective , Attribute , Position , and Relation .
Figure 3: Compositional structure and category definitions in MultihopSpatial.
Figure 4: Example MultihopSpatial questions for 2-hop and 3-hop reasoning under ego-centric (top) and exo-centric (bottom) views . We omit the phrase “ and provide the bounding box coordinate of the region related to your answer ” for brevity.
| Model | Overall | 3Hop-Ego | 3Hop-Exo | 2Hop-Ego | 2Hop-Exo | 1Hop-Ego | 1Hop-Exo | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Acc. | Acc@50 | avg. IoU | Acc. | Acc@50 | Acc. | Acc@50 | Acc. | Acc@50 | Acc. | Acc@50 | Acc. | Acc@50 | Acc. | Acc@50 | |
| Proprietary Models – Instant | |||||||||||||||
| Claude-Opus-4.5 [ 4 ] | 45.1 | 3.2 | 13.3 | 25.7 | 2.0 | 48.5 | 3.6 | 33.7 | 2.0 | 58.1 | 4.8 | 43.2 | 3.5 | 61.1 | 3.1 |
| Claude-Sonnet-4.5 [ 5 ] | 20.9 | 0.5 | 4.2 | 6.4 | 0.4 | 18.5 | 0.9 | 12.3 | 0.0 | 34.7 | 1.6 | 13.5 | 0.0 | 40.0 | 0.1 |
| GPT-5.2 [ 30 ] | 19.3 | 2.0 | 11.8 | 5.3 | 0.7 | 18.0 | 4.9 | 10.8 | 0.1 | 30.4 | 5.9 | 8.1 | 0.0 | 43.3 | 0.4 |
| Open-weight Models – Instant | |||||||||||||||
Table 1: Benchmark results across different hop counts and Ego/Exo perspectives . Green cells indicate the best performance within each model group.
Figure 5: All model performance across hop count and ego/exo perspectives.
Figure 10: Grounded (Acc@50IoU) vs. Ungrounded accuracy per model. Solid bars indicate grounded accuracy (Acc@50IoU) and hatched regions indicate ungrounded responses. Percentages denote the ungrounded ratio.
Figure 13: Effect of model scale on MCQ Accuracy (top) and Acc@50IoU (bottom) across overall and per-hop settings. Solid lines indicate exo-centric and dashed lines indicate ego-centric performance.
Figure 14: Average Error Rates ( ) by tag combination across model categories. A, R, and P denote Attribute, Relation, and Position, respectively.
| MultihopSpatial | Out-of-Domain | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Acc. | Acc@50IoU | avg. IoU | BLINK [ 14 ] | 3DSRBench [ 26 ] | OmniSpatial [ 19 ] | VSI-Bench [ 40 ] | SpatialMQA [ 24 ] |
| Qwen3-VL-4B-Instruct | 37.8 | 31.0 | 69.9 | 82.5 | 56.1 | 42.7 | 62.8 | 39.6 |
| w/ MultihopSpatial-Train | 62.9 | 53.8 | 72.6 | 85.3 | 56.3 | 43.9 | 63.2 | 41.1 |
Table 2: Impact of MultihopSpatial-Train on spatial reasoning across diverse benchmarks.
| VLM backbone | Task-1 | Task-2 | Task-3 | Task-4 | Task-5 | Calvin | Libero |
|---|---|---|---|---|---|---|---|
| Qwen3-VL-4B-Instruct | 92.4 | 81.8 | 74.1 | 66.8 | 59.9 | 3.75 | 35.8 |
| w/ MultihopSpatial-Train | 93.0 | 85.4 | 79.3 | 73.2 | 66.9 | 3.98 | 40.0 |
Table 3: Vision-Language-Action evaluation on Calvin ABC-D [ 28 ] and Libero [ 23 ] benchmark. Each task (Task-#) measures success rate (%) on Calvin and Calvin denotes the average number of successfully completed tasks per sequence.
Figure 15: Qualitative failure analysis on a 3-hop ego-centric question. All three models recognized the position condition “in front of” , but failed to incorporate it into their final predictions. Full reasoning traces are provided in Appendix Table 8 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | MCQ Acc. | Acc@50IoU | avg. IoU |
|---|---|---|---|
| Qwen3-VL-4B-Instruct | 28.9 | 23.3 | 67.4 |
| SFT | 58.1 | 49.0 | 71.5 |
| GRPO ( ) | 61.8 | 51.6 | 71.4 |
| GRPO ( ) | 61.1 | 52.3 | 72.4 |
Table 4: Ablation Study: RL vs. SFT Post-training on MultihopSpatial-Train.
| MCQ Acc. | Acc@50IoU | avg. IoU | |
|---|---|---|---|
| 1 & 1 | 61.1 | 52.3 | 72.4 |
| 1 & 2 | 60.6 | 51.1 | 71.6 |
| 2 & 1 | 52.5 | 45.3 | 72.8 |
| 2 & 2 | 59.9 | 51.8 | 72.6 |
Table 5: Ablation Study: & in Reward Function.
| Model | MultihopSpatial (ID) | Out-Of-Domain (OOD) | ||||||
|---|---|---|---|---|---|---|---|---|
| Acc. | Acc@50 | BLINK | 3DSR | Omni | VSI | SpaMQA | OOD Avg. | |
| Qwen3-VL-4B | 37.8 | 31.0 | 82.5 | 56.1 | 42.7 | 62.8 | 39.6 | 56.7 |
| + OmniSpatial | 37.1 | 30.6 | 78.3 | 56.8 | 43.1 | 63.7 | 42.8 | 56.9 |
| + MultihopSpatial | 62.9 | 53.8 | 85.3 | 56.3 | 43.9 | 63.2 | 41.1 | 58.0 |
| Qwen3-VL-8B | 38.0 | 31.3 | 85.3 | 56.3 | 47.5 | 62.5 | 39.1 | 58.2 |
| + OmniSpatial [ 19 ] | 38.1 | 31.0 | 79.7 | 56.4 | 47.1 | 63.0 | 43.7 | 58.0 |
Table 6: Training corpus comparison with different model scales. ID: In-domain. OOD Avg. denotes the mean performance across the five OOD benchmarks.
Figure 16: Rank Correlations between metrics.
| Category | Definition | Examples |
|---|---|---|
| ATT | Visual properties of an object (color, shape, state, etc.) | red object, sitting person, round object |
| POS | Spatial location of an object | left side of the image, in front of the man, on the desk |
| REL | Spatial relationship between objects (comparative, superlative) | highest, closest, farthest |
Table 7: Definitions and examples of the three spatial reasoning categories used in MultihopSpatial.
Figure 17: Screenshot of the data annotation interface. Annotators write a question-answer pair, select a view type, and draw a bounding box.
Figure 18: Screenshot of the data verification interface. Reviewers verify samples based on the criteria and provide corrections and comments.
Figure 19: Qualitative failure analysis on a 2-hop ego-centric question. GPT, Gemini, and Claude fail on perspective, position, and relation, respectively.
Figure 20: Qualitative failure analysis on a 2-hop ego-centric question. GPT and Gemini fail on position recognition, whereas Claude fails on attribute recognition.
Figure 21: Qualitative failure analysis on a 2-hop ego-centric question. All three models fail on relation recognition.
Figure 22: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to recognize the attribute condition.
Figure 23: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to correctly determine the spatial relation based on the 3D distance.
Figure 24: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to recognize the attribute condition.
Figure 25: Qualitative failure analysis on a 3-hop ego-centric question. All three models fail to correctly identify the perspective.
Figure 26: Qualitative failure analysis on a 3-hop ego-centric question. All three models fail to correctly identify the perspective.
Figure 27: Qualitative failure analysis on a 3-hop exo-centric question. GPT and Gemini fail on attribute condition recognition, whereas Claude fails on position condition recognition.
Figure 28: Qualitative failure analysis on a 3-hop exo-centric question. GPT and Claude fail to recognize the attribute condition, whereas Gemini fails on relation recognition.
| Model | Reasoning Trace |
|---|---|
| GPT-5.2 | The sitting person is at the table with a gray laptop directly in front of them, and a black cellphone lying on the table closer to the camera. The framed picture is on the back wall above and behind the person, and the white door is also on the back/right wall. Among these square or rectangular objects, the one farthest from the sitting person is the wall-mounted framed picture, since it is clearly behind them and farther than the nearby door and table objects. Answer: (a) framed picture with a drawing of a person Bounding Box: { "bbox_2d" : [0.364, 0.035, 0.471, 0.190]} |
| Gemini3.0-Flash | Based on the visual information in the image, the man is sitting at a table and facing toward the camera, as indicated by his face being visible and the back of his laptop’s lid facing the viewer. From the man’s perspective, the objects mentioned in the choices are positioned as follows: (d) the gray laptop is directly in front of him on the table, likely only a foot or two away; (b) the black cellphone is also on the table, farther from him than the laptop and closer to the camera; (a) the framed picture with a drawing of a person is on the wall directly behind him; and (c) the white door is located on a far wall in the background of the scene, significantly farther away from him in 3D space than any of the items on the table. Even though the door is technically behind the plane of his body, it is the most distant object from him among the choices given. In many visual question-answering tasks, “in front of them” is used broadly to describe objects visible in the scene or from the camera’s vantage point facing the person. Answer: (c) white door Bounding Box: { "bbox_2d" : [0.273, 0.784, 0.571, 0.929]} |
| Claude Sonnet 4.5 | I need to identify which rectangular or square object in front of the sitting person is farthest from them. Let me analyze the objects mentioned in the choices: (a) framed picture with a drawing of a person, visible on the shelf in the background; (b) black cellphone, on the table in front of the person; (c) white door, visible on the right side of the image; and (d) gray laptop, on the table directly in front of the person. From the person’s perspective sitting at the table, the laptop is closest because it is right in front of them. The cellphone is on the table and relatively close. The framed picture is on the shelf above and behind the fireplace, while the white door is on the right wall. The framed picture appears to be the farthest object from the sitting person among the rectangular or square objects listed. It is located on a shelf in the background, above the fireplace. Answer: (a) framed picture with a drawing of a person Bounding Box: { "bbox_2d" : [0.3031, 0.0669, 0.4031, 0.1736]} |
Table 8: Full reasoning traces from GPT-5.2, Gemini3.0-Flash, and Claude Sonnet 4.5 for the example shown in Fig. 15 .
More in this topic
Spatial Reasoning Benchmarks
Spatial reasoning benchmarks test whether AI systems can perceive, represent, and act on geometric, relational, and viewpoint-dependent information.
See all 155 papers