Organizations: Electronics and Telecommunications Research Institute, South Korea · Korea Advanced Institute of Science and Technology, South Korea · Sungkyunkwan University, South Korea · DeepAuto, South Korea
Spatial reasoning is foundational for Vision-Language Models (VLMs), particularly when deployed as Vision-Language-Action (VLA) agents in physical environments. However, existing benchmarks predominantly focus on elementary, single-hop relations, neglecting the multi-hop compositional reasoning and precise visual grounding essential for real-world scenarios. To address this, we introduce MultihopSpatial, offering three key contributions: (1) A comprehensive benchmark designed for multi-hop and compositional spatial reasoning, featuring 1- to 3-hop complex queries across diverse spatial perspectives. (2) Acc@50IoU, a complementary metric that simultaneously evaluates reasoning and visual grounding by requiring both answer selection and precise bounding box prediction - capabilities vital for robust VLA deployment. (3) MultihopSpatial-Train, a dedicated large-scale training corpus to foster spatial intelligence. Extensive evaluation of 37 state-of-the-art VLMs yields eight key insights, revealing that compositional spatial reasoning remains a formidable challenge. Finally, we demonstrate that reinforcement learning post-training on our corpus enhances both intrinsic VLM spatial reasoning and downstream embodied manipulation performance.
Figures & tables
Figure 1: A multi-hop spatial reasoning example for an embodied agent in the real-world scenario.
Figure 2: Comparison of existing benchmarks (single-hop) and MultihopSpatial benchmark. In the question text, colored spans denote the queried reasoning components: Perspective , Attribute , Position , and Relation .
Figure 3: Compositional structure and category definitions in MultihopSpatial.
Figure 4: Example MultihopSpatial questions for 2-hop and 3-hop reasoning under ego-centric (top) and exo-centric (bottom) views . We omit the phrase “ and provide the bounding box coordinate of the region related to your answer ” for brevity.
Model
Overall
3Hop-Ego
3Hop-Exo
2Hop-Ego
2Hop-Exo
1Hop-Ego
1Hop-Exo
Acc.
Acc@50
avg. IoU
Acc.
Acc@50
Acc.
Acc@50
Acc.
Acc@50
Acc.
Acc@50
Acc.
Acc@50
Acc.
Acc@50
Proprietary Models – Instant
Claude-Opus-4.5 [ 4 ]
45.1
3.2
13.3
25.7
2.0
48.5
3.6
33.7
2.0
58.1
4.8
43.2
3.5
61.1
3.1
Claude-Sonnet-4.5 [ 5 ]
20.9
0.5
4.2
6.4
0.4
18.5
0.9
12.3
0.0
34.7
1.6
13.5
0.0
40.0
0.1
GPT-5.2 [ 30 ]
19.3
2.0
11.8
5.3
0.7
18.0
4.9
10.8
0.1
30.4
5.9
8.1
0.0
43.3
0.4
Open-weight Models – Instant
Table 1: Benchmark results across different hop counts and Ego/Exo perspectives . Green cells indicate the best performance within each model group.
Figure 5: All model performance across hop count and ego/exo perspectives.
Figure 7Figure 8
Figure 10: Grounded (Acc@50IoU) vs. Ungrounded accuracy per model. Solid bars indicate grounded accuracy (Acc@50IoU) and hatched regions indicate ungrounded responses. Percentages denote the ungrounded ratio.
Figure 10
Figure 13: Effect of model scale on MCQ Accuracy (top) and Acc@50IoU (bottom) across overall and per-hop settings. Solid lines indicate exo-centric and dashed lines indicate ego-centric performance.
Figure 14: Average Error Rates ( ↓,% ) by tag combination across model categories. A, R, and P denote Attribute, Relation, and Position, respectively.
MultihopSpatial
Out-of-Domain
Model
Acc.
Acc@50IoU
avg. IoU
BLINK [ 14 ]
3DSRBench [ 26 ]
OmniSpatial [ 19 ]
VSI-Bench [ 40 ]
SpatialMQA [ 24 ]
Qwen3-VL-4B-Instruct
37.8
31.0
69.9
82.5
56.1
42.7
62.8
39.6
w/ MultihopSpatial-Train
62.9
53.8
72.6
85.3
56.3
43.9
63.2
41.1
Table 2: Impact of MultihopSpatial-Train on spatial reasoning across diverse benchmarks.
VLM backbone
Task-1
Task-2
Task-3
Task-4
Task-5
Calvin
Libero
Qwen3-VL-4B-Instruct
92.4
81.8
74.1
66.8
59.9
3.75
35.8
w/ MultihopSpatial-Train
93.0
85.4
79.3
73.2
66.9
3.98
40.0
Table 3: Vision-Language-Action evaluation on Calvin ABC-D [ 28 ] and Libero [ 23 ] benchmark. Each task (Task-#) measures success rate (%) on Calvin and Calvin denotes the average number of successfully completed tasks per sequence.
Figure 15: Qualitative failure analysis on a 3-hop ego-centric question. All three models recognized the position condition “in front of” , but failed to incorporate it into their final predictions. Full reasoning traces are provided in Appendix Table 8 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Model
MCQ Acc.
Acc@50IoU
avg. IoU
Qwen3-VL-4B-Instruct
28.9
23.3
67.4
+ SFT
58.1
49.0
71.5
+ GRPO ( Rmcq )
61.8
51.6
71.4
+ GRPO ( Rmcq+Rbbox )
61.1
52.3
72.4
Appendix
Table 4: Ablation Study: RL vs. SFT Post-training on MultihopSpatial-Train.
α&β
MCQ Acc.
Acc@50IoU
avg. IoU
1 & 1
61.1
52.3
72.4
1 & 2
60.6
51.1
71.6
2 & 1
52.5
45.3
72.8
2 & 2
59.9
51.8
72.6
Appendix
Table 5: Ablation Study: α & β in Reward Function.
Model
MultihopSpatial (ID)
Out-Of-Domain (OOD)
Acc.
Acc@50
BLINK
3DSR
Omni
VSI
SpaMQA
OOD Avg.
Qwen3-VL-4B
37.8
31.0
82.5
56.1
42.7
62.8
39.6
56.7
+ OmniSpatial
37.1
30.6
78.3
56.8
43.1
63.7
42.8
56.9
+ MultihopSpatial
62.9
53.8
85.3
56.3
43.9
63.2
41.1
58.0
Qwen3-VL-8B
38.0
31.3
85.3
56.3
47.5
62.5
39.1
58.2
+ OmniSpatial [ 19 ]
38.1
31.0
79.7
56.4
47.1
63.0
43.7
58.0
Appendix
Table 6: Training corpus comparison with different model scales. ID: In-domain. OOD Avg. denotes the mean performance across the five OOD benchmarks.
Figure 16: Rank Correlations between metrics.
Category
Definition
Examples
ATT
Visual properties of an object (color, shape, state, etc.)
red object, sitting person, round object
POS
Spatial location of an object
left side of the image, in front of the man, on the desk
REL
Spatial relationship between objects (comparative, superlative)
highest, closest, farthest
Appendix
Table 7: Definitions and examples of the three spatial reasoning categories used in MultihopSpatial.
Figure 17: Screenshot of the data annotation interface. Annotators write a question-answer pair, select a view type, and draw a bounding box.
Figure 18: Screenshot of the data verification interface. Reviewers verify samples based on the criteria and provide corrections and comments.
Figure 19: Qualitative failure analysis on a 2-hop ego-centric question. GPT, Gemini, and Claude fail on perspective, position, and relation, respectively.
Figure 20: Qualitative failure analysis on a 2-hop ego-centric question. GPT and Gemini fail on position recognition, whereas Claude fails on attribute recognition.
Figure 21: Qualitative failure analysis on a 2-hop ego-centric question. All three models fail on relation recognition.
Figure 22: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to recognize the attribute condition.
Figure 23: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to correctly determine the spatial relation based on the 3D distance.
Figure 24: Qualitative failure analysis on a 2-hop exo-centric question. All three models fail to recognize the attribute condition.
Figure 25: Qualitative failure analysis on a 3-hop ego-centric question. All three models fail to correctly identify the perspective.
Figure 26: Qualitative failure analysis on a 3-hop ego-centric question. All three models fail to correctly identify the perspective.
Figure 27: Qualitative failure analysis on a 3-hop exo-centric question. GPT and Gemini fail on attribute condition recognition, whereas Claude fails on position condition recognition.
Figure 28: Qualitative failure analysis on a 3-hop exo-centric question. GPT and Claude fail to recognize the attribute condition, whereas Gemini fails on relation recognition.
Model
Reasoning Trace
GPT-5.2
The sitting person is at the table with a gray laptop directly in front of them, and a black cellphone lying on the table closer to the camera. The framed picture is on the back wall above and behind the person, and the white door is also on the back/right wall. Among these square or rectangular objects, the one farthest from the sitting person is the wall-mounted framed picture, since it is clearly behind them and farther than the nearby door and table objects. Answer: (a) framed picture with a drawing of a person Bounding Box: { "bbox_2d" : [0.364, 0.035, 0.471, 0.190]}
Gemini3.0-Flash
Based on the visual information in the image, the man is sitting at a table and facing toward the camera, as indicated by his face being visible and the back of his laptop’s lid facing the viewer. From the man’s perspective, the objects mentioned in the choices are positioned as follows: (d) the gray laptop is directly in front of him on the table, likely only a foot or two away; (b) the black cellphone is also on the table, farther from him than the laptop and closer to the camera; (a) the framed picture with a drawing of a person is on the wall directly behind him; and (c) the white door is located on a far wall in the background of the scene, significantly farther away from him in 3D space than any of the items on the table. Even though the door is technically behind the plane of his body, it is the most distant object from him among the choices given. In many visual question-answering tasks, “in front of them” is used broadly to describe objects visible in the scene or from the camera’s vantage point facing the person. Answer: (c) white door Bounding Box: { "bbox_2d" : [0.273, 0.784, 0.571, 0.929]}
Claude Sonnet 4.5
I need to identify which rectangular or square object in front of the sitting person is farthest from them. Let me analyze the objects mentioned in the choices: (a) framed picture with a drawing of a person, visible on the shelf in the background; (b) black cellphone, on the table in front of the person; (c) white door, visible on the right side of the image; and (d) gray laptop, on the table directly in front of the person. From the person’s perspective sitting at the table, the laptop is closest because it is right in front of them. The cellphone is on the table and relatively close. The framed picture is on the shelf above and behind the fireplace, while the white door is on the right wall. The framed picture appears to be the farthest object from the sitting person among the rectangular or square objects listed. It is located on a shelf in the background, above the fireplace. Answer: (a) framed picture with a drawing of a person Bounding Box: { "bbox_2d" : [0.3031, 0.0669, 0.4031, 0.1736]}
Appendix
Table 8: Full reasoning traces from GPT-5.2, Gemini3.0-Flash, and Claude Sonnet 4.5 for the example shown in Fig. 15 .
Vision-Language Models (VLMs) have been increasingly applied in real-world scenarios due to their outstanding understanding and reasoning capabilities. Although VLMs have already demonstrated impressive capabilities in common visual question answering and logical reasoning, they still lack the ability to make reasonable decisions in complex real-world environments. We define this ability as spatial logical reasoning, which not only requires understanding the spatial relationships among objects in complex scenes, but also the logical dependencies between steps in multi-step tasks. To bridge this gap, we introduce Spatial Logical Question Answering (SpatiaLQA), a benchmark designed to evaluate the spatial logical reasoning capabilities of VLMs. SpatiaLQA consists of 9,605 question answer pairs derived from 241 real-world indoor scenes. We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning. To address this issue, we propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs, thereby enhancing the spatial logical reasoning ability of VLMs, outperforming all previous methods. Code and dataset are available at https://github.com/xieyc99/SpatiaLQA.
Yuechen Xie, Xiaoyan Zhang, Yicheng Shan +6
Zhejiang University · The University of Sydney · ManyCore +2
Humans can effortlessly perceive spatial layouts, form cognitive representations, reason about spatial relations, and translate such reasoning into actions in everyday 3D environments. Although recent vision-language models (VLMs) have shown promising performance on observation-conditioned spatial perception and reasoning tasks, it remains unclear whether they can build coherent spatial understanding, act upon it, and refine their actions through multi-turn feedback. To study this problem, we introduce SpatialAct, a simulator-grounded benchmark for probing action-conditioned spatial reasoning in 3D scenes. Starting from the most challenging setting, Multi-turn Interactive Refinement, we further design its decomposed counterpart, Single-step Error Detection and Fix, together with five fundamental spatial ability tasks to diagnose the underlying causes of model failures. Experiments reveal a clear reasoning-to-action gap: current VLMs can perform well on isolated spatial reasoning tasks, but struggle to maintain coherent spatial beliefs and produce reliable actions during multi-turn feedback, substantially underperforming humans. These results suggest that current VLM agents still lack robust spatial state tracking under action-induced environment changes, even when low-level control is abstracted away.
Tianhui Liu, Jie Feng, Zhiheng Zheng +6
1The Hong Kong University of Science and Technology (Guangzhou) · 2Zhongguancun Academy · 3Tsinghua University +1
Metric reasoning is a critical and challenging task for Vision Language Models (VLMs), playing a pivotal role in embodied AI tasks such as robotic manipulation and autonomous navigation. However, current spatial reasoning remains bottlenecked by rigid pixel-level supervision; such localized optimization often compromises general multimodal intelligence, triggering performance degradation or catastrophic forgetting of broad reasoning capabilities. To address these limitations, we introduce Metric-Bench, a focused benchmark designed to guide metric-spatial reasoning using contextual information. By incorporating in-image reference objects with known physical dimensions, Metric-Bench guides models to implicitly learn the 2D-to-3D mapping without camera intrinsics. We further present MetricReasoner, a task-adapted reinforcement fine-tuning recipe for reference-grounded metric reasoning, using structured prompts and verifiable numerical rewards. Extensive experiments on Metric-Bench demonstrate that our approach significantly enhances spatial metric understanding, outperforming existing and even larger proprietary models by 43.1%, while improving downstream embodied performance over a spatial-specialized counterpart by 30.4% on RoboSpatial overall accuracy and 9.3% on ERQA, and additionally delivering consistent gains on general benchmarks (15.9% on V⋆Bench, 88.9% on BLINK), indicating that the proposed adaptation does not necessarily compromise general VLM capabilities.
Yuling Xi, Haokai Zhang, Muzhi Zhu +10
State Key Lab of CAD & CG, Zhejiang University, China · Zhejiang University of Technology · Hangzhou Dianzi University, China +1