SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models

Yuechen Xie; Xiaoyan Zhang; Yicheng Shan; Hao Zhu; Rui Tang; Rong Wei; Mingli Song; Yuanyu Wan

cs.CVFeb 24, 2026

SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models

Authors: Yuechen Xie, Xiaoyan Zhang, Yicheng Shan, Hao Zhu, Rui Tang, Rong Wei, Mingli Song, Yuanyu Wan,

Organizations: Zhejiang University · The University of Sydney · ManyCore · State Key Laboratory of Blockchain and Security, Zhejiang University ·

Abstract

Vision-Language Models (VLMs) have been increasingly applied in real-world scenarios due to their outstanding understanding and reasoning capabilities. Although VLMs have already demonstrated impressive capabilities in common visual question answering and logical reasoning, they still lack the ability to make reasonable decisions in complex real-world environments. We define this ability as spatial logical reasoning, which not only requires understanding the spatial relationships among objects in complex scenes, but also the logical dependencies between steps in multi-step tasks. To bridge this gap, we introduce Spatial Logical Question Answering (SpatiaLQA), a benchmark designed to evaluate the spatial logical reasoning capabilities of VLMs. SpatiaLQA consists of 9,605 question answer pairs derived from 241 real-world indoor scenes. We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning. To address this issue, we propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs, thereby enhancing the spatial logical reasoning ability of VLMs, outperforming all previous methods. Code and dataset are available at https://github.com/xieyc99/SpatiaLQA.

Figures & tables

Figure 1: Common VQA [ 21 ] typically involve recognizing visual content and factual knowledge, while common logical reasoning [ 71 ] focuses on abstract, symbolic problem-solving. Spatial logical reasoning, in contrast, requires integrating both spatial understanding and multi-step logical reasoning to accomplish tasks in real-world scenes.

Benchmarks	Modality	SU	LR	Real Scene	Answer Type	Multi-step	Precondition	LLM/VLM Scoring	Size
CLEVR [ 24 ]	I	✓	✗	✗	Open	✗	✗	✗	853.6K
GQA [ 21 ]	I	✓	✗	✓	Open	✗	✗	✗	$>$ 1M
MMBench [ 36 ]	I	✓	✗	✓	MC	✗	✗	✓	3.2K
Spatial-MM [ 50 ]	I	✓	✗	✓	MC	✗	✗	✗	2.3K
SpatialRGPT-Bench [ 14 ]	I	✓	✗	✓	MC	✗	✗	✗	1.5K
Open3DVQA [ 72 ]	I	✓	✗	✗	Open	✗	✗	✓	9.0K

Table 1: Comparison between SpatiaLQA and other benchmarks ( VQA , logical reasoning and EQA ). ‘SU’ and ‘LR’ denote spatial understanding and long-range reasoning, respectively. ‘I’, ‘P’ and ‘V’ denote image, point cloud and video, respectively. ‘Open’ and ‘MC’ stand for open-vocabulary and multiple-choice respectively. ‘Multi-step’ specifies whether the answers involve multiple steps. ‘Precondition’ indicates whether each step in the answer is annotated with its preconditions (i.e., which steps must be completed beforehand).

Figure 2: Prompt template and examples of several indoor scenes.

Figure 3: The distributions of answer step counts, scene categories, and partial object categories in SpatiaLQA. The x-axes of the three plots represent the number of answer steps, indoor scene categories, and object categories, while the y-axes indicate the number of samples.

Figure 4: The data collection pipeline for SpatiaLQA. Note that although the graph expansion augmentation in the figure is applied only to the data from subgraph extraction augmentation, we actually also applied graph expansion augmentation to the manually annotated data.

Figure 5: The matching process between the predicted and annotated steps. We first use GPT-4o to match the predicted steps and annotated steps in pairs based on the image (allowing one-to-many matches), resulting in a matching matrix. Then, we apply the Hungarian algorithm to filter the matching matrix, removing redundant matches to achieve the maximum one-to-one matches.

	Size	$R_{c}$	$P_{c}$	$F_{c}$	$R_{p}$	$P_{p}$	$F_{p}$
human	-	97.6	97.6	97.6	92.3	92.7	92.5
BLIP2-OPT [ 27 ]	4B	24.6	99.9	39.5	0.0	0.0	-
BLIP2-Flan-T5-xl [ 27 ]	4B	24.0	74.3	36.3	0.1	0.2	0.1
BLIP2-Flan-T5-xxl [ 27 ]	12B	41.3	67.7	51.3	0.0	0.2	0.0
LLaVA1.5-7B [ 34 ]	7B	40.3	39.8	40.0	6.4	9.0	7.5
LLaVA1.5-13B [ 34 ]	13B	39.1	49.4	42.5	9.1	19.3	12.3

Table 2: The evaluation results of 41 VLMs. ‘Ins’ indicates that the model is an ‘instruction-tuned’ version. Recall and precision are used as reference metrics and are marked in gray . The best and second-best F1 scores (excluding human results) are marked in red and blue , respectively, and we use a dashed line to separate open-source VLMs (above) and proprietary VLMs (below).

Figure 6: Evaluation results of human and different scoring VLMs. The x-axis represents the eight representative VLMs being evaluated. Each point with the same marker shape denotes the F1 scores obtained by the same scoring VLM or by human evaluators. The solid lines indicate the F1 scores for content, while the dashed lines represent the F1 scores for preconditions.

Scoring VLMs	$\rho_{c}$	$\rho_{p}$	$s$	$\Delta$
Qwen3-VL-8B-Ins	0.96	0.89	0.98	13.4
InternVL3.5-14B-Ins	0.96	0.78	0.99	14.9
Qwen-VL-Max	0.97	0.86	0.99	3.5
GPT-4o	0.99	0.96	0.99	3.0

Table 3: Comparison between scoring VLM evaluation results and human evaluation results.

\rho_{c}

(

\rho_{p}

) represents the Pearson correlation coefficient between the content (precondition) F1 scores obtained by VLMs and those obtained by humans.

s

(

\Delta

) denotes the cosine similarity (mean absolute error) between the F1 scores obtained by VLMs and those obtained by humans.

Figure 7: Dimension-wise analysis. Each dot represents the F1 score of a VLM under specific conditions, including different numbers of answer steps, annotation methods, and scene categories. ‘Human’, ‘Subgraph’, and ‘Graph Expansion’ correspond to data from ‘manual annotations’, ‘subgraph extraction augmentation’, and ‘graph expansion augmentation’, respectively.

Figure 8: The overview of RSGAR. Scene graph generation and question answering are performed by the same VLM.

Table 4: (a) The results of various methods. Bold and underlined scores denote the best and second-best results. ‘+ depth’, ‘+ seg’, and ‘+ depth&seg’ represent vanilla reasoning enhanced with depth maps, segmentation maps, and both, respectively. (b) RSGAR’s

F_{c}

and

F_{p}

across answer step counts. The values in parentheses represent the changes relative to vanilla reasoning.

Table 5: Ablation studies on GPT-4o. The default settings are indicated in italics . ‘seg’ and ‘depth’ denote the segmentation map and depth map, respectively.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Method	$\tau$	$F_{c}$	$F_{p}$
Vanilla Reasoning	27.4h	67.4	25.1
+ depth	27.9h	64.1	19.0
+ seg	29.4h	50.3	14.7
+ seg&depth	30.1h	50.5	14.5
PhysAgent	31.5h	64.7	22.2
CoT	27.5h	67.6	27.0

Appendix

Table A1: Efficiency analysis.

\tau

is the time taken by each method to perform one validation on SpatiaLQA.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 841 papers

SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Aug 3, 2026Jing Wu, Jianhua Wu, Jiayi Guan +5Vision-Language Models Visual Spatial Reasoning
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher
MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model
Mar 19, 2026Youngwan Lee, Soojin Jang, Yoorhim Cho +3Spatial Reasoning Benchmarks Spatial Reasoning
Electronics and Telecommunications Research Institute, South Korea · Korea Advanced Institute of Science and Technology, South Korea · Sungkyunkwan University, South Korea +1
Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA
Mar 13, 2026Nahid Alam, Leema Krishna Murali, Siddhant Bharadwaj +7Spatial Reasoning in Language Models Visual Spatial Reasoning
Cohere Labs Community · Eisai Inc. · Indian Institute of Science, Bangalore +4

Aug 3, 2026J/K move · Enter open · S save

SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

Jing Wu, Jianhua Wu, Jiayi Guan, Jiahong Chen+4

Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher

Vision-Language Models Visual Spatial Reasoning VLM Reasoning Spatial Reasoning in Language Models

Open paper AskPDF arXiv

Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient Spatio-vision Language Models (SpatioLM), that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at \faGithub~spatio-lm.

Abstract

VLM Reasoning

Explore similar work

SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model

Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA