Make Geometry Matter for Spatial Reasoning

Shihua Zhang; Qiuhong Shen; Shizun Wang; Tianbo Pan; Xinchao Wang

cs.CVMar 27, 2026

Make Geometry Matter for Spatial Reasoning

Authors: Shihua Zhang, Qiuhong Shen, Shizun Wang, Tianbo Pan, Xinchao Wang

Organizations: National University of Singapore, Singapore

Abstract

Empowered by large-scale training, vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited. Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs. Nevertheless, we observe that naive token fusion followed by standard fine-tuning in this line of work often leaves such geometric cues underutilized for spatial reasoning, as VLMs tend to rely heavily on 2D visual cues. In this paper, we propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens. GeoSR introduces two key components: (1) Geometry-Unleashing Masking, which strategically masks portions of 2D vision tokens during training to weaken non-geometric shortcuts and force the model to consult geometry tokens for spatial reasoning; and (2) Geometry-Guided Fusion, a gated routing mechanism that adaptively amplifies geometry token contributions in regions where geometric evidence is critical. Together, these designs unleash the potential of geometry tokens for spatial reasoning tasks. Extensive experiments on both static and dynamic spatial reasoning benchmarks demonstrate that GeoSR consistently outperforms prior methods and establishes new state-of-the-art performance by effectively leveraging geometric information. The project page is available at https://suhzhang.github.io/GeoSR/.

Figures & tables

Panel of Figure 1 — Figure 1: Geometry injection can be underutilized and even harmful for spatial reasoning. We compare three variants: w/o Geo. (removing the geometry branch), w/ Geo. (injecting geometry tokens via naive fusion with standard fine-tuning), and Ours. The plots reveal a counterintuitive yet reproducible pattern: in static scenes, w/ Geo. brings only marginal gains over w/o Geo., whereas in dynamic videos, w/ Geo. can even underperform the w/o Geo. baseline. This suggests that VLMs often fall back to appearance-driven shortcuts in 2D visual tokens and treat geometry as a dispensable side signal. These observations motivate GeoSR (Ours), which compels models to use geometry as actionable evidence and yields consistent improvements across both static and dynamic spatial reasoning.

Figure 2: Overview of the Geometry-Aware Framework for spatial reasoning. It augments a VLM with an additional geometry branch. A pretrained geometric tokenizer extracts geometry tokens $\boldsymbol{F}_{\mathrm{G}}$ from the input video, which are fused with standard vision tokens $\boldsymbol{F}_{\mathrm{V}}$ by a fusion module to form $\boldsymbol{F}$ . The VLM answers the query based on the fused evidence together with the text tokens $\boldsymbol{F}_{\mathrm{P}}$ . Snow and flame icons denote frozen and trainable components, respectively. This paradigm serves as our baseline. Our key observation is that naive fusion under standard fine-tuning can leave $\boldsymbol{F}_{\mathrm{G}}$ underutilized, which motivates GeoSR to make geometry really matter for spatial reasoning.

Figure 3: Overview of the proposed strategies in GeoSR . (a) Geometry-Unleashing Masking suppresses appearance shortcuts during training by masking a subset of vision tokens $\boldsymbol{F}_{\mathrm{V}}$ . For static settings, the mask is sampled randomly. For dynamic settings, bottleneck tokens $\boldsymbol{B}$ first attend to text tokens $\boldsymbol{F}_{\mathrm{P}}$ to obtain $\boldsymbol{F}_{\mathrm{B}}$ , which then attend to geometry tokens $\boldsymbol{F}_{\mathrm{G}}$ to produce question-relevant geometry evidence $\boldsymbol{Z}_{\mathrm{G}}$ and a relevance score used for TopK masking. (b) Geometry-Guided Fusion redistributes the compact evidence $\boldsymbol{Z}_{\mathrm{G}}$ back to token-level geometry features (if applicable) and applies a learned gate $\boldsymbol{\alpha}$ to control the contributions of masked vision features $\tilde{\boldsymbol{F}}_{\mathrm{V}}$ and geometry features $\tilde{\boldsymbol{F}}_{\mathrm{G}}$ , producing fused tokens $\boldsymbol{F}$ for the VLM backbone.

Models	Subtask Types								Avg.
	Numerical Questions				Multiple-Choice Questions
	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
Proprietary Models (API)
GPT-4o [ 13 ]	43.1	34.1	68.6	64.2	48.3	43.1	29.4	51.3	47.8
Gemini-1.5 Flash [ 27 ]	50.8	33.6	56.5	45.2	48.0	39.8	32.7	59.2	45.7
Gemini-1.5-Pro [ 27 ]	49.6	28.8	58.6	49.4	46.0	48.1	42.0	68.0	48.8

Table 1: Performance on static spatial reasoning. Best , second-best , and third-best results are highlighted.

Models	Subtask Types													Avg.
	Absolute						Relative						Non. Temp.
	Dist.	Dir.	Ori.	Spd.	Spd. Comp.	Dir. Pred.	Dist.	Dir.	Ori.	Spd.	Spd. Comp.	Dir. Pred.	Non. Temp.
Proprietary Models (API)
GPT-4o [ 13 ]	18.8	29.2	26.8	29.7	21.5	26.2	24.1	23.8	17.2	32.3	22.8	24.4	34.7	26.4
GPT-5 [ 24 ]	21.1	41.5	48.7	34.5	33.3	34.7	17.2	44.3	41.9	21.2	25.0	30.9	26.7	30.8
Gemini-2.5-Flash [ 6 ]	18.8	27.6	19.5	25.0	23.6	22.0	11.2	28.4	30.8	23.2	17.1	22.6	38.8	24.9

Table 2: Performance on dynamic spatial reasoning. Best , second-best , and third-best results are highlighted.

Setting	Module			Subtask Types								Avg.
Setting	Geo. Mask.	Geo. Fus.	Ori. Fus.	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order	Avg.
(a)	$\checkmark$	$\checkmark$		68.3	38.7	57.4	62.3	48.7	44.4	35.6	59.5	51.9
(b)	$\checkmark$		$\checkmark$	67.7	38.5	57.7	61.6	46.5	40.9	34.0	52.9	50.0
(c)	$\checkmark$			68.1	38.8	57.7	60.5	46.8	42.1	26.9	55.7	49.6
(d)		$\checkmark$		68.2	38.0	58.7	57.6	45.6	42.5	34.5	61.8	50.9
(e)			$\checkmark$	67.2	37.8	58.0	60.9	44.9	41.8	34.0	56.6	50.2

Table 3: Ablations on GeoSR components for static spatial reasoning.

Setting	Module			Subtask Types													Avg.
Setting	Geo. Mask.	Geo. Fus.	Ori. Fus.	Abs. Dist.	Abs. Dir.	Abs. Ori.	Abs. Spd.	Abs. Spd. Comp.	Abs. Dir. Pred.	Rel. Dist.	Rel. Dir.	Rel. Ori.	Rel. Spd.	Rel. Spd. Comp.	Rel. Dir. Pred.	Non. Temp.	Avg.
(a)	$\checkmark$	$\checkmark$		88.0	84.4	87.8	83.3	78.5	41.0	79.5	83.0	83.8	76.5	48.9	39.8	52.9	66.1
(b)	$\checkmark$		$\checkmark$	86.7	81.3	87.8	78.6	78.5	41.9	78.6	84.1	83.8	75.5	43.9	34.9	53.8	64.7
(c)	$\checkmark$			86.7	67.2	80.5	76.2	67.7	40.2	77.7	75.0	71.3	55.1	36.0	32.5	48.7	58.1
(d)		$\checkmark$		88.0	84.4	89.0	77.4	78.5	41.9	77.7	87.5	82.5	70.4	41.7	38.6	54.2	64.9
(e)			$\checkmark$	88.0	78.1	86.6	75.0	76.3	40.2	75.9	79.5	81.3	69.4	41.7	35.5	53.4	62.8

Table 4: Ablations on GeoSR components for dynamic spatial reasoning.

Table 8

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Visualization of the static spatial reasoning results on VSI-Bench [ 41 ] .

Figure 5: Visualization of the dynamic spatial reasoning results on DSR-Bench [ 53 ] .

Figure 6: Example illustrating ambiguity in the benchmark. The question asks for a speed comparison between the elephant and the person, but the visual evidence is ambiguous from a geometric perspective. The ground-truth answer is “(1) The latter is faster. (2) The former is faster.” However, in the video segment from 13.0s to 19.0s, it is difficult to determine which one is faster, since the person and the elephant appear to move at similar speeds.

Figure 7: Example illustrating ambiguity in the benchmark. The question asks about the relative spatial relation among the table and the two cartoon characters, and the ground-truth answer states that “No, during some moments, the table is not between them and is located behind the character at [102,28,364,319].” However, in the video segment from 8.5s to 11.0s, this relation remains difficult to judge. Although the purple character moves toward the table, the character is partially occluded by it, making it unclear whether the table is indeed behind the character.

Models	Subtask Types								Avg.
	Numerical Questions				Multiple-Choice Questions
	Obj. Cnt.	Abs. Dist.	Obj. Size	Room Size	Rel. Dist.	Rel. Dir.	Route Plan	Appr. Order
GeoSR	68.3	38.7	57.4	62.3	48.7	44.4	35.6	59.5	51.9
+QFormer	68.6	38.8	57.8	60.6	49.0	43.5	35.6	60.2	51.8

Appendix

Table 7: Performance of GeoSR and GeoSR with QFormer on static spatial reasoning.

Visual Spatial Reasoning

Visual spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and physical change from images and video.

See all 250 papers

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
Jun 16, 2026Yatai Ji, An-Chieh Cheng, Yang Fu +13Visual Spatial Reasoning 3D Spatial Reasoning
The University of Hong Kong · NVIDIA · University of California, San Diego
Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning
Sep 5, 2026Gege Zhang, Shuaicheng Niu, Gang Dai +2Spatial Reasoning 3D Spatial Reasoning
Nanyang Technological University Singapore, Singapore · South China University of Technology Guangzhou, China
GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning
May 21, 2026Deshui Miao, Xingsen Huang, Yameng Gu +3Visual Spatial Reasoning Spatiotemporal Reasoning
Harbin Institute of Technology, Shenzhen · Hainan University · Pengcheng Laboratory +2

Jun 16, 2026J/K move · Enter open · S save

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

Yatai Ji, An-Chieh Cheng, Yang Fu, Yukang Chen+12

The University of Hong Kong · NVIDIA · University of California, San Diego

Visual Spatial Reasoning 3D Spatial Reasoning VLM Reasoning Visually Grounded Reasoning RL for VLMs

Open paper AskPDF arXiv

Spatial VLMs have made substantial progress in geometric perception, yet complex spatial reasoning requiring multi-step inference over depth, distance, and scene relations remains challenging. Moreover, different spatial queries call for fundamentally different strategies: some are best addressed through purely linguistic, step-by-step deduction, while others require explicit 3D grounding before quantitative inference. We present Dual-Path Spatial Reasoning via Reinforcement Learning for Spatial VLMs (SR-REAL), a unified framework that equips a spatial VLM with two complementary reasoning paths: Language-Only Reasoning (LOR), which performs step-by-step linguistic deduction, and Detect-Then-Reason (DTR), which detects 3D geometric cues (e.g., centers or bounding boxes) via region tokens before explicit geometric inference. SR-REAL begins with a cold-start supervised fine-tuning stage that constructs LOR and DTR chain-of-thought supervision and exposes a region-to-3D interface, followed by RL that optimizes the policy model with accuracy and format rewards; for DTR, a discrete center-based detection reward further refines geometric alignment. Across diverse spatial benchmarks, SR-REAL significantly outperforms spatial VLM baselines: (i) a single RL-trained model supports both reasoning paths, with DTR excelling in region-aware tasks through precise 3D localization and LOR enhancing general spatial reasoning; (ii) jointly training both paths fosters mutual reinforcement; (iii) high-quality, blended cold-start data is crucial for stable RL optimization; and (iv) the model generalizes across datasets and domains without per-task tuning, demonstrating positive transfer between LOR and DTR.

Abstract

Visual Spatial Reasoning

Explore similar work

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning

GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning