Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning

Gege Zhang; Shuaicheng Niu; Gang Dai; Lei Sun; Shuangping Huang

cs.CVSep 5, 2026

Geometry-Aware Test-Time Learning for Quantitative Spatial Reasoning

Authors: Gege Zhang, Shuaicheng Niu, Gang Dai, Lei Sun, Shuangping Huang

Organizations: Nanyang Technological University Singapore, Singapore · South China University of Technology Guangzhou, China

Abstract

Quantitative spatial reasoning in visual-language models (VLMs) aims to infer spatial distances and directional relationships among objects in 3D space from a 2D image and a natural language query. Despite recent progress, VLM spatial reasoning remains brittle under distribution shifts, largely due to the high cost of 3D supervision. As a result, models often produce inconsistent or contradictory predictions when faced with novel object configurations or rephrased spatial queries, revealing a misalignment between learned representations and underlying geometry. To address this, we propose TTL-SR, a geometry-aware Test-Time Learning framework for quantitative Spatial Reasoning that leverages geometric consistency constraints and unlabeled test data to adapt models to target domains. Specifically, TTL-SR augments the input query with geometrically coupled auxiliary queries, filters unreliable predictions via adaptive geometric triggering to construct structured token-level pseudo-labels, and updates model parameters under a geometry-aware multi-objective loss using only test data. Experimental results demonstrate that TTL-SR significantly boosts spatial reasoning performance, yielding 6.47% and 9.41% accuracy gains for Qwen3-VL-4B-Instruct and SpatialRGPT-VILA-1.5-8B on Q-Spatial-ScanNet dataset, respectively.

Figures & tables

Figure 1. Comparison of spatial reasoning paradigms. (a) Standard VLM inference is static, often leading to inconsistent and physically implausible predictions. (b) Our proposed approach leverages geometric consistency at test time, enabling coherent and physically plausible spatial reasoning.

Figure 2. Overview of the TTL-SR pipeline. The framework operates in three phases: (1) Spatial Query Augmentation generates geometrically coupled auxiliary queries via predefined templates; (2) Adaptive Geometric Consistency Modeling filters raw numerical estimates through reliability triggers to construct structured token-level pseudo-labels; and (3) Test-Time Optimization iteratively updates VLM parameters through a multi-objective loss.

Figure 3. Preliminary analysis of feature space evolution and reasoning performance on the Q-Spatial-ScanNet dataset ( Liao et al., 2024 ) . Bars (left y-axis) show the average pairwise

L_{2}

distance between sample features extracted from the last hidden layer of Qwen3-VL-4B-Instruct ( Yang et al., 2025a ) , while markers and lines (right y-axis) indicate the corresponding accuracy, respectively.

Method	Q-Spatial-ScanNet						Q-Spatial++	SPAR-Bench
Method	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Average	Average	Dist. oo Average
Doubao-Seed-1.8	22.2 / 0.84	6.7 / 3.44	13.8 / 1.40	56.5 / 0.43	59.1 / 0.24	24.71	22.77	44.6 / 0.37
GPT-5.2	47.2 / 0.38	35.0 / 1.12	34.5 / 0.70	60.9 / 0.29	63.6 / 0.29	44.71	26.73	49.4 / 0.29
Gemini-3-Flash	58.3 / 0.24	28.3 / 0.58	44.8 / 0.66	60.9 / 0.34	90.9 / 0.13	50.00	38.61	58.4 / 0.28
Qwen3-VL-Plus	38.9 / 0.52	16.7 / 1.74	24.1 / 1.86	56.5 / 0.36	77.3 / 0.21	35.88	23.76	42.1 / 0.41
Qwen3-VL-2B-Instruct	38.9 / 0.31	31.7 / 1.30	20.7 / 0.66	52.2 / 0.38	40.9 / 0.55	35.29	31.68	39.9 / 0.37

Table 1. Quantitative comparison on Q-Spatial-ScanNet, Q-Spatial++, and the SPAR-Bench object-to-object distance task. Results are reported as Accuracy (%)

\uparrow

/ MARE

\downarrow

. The Average column denotes the mean accuracy across all categories.

Method	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Direction	Average
Doubao-Seed-1.8	24.3 / 0.67	21.3 / 1.38	33.0 / 0.52	57.1 / 0.32	68.4 / 2.90	55.1 / 55.79 ∘	43.12
GPT-5.2	15.5 / 0.57	27.1 / 1.14	20.8 / 0.46	59.7 / 0.33	71.4 / 1.91	64.5 / 42.06 ∘	42.72
Gemini-3-Flash	17.6 / 0.59	31.2 / 0.75	17.9 / 0.52	63.9 / 0.29	68.4 / 1.77	57.0 / 47.10 ∘	42.72
Qwen3-VL-Plus	31.8 / 0.48	32.0 / 1.11	31.1 / 0.56	51.1 / 0.38	69.2 / 1.79	49.1 / 60.00 ∘	44.25
Qwen3-VL-2B-Instruct	16.2 / 0.55	30.3 / 0.51	17.0 / 0.61	32.3 / 0.45	37.6 / 2.10	30.7 / 78.10 ∘	27.32
Qwen3-VL-4B-Instruct	29.1 / 0.51	35.3 / 0.96	29.3 / 0.74	39.9 / 0.42	64.7 / 1.25	41.1 / 61.70 ∘	40.05

Table 2. Quantitative comparison of spatial reasoning performance on SpatialRGPT-Bench. Results are reported as Accuracy (%)

\uparrow

/ MARE

\downarrow

. The Direction metric are reported as Accuracy (%)

\uparrow

/ Mean Angular Error ( ∘ )

\downarrow

Figure 4. Qualitative comparison of spatial reasoning between the baseline Qwen3-VL-4B-Instruct and our TTL-SR.

Ent	KL	Geo	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Average
			52.8 / 0.34	21.7 / 1.94	34.5 / 1.33	56.5 / 0.44	59.1 / 0.46	40.00
✓			58.3 / 0.28	23.3 / 1.44	37.9 / 1.20	60.9 / 0.50	45.5 / 0.44	40.18
✓	✓		55.6 / 0.29	21.7 / 1.61	37.9 / 1.23	60.9 / 0.49	50.0 / 0.44	40.59
	✓	✓	55.6 / 0.25	28.3 / 1.62	37.9 / 1.38	52.2 / 0.46	59.1 / 0.41	42.94
✓		✓	61.1 / 0.25	25.0 / 1.58	37.9 / 1.25	52.2 / 0.45	63.6 / 0.30	43.53
✓	✓	✓	63.9 / 0.24	28.3 / 1.48	34.5 / 1.23	60.9 / 0.43	68.2 / 0.31	46.47

Table 3. Ablation study of TTL-SR loss components.

Constraints	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Average
Baseline (Ent+KL)	55.6 / 0.29	21.7 / 1.61	37.9 / 1.23	60.9 / 0.49	50.0 / 0.44	40.59
w/ $\mathcal{L}_{geo}$ (Same)	58.3 / 0.26	28.3 / 1.61	34.5 / 1.42	56.5 / 0.46	54.6 / 0.41	42.94
w/ $\mathcal{L}_{geo}$ (Direct)	58.3 / 0.29	23.3 / 1.61	31.0 / 1.57	60.9 / 0.46	59.1 / 0.40	41.76
w/ $\mathcal{L}_{geo}$ (Vert.)	61.1 / 0.27	25.0 / 1.66	37.9 / 1.30	60.9 / 0.47	50.0 / 0.44	42.94
w/ $\mathcal{L}_{geo}$ (Horiz.)	58.3 / 0.29	26.7 / 1.65	37.9 / 1.23	60.9 / 0.50	50.0 / 0.44	42.94
w/ $\mathcal{L}_{geo}$ (Width)	58.3 / 0.28	25.0 / 1.64	31.0 / 1.39	60.9 / 0.48	50.0 / 0.44	41.18

Table 4. Impact of individual geometric constraints.

Method	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Average
w/o AT	61.1 / 0.25	23.3 / 1.50	34.5 / 1.27	56.5 / 0.44	68.2 / 0.32	43.53
w/o SPL	58.3 / 0.27	28.3 / 1.64	27.6 / 1.45	60.9 / 0.45	63.6 / 0.32	43.53
w/o UL	58.3 / 0.25	28.3 / 1.57	31.0 / 1.46	60.9 / 0.43	63.6 / 0.32	44.12
Ours	63.9 / 0.24	28.3 / 1.48	34.5 / 1.23	60.9 / 0.43	68.2 / 0.31	46.47

Table 5. Component-wise ablation of TTL-SR framework.

Param.	Value	Direct Dist.	Horiz. Dist.	Vert. Dist.	Width	Height	Average
$\tau$	0.00	61.1 / 0.25	23.3 / 1.50	34.5 / 1.27	56.5 / 0.44	68.2 / 0.32	43.53
	0.10	58.3 / 0.26	26.7 / 1.52	37.9 / 1.23	56.5 / 0.44	63.6 / 0.31	44.12
	0.20	63.9 / 0.24	28.3 / 1.48	34.5 / 1.23	60.9 / 0.43	68.2 / 0.31	46.47
	0.40	61.1 / 0.26	25.0 / 1.59	34.5 / 1.40	52.2 / 0.46	63.6 / 0.32	42.94
$\tau_{label}$	0.10	61.1/ 0.25	25.0 / 1.56	37.9 / 1.25	47.8 / 0.46	68.2 / 0.33	43.53
	0.20	63.9 / 0.24	28.3 / 1.48	34.5 / 1.23	60.9 / 0.43	68.2 / 0.31	46.47

Table 6. Sensitivity analysis of key hyper-parameters in TTL-SR, including the trigger threshold

\tau

, the soft label tolerance interval

\tau_{label}

, and the sampling top-

k

for UL.

Spatial Reasoning

Spatial reasoning studies how systems represent geometry, relations, viewpoints, and motion to understand scenes, plan actions, and solve spatial tasks.

See all 159 papers

Make Geometry Matter for Spatial Reasoning
Mar 27, 2026Shihua Zhang, Qiuhong Shen, Shizun Wang +2Visual Spatial Reasoning 3D Spatial Reasoning
National University of Singapore, Singapore
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
Jun 16, 2026Yatai Ji, An-Chieh Cheng, Yang Fu +13Visual Spatial Reasoning 3D Spatial Reasoning
The University of Hong Kong · NVIDIA · University of California, San Diego
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
May 28, 2026Chun-Hsiao Yeh, Shengyi Qian, Manchen Wang +3Vision-Language Models 3D Spatial Reasoning
FAIR at Meta · UC Berkeley · HKU

Mar 27, 2026J/K move · Enter open · S save

Make Geometry Matter for Spatial Reasoning

Shihua Zhang, Qiuhong Shen, Shizun Wang, Tianbo Pan+1

National University of Singapore, Singapore

Visual Spatial Reasoning 3D Spatial Reasoning Vision-Language Models 3D VLMs

Open paper AskPDF arXiv

Empowered by large-scale training, vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited. Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs. Nevertheless, we observe that naive token fusion followed by standard fine-tuning in this line of work often leaves such geometric cues underutilized for spatial reasoning, as VLMs tend to rely heavily on 2D visual cues. In this paper, we propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens. GeoSR introduces two key components: (1) Geometry-Unleashing Masking, which strategically masks portions of 2D vision tokens during training to weaken non-geometric shortcuts and force the model to consult geometry tokens for spatial reasoning; and (2) Geometry-Guided Fusion, a gated routing mechanism that adaptively amplifies geometry token contributions in regions where geometric evidence is critical. Together, these designs unleash the potential of geometry tokens for spatial reasoning tasks. Extensive experiments on both static and dynamic spatial reasoning benchmarks demonstrate that GeoSR consistently outperforms prior methods and establishes new state-of-the-art performance by effectively leveraging geometric information. The project page is available at https://suhzhang.github.io/GeoSR/.

Abstract

Spatial Reasoning

Explore similar work

Make Geometry Matter for Spatial Reasoning

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning