Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models

Junjie Wang; Zhihao Yuan; Yunhan Tang; Sheng Xu; Jiaqi Mao; Xiaoyu Ye; Yueru Luo; Xu Yan

cs.CVOct 21, 2026

Think-Capture: Active 3D Geometry Acquisition for Spatial Reasoning in Vision-Language Models

Authors: Junjie Wang, Zhihao Yuan, Yunhan Tang, Sheng Xu, Jiaqi Mao, Xiaoyu Ye, Yueru Luo, Xu Yan,

Organizations: FNii-Shenzhen · CUHK-Shenzhen · University of Electronic Science and Technology of China · University College Dublin

Abstract

Vision-Language Models (VLMs) excel at 2D recognition but often fail to maintain consistent 3D relations across sparse viewpoints, leading to spatial hallucinations under occlusion and perspective changes. Existing methods commonly fuse all frames with dense geometry, incurring substantial computation and context overhead while encouraging blind reliance on imperfect estimators. We present Think-Capture, a query-conditioned framework that first identifies task-critical objects and then acquires their geometry on demand. A 3D toolkit lifts zero-shot masks into a common coordinate frame, and a Symbolic 3D State Abstraction compresses object and camera states into structured text. Hierarchical Group Relative Policy Optimization (GRPO) trains the VLM on cases in which geometry either helps or misleads, teaching it when to use tool evidence and when to fall back on visual cues. Think-Capture-4B reaches 54.4% on MindCube-Tiny (+21.5 points over its backbone), 39.1% on the MMSI-Bench positional subset (+7.9), and 42.8% on ViewSpatial (+5.2). It also remains robust under coordinate and mask perturbations. The full stack fits one 24 GB GPU and serves as an asynchronous high-level reasoner on an Agibot G1. Our code is available at https://github.com/819346675/Think-Capture.

Figures & tables

Figure 1: Spatial reasoning paradigms. (a) Passive methods fuse 2D and 3D features for every query. (b) Think-Capture first selects task-critical targets, acquires their geometry, and reasons jointly over visual tokens and explicit 3D state.

Figure 2: Think-Capture overview. Stage 1 selects query-relevant targets. SAM3 and $\pi^{3}$ produce the object and camera state used with the images in Stage 2. In the training panel, the accuracy-branch labels denote $r_{\mathrm{acc}}$ , with $+1/+2$ for correct positive/negative examples (Sec. 3.4 ). The example annotations illustrate the interface.

Proprietary Models
Model	MindCube-Tiny				MMSI-Bench-Positional Relationship Subset							ViewSpatial
Model	Among	Around	Rotation	Overall	Cam-Cam	Cam-Obj	Obj-Reg	Cam-Reg	Reg-Reg	Obj-Obj	Overall	Overall
GPT-4o	44.6	58.4	39.0	46.8	34.4	19.8	37.6	27.7	23.5	24.5	28.0	35.0
Gemini-2.5 Pro	40.2	56.7	84.3	52.5	39.7	45.3	35.2	43.3	39.5	31.9	39.0	43.6
Doubao-1.5	46.0	38.4	75.1	49.7	25.8	33.7	49.4	39.8	33.3	33.0	35.6	-
GLM-4.5V	44.1	23.5	46.9	39.7	38.7	30.2	40.0	39.8	29.6	35.1	35.6	-

Table 1: Comprehensive performance comparison on MindCube-Tiny, MMSI-Bench (Positional Relationship Subset), and ViewSpatial-Bench. Metrics are reported as accuracy (%). We grey out the proprietary models. The best performance among open-source models is highlighted in bold . The improvement of Think-Capture over the baseline ( Qwen3-VL-4B ) is shown in blue below the score.

Model setup	Clean	$\pi^{3}$ coordinate noise		SAM3 mask dropout
Model setup	Clean	0.2 m	0.4 m	15%	30%
Zero-shot tool use	39.9	32.8	26.7	37.2	36.2
w/o hierarchical GRPO	51.2	44.3	40.3	49.2	49.0
Think-Capture (full)	54.4	50.7	46.2	53.8	52.1

Table 2: Robustness to controlled tool perturbations on MindCube-Tiny (accuracy, %). Coordinate noise is zero-mean Gaussian perturbation of

\pi^{3}

outputs; mask noise randomly drops SAM3 foreground pixels. Hierarchical GRPO improves both clean accuracy and tolerance to corrupted evidence.

Size	MindCube		MMSI full
Size	Base	Ours	Base	Ours
Qwen3-VL-2B	29.3	49.2 (+19.9)	25.8	34.7 (+8.9)
Qwen3-VL-4B	32.9	54.4 (+21.5)	28.9	38.9 (+10.0)
Qwen3-VL-8B	33.4	54.2 (+20.8)	29.6	38.4 (+8.8)

Table 3: Backbone scaling on MindCube-Tiny and full MMSI-Bench (accuracy, %). Parentheses give absolute gains over each matched backbone.

Stage-2 input	MindCube	MMSI full	ViewSpatial
2D boxes	36.9	31.8	37.9
Symbolic 3D state	54.4	38.9	42.8

Table 4: Matched visual-prompting control (accuracy, %). Both variants use the same Stage 1 target proposals and SAM3 masks.

Figure 3: Resolving egocentric spatial ambiguity on MindCube-Tiny. The baseline equates the background visible beyond the blue table with the observer’s rear and answers “sofa.” Think-Capture identifies the blue table as the facing anchor, relates the surrounding furniture in the common 3D frame, and correctly recovers the unseen seating bench behind the observer.

Base Model	TC	GRPO	Format	Tokens	Among	Around	Rotation	Overall
Qwen3-VL-4B			-	-	26.5	44.6	37.5	32.9
Qwen3-VL-4B		✓	-	-	28.9	41.2	36.4	33.3
Qwen3-VL-4B	✓		-	205	42.0	40.0	33.5	39.9
Qwen3-VL-4B	✓	✓	Unified	202	50.5	57.6	38.0	49.8
Qwen3-VL-4B	✓	✓	Template	100	57.6	54.1	45.0	54.4

Table 5: Ablation study on MindCube-Tiny. TC denotes Think-Capture; Tokens is the mean length of the 3D representation.

Figure 4: Cross-view perspective taking on MMSI-Bench. The baseline describes the mirror relative to the scene or to the wrong view and predicts “back left.” Think-Capture’s response uses the photographer-relative description and gives the correct front-left answer.

Figure 5: Real-world deployment on Agibot G1. Four snapshots show the initial observation, target approach, and medicine grasp. Think-Capture uses task-relevant symbolic geometry to supply a global waypoint to the high-frequency StarVLA motor policy.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

GRPO Configuration
Parameter	Value
Base Model	Qwen3-VL-4B-Instruct
Trainable Params	4,022M / 4,438M (vision encoder frozen)
Optimizer	AdamW
Learning Rate	$5\times 10^{-7}$ , Cosine Schedule + 50-step Warmup
Generations ( $G$ )	8 per prompt

Appendix

Table S1 : Detailed hyperparameters for GRPO training.

Figure S1 : GRPO training dynamics: accuracy on positive/negative samples and mean reward.

Figure S2 : Training loss over the GRPO optimization trajectory.

Model setup	Clean	$\pi^{3}$ noise		SAM3 noise
Model setup	Clean	0.2 m	0.4 m	15%	30%
Zero-shot model	39.9	32.8	26.7	37.2	36.2
w/o hierarchical GRPO	51.2	44.3	40.3	49.2	49.0
Ours (full)	54.4	50.7	46.2	53.8	52.1

Appendix

Table S2 : Accuracy (%) under controlled tool perturbations on MindCube-Tiny.

Model size	MindCube-Tiny		MMSI full
Model size	Base	Ours	Base	Ours
Qwen3-VL-2B	29.3	49.2 (+19.9)	25.8	34.7 (+8.9)
Qwen3-VL-4B	32.9	54.4 (+21.5)	28.9	38.9 (+10.0)
Qwen3-VL-8B	33.4	54.2 (+20.8)	29.6	38.4 (+8.8)

Appendix

Table S3 : Backbone scaling on MindCube-Tiny and full MMSI-Bench. Parentheses show absolute gains over the corresponding base model.

Stage-2 input	MindCube	MMSI-Bench	ViewSpatial
2D boxes	36.9	31.8	37.9
Symbolic 3D state	54.4	38.9	42.8

Appendix

Table S4 : Comparison with 2D-box visual prompting (accuracy, %).

Stage	Component	Avg. Time	Description
Stage 1	Planner & Proposal	6.8s	VLM analyzes query & proposes targets
Tool	Geometry & Perception	3.1s	Parallel $\pi^{3}$ + targeted SAM3
Stage 2	Spatial Reasoning	11.9s	CoT reasoning
Total	End-to-End	21.8s	Median: 20.4s

Appendix

Table S5 : Mean inference latency on MindCube-Tiny.

Visual Spatial Reasoning

Visual spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and physical change from images and video.

See all 251 papers

3D-Aware VLMs with Implicit and Explicit Geometries
Jul 23, 2026Wenhao Li, Xueying Jiang, Quanhao Qian +43D Spatial Reasoning 3D Object Detection
1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1
Grounded 3D-Aware Spatial Vision-Language Modeling
May 28, 2026An-Chieh Cheng, Yang Fu, Yatai Ji +12Visually Grounded Reasoning 3D Spatial Reasoning
UCSD · NVIDIA · MIT
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
May 28, 2026Chun-Hsiao Yeh, Shengyi Qian, Manchen Wang +3Vision-Language Models 3D Spatial Reasoning
FAIR at Meta · UC Berkeley · HKU

Jul 23, 2026J/K move · Enter open · S save

3D-Aware VLMs with Implicit and Explicit Geometries

Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao+3

1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab+1

3D Spatial Reasoning 3D Object Detection 3D Representation Learning 3D VLMs 3D Visual Grounding

Open paper AskPDF arXiv

Despite rapid progress, most existing vision-language models (VLMs) built from 2D visual inputs often struggle when handling various 3D tasks that require fine-grained spatial understanding and reasoning. To bridge this gap, we present VLM-IE3D, a unified framework that enhances the 3D spatial awareness of VLMs by equipping them with both implicit and explicit 3D geometries learned from RGB videos. Our VLM-IE3D introduces Implicit Geometry Tokens (IGTs) that capture high-level geometric priors from input videos, as well as complementary Explicit Geometry Tokens (EGTs) that encode detailed geometric structures from reconstructed 3D attributes. On top of that, VLM-IE3D comes with a 3D-aware adapter that effectively fuses the two types of geometric representations with 2D visual cues. This RGB-only design injects strong 3D inductive biases for fine-grained spatial understanding and reasoning without requiring any additional 3D inputs. Extensive experiments show that VLM-IE3D achieves superior performance consistently across various 3D tasks including 3D video detection, 3D visual grounding, 3D dense captioning, and spatial reasoning. Code and models are available at https://github.com/Vegetebird/VLM-IE3D.

Abstract

Visual Spatial Reasoning

Explore similar work

3D-Aware VLMs with Implicit and Explicit Geometries

Grounded 3D-Aware Spatial Vision-Language Modeling

Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning