Think3D: Thinking with Space for Spatial Reasoning

Zaibin Zhang; Yuhan Wu; Lianjie Jia; Yifan Wang; Zhongbo Zhang; Yijiang Li; Binghao Ran; Fuxi Zhang

cs.CVJan 19, 2026

Think3D: Thinking with Space for Spatial Reasoning

Authors: Zaibin Zhang, Yuhan Wu, Lianjie Jia, Yifan Wang, Zhongbo Zhang, Yijiang Li, Binghao Ran, Fuxi Zhang,

Organizations: Dalian University of Technology · University of California San Diego · Independent Researcher · University of Oxford

Abstract

While Vision-Language Models (VLMs) excel at 2D visual understanding, they remain constrained by 2D-centric paradigm that severely limits genuine 3D spatial reasoning. To bridge this gap, we introduce Think3D, a novel framework that equips VLM agents with interactive, 3D chain-of-thought reasoning capabilities. By integrating a suite of 3D manipulation tools, Think3D transforms perception into active spatial exploration, mirroring human geometric reasoning. Think3D consistently improves proprietary models, including GPT-4.1 and Gemini 2.5 Pro, across BLINK Multi-view, MindCube-1K, and VSI-Bench-Tiny. We further propose Think3D-RL to teach smaller open-weight models how to manipulate 3D space effectively. Using only final-answer rewards, without process supervision or handcrafted exploration trajectories, Think3D-RL enables Qwen3-VL-4B to autonomously learn effective 3D exploration strategies. After training, the model exhibits tool-use patterns similar to those of stronger proprietary models, while shifting the effect of 3D tool use on MindCube-1K from a performance drop to a substantial improvement. These results show that active exploration in 3D space provides an effective and general paradigm for improving spatial reasoning in multimodal agents. Code, models, and data are available at https://github.com/zhangzaibin/spagent.

Figures & tables

Figure 2: The Think3D pipeline. The VLM interacts with the 3D scene through iterative calls to the 3D Manipulation Toolkit, issuing viewpoint-manipulation actions that control camera pose and rendering parameters. Each rendered image is appended to the agent’s memory and informs the next reasoning step, forming a repeated cycle of observe $\rightarrow$ manipulate $\rightarrow$ reflect.

Parameter	Setting
Backbone	Qwen3-VL-4B-Instruct
Training examples	977
Training epochs	1
Rollouts per prompt	8
Training batch size	8
Gradient accumulation	4

Table 3: Optimization and execution settings for Think3D-RL.

3D Rec.	Cam. Anchor	Cam. Cho.	Ego View	BLINK	MindCube
				36.84	47.58
✓				41.17	50.16
✓	✓			55.46	51.90
✓	✓	✓		61.65	52.46
✓	✓	✓	✓	63.91	55.25

Table 4: Ablation on different 3D reasoning components. All results are reported as accuracy (%). 3D Rec. denotes reasoning with reconstructed 3D geometry; Cam. Anchor uses the camera pose as the manipulation anchor; Cam. Cho. enables camera selection; and Ego-view indicates whether the model may request ego-centric views. We report results on BLINK (multi-view) and MindCube-1K.

Panel of Figure 3 — Figure 3: Spatial exploration patterns in viewpoint selection. Strong models concentrate on informative viewing angles, such as oblique and top-down views. After RL fine-tuning, the viewpoint distribution of Qwen3-VL-4B T3RL shifts toward a similar pattern. Exploration strategies also vary substantially across tasks; for example, route planning favors top-down views near $(0,60)$ . The distributions summarize the viewpoints selected through the tool interface.

Figure 4: Reinforcement Learning Dynamics. As RL fine-tuning progresses, the model learns when extra 3D tool calls are worthwhile, shifting from shorter but less accurate trajectories to more informative explorations with higher reward.

Exploration strategy	BLINK Multi-view
Random	45.86
Heuristic	47.37
RL random	45.11
RL	53.39

Table 5: Effect of viewpoint-selection strategies on BLINK Multi-view. All results are accuracy (%).

Panel of Figure 5 — Figure 5: GPT-4.1’s view-mode usage and tool-call iteration distributions across spatial reasoning tasks.

Figure 6: Spatial exploration behavior of Think3D. The agent autonomously selects viewpoints and switches between global and ego-centric views; after RL training, it explores angles more systematically than the untuned baseline.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: The system prompt. Instruction prompt detailing tool invocation rules and the multi-step workflow for iterative 3D viewpoint exploration, including tool-call format, recommended angles, and guidelines for reasoning with reconstructed camera poses.

Figure 8: The Pi3 Tool Prompt. The prompt specifies the tool’s capabilities, key control parameters, and multi-angle query usage strategies to support comprehensive spatial understanding.

Figure 9: Multi-step prompt for iterative 3D viewpoint exploration. Including angle selection, camera rotation controls, tool invocation rules to refine spatial reasoning.

Figure 10: The RL system prompt. Instruction prompt defining the eight-candidate-view 3D analysis workflow, including tool-call format, angle selection rules (left, right, top, and oblique), and iterative reasoning steps for viewpoint-guided spatial understanding.

Figure 11: The RL continuation prompt used during non-final turns. Iterative-step instruction prompt outlining eight allowed viewpoint choices, tool-call rules, and the decision process for progressing or concluding 3D spatial analysis.

Figure 12: The RL continuation prompt used in the final turn. Final-turn instruction prompt specifying the no-tool phase, requiring explicit reasoning and a final answer based solely on previously generated 3D views and the original image.

Figure 13: The prompt without tools. Base instruction prompt for direct image-question analysis, requiring explicit reasoning and final answer formatting without tool interactions.

Figure 14: The critique prompt and refinement prompt used in the self-refine experiment.

Parameter	Setting
Foundation model	Qwen3-VL-4B-Instruct
Number of trained agents	1
Number of solution rounds	3
Number of evaluation rounds	3
Horizon for discussion history	1
Token limit for prompts	180000

Appendix

Table 6: Training and evaluation parameters used in both the RL optimization process and subsequent evaluation.

3D Spatial Reasoning

3D spatial reasoning studies how AI systems infer geometry, object relations, viewpoints, and motion to understand and act in three-dimensional scenes.

See all 201 papers

Learning to Think and Explore in 3D World
Oct 15, 2026Yijiang Li, Bingyang Wang, Nuno VasconcelosVLM Reasoning 3D VQA
University of California San Diego · Georgia Institute of Technology
Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning
Apr 8, 2026Jiahua Chen, Qihong Tang, Weinong Wang +13D Spatial Reasoning VLM Reasoning
Tsinghua University · Nanjing University · Tencent
3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training
Jun 3, 2026Jiaxin Shi, Xidong Zhang, Fucai Zhu +33D Spatial Reasoning Robotic Manipulation
Shanghai Jiao Tong University · Daimon Robotics · Harbin Institute of Technology +4

Oct 15, 2026J/K move · Enter open · S save

Learning to Think and Explore in 3D World

Yijiang Li, Bingyang Wang, Nuno Vasconcelos

University of California San Diego · Georgia Institute of Technology

VLM Reasoning 3D VQA Active Visual Perception RL for VLMs

Open paper AskPDF arXiv

We study how vision-language models (VLMs) can learn to think and actively explore in 3D space. Instead of providing multi-view images, videos, or 3D representations to the model, we preserve the native visual encoding of VLMs, i.e., a single image, one at a time, just like humans, and treat the 3D representation only as an environment with which the model can interact through actions (e.g., move left) and observations (rendered images). Each rollout begins from an initial first-person view and a visual question. The model can then iteratively reason, navigate, and reorient its viewpoint before committing to a final answer. Following an initial trajectory-supervised finetuning, we apply reinforcement learning (RL) with rewards driven primarily by final-answer correctness. Crucially, RL provides no direct supervision over which viewpoints the model should visit or which exploration actions it should take. Instead, the model must learn an effective exploration policy solely from the extent to which its interactions improve performance. Across six 3D benchmarks, our model ACT3D achieves an average accuracy of 53.2% that improves its base Qwen model by 22.0% and surpasses proprietary Gemini 3.6 Flash. ACT3D also outperforms existing open-source 3D methods while using 50-70% fewer input frames, and generalizes to unseen domains with an average improvement of 5.9% over its supervised initialization. Further analysis reveals adaptive, question-conditioned exploration. Performance gains over the base model increase from 0.6% on questions answerable from the initial view to 16.3% on those requiring extensive exploration. ACT3D searches for occluded targets, avoids unnecessary exploration, and acquires complementary viewpoints for spatial comparisons. These findings demonstrate that outcome-based RL enables VLMs not only to reason over visual evidence, but also to learn when and how to acquire it.

Abstract

3D Spatial Reasoning

Explore similar work

Learning to Think and Explore in 3D World

Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning

3DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-training