cs.CVOct 17, 2026

From Views to Worlds: Active Exploration over 3D Worlds for Vision-Language Models

Authors: Qijian Tian, Jiayu Ying, Ke Fan, Lizhuang Ma, Xin Tan

Organizations: Shanghai Jiao Tong University, Shanghai, China · East China Normal University, Shanghai, China

Abstract

Vision-language models (VLMs) have achieved remarkable progress in 2D visual understanding tasks, yet the passive view-centric perception paradigm limits their capacity for 3D spatial reasoning. Such reasoning often requires actively acquiring spatial evidence beyond the currently visible views, such as camera poses and cross-view spatial relationships, to infer 3D relationships that are not explicitly represented in 2D images. However, existing methods either lack active exploration or merely provide additional views during inference, without explicitly acquiring such spatial evidence beyond 2D images. In this paper, we propose View-to-World (V2W), an active exploration framework that enables VLMs to actively explore 3D worlds and lifts VLMs from passive view-centric perception to active world-centric spatial reasoning. Given a spatial reasoning task with multi-view images, V2W first reconstructs an explicit 3D world with geometry, camera poses, and language-grounded semantics, and exposes it to VLMs through visual and linguistic interfaces. Through multi-turn interaction with these interfaces, VLMs iteratively acquire spatial evidence from the constructed 3D world, enabling active world-centric spatial reasoning. V2W can be integrated with existing VLMs in a training-free manner and can be further enhanced by optimizing the exploration policy via reinforcement learning. Experiments on spatial mental modeling benchmarks demonstrate that V2W substantially improves VLM baselines in the training-free setting and achieves state-of-the-art performance with the learned exploration policy, highlighting the importance of active exploration for spatial intelligence. Project page: https://fangzhou2000.github.io/projects/v2w.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learning to Think and Explore in 3D World

    Oct 15, 2026Yijiang Li, Bingyang Wang, Nuno VasconcelosVLM Reasoning3D VQA

  2. Seek-and-View Reasoning for Multi-View Spatial Understanding

    Oct 8, 2026Qixiang Chen, Cheng Zhang, Fucai Ke +3Multimodal Spatial ReasoningVisual Spatial Reasoning

  3. World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning

    Apr 29, 2026Wanyue Zhang, Wenxiang Wu, Wang Xu +6Visual Spatial ReasoningVLM Adaptation