3D VQA

VQA: Visual Question Answering

Three-dimensional visual question answering (VQA) asks models to answer questions that require understanding objects, layouts, distances, viewpoints, or events in 3D scenes and volumes. Systems combine language models with point clouds, RGB-D images, multiview video, or reconstructed scene representations such as occupancy maps and scene graphs; some also use geometric tools to measure or retrieve relevant evidence. Benchmarks such as ScanQA and SQA3D test scene understanding and spatial reasoning, while medical VQA evaluates reasoning over volumetric scans. A central challenge is ensuring answers are grounded in 3D evidence rather than language or visual shortcuts, while making reasoning over many views accurate and efficient.

Latest papers

51 papers9 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Zhengrong-Spatial-Aware 1.0: Advancing 3D Multimodal Intelligence
Oct 27, 2026Yu Zhang, Xiaogang Wang, Yuhang Cheng +23D VQA Multimodal Spatial Reasoning
Beijing Freedo Technology Co., Ltd. · Southwest University
3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement
Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding 3D VLMs
School of Artificial Intelligence, Dalian University of Technology, Dalian 116081, China. · School of Information and Communication Engineering, Dalian University of Technology, Dalian 116081, China.
TP-JSCC: Efficient Token Pruning and Joint Source-Channel Coding for LVLMs-based 3D-VQA in Cloud-Edge Collaboration
Oct 26, 2026Zhengxin Li, Chongzhen Tian, Hui Yuan +2Visual Token Compression 3D VQA
School of Control Science and Engineering, Shandong University, Jinan 250061, China · Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan 250061, China · Shandong Key Laboratory of Intelligent Communication and Sensing-Computing Integration, Jinan 250100, China +2
P $^3$ -VLM: A Point-based Alternative for Grounded 3D Vision-Language Models
Oct 20, 2026Anna-Maria Halacheva, Jan-Nico Zaech, Sombit Dey +23D VLMs 3D Visual Grounding
INSAIT, Sofia University “St. Kliment Ohridski”
Do 3D Large Language Models Really Use 3D Evidence?
Oct 17, 2026Yifei Zhao, Fanyu Zhao, Han Zhang +63D VLMs 3D VQA
Fudan University · University of Science and Technology of China
Learning to Think and Explore in 3D World
Oct 15, 2026Yijiang Li, Bingyang Wang, Nuno VasconcelosVLM Reasoning 3D VQA
University of California San Diego · Georgia Institute of Technology
Holi-Spatial 360: Generative Reconstruction from Sparse Panoramas for Spatial Rendering and Intelligence
Oct 14, 2026Yuanyuan Gao, Zhirui Liu, Yuning Gong +9Indoor Scene Reconstruction Panoramic 3D Reconstruction
The Hong Kong University of Science and Technology · ShanghaiTech University · Tencent +4
EgoAlign: Situated 3D Understanding without Situation Descriptions
Oct 12, 2026Yiyang Li, Tianyi Ma, Zehong Wang +13D VQA 3D VLMs
Department of Computer Science and Engineering, University of Notre Dame
Lens3D: Target-Conditioned Visual Foveation for Fine-Grained 3D Understanding
Oct 5, 2026Junming Huang, Zini Chen, Shuaiying Hou +3Active View Selection 3D VQA
Zhejiang University · LIGHTSPEED
Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMs
Oct 4, 2026Yuqun Wu, Yao Xiao, Chuhang Zou +2Visual Spatial Reasoning 3D VQA
University of Illinois at Urbana-Champaign · Meta
Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding
Sep 29, 2026Jiayu Ying, Qijian Tian, Ruijie Xu +4Visual Question Answering Multimodal Large Language Models
East China Normal University · Shanghai Jiao Tong University · Fudan University +1
Long Time No See: Benchmarking VLMs for Out-of-Sight Spatiotemporal Reasoning in Egocentric Videos
Sep 28, 2026Fangzhou Ma, Ivo Alexander Ban, Eren Homburg +5Temporal Video Grounding Visual Spatial Reasoning
ETH Zurich · Microsoft Spatial AI Lab · Bocconi University
SpatialSpeak: QA-Native Reconstruction with Local and Global Context for Spatial Chain-of-Thought Reasoning
Sep 27, 2026Yang Cao, Jiaxin Zhang, Dave Zhenyu Chen +4Visual Spatial Reasoning Multi-View 3D Reconstruction
Hong Kong University of Science and Technology · Harbin Institute of Technology · Huawei Noah’s Ark Lab
SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding
Sep 27, 2026Xiangqi Li, Libo Huang, Jiarui Zhao +43D Scene Representation 3D VQA
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences
CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environments
Sep 26, 2026Boao Yu, Yingzhen Nie, Yue Hu +23D VQA Tool Use in VLMs
College of Systems Engineering, National University of Defense Technology · National Key Laboratory of Digital Intelligent Modeling and Simulation
SparseTalk - Sparsifying 3D Gaussian Language Fields for Efficient 3D Visual Question Answering
Sep 14, 2026Davit Soselia, Joseph JaJa, Amitabh Varshney3DGS Compression Language-Embedded 3DGS
University of Maryland, College Park, MD, USA
GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
Sep 3, 2026Junqing Du, Fernando Ropero, Erkin Turkoz +23D Spatial Reasoning Multimodal Large Language Models
Riemann Lab, Huawei Technologies
MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT
Sep 3, 2026Chenguang Zheng, Le Xue, Yichi Zhang +8PET/CT Imaging Medical VLMs
Huashan Hospital & Human Phenome Institute, Fudan University, Shanghai, China · Shanghai Academy of Artificial Intelligence for Science, Shanghai, China · Artificial Intelligence Innovation and Incubation Institute, Fudan University, Shanghai, China +1
Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering
Aug 28, 2026Shaozu Ding, Linan Song, Dajiang Suo3D VQA Spatial Reasoning Benchmarks
The Polytechnic School, Arizona State University
Memory Tree Guided Key Frame Querying for Efficient 3D Question Answering
Aug 18, 2026Hsiang-Wei Huang, Fu-Chen Chen, Li-Wu Tsao +7Embodied QA 3D VQA
University of Washington, United States · Amazon, United States · The University of Texas at Austin, United States
Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding
Aug 4, 2026Lucy Lin, Ayush Jain, Yifan Liu +13D Foundation Models 3D VQA
Carnegie Mellon University
3DZip: Spatial-Aware Feature Diversity-Guided Token Compression for 3D Question Answering
Aug 2, 2026Changwoo Baek, Kyeongbo KongMultimodal Token Compression 3D VQA
Department of Electrical and Electronics Engineering, Pusan National University, Busan, Republic of Korea
ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
Jul 30, 2026Ping-Kun Chiang, Kun-Ru Wu, Po-han Li +33D Spatial Reasoning VLM Reasoning
Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering
Jul 5, 2026Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2Efficient VLM Inference Multimodal QA
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
Dense Reward for Multi-View 3D Reasoning with Global Maps and Local Views
Jun 22, 2026Jiho Choi, Seonho Lee, Seojeong Park +1Visual Question Answering Active Perception
Graduate School of Artificial Intelligence, KAIST, Republic of Korea · KRAFTON, Republic of Korea
Occ-VLM: Occupancy Grounded Vision Language Model for Indoor Scene Understanding
Jun 18, 2026Jianing Li, Zhou Fang, Yijiang Liu +13D Occupancy Prediction 3D VQA
School of Electronic Science and Engineering, Nanjing University
UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQA
Jun 10, 2026Mengzhuo Chen, Yan Shu, Chi Liu +4Medical VQA VLM Reasoning
IQuest Research
Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors
Jun 5, 2026Hanxun Yu, Xuan Qu, Lei Ke +43D Spatial Reasoning 3D VQA
1Zhejiang University · 2Tencent Hunyuan · 3HKUST +1
PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding
Jun 4, 2026Shaohui Dai, Yansong Qu, You Shen +23D Part Segmentation 3D Representation Learning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University
Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportation
Jun 2, 2026Dongsheng Wang, Dawei Su, Hui Huang3D VQA Visual Token Compression
College of Computer Science and Software Engineering, Shenzhen University, China.

Oct 27, 2026J/K move · Enter open · S save

Zhengrong-Spatial-Aware 1.0: Advancing 3D Multimodal Intelligence

Yu Zhang, Xiaogang Wang, Yuhang Cheng, Jingrui Li+1

Beijing Freedo Technology Co., Ltd. · Southwest University

3D VQA Multimodal Spatial Reasoning 3D VLMs Multimodal Large Language Models

Open paper AskPDF arXiv

This report presents the implementation and validation of two end-to-end multimodal adaptation pipelines for 3D-aware visual question answering on a custom spatial dataset. The first pipeline fine-tunes HiSpatial-3B, which augments RGB observations with metric-scale 3D point maps; the second adapts Qwen3-VL through memory-efficient QLoRA-based supervised fine-tuning. The completed engineering stack covers dataset conversion, image-annotation alignment, monocular geometry preprocessing, instruction construction, mixed-domain training, checkpoint management, batch inference, and automated evaluation. On the target spatial benchmark, HiSpatial adaptation increases overall accuracy from 1.73% before fine-tuning to 79.89% at 3,600 steps. On a separately collected styled split, accuracy improves from 14.9% to 55.1%, indicating transfer beyond the original training set. The Qwen3-VL pipeline is likewise implemented end-to-end for both 8B- and 32B-scale models using mixed-domain QLoRA adaptation. The resulting accuracy improves from 20.74% to 45.27% for Qwen3-VL-8B and from 30.79% to 47.60% for Qwen3-VL-32B. Together, these results establish two complementary and reproducible adaptation pipelines for geometry-aware and general-purpose multimodal spatial reasoning. These results establish a reproducible engineering foundation for subsequent work on metric 3D reasoning, spatial grounding, and domain-adapted multimodal systems, while also identifying training-data composition as a central factor in preserving general multimodal capability during specialization.