3D VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 118

All topics

Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Models
Jun 19, 2026Marwane Hariat, Gianni Franchi, David Filliat +1Multi-View Consistency Bayesian Inference
U2IS, ENSTA – Institut Polytechnique de Paris, Palaiseau, France · Pôle Recherche, Agence Ministérielle pour l’IA de Défense, Palaiseau, France
SpatialSV: Internalizing Interpretable 3D Spatial Awareness in MLLMs via Task-Oriented Visual Supervision
Jun 18, 2026Jiayu Tang, Yuchen Zhou, Chao Gou3D Spatial Reasoning VLM Interpretability
School of Intelligent Systems Engineering, Sun Yat-sen University
3D-PLOT-LLM: Part-Level Object Tokens for 3D Large Language Models
Jun 18, 2026Jintang Xue, Xinyu Wang, Yixing Wu +23D Representation Learning Multimodal Foundation Models
University of Southern California Los Angeles, California, USA · Ohio State University Columbus, Ohio, USA
Occ-VLM: Occupancy Grounded Vision Language Model for Indoor Scene Understanding
Jun 18, 2026Jianing Li, Zhou Fang, Yijiang Liu +13D Occupancy Prediction 3D VQA
School of Electronic Science and Engineering, Nanjing University
OneCanvas: 3D Scene Understanding via Panoramic Reprojection
Jun 17, 2026Bartłomiej Baranowski, Dave Zhenyu Chen, Matthias Nießner3D Spatial Reasoning Embodied QA
Technical University of Munich · Huawei
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Jun 16, 2026Sneha Paul, Zachary Patterson, Nizar BouguilaMultimodal Robustness VLM Adaptation
Concordia University, Canada
Segment and Select: Vision-Language Segmentation in 3D Scenarios
Jun 9, 2026Yulin Chen, Zhihang Zhong, Yuenan Hou3D Point Cloud Segmentation Referring Expression Segmentation
Shanghai AI Laboratory · University of Science and Technology of China · Shanghai Jiaotong University
3D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis
Jun 9, 2026Yuhao Wang, Puyi Wang, Linjie Li +33D Asset Generation 3D Scene Editing
1Shanghai Jiao Tong University · 2The Chinese University of Hong Kong · 3Microsoft +1
DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation
Jun 8, 2026Nanshan Jia, Zhenyu Zhao, Junbo Peng +3VLM Evaluation 3D VLMs
University of California, Berkeley · Roblox Corporation
Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors
Jun 5, 2026Hanxun Yu, Xuan Qu, Lei Ke +43D Spatial Reasoning 3D VQA
1Zhejiang University · 2Tencent Hunyuan · 3HKUST +1
PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understanding
Jun 4, 2026Shaohui Dai, Yansong Qu, You Shen +23D Part Segmentation 3D Representation Learning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University
BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding
Jun 3, 2026Muhammad Usama, Didier Stricker, Mohammad Sadil Khan +1Multimodal Pretraining Multimodal Contrastive Learning
1DFKI, Germany · 2RPTU Kaiserslautern-Landau, Germany
MASER: Modality-Adaptive Specialist Routing for Embodied 3D Spatial Intelligence
Jun 1, 2026Hilton Raj, Vishnuram AVEmbodied QA Adapter Tuning
Boston University
HOLA: Holistic Multi-Modal Alignment for Open-Set 3D Recognition
May 31, 2026Koby Aharonov, Oren Shrout, Ayellet TalContrastive Learning Open-Set Recognition
Technion – Israel Institute of Technology
Distilling Neuro-Symbolic Programs into 3D Multi-modal LLMs
May 31, 2026Wentao Mo, Yang LiuCoT Distillation 3D Spatial Reasoning
Wangxuan Institute of Computer Technology, Peking University
VLM3: Vision Language Models Are Native 3D Learners
May 28, 2026Zhipeng Cai, Zhuang Liu, Yunyang Xiong +3Camera Pose Estimation Monocular Depth Estimation
Meta · Princeton University
Grounded 3D-Aware Spatial Vision-Language Modeling
May 28, 2026An-Chieh Cheng, Yang Fu, Yatai Ji +123D Spatial Reasoning Visually Grounded Reasoning
UCSD · NVIDIA · MIT
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
May 28, 2026Chun-Hsiao Yeh, Shengyi Qian, Manchen Wang +3Vision-Language Models 3D Spatial Reasoning
FAIR at Meta · UC Berkeley · HKU
Planning with the Views
May 28, 2026Kangrui Wang, Linjie Li, Zhengyuan Yang +7Next-Best-View Planning RL for VLMs
Northwestern University · University of Washington · Microsoft +2
SSR3D-LLM: Structured Spatial Reasoning via Latent Steps for Fine-Grained Grounding in Unified 3D-LLMs
May 27, 2026Jiawei Li, Ziyi Liu, Weijie Shi +33D Spatial Reasoning Latent Visual Reasoning
The Hong Kong University of Science and Technology · Soochow University
PointQ-Bench: Benchmarking Diagnostic and Interpretable Point Cloud Quality Assessment
May 27, 2026Duanchu Wang, Cheng Li, Junjie Yang +5VLM Evaluation Point Cloud Understanding
Xidian University · Xi’an Jiaotong University · University of Chinese Academy of Sciences +1
TrackRef3D: Multi-View Consistent Track-then-Label for Open-World Referring Segmentation in 3D Gaussian Splatting
May 26, 2026Yuyang Tan, Renhe Zhang, Hang Zhang +2Multi-View Consistency 3D Gaussian Splatting
East China Normal University, Shanghai, China · Shanghai AI Laboratory · University of Electronic Science and Technology of China, Chengdu, China
PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought
May 21, 2026Chaoqi Chen, Qile Xu, Wenjun Zhou +1CoT Reasoning Point Cloud Understanding
Visual Computing Research Center (VCC), College of Computer Science and Software Engineering (CSSE), Shenzhen University, China · VCC, CSSE, Shenzhen University, China
PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects
May 20, 2026Ziang Cao, Yinghao Liu, Haitian Li +53D Asset Generation Physics-Based Simulation
S-Lab, Nanyang Technological University · ACE Robotics
CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models
May 19, 2026Hsiang-Wei Huang, Junbin Lu, Kuang-Ming Chen +3VLM Evaluation 3D Spatial Reasoning
Department of Electrical and Computer Engineering, University of Washington, Seattle, USA
Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence
May 19, 2026Zebin He, Mingxin Yang, Shuhui Yang +43D Foundation Models Cross-Modal Alignment
1HKUST · 2Tencent Hunyuan
Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs
May 19, 2026Xueying Jiang, Wenhao Li, Quanhao Qian +43D Object Detection 3D VLMs
1Nanyang Technological University · 2DAMO Academy, Alibaba Group · 3HuPan Lab +1
3D Segmentation Using Viewpoint-Dependent Spatial Relationships
May 15, 2026Ayaka Nanri, Klara Reichard, Mert Kiray +33D Spatial Reasoning 3D Scene Understanding
Institute of Science Tokyo · Technical University of Munich · BMW Group +12
PanoWorld: Towards Spatial Supersensing in 360 $^\circ$ Panorama World
May 13, 2026Changpeng Wang, Xin Lin, Junhan Liu +53D Spatial Reasoning 3D Scene Understanding
1Zhejiang University · University of California, San Diego · University of California, Irvine +1
Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoT
May 10, 2026Alaa Asfour, Christopher Indris, Leihan Chen +2Cross-Modal Knowledge Distillation 3D Spatial Reasoning
Department of Computer Science, Toronto Metropolitan University, Toronto, ON, Canada

Jun 19, 2026, cs.CVJ/K move · Enter open · S save

Lightweight 3D Feature Pretraining by Bayesian Inversion of 2D Foundation Models

Marwane Hariat, Gianni Franchi, David Filliat, Antoine Manzanera

U2IS, ENSTA – Institut Polytechnique de Paris, Palaiseau, France · Pôle Recherche, Agence Ministérielle pour l’IA de Défense, Palaiseau, France

Multi-View Consistency Bayesian Inference 3D Representation Learning 3D VLMs

Open paper AskPDF arXiv

We present Casper3D, a lightweight probabilistic framework for converting noisy multi-view 2D foundation-model embeddings into a latent 3D semantic representation. We model view-level semantic features as noisy observations of an underlying 3D semantic state and infer this state with a set-based variational model that incorporates relative pose during multi-view reasoning. Casper3D is trained by predicting held-out semantic observations from novel viewpoints, while remaining aligned with visual and text semantic spaces for open-vocabulary 3D understanding. The framework is backbone-agnostic and applies to both language-aligned and self-supervised embeddings. Experiments show that Casper3D produces more stable 3D semantics than simple multi-view pooling, especially in ambiguous and noisy settings.

Momentum

Related topics

Latest papers 118