3D VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 118

All topics

Curvature-Aware Captioning:Leveraging Geodesic Attention for 3D Scene Understanding
May 9, 2026Ziyao He, Yingjie Liu, ZhangYangRui +3Attention Mechanisms Hyperbolic Representation Learning
East China Normal University
Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment
May 8, 2026Jerry Jiang, Haowen Sun, Denis Gudovskiy +43D Spatial Reasoning 3D Representation Learning
Tsinghua University · Panasonic AI Lab · Panasonic DX-CPS +1
Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models
May 8, 2026Haoming Wang, Wei GaoVisual Spatial Reasoning VLM Interpretability
Department of Electrical and Computer Engineering University of Pittsburgh Pittsburgh, PA 15261
VoxAfford: Multi-Scale Voxel-Token Fusion for Open-Vocabulary 3D Affordance Detection
May 2, 2026Haowen Sun, Shaolong Zhang, Mingyang Li +63D Point Cloud Segmentation 3D Scene Understanding
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049
Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding
Apr 29, 2026Yufei Yin, Jie Zheng, Qianke Meng +73D VLMs 3D Visual Grounding
Hangzhou Dianzi University Hangzhou, Zhejiang, China · Zhejiang University Hangzhou, Zhejiang, China · Alibaba Group Hangzhou, Zhejiang, China
Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment
Apr 23, 2026Jingkun Chen, Ruoshi Xu, Mingqi Gao +23D Spatial Reasoning VLM Hallucination
Northwestern Polytechnical University China · Southern University of Science and Technology China · The University of Sheffield United Kingdom +2
CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models
Apr 21, 2026Shengli Zhou, Xiangchen Wang, Guanhua Chen +13D Spatial Reasoning VLM Reasoning
1Southern University of Science and Technology · 2SpatialTemporal AI
XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments
Apr 20, 2026Kangan Qian, ChuChu Xie, Yang Zhong +133D Spatial Reasoning Embodied QA
Tsinghua University · Automotive and Robotics, Xiaomi Corporation · National University of Singapore +2
Geometry-Guided 3D Visual Token Pruning for Video-Language Models
Apr 20, 2026Han Li, Zehao Huang, Jiahui Fu +2Visual Token Pruning Visual Token Compression
Beihang University · Zhongguancun Academy
Medical Image Understanding Improves Survival Prediction via Visual Instruction Tuning
Apr 20, 2026Xixi Liu, Jorge Lazo, Andreas Hallqvist +8Clinical Outcome Prediction Medical VLMs
Chalmers University of Technology, Gothenburg, Sweden
E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes
Apr 20, 2026Koya Sakamoto, Taiki Miyanishi, Daichi Azuma +6Active Perception Next-Best-View Planning
The University of Tokyo, Japan · National Institute of Informatics, Japan · Institute of Science Tokyo, Japan +3
P3T: Prototypical Point-level Prompt Tuning with Enhanced Generalization for 3D Vision-Language Models
Apr 17, 2026Geunyoung Jung, Soohong Kim, Kyungwoo Song +1Point Cloud Classification VLM Adaptation
Department of Artificial Intelligence, University of Seoul, South Korea · Department of Applied Statistics, Yonsei University, South Korea
RGB-Pointmap Pretraining for Unified 3D Scene Understanding
Apr 2, 2026Ye Mao, Weixun Luo, Ranran Huang +23D Foundation Models Multi-View Learning
Imperial College London
LESV: Language Embedded Sparse Voxel Fusion for Open-Vocabulary 3D Scene Understanding
Apr 1, 2026Fusang Wang, Nathan Piasco, Moussab Bennehar +3Point Cloud Understanding 3D Scene Understanding
Huawei Noah’s Ark Lab · CAOR, Mines Paris-PSL, France
SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes
Mar 31, 2026Léopold Maillard, Francis Engelmann, Tom Durand +43D Spatial Reasoning Embodied AI
Ecole Polytechnique · Dassault Syst`emes · Stanford University +1
TSegAgent: Zero-Shot Tooth Segmentation via Geometry-Aware Vision-Language Agents
Mar 20, 2026Shaojie Zhuang, Lu Yin, Guangshun Wei +3Medical Imaging Medical Image Segmentation
Shandong University, Jinan 250010, China · University of Surrey, Guildford, Surrey, United Kingdom · King’s College London, London, United Kingdom
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding
Mar 19, 2026Xianjin Wu, Dingkang Liang, Tianrui Feng +53D Spatial Reasoning 3D Scene Understanding
Huazhong University of Science and Technology, China · Baidu Inc., China
X-GS: An Extensible Framework for Perceiving and Thinking with 3D Gaussian Splatting
Mar 10, 2026Yueen Ma, Zenglin Xu, Irwin King3D Gaussian Splatting 3DGS SLAM
The Chinese University of Hong Kong · Shanghai Academy of AI for Science · Fudan University
JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
Mar 6, 2026Sandeep Inuganti, Hideaki Kanayama, Kanta Shimizu +43D Point Cloud Segmentation Open-Vocabulary 3D Segmentation
German Research Center for Artificial Intelligence, DFKI, Germany · RPTU Kaiserslautern, Germany · Ricoh Company, Ltd. Japan +1
TotalFM: An Organ-Separated 3D-CT Foundation Model Leveraging Large-Scale Routine Clinical Radiology Data
Jan 1, 2026Kohei Yamamoto, Tomohiro Kikuchi3D Foundation Models Medical Imaging Foundation Models
Department of Radiology, Jichi Medical University · Data Science Center, Jichi Medical University
OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
Dec 28, 2025Wenyuan Huang, Zhenyu Zhang, Zhao Wang +43D VLMs 3D Visual Grounding
Nanjing University, School of Intelligent Science and Technology, Nanjing, China · China Mobile Zijin Innovation Institute, Nanjing, China
BrepLLM: Enabling Large Language Models to Understand Boundary Representations
Dec 18, 2025Liyuan Deng, Hao Guo, Yongkang Dai +5Multimodal Large Language Models 3D Representation Learning
Northwestern Polytechnical University · National University of Singapore · Hong Kong University of Science and Technology +1
Think, Then Look: Active Spatial Reasoning for House-Scale 3D Scene Understanding
Dec 2, 2025Hongpei Zheng, Shijie Li, Lin Qian +4Spatial Reasoning Benchmarks 3D Spatial Reasoning
University of Manchester · Institute for Infocomm Research (I2R), A*STAR, Singapore · University of Bedfordshire
Error-Driven Scene Editing for 3D Grounding in Large Language Models
Nov 18, 2025Yue Zhang, Zun Wang, Han Lin +53D Scene Editing 3D Spatial Reasoning
UNC Chapel Hill · University of Michigan · Google Research
VLEM: Real-Time 3D Vision-Language Embedding Mapping
Aug 8, 2025Christian Rauch, Björn Ellensohn, Linus Nwankwo +2Open-Vocabulary 3D Segmentation 3D Scene Representation
Technical University of Leoben, Franz Josef-Straße 18, 8700 Leoben, Austria
Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos
Jul 29, 2025Ziren Gong, Xiaohan Li, Fabio Tosi +4Open-Vocabulary 3D Segmentation 3D Reconstruction
University of Bologna · USTC · BIT +1
From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation
May 31, 2025Daniele Molino, Camillo Maria Caruso, Filippo Ruffini +2Latent Diffusion Models Medical Image Synthesis
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Rome, Italy · Department of Diagnostics and Intervention, Biomedical Engineering and Radiation Physics, Umeå University, Umeå, Sweden · UniCamillus – Saint Camillus International University of Health Sciences, Rome, Italy
3D-MoE: Towards Spatial Intelligence with Mixture-of-Experts for 3D Reasoning and Action Generation
Jan 28, 2025Yueen Ma, Zenglin Xu, Irwin King3D Spatial Reasoning Mixture-of-Experts Inference
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR

May 9, 2026, cs.CVJ/K move · Enter open · S save

Curvature-Aware Captioning:Leveraging Geodesic Attention for 3D Scene Understanding

Ziyao He, Yingjie Liu, ZhangYangRui, Mingsong Chen+2

East China Normal University

Attention Mechanisms Hyperbolic Representation Learning 3D Scene Understanding 3D VLMs 3D Visual Grounding

Open paper AskPDF arXiv

Accurate 3D scene description is fundamental to robotic navigation and augmented reality, yet current dense captioning methods face significant limitations in processing sparse point cloud data. % Existing approaches that apply Euclidean embedding spaces struggle to simultaneously preserve fine-grained local geometric details and model exponentially growing global semantic hierarchies, leading to either inaccurate localization or disjointed, shallow scene descriptions. % In this work, we propose a novel \textbf{\textsc{Curvature-Aware Captioning}} framework, integrating novel non-Euclidean geodesic attention mechanisms, to resolve the localization-contextualization conflict. % Specifically, self-attention within Oblique space enforces dimensional homogeneity while establishing long-range dependencies. Bidirectional geodesic cross-attention within Lorentz space models hierarchical semantic relationships across scene instances, enabling simultaneous precision in object localization and coherence in scene descriptions. % Theoretical analysis confirms that the curvature complementarity between the Oblique manifold and Lorentz hyperboloid resolves the Euclidean-hyperbolic conflict, ensuring feature stability via isotropic optimization while preserving inherent hierarchical relationships. Extensive experiments on ScanRefer and Nr3D benchmarks demonstrate state-of-the-art performance, with significant gains in both localization accuracy and descriptive richness.

Momentum

Related topics

Latest papers 118