3D Representation Learning

Follow this topic Ask about this topic

3D representation learning studies how to encode point clouds, meshes, voxel grids, implicit fields, Gaussian splats, and multi-view images into features that preserve shape, spatial relations, and meaning. Methods include masked and contrastive self-supervised learning, rotation-equivariant networks, neural fields, generative latent models, and alignment with image, text, or action features. Systems are tested on tasks such as ShapeNet object recognition and reconstruction, ScanNet scene understanding, and robotic manipulation benchmarks. A central challenge is balancing compact, efficient features with the geometric detail, topology, and robustness needed for sparse views, occlusion, and transfer across datasets or tasks.

Latest papers

191 papers21 papers in the last four weeks, up 24% on the four weeks before. 0.1% of all new papers.

Spa3R: Predictive Spatial Field Modeling for 3D Visual Reasoning
Feb 24, 2026Haoyi Jiang, Liu Liu, Xinjie Wang +53D Spatial Reasoning 3D VLMs
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · Horizon Robotics, Beijing 100094, China · D-Robotics, Shenzhen 518055, China
3DSPA: A 3D Semantic Point Autoencoder for Evaluating Video Realism
Feb 23, 2026Bhavik Chandna, Kelsey R. AllenVideo Quality Assessment Video Generation Evaluation
University of California, San Diego, CA, USA · Vector Institute, Toronto, ON, Canada · University of British Columbia, Vancouver, BC, Canada
Universal Pose Pretraining for Generalizable Vision-Language-Action Policies
Feb 23, 2026Haitao Lin, Hanyang Yu, Jingshun Huang +5Robot Policy Learning Vision-Language-Action Models
Tencent Robotics X · Futian Laboratory · The Hong Kong University of Science and Technology +2
HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulation
Feb 21, 2026Chongyang Xu, Shen Cheng, Haipeng Li +3Robotic Manipulation Robot Imitation Learning
College of Computer Science, Sichuan University · Dexmal · School of Information and Communication Engineering, University of Electronic Science and Technology of China
Cryo-SWAN: the Multi-Scale Wavelet-decomposition-inspired Autoencoder Network for molecular density representation of molecular volumes
Feb 18, 2026Rui Li, Artsemi Yushkevich, Mikhail Kudryashev +13D Representation Learning Molecular Representation Learning
Center for Advanced Systems Understanding (CASUS), Görlitz, Germany. · Helmholtz-Zentrum Dresden-Rossendorf e. V. (HZDR), Dresden, Germany. · In situ Structural Biology, Max Delbrück Center for Molecular Medicine in the Helmholtz Association, Berlin, Germany. +4
Language and Geometry Grounded Sparse Voxel Representations for Holistic Scene Understanding
Feb 17, 2026Guile Wu, David Huang, Bingbing Liu +13D Scene Understanding 3D Scene Representation
Huawei Noah’s Ark Lab · University of Toronto
BrepLLM: Enabling Large Language Models to Understand Boundary Representations
Dec 18, 2025Liyuan Deng, Hao Guo, Yongkang Dai +5Multimodal Large Language Models 3D Representation Learning
Northwestern Polytechnical University · National University of Singapore · Hong Kong University of Science and Technology +1
InStyle: Instant Appearance Stylization of 3D Shapes
Dec 3, 2025Melis Ocal, Xiaoyan Xing, Yue Li +33D Shape Editing 3D Representation Learning
University of Amsterdam · Bosch Center for AI
FUSE: A Flow-based Mapping Between Shapes
Nov 17, 2025Lorenzo Olearo, Giulio Viganò, Daniele Baieri +2Flow Matching Non-Rigid Shape Correspondence
University of Milano-Bicocca, Milan, Italy · University of Bonn, Bonn, Germany · Lamarr Institute, Germany +1
Geometry-Centered 3D Latent World Models for Growing Surfaces
May 29, 2025Xiaoyi Liu, Hao TangWorld Model Learning Latent World Models
Department of Computer Science Brown University · School of Computer Science Peking University
Streaming Dense Voxel Representations for 3D Occupancy Prediction
Mar 28, 2025Seokha Moon, Janghyun Baek, Yujin Jeong +53D Occupancy Prediction 3D Representation Learning
1Korea University · 5NAVER LABS · † Work done during an internship at NAVER LABS. +4
Point-Voxel Absorbing Graph Representation Learning for Event Stream based Recognition
Jun 8, 2023Yuxiang Zhang, Chengguo Yuan, Xiao Wang +2Event-Based Vision Graph Neural Networks
aSchool of Computer Science and Technology, Anhui University, Hefei 230601, China.

Feb 24, 2026J/K move · Enter open · S save

Spa3R: Predictive Spatial Field Modeling for 3D Visual Reasoning

Haoyi Jiang, Liu Liu, Xinjie Wang, Yonghao He+4

School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · Horizon Robotics, Beijing 100094, China · D-Robotics, Shenzhen 518055, China

3D Spatial Reasoning 3D VLMs 3D Scene Representation 3D Representation Learning Self-Supervised Learning

Open paper AskPDF arXiv

Vision-language models excel at 2D visual understanding but remain limited in 3D spatial reasoning. Existing approaches either depend on explicit 3D modalities, which limits scalability, or inject partial, view-conditioned geometric priors and leave the language model to recover global scene structure from sparse cues. We introduce Spa3R, a self-supervised framework that learns a unified, view-invariant spatial representation from unposed multi-view RGB images. Its Predictive Spatial Field Modeling objective compresses context views into a compact latent representation and predicts aligned geometric and semantic feature fields at novel viewpoints, thereby encouraging coherent encoding of scene geometry and layout. We integrate the pre-trained Spa3R Encoder into a vision-language model through a lightweight residual cross-attention adapter, yielding Spa3-VLM and grounding language reasoning in global spatial context. Spa3-VLM achieves an average score of 58.6% on VSI-Bench and delivers leading or competitive performance across three additional spatial reasoning benchmarks. These results demonstrate that predictive spatial representation learning provides an effective visual foundation for 3D reasoning.