3D Representation Learning

Ask about this topic

Overview

3D representation learning studies how to encode point clouds, meshes, voxel grids, implicit fields, Gaussian splats, and multi-view images into features that preserve shape, spatial relations, and meaning. Methods include masked and contrastive self-supervised learning, rotation-equivariant networks, neural fields, generative latent models, and alignment with image, text, or action features. Systems are tested on tasks such as ShapeNet object recognition and reconstruction, ScanNet scene understanding, and robotic manipulation benchmarks. A central challenge is balancing compact, efficient features with the geometric detail, topology, and robustness needed for sparse views, occlusion, and transfer across datasets or tasks.

Latest papers

CardsList

192 papers21 papers in the last four weeks, up 24% on the four weeks before. 0.1% of all new papers.

  1. To View Transform or Not to View Transform: NeRF-based Pre-training Perspective

    Mar 30, 2026Hyeonjun Jeong, Juyeb Shin, Dongsuk Kum3D ReconstructionAutonomous Driving Perception

    KAIST, Daejeon, Korea

  2. 3-D Representations for Hyperspectral Flame Tomography

    Mar 29, 2026Nicolas Tricard, Zituo Chen, Sili DengComputed TomographyImplicit Neural Representations

    Department of Mechanical Engineering, Massachusetts Institute of Technology 77 Massachusetts Ave, Cambridge, MA 02139, United States

  3. 3D-IDE: 3D Implicit Depth Emergent

    Mar 28, 2026Chushan Zhang, Ruihan Lu, Jinguang Tong +23D VLMs3D Scene Understanding

    School of Computing, Australian National University · FreiNexus · School of EECS, The University of Queensland +1

  4. FAST3DIS: Feed-forward Anchored Scene Transformer for 3D Instance Segmentation

    Mar 27, 2026Changyang Li, Xueqing Huang, Shin-Fang Chng +3Multi-View Contrastive Learning3D Instance Segmentation

    Goertek Alpha Labs

  5. Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds

    Mar 26, 2026Bin Yang, Mohamed Abdelsamad, Miao Zhang +1Self-Supervised Point Cloud LearningPoint Cloud Learning

    Bosch Center for AI, Stuttgart, Germany · Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany · Robert Learning Lab, University of Freiburg, Freiburg, Germany

  6. PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning

    Mar 25, 2026Yankai Wang, Yiding Sun, Qirui Wang +3Point Cloud LearningRL Fine-Tuning

    School of Software Engineering, Xi’an Jiaotong University, Xi’an 710048, China · International School, Beijing University of Posts and Telecommunications, Beijing 100876, China

  7. Patchwork: A compact representation for 3D polygonal shapes

    Mar 24, 2026Ruichen Zheng, Biao Zhang, Michael Birsak +2Geometric Representation Learning3D Representation Learning

    King Abdullah University of Science and Technology, Thuwal, 23955, Saudi Arabia

  8. SpatialBoost: Enhancing Visual Representation through Language-Guided Reasoning

    Mar 23, 2026Byungwoo Jeon, Dongyoung Kim, Huiwon Jang +2Visual Representation Learning3D Spatial Reasoning

    KAIST · RLWRLD · NAVER Cloud

  9. Suiren-1.0 Technical Report: A Family of Molecular Foundation Models

    Mar 23, 2026Junyi An, Xinyu Lu, Yun-Fei Shi +5Molecular Foundation ModelsMolecular Representation Learning

    Shanghai Academy of AI for Science (SAIS)

  10. Back to Point: Exploring Point-Language Models for Zero-Shot 3D Anomaly Detection

    Mar 23, 2026Kaiqiang Li, Gang Li, Mingle Zhou +3Point Cloud Anomaly DetectionIndustrial Inspection

    Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.

  11. LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis

    Mar 20, 2026Stanislaw Szymanowicz, Minghao Chen, Jianyuan Wang +2Novel View Synthesis3D Representation Learning

    Visual Geometry Group, University of Oxford · Meta AI

  12. VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation

    Mar 19, 2026Chinmay Prabhakar, Bastian Wittmann, Tamaz Amiranashvili +6Graph GenerationGraph Representation Learning

    University of Zurich, Switzerland · ETH AI Center, Zurich, Switzerland · Technical University of Munich, Germany +1

  13. LoST: Level of Semantics Tokenization for 3D Shapes

    Mar 18, 2026Niladri Shekhar Dutt, Zifan Shi, Paul Guerrero +4Autoregressive 3D Scene Generation3D Shape Generation

    University College London · Adobe Research

  14. VolumeDP: Modeling Volumetric Representation for Manipulation Policy Learning

    Mar 18, 2026Tianxing Zhou, Feiyang Xue, Zhangchen Ye +3Visuomotor Policy LearningRobotic Manipulation

    1IIIS, Tsinghua University · 2Galaxea AI

  15. Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshes

    Mar 18, 2026Umangi Jain, Vladimir Kim, Matheus Gadelha +23D Representation Learning3D Part Segmentation

    University of Toronto · Adobe Research

  16. OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder

    Mar 17, 2026Sensen Gao, Zhaoqing Wang, Qihang Cao +53D Scene Generation3D Diffusion Models

    Mohamed bin Zayed University of Artificial Intelligence · AISphere · Shanghai Jiao Tong University +3

  17. IConE: Batch Independent Collapse Prevention for Self-Supervised Representation Learning

    Mar 16, 2026Konstantinos Almpanakis, Anna KreshukSelf-Supervised LearningRepresentation Collapse

    European Molecular Biology Laboratory, Heidelberg, Germany

  18. ReMAP-DP: Reprojected Multi-view Aligned PointMaps for Diffusion Policy

    Mar 16, 2026Xinzhang Yang, Renjun Wu, Jinyan Liu +1Vision-Based Robot ControlDiffusion Policy

    School of Computer Science, Beijing Institute of Technology, China.

  19. Masked BRep Autoencoder via Hierarchical Graph Transformer

    Mar 16, 2026Yifei Li, Kang Wu, Wenming Wu +13D Representation LearningGraph SSL

    University of Science and Technology of China · Hefei University of Technology

  20. LiTo: Surface Light Field Tokenization

    Mar 11, 2026Jen-Hao Rick Chang, Xiaoming Zhao, Dorian Chan +13D Representation Learning3D Scene Representation

    Apple

  21. Pointy - A Lightweight Transformer for Point Cloud Foundation Models

    Mar 11, 2026Konrad Szafer, Marek Kraft, Dominik BelterPoint Cloud Learning3D Foundation Models

    Institute of Robotics and Machine Intelligence, Poznan University of Technology

  22. CanoVerse: 3D Object Scalable Canonicalization and Dataset for Generation and Pose

    Mar 7, 2026Li Jin, Yuchen Yang, Weikai Chen +113D Shape Retrieval3D Representation Learning

    SDU · LIGHTSPEED · UNC Chapel Hill +2

  23. Optimizing Multi-Modal Models for Image-Based Shape Retrieval: The Role of Pre-Alignment and Hard Contrastive Learning

    Mar 7, 2026Paul Julius Kühn, Cedric Spengler, Michael Weinmann +23D Shape RetrievalCross-Modal Retrieval

    Fraunhofer IGD, 64283 Darmstadt, Germany · Delft University of Technology, 2628 CD Delft, Netherlands

  24. LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents

    Mar 6, 2026Tianhao Zhao, Youjia Zhang, Hang Long +73D Mesh Generation3D Representation Learning

    Huazhong University of Science and Technology · Peking University · Independent Researcher +1

  25. Hyperbolic Multiview Pretraining for Robotic Manipulation

    Mar 5, 2026Jin Yang, Ping Wei, Yixin Chen +1Robotic Manipulation3D Representation Learning

    National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University

  26. PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters

    Mar 4, 2026Yinghong Yu, Guangyuan Li, Jiancheng Yang3D Representation Learning3D Foundation Models

    ELLIS Institute Finland, Espoo, 02150, Finland · Aalto University, Espoo, 02150, Finland

  27. HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding

    Mar 2, 2026Lei Yao, Yong Chen, Yuejiao Su +33D Affordance Grounding3D Visual Grounding

    The Hong Kong Polytechnic University · Huazhong University of Science and Technology

  28. A 3D mesh convolution-based autoencoder for geometry compression

    Mar 2, 2026Germain Bregeon, Marius Preda, Radu Ispas +1Convolutional Autoencoder3D Mesh Reconstruction

    Telecom SudParis, Institut Polytechnique de Paris, France · Keyrus, Paris, France

  29. PointAlign: Feature-Level Alignment Regularization for 3D Vision-Language Models

    Feb 28, 2026Yuanhao Su, Shaofeng Zhang, Xiaosong Jia +13D VLMsVision-Language Alignment

    University of Science and Technology of China · Fuzhou University · Fudan University +1

  30. FoV-Net: Rotation-Invariant CAD B-rep Learning via Field-of-View Ray Casting

    Feb 27, 2026Matteo Ballegeer, Dries F. BenoitBoundary Representation Modeling3D Representation Learning