3D Foundation Models

Follow this topic Ask about this topic

3D foundation models aim to represent the shape, structure, and meaning of objects and scenes in forms that transfer across tasks such as reconstruction, segmentation, robotics, and generation. Common approaches pretrain on point clouds, multi-view images, meshes, or medical volumes using self-supervision, masked prediction, and alignment with images or language; models are then adapted with fine-tuning, lightweight prompts, or distillation. Generative models also use these representations to create or edit 3D assets and scenes. A central challenge is balancing broad transfer with reliable performance across different sensors, domains, and scales, while improving efficiency and evaluating geometric consistency and robustness.

Latest papers

69 papers13 papers in the last four weeks, up 63% on the four weeks before. 0.1% of all new papers.

Pointy - A Lightweight Transformer for Point Cloud Foundation Models
Mar 11, 2026Konrad Szafer, Marek Kraft, Dominik BelterPoint Cloud Learning 3D Foundation Models
Institute of Robotics and Machine Intelligence, Poznan University of Technology
PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters
Mar 4, 2026Yinghong Yu, Guangyuan Li, Jiancheng Yang3D Representation Learning 3D Foundation Models
ELLIS Institute Finland, Espoo, 02150, Finland · Aalto University, Espoo, 02150, Finland
Utonia: Toward One Encoder for All Point Clouds
Mar 3, 2026Yujia Zhang, Xiaoyang Wu, Yunhan Yang +63D Foundation Models 3D ViTs
The University of Hong Kong · Xiaomi · The Chinese University of Hong Kong
MERG3R: A Divide-and-Conquer Approach to Large-Scale Neural Visual Geometry
Mar 2, 2026Leo Kaixuan Cheng, Abdus Shaikh, Ruofan Liang +3Large-Scale 3D Reconstruction 3D Reconstruction
University of Toronto · Vector Institute
Zatom-1: Towards a Multimodal Foundation Model for 3D Molecules and Materials
Feb 24, 2026Alex Morehead, Miruna Cretu, Antonia Panescu +143D Foundation Models Multimodal Foundation Models
LBNL · ICSI · University of Cambridge +3
Depth Completion as Parameter-Efficient Test-Time Adaptation
Feb 16, 2026Bingxin Ke, Qunjie Zhou, Jiahui Huang +5Depth Completion Test-Time Adaptation
NVIDIA · ETH Zürich
Scalable Adaptation of 3D Geometric Foundation Models via Weak Supervision from Internet Video
Feb 8, 2026Zihui Gao, Ke Liu, Donny Y. Chen +53D Foundation Models 3D Reconstruction
Zhejiang University, State Key Lab of CAD & CG · Independent Researcher · Nanyang Technological University +1
Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardware
Jan 31, 2026Brandon Leblanc, Charalambos Poullis3D Foundation Models Knowledge Distillation
Immersive and Creative Technologies Lab Concordia University Montreal, Canada
TotalFM: An Organ-Separated 3D-CT Foundation Model Leveraging Large-Scale Routine Clinical Radiology Data
Jan 1, 2026Kohei Yamamoto, Tomohiro Kikuchi3D Foundation Models Medical Imaging Foundation Models
Department of Radiology, Jichi Medical University · Data Science Center, Jichi Medical University

Mar 11, 2026J/K move · Enter open · S save

Pointy - A Lightweight Transformer for Point Cloud Foundation Models

Konrad Szafer, Marek Kraft, Dominik Belter

Institute of Robotics and Machine Intelligence, Poznan University of Technology

Point Cloud Learning 3D Foundation Models 3D Representation Learning 3D ViTs

Open paper AskPDF arXiv

Foundation models for point cloud data have recently grown in capability, often leveraging extensive representation learning from language or vision. In this work, we take a more controlled approach by introducing a lightweight transformer-based point cloud architecture. In contrast to the heavy reliance on cross-modal supervision, our model is trained only on 39k point clouds - yet it outperforms several larger foundation models trained on over 200k training samples. Interestingly, our method approaches state-of-the-art results from models that have seen over a million point clouds, images, and text samples, demonstrating the value of a carefully curated training setup and architecture. To ensure rigorous evaluation, we conduct a comprehensive replication study that standardizes the training regime and benchmarks across multiple point cloud architectures. This unified experimental framework isolates the impact of architectural choices, allowing for transparent comparisons and highlighting the benefits of our design and other tokenizer-free architectures. Our results show that simple backbones can deliver competitive results to more complex or data-rich strategies. The implementation, including code, pre-trained models, and training protocols, is available at https://github.com/KonradSzafer/Pointy.