3D Foundation Models

Follow this topic Ask about this topic

3D foundation models aim to represent the shape, structure, and meaning of objects and scenes in forms that transfer across tasks such as reconstruction, segmentation, robotics, and generation. Common approaches pretrain on point clouds, multi-view images, meshes, or medical volumes using self-supervision, masked prediction, and alignment with images or language; models are then adapted with fine-tuning, lightweight prompts, or distillation. Generative models also use these representations to create or edit 3D assets and scenes. A central challenge is balancing broad transfer with reliable performance across different sensors, domains, and scales, while improving efficiency and evaluating geometric consistency and robustness.

Latest papers

69 papers13 papers in the last four weeks, up 63% on the four weeks before. 0.1% of all new papers.

ProMo3D: Probing Motion Cues in Frozen 3D Foundation Models
Oct 24, 2026Xiaoang Zhang, Mert Kiray, Yordanka Velikova +2Motion Segmentation 3D Foundation Models
Technical University of Munich (TUM), Germany · Munich Center for Machine Learning (MCML), Germany · Obsphera, Munich, Germany
Plausible but Unimplantable: A Trustworthiness Audit of Shape Completion Foundation Models for Cranial Defect Reconstruction
Oct 21, 2026Marek Wodzinski, Kamil Kwarciak, Daria Hemmerling +1Trustworthy AI 3D Shape Completion
AGH University of Krakow, Krakow, Poland · Sano Centre for Computational Medicine, Krakow, Poland
Foundation-Model-Driven 3D Scene Graphs for Mobile Manipulation
Oct 16, 2026Dominic Maggio, Lorenzo Shaikewitz, Aryan Naveen +2Robotic Spatial Memory 3D Mapping
Laboratory for Information & Decision Systems, Massachusetts Institute of Technology Cambridge, MA, USA. · Samsung Research America
Towards Reconstructing Geographically Diverse Architecture with 3D Foundation Models
Oct 15, 2026Nicky Kriplani, Yiwen Zhang, Angelina Wang +13D Foundation Models 3D Reconstruction
Cornell Tech, Cornell University
Remember, Revisit, Revise: Scaffold Memory for Long-Horizon 3D Geometry
Oct 14, 2026Soumick Sarker, Mengmi Zhang3D Reconstruction Streaming 3D Reconstruction
Deep NeuroCognition Lab, Nanyang Technological University, Singapore
UniMetric: Universal Metric Scale Recovery for Visual Geometry Foundation Models
Oct 12, 2026Lei Yang, Yuner Zhang, Ziyang Liu +53D Reconstruction 3D Foundation Models
Nanyang Technological University · Carnegie Mellon University · The Hong Kong Polytechnic University +3
Drive3R: Block-wise Streaming Geometric Foundation Models for Autonomous Driving
Oct 10, 2026Tianyi Yang, Haojun Yang, Jipeng Xiao +43D Foundation Models Streaming 3D Reconstruction
AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · Voyager Research, Didi Chuxing · The Chinese University of Hong Kong, Shenzhen
Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction
Oct 8, 2026Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh +33D Foundation Models Feed-Forward 3D Reconstruction
Advanced Micro Devices Inc. (AMD)
Pumpire: Unified Benchmark for Metric Distance Estimation
Oct 8, 2026Siyu Chen, Zehan Wang, Jiayang Xu +63D Foundation Models Metric Depth Estimation
Zhejiang University · Zhejiang University of Technology · Shanghai Jiaotong University
RDGSplat: Render-Dedicated Geometry for Novel View Synthesis
Oct 6, 2026Zhijie Zheng, Xinhao Xiang, Jiawei ZhangNovel View Synthesis 3D Gaussian Splatting
University of California, Davis CA 95616, USA
Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Foundation Models
Oct 5, 2026Zhimin Shao, Xijun Liu, Zhaoliang Zhang +43D Foundation Models Token Merging
Department of Electrical and Computer Engineering Johns Hopkins University · Department of Data Science University of Virginia
CLoSeR: Closing the Loop for Long-Context Streaming Reconstruction
Oct 1, 2026Moyang Li, Zihan Zhu, Wei Zhang +23D Foundation Models Streaming 3D Reconstruction
ETH Zurich · University of Stuttgart · Microsoft
TaoFlowForge: Progressive Native Mesh Generation via Cascaded Flow Matching
Sep 29, 2026Xianze Fang, Qiyuan Feng, Dongfang Sun +63D Foundation Models 3D Mesh Generation
Taobao3D Team, Alibaba Group
nnFoundation: 3D Foundation Models for Radiology
Sep 22, 2026Constantin Ulrich Harsy, Tassilo Wald, Karol Gotkowski +803D Foundation Models Medical Imaging Foundation Models
Toward a foundation model for forest point clouds
Sep 21, 2026Yuanwen Yue, Stefano Puliti, Damien Robert +73D Foundation Models Point Cloud Classification
ETH Zurich · University of Oxford · Norwegian Institute of Bioeconomy Research (NIBIO) +1
Adaptive World Memory 3D Foundation Model for Scalable 3D Mapping, Localization, and Rendering
Sep 18, 2026Tianchen Deng, Guole Shen, Yilin Shen +73D Mapping 3D Foundation Models
School of Automation and Intelligent Sensing, Shanghai Jiao Tong university and State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Key Laboratory of Navigation and Location Based Services, Shanghai 200240, China · Nanyang Technological University, Singapore · University of Technology Nuremberg, Germany
GeoCond: A Conditioning-Aware Reliability Adapter for Feed-Forward 3D Reconstruction
Sep 16, 2026David Ahmedt-Aristizabal, Mohammad Ali Armin, Russell Tsuchida +1UQ for 3D Reconstruction 3D Reconstruction
CSIRO, Australia · Monash University, Australia
Beyond Argmax: A Mechanistic Study of Semantic Retention in Frozen Foundation-Model Composition for Generalized Few-Shot 3D Segmentation
Sep 10, 2026Silas Kwabla Gah, Ebenezer Owusu3D Foundation Models Ensemble Learning
Department of Computer Science University of Ghana, Legon
Zero-Shot Novel Depth Synthesis Using 3D Foundation Models Scene Representations
Sep 3, 2026Denis M. Akola, David F. Fouhey3D Foundation Models 3D Reconstruction
New York University, Tandon School of Engineering, 1 MetroTech Center, Brooklyn, NY 11201, USA · Courant Institute of Mathematical Sciences, 251 Mercer St, New York, NY 10012
VI3: Grounding Pretrained 3D Foundation Models with Inertial Cues
Sep 3, 2026Ernesto Lozano, Alberto Jaenal, Javier Civera3D Foundation Models
I3A, Universidad de Zaragoza, Spain
Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation
Sep 1, 2026R. James Cotton, Divya Joshi, Colleen Peyton3D Foundation Models Markerless Motion Capture
1Shirley Ryan AbilityLab · Department of Physical Medicine and Rehabilitation, Northwestern University · Department of Physical Therapy and Human Movement Science, Northwestern University +1
GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame
Aug 30, 2026Zhe Dong, Wanqing Wu, Yuzhe Sun +53D Foundation Models 3D Reconstruction
School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China · 54th Research Institute, China Electronics Technology Group Corporation, Shijiazhuang 050081, China · State Key Laboratory of Comprehensive PNT Network and Equipment Technology, Shijiazhuang 050081, China +1
Lift, Associate, and Fuse: A Decision-Centric Framework for 2D-to-3D Foundation Model Transfer
Aug 21, 2026Wentao Sun, Yiping Chen, John S. Zelek +1Transfer Learning 3D Semantic Segmentation
Department of Systems Design Engineering, University of Waterloo, N2L 3G1 Waterloo, Canada · School of Geospatial Engineering and Science, Sun Yat-sen University, 519082 Zhuhai, China
Do 3D Medical Foundation Models See Through MRI Artifacts? A Controlled Study of Representation Robustness
Aug 6, 2026Julia Anna Mielcarz, Daniel Klaaby, Mostafa Mehdipour Ghazi3D Foundation Models Medical Imaging Foundation Models
Pioneer Centre for AI, University of Copenhagen, Copenhagen, Denmark
Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models
Aug 6, 2026Maulik Chevli, Johannes Brandt, Rickmer Braren +23D Foundation Models Medical Imaging Foundation Models
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Dept. of Diagnostic and Interventional Radiology, UKE Hamburg, Germany · Dept. of Computing, Imperial College London, UK +1
Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding
Aug 4, 2026Lucy Lin, Ayush Jain, Yifan Liu +13D Foundation Models 3D VQA
Carnegie Mellon University
Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editing
Aug 3, 2026Junliang Ye, Kenkun Liu, Guocun Wang +133D VLMs Part-Level 3D Generation
Tencent Hunyuan
UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routing
Jul 27, 2026Zefan Qu, Zhenwei Wang, Gerhard Petrus Hancke +13D Foundation Models Cross-Attention
Department of Computer Science City University of Hong Kong
Latent Riemannian Flow Matching for Geometry-Grounded 3D Foundation Models
Jul 21, 2026Lisa Weijler, Irene Ballester, Guofeng Mei +23D Foundation Models Flow Matching
TU Wien · Fondazione Bruno Kessler · Imperial College London
ABot-3DWorld 0: A Universal World Model to Explore Any 3D Space
Jul 13, 2026Mingchao Sun, Luyang Tang, Yu Liu +343D Scene Representation Panoramic 3D Reconstruction
AMAP CV Lab Alibaba Group

Oct 24, 2026J/K move · Enter open · S save

ProMo3D: Probing Motion Cues in Frozen 3D Foundation Models

Xiaoang Zhang, Mert Kiray, Yordanka Velikova, Manoj Biswanath+1

Technical University of Munich (TUM), Germany · Munich Center for Machine Learning (MCML), Germany · Obsphera, Munich, Germany

Motion Segmentation 3D Foundation Models Video Object Segmentation Representation Probing

Open paper AskPDF arXiv

Feed-forward 3D foundation models have recently reshaped multi-view reconstruction, yet dynamic 4D understanding still often relies on large-scale motion supervision or hand-crafted, architecture-specific heuristics. We ask whether frozen 3D foundation model tokens already encode motion-relevant information that can be exposed by lightweight readout functions, rather than re-learning motion-specific features with high-capacity decoders. Through probing on three representative backbones, we find that static--dynamic separation is decodable from frozen token representations. Our rank-restricted and spectral analyses on

π^3

X reveal that the recovered motion cues are concentrated in a low-dimensional subspace. Building on this, we propose Probing-based Motion Distillation (PMD), which converts pairwise motion-state compatibility scores against a fixed anchor token into dense motion masks. Trained on only 160 YouTube-VOS videos using a single 16 GB GPU, PMD transfers zero-shot to DAVIS, SegTrackv2, and FBMS-59 without target-benchmark fine-tuning. With multi-view pretrained geometric backbones, PMD outperforms training-free heuristics and approaches substantially heavier supervised and self-supervised methods, suggesting that motion decoding from frozen 3D foundation models can be lightweight and data-efficient.