Real World 3D
Real-world 3D scene understanding aims to accurately perceive, reconstruct, and interact with three-dimensional environments using computational methods. Current research heavily focuses on developing robust and efficient algorithms, particularly leveraging diffusion models and neural fields, to address challenges like occlusion, varying data density, and the need for data-efficient learning. This field is crucial for advancing applications in robotics, autonomous driving, augmented reality, and medical imaging, driving the development of large-scale, multi-modal datasets and improved model architectures for tasks such as 3D object detection, segmentation, and reconstruction. The resulting improvements in 3D perception will have significant impact across numerous industries.
Papers
From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos
Matthew Wallingford, Anand Bhattad, Aditya Kusupati, Vivek Ramanujan, Matt Deitke, Sham Kakade, Aniruddha Kembhavi, Roozbeh Mottaghi, Wei-Chiu Ma, Ali Farhadi
LoRA3D: Low-Rank Self-Calibration of 3D Geometric Foundation Models
Ziqi Lu, Heng Yang, Danfei Xu, Boyi Li, Boris Ivanovic, Marco Pavone, Yue Wang