3D ViTs

3D Vision Transformers apply attention to volumetric images, point clouds, meshes, and multi-view images to learn spatial structure for tasks such as medical segmentation, object detection, and scene reconstruction. Common approaches include voxel or point-token encoders, query-based decoders, camera- or geometry-aware positional encodings, and fusion across views or modalities. They are evaluated on benchmarks including ScanNet, nuScenes, BraTS, and ShapeNet, with newer work also adapting pretrained geometry models to streaming video and long sequences. A central tension is how to preserve fine geometric detail and cross-view consistency while reducing the memory and computation required by attention, especially when transferring across sensors and domains.