3D ViTs
ViT: Vision Transformer
Overview
3D Vision Transformers apply attention to volumetric images, point clouds, meshes, and multi-view images to learn spatial structure for tasks such as medical segmentation, object detection, and scene reconstruction. Common approaches include voxel or point-token encoders, query-based decoders, camera- or geometry-aware positional encodings, and fusion across views or modalities. They are evaluated on benchmarks including ScanNet, nuScenes, BraTS, and ShapeNet, with newer work also adapting pretrained geometry models to streaming video and long sequences. A central tension is how to preserve fine geometric detail and cross-view consistency while reducing the memory and computation required by attention, especially when transferring across sensors and domains.
Latest papers
145 papers14 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.