3D ViTs

ViT: Vision Transformer

Overview

3D Vision Transformers apply attention to volumetric images, point clouds, meshes, and multi-view images to learn spatial structure for tasks such as medical segmentation, object detection, and scene reconstruction. Common approaches include voxel or point-token encoders, query-based decoders, camera- or geometry-aware positional encodings, and fusion across views or modalities. They are evaluated on benchmarks including ScanNet, nuScenes, BraTS, and ShapeNet, with newer work also adapting pretrained geometry models to streaming video and long sequences. A central tension is how to preserve fine geometric detail and cross-view consistency while reducing the memory and computation required by attention, especially when transferring across sensors and domains.

Latest papers

CardsList

157 papers13 papers in the last four weeks, down 19% on the four weeks before. 0.1% of all new papers.

  1. Axial-Centric Cross-Plane Attention for 3D Medical Image Classification

    Feb 25, 2026Doyoung Park, Jinsoo Kim, Lohendran BaskaranMedical Image Classification3D Medical Imaging

  2. Boosting Instance Awareness via Cross-View Correlation with 4D Radar and Camera for 3D Object Detection

    Feb 24, 2026Xiaokai Bai, Lianqing Zheng, Si-Yuan Cao +63D Object DetectionCamera-Radar Fusion

  3. RefineFormer3D: Efficient 3D Medical Image Segmentation via Adaptive Multi-Scale Transformer with Cross Attention Fusion

    Feb 18, 2026Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal3D Medical Image Segmentation3D ViTs

  4. MIC-BEV: Multi-Infrastructure Camera Bird's-Eye-View Transformer with Relation-Aware Fusion for 3D Object Detection

    Oct 28, 2025Yun Zhang, Zhaoliang Zheng, Johnson Liu +5Multi-View 3D Object Detection3D Object Detection

  5. CATSplat: Context-Aware Transformer with Spatial Guidance for Generalizable 3D Gaussian Splatting from A Single-View Image

    Dec 17, 2024Wonseok Roh, Hwanhee Jung, Jong Wook Kim +63D Gaussian Splatting3D ViTs

  6. Tackling fluffy clouds: robust agricultural field boundary delineation from Sentinel-1 and Sentinel-2 satellite image time series

    Sep 20, 2024Foivos I. Diakogiannis, Zheng-Shu Zhou, Jeff Wang +133D ViTsRemote Sensing Image Segmentation

  7. Multi-view Hand Reconstruction with a Point-Embedded Transformer

    Aug 20, 2024Lixin Yang, Licheng Zhong, Pengxiang Zhu +43D ViTsMarkerless Motion Capture