3D ViTs

ViT: Vision Transformer

Overview

3D Vision Transformers apply attention to volumetric images, point clouds, meshes, and multi-view images to learn spatial structure for tasks such as medical segmentation, object detection, and scene reconstruction. Common approaches include voxel or point-token encoders, query-based decoders, camera- or geometry-aware positional encodings, and fusion across views or modalities. They are evaluated on benchmarks including ScanNet, nuScenes, BraTS, and ShapeNet, with newer work also adapting pretrained geometry models to streaming video and long sequences. A central tension is how to preserve fine geometric detail and cross-view consistency while reducing the memory and computation required by attention, especially when transferring across sensors and domains.

Latest papers

CardsList

157 papers13 papers in the last four weeks, down 19% on the four weeks before. 0.1% of all new papers.

  1. RETR: A Structure-Preserving RGB-Event Transformer for Robust 3D Lane Detection

    Nov 14, 2026Jingtao Dong, Hao Zhuang, Hao Yang +2Lane DetectionAutonomous Driving Perception

  2. Training-Free Acceleration of Multi-View 3D Transformers via Regime-Aware Attention Approximation

    Nov 13, 2026Muhammed Izzet Sezer, Sevval Belkis Dikkaya3D ViTsEfficient ViTs

  3. Rotation Is Token-Blind: A Structural Criterion for Post-Training Quantization of Feed-Forward Geometry Transformers

    Nov 5, 2026Huanfei He, Bohao Zhang, Jiaxin Zhang +4Post-Training QuantizationNeural Network Quantization

  4. Georeferenced Cross-View 3D Reconstruction from Ground and Satellite Images

    Nov 5, 2026David G. Shatwell, Mubarak Shah3D ReconstructionMulti-View 3D Reconstruction

  5. VINO: Revisiting 3D Representation Learning with Volume Transformers

    Nov 4, 2026Adrian Kruse, Kadir Yilmaz, Daan de Geus +23D Representation LearningRepresentation Learning

  6. GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats

    Nov 3, 2026Shahaf Valencia Levy, Sagie BenaimNeural Rendering3D Gaussian Splatting

  7. DUNEDepth: Bronchoscopic Depth and Pose Estimation via Hybrid Adaptation of a 3D-Vision Foundation Model

    Nov 2, 2026Qi Chang, Austin Kao, Ange Lou +1Depth EstimationCamera Pose Estimation

  8. Solv3R: 3D Looped Transformer as a Multi-view Solver

    Nov 2, 2026Jeong-gi Kwak, Andrea Tagliasacchi, Sho Kagami +23D ReconstructionMulti-View 3D Reconstruction

  9. Coarse3D: Controllable and Faithful Sparse Structure Generation for Image-to-3D

    Oct 30, 2026Gaige Wang, Bowen Pan, Hao Ren +23D ViTs

  10. GPA-VGGT: Geometry-Preserving Acceleration of VGGT

    Oct 27, 2026Muyeol Sung, Jangwon Kim, Jongwoo LimEfficient Transformer Inference3D ViTs

  11. BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

    Oct 21, 2026Shujun Yao, Junhui Lin, Congying Han +3Multi-View 3D ReconstructionFeed-Forward 3D Reconstruction

  12. ReDT: Resolution-Decoupled Visual Geometry Transformer

    Oct 18, 2026Abteen Arab, Guile Wu, Dongfeng Bai3D ViTs3D Reconstruction

  13. Motion- and Gravity-Aware Feedforward 3D Reconstruction

    Oct 15, 2026Shiyao Qian, Yuan Ren, Dongfeng BaiMulti-View 3D ReconstructionFeed-Forward 3D Reconstruction

  14. BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates

    Oct 12, 2026Lijian Li, Yuanpeng He, Yudian Zheng +6Streaming 3D ReconstructionBayesian Filtering

  15. GNFormer: Gaussian Neural Field Transformer for Efficient Camera-based Occupancy Prediction

    Oct 11, 2026Guangcheng Chen, Li He, Hong Zhang3D Occupancy Prediction3D ViTs

  16. Hardware-aware Calibrated Clustered Attention for Efficient Visual Geometric Transformers

    Oct 7, 2026Weitian Wang, Shubham Rai, Cecilia De La Parra +1Neural Network Hardware AccelerationEfficient ViTs

  17. S2Tok: Streaming 3D Gaussian Reconstruction with Persistent Spatial Tokens

    Oct 6, 2026Fang Li, Jiraphon Yenphraphai, Quentin Herau +5Streaming 3D Reconstruction3D Scene Representation

  18. VGGT-Bridge: Beyond Sequential Pose Graphs via Coarse-Stride Skip Edges

    Oct 5, 2026Sungjae Choi, Hanna Bae, Sunghyun Baek +13D ViTsStreaming 3D Reconstruction

  19. Deep Prior Learning for Embodied Perception

    Oct 4, 2026Yimou Wu, Jiaxin Guo, Yun-hui Liu +13D ViTsCamera Pose Estimation

  20. VASC: Value-Aware Sparse Attention with Cross-Layer Memory for Efficient 3D Reconstruction

    Oct 1, 2026Junyi Wu, Fanqing Kong, Leyang Chen +23D ViTs3D Reconstruction

  21. RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation

    Oct 1, 2026Minsu Kim, Jaesung Choe, Jiwoo Lee +23D ViTs3D Spatial Reasoning

  22. ReSS: Residual-Restoring Sparse Attention for 3D Vision Transformers

    Sep 28, 2026Yongsung Kim, Jaehoon Lee, Minjun Park +33D ViTsEfficient ViTs

  23. PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection

    Sep 26, 2026Hui Ye, Yudong Liu, Yiran Chen +2Multi-View 3D Object Detection3D Object Detection

  24. Revisiting Multi-View Stereo: A Sequence-to-Sequence Formulation

    Sep 21, 2026Aoxiang Fan, Corentin Dumery, Nicolas Talabot +1Multi-View Depth Estimation3D ViTs

  25. STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

    Sep 21, 2026Mena Kamel, Natalie Won, Amrut Sarangi +23D ViTsMulti-View Learning

  26. VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers

    Sep 20, 2026Abteen Arab, Guile Wu, Chengjie Huang +13D ViTsMulti-View 3D Reconstruction