Efficient ViTs

ViT: Vision Transformer

Overview

Efficient Vision Transformer research seeks to make visual models practical at high resolutions and on limited hardware without losing the accuracy needed for classification, detection, and segmentation. Common approaches replace or constrain full self-attention with linear, windowed, or hybrid attention, and reduce work through token pruning or merging, structured pruning, quantization, and adaptive early exits. Researchers evaluate these methods on benchmarks such as ImageNet, COCO, and ADE20K, as well as on edge devices and dense prediction tasks where fine details matter. A central tension is that fewer FLOPs or tokens do not always mean faster real-world inference, and aggressive compression can harm accuracy, robustness, or small-region detail.

Latest papers

CardsList

255 papers19 papers in the last four weeks, down 44% on the four weeks before. 0.1% of all new papers.

  1. AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency

    Nov 18, 2026Zecheng Hao, Zihan Huang, Shenghao Xie +3Spiking TransformersSpiking Neural Networks

  2. SWAM: Saliency-Weighted Asymmetric Merging for Efficient Vision Transformers

    Nov 13, 2026Utkarsh Kumar Singh, Manish Das, Shiv Ram Dubey +1Token MergingEfficient ViTs

  3. Training-Free Acceleration of Multi-View 3D Transformers via Regime-Aware Attention Approximation

    Nov 13, 2026Muhammed Izzet Sezer, Sevval Belkis Dikkaya3D ViTsEfficient ViTs

  4. FoveaDet: Software-Hardware Co-Design of Foveated CNN-Transformer for Object Detection

    Nov 11, 2026Deepak Mewada, Apporva Saraf, Ashwin LeleEfficient Object DetectionObject Detection

  5. Differentiable Virtual-Slot Partitioning for Adaptive Tokenization in Vision Transformers

    Nov 8, 2026Gaoxiang He, Bin Zhu, Bo Xie +3Adaptive TokenizationEfficient ViTs

  6. Beyond Quadratic Attention: Separable Gaussian Operators for ODE Vision Transformers

    Nov 5, 2026Carlos Boned Riera, David Romero Sanchez, Oriol Ramos TerradesTransformer AttentionEfficient ViTs

  7. ChanSFormer: A Channel Agnostic Vision Transformer for Multi-Channel Cell Painting Images

    Nov 4, 2026Jingwei Zhang, Srinivasan Sivanandan, Dimitris SamarasVision TransformerEfficient ViTs

  8. Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers

    Nov 3, 2026Aditya Gupta, Jean Sebastien Dandurand, Kai Qiu +2Vision TransformerEfficient ViTs

  9. Exploiting Biases in Attention for Transformers

    Nov 3, 2026Charles Herr, Gabriele Spadaro, Enzo TartaglioneTransformer AttentionAttention Mechanisms

  10. LiSA: Linear Stochastic Attention for Long-Range Visual Modeling

    Nov 3, 2026Sahar Dastani, Fereshteh Shakeri, Samuel Barbeau +4Linear AttentionEfficient ViTs

  11. Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers

    Oct 29, 2026Jiongfan Yang, Weichen Fan, Haiwen Diao +2Efficient ViTsTokenization

  12. ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

    Oct 29, 2026Wenxi Li, Jingchen Huang, Chenyang Lyu +4Object DetectionEfficient Object Detection

  13. SSQA: Sibling-Selective Quadtree Attention for Hierarchical Modeling in Perception Tasks

    Oct 28, 2026Yufan Chen, Arnav Bali, Angela Liu +2Efficient ViTsHierarchical Representation Learning

  14. ConvRoute-ViT: Convolutional Token Routing for Efficient Vision Transformers

    Oct 28, 2026Manish Das, Shiv Ram Dubey, Satish Kumar SinghVision TransformerEfficient ViTs

  15. ACSAN: All-Convolutional Self-Attention Network Integrating Convolutional Operations with Vision Transformer

    Oct 26, 2026Zimian Wei, Tianqi Zhang, Hengyue PanHybrid CNN-Transformer ArchitecturesEfficient ViTs

  16. Promise and Limits of Cross-Architecture Knowledge Distillation for Lightweight Medical Vision Transformers (LiteMed-ViT)

    Oct 24, 2026Shuaibu Shehu, Elijah M. Flomo, Mohammad Safiya +2Efficient ViTsCross-Architecture Knowledge Distillation

  17. Consommé: Conditioned Token Merging for Split Inference

    Oct 23, 2026Man Ngo, Jan S. RellermeyerSplit InferenceEfficient ViTs

  18. BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

    Oct 21, 2026Shujun Yao, Junhui Lin, Congying Han +3Multi-View 3D ReconstructionFeed-Forward 3D Reconstruction

  19. Client-Aware Security Adaptive Early Exit Vision Transformers for Heterogeneous Federated Learning

    Oct 16, 2026Sahar Refaei Ghalibafan, Samira Nazari, Christian Herglotz +2Early-Exit Neural NetworksHeterogeneous Federated Learning

  20. MaskMerge: Structure-aware Token Merging for Efficient Instance Segmentation

    Oct 16, 2026Bavesh Balaji, Letian Jiang, Alexander Wong +2Token MergingVisual Token Merging

  21. SCMerge: Spatial-Consistency-Aware Token Pair Merging for Efficient Semantic Segmentation

    Oct 15, 2026Junfeng Ren, Yi Gong, Yuan ZengVisual Token MergingEfficient ViTs

  22. Adaptive Observation Control for Active Visual Recognition

    Oct 14, 2026Dikai Li, Weijia Wang, Yuan-Gen WangActive Visual PerceptionAdaptive Computation

  23. Prune-and-Merge: Training-Free Scaling of Vision Transformers

    Oct 13, 2026Vatsal, Soumil Malhotra, Priyanshu Chahal +1Vision TransformerEfficient ViTs

  24. Bottlenecked semantic compression by representation reconstruction

    Oct 12, 2026Xiaoya Tang, Shilong Liu, Daniel Smolyak +3Visual Token CompressionEfficient ViTs

  25. ConDenseDrive: Knowledge-Dense, Token-Sparse Visual Encoder for End-to-End Autonomous Driving

    Oct 12, 2026Shengzhen Li, Yongkang Li, Hao Gao +5End-to-End Autonomous DrivingEfficient ViTs

  26. Growing to See: Efficient Pretraining of Self-Supervised Vision Transformers

    Oct 12, 2026Amir Mohammad Karimi Mamaghan, Ferdinand Kapl, Andrea Dittadi +4Self-Supervised Visual Representation LearningSelf-Supervised Pre-Training

  27. BARTER: Compact Dynamic Sparse Training with Standard Dense Kernels

    Oct 12, 2026Yueyuan Sui, Payal Mohapatra, Qi Zhu +1Dynamic Sparse TrainingNeural Network Pruning