Efficient ViTs

ViT: Vision Transformer

Ask about this topic

Overview

Efficient Vision Transformer research seeks to make visual models practical at high resolutions and on limited hardware without losing the accuracy needed for classification, detection, and segmentation. Common approaches replace or constrain full self-attention with linear, windowed, or hybrid attention, and reduce work through token pruning or merging, structured pruning, quantization, and adaptive early exits. Researchers evaluate these methods on benchmarks such as ImageNet, COCO, and ADE20K, as well as on edge devices and dense prediction tasks where fine details matter. A central tension is that fewer FLOPs or tokens do not always mean faster real-world inference, and aggressive compression can harm accuracy, robustness, or small-region detail.

Latest papers

CardsList

243 papers29 papers in the last four weeks, up 32% on the four weeks before. 0.2% of all new papers.

  1. Read, Allocate, Route: Role-Coupled Tokenization for Pixel-Space Generative Transformers

    Oct 29, 2026Jiongfan Yang, Weichen Fan, Haiwen Diao +2Efficient ViTsTokenization

    Peking University · SenseTime · Nanyang Technological University

  2. ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer

    Oct 29, 2026Wenxi Li, Jingchen Huang, Chenyang Lyu +4Object DetectionEfficient Object Detection

    KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China +3

  3. SSQA: Sibling-Selective Quadtree Attention for Hierarchical Modeling in Perception Tasks

    Oct 28, 2026Yufan Chen, Arnav Bali, Angela Liu +2Efficient ViTsHierarchical Representation Learning

    Thomas Lord Department of Computer Science, University of Southern California, Los Angeles, CA, USA. · South River High School, Edgewater, MD, USA. · Department of Computer Science, University of Maryland, College Park, MD, USA.

  4. ConvRoute-ViT: Convolutional Token Routing for Efficient Vision Transformers

    Oct 28, 2026Manish Das, Shiv Ram Dubey, Satish Kumar SinghVision TransformerEfficient ViTs

    IIIT Allahabad, Prayagraj, India

  5. ACSAN: All-Convolutional Self-Attention Network Integrating Convolutional Operations with Vision Transformer

    Oct 26, 2026Zimian Wei, Tianqi Zhang, Hengyue PanHybrid CNN-Transformer ArchitecturesEfficient ViTs

    College of Computer Science and Technology, National University of Defense Technology, 109 Deya Road, Changsha, 410073, Hunan, China

  6. Promise and Limits of Cross-Architecture Knowledge Distillation for Lightweight Medical Vision Transformers (LiteMed-ViT)

    Oct 24, 2026Shuaibu Shehu, Elijah M. Flomo, Mohammad Safiya +2Efficient ViTsCross-Architecture Knowledge Distillation

    SR University · GITAM University · Dr. KNS Memorial Institute of Medical Sciences

  7. Consommé: Conditioned Token Merging for Split Inference

    Oct 23, 2026Man Ngo, Jan S. RellermeyerSplit InferenceEfficient ViTs

    Leibniz University Hannover Hannover, Germany

  8. TileShare: Training-Free Acceleration of Overlapping Tiled ViT Decoding

    Oct 23, 2026Xiangzhong Ye, Zhongshen Li, Shen You +6Efficient ViTsVision Transformer

  9. BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

    Oct 21, 2026Shujun Yao, Junhui Lin, Congying Han +3Multi-View 3D ReconstructionFeed-Forward 3D Reconstruction

    University of Chinese Academy of Sciences, Beijing, China · the University of Maryland, College Park, MD, USA · Beijing University of Posts and Telecommunications, Beijing, China

  10. Client-Aware Security Adaptive Early Exit Vision Transformers for Heterogeneous Federated Learning

    Oct 16, 2026Sahar Refaei Ghalibafan, Samira Nazari, Christian Herglotz +2Early-Exit Neural NetworksHeterogeneous Federated Learning

    Brandenburg University of Technology, Cottbus, Germany · University of Zanjan, Zanjan, Iran · Tallinn University of Technology, Tallinn, Estonia +1

  11. MaskMerge: Structure-aware Token Merging for Efficient Instance Segmentation

    Oct 16, 2026Bavesh Balaji, Letian Jiang, Alexander Wong +2Token MergingVisual Token Merging

    University Of Waterloo

  12. SCMerge: Spatial-Consistency-Aware Token Pair Merging for Efficient Semantic Segmentation

    Oct 15, 2026Junfeng Ren, Yi Gong, Yuan ZengVisual Token MergingEfficient ViTs

    Southern University of Science and Technology · Shenzhen Technology University

  13. Adaptive Observation Control for Active Visual Recognition

    Oct 14, 2026Dikai Li, Weijia Wang, Yuan-Gen WangActive Visual PerceptionAdaptive Computation

    School of Artificial Intelligence, Guangzhou University, Guangzhou, China

  14. Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression

    Oct 13, 2026Vatsal, Soumil Malhotra, Priyanshu Chahal +1Vision TransformerEfficient ViTs

    Department of Mathematics, Indian Institute of Technology Roorkee

  15. Bottlenecked semantic compression by representation reconstruction

    Oct 12, 2026Xiaoya Tang, Shilong Liu, Daniel Smolyak +3Visual Token CompressionEfficient ViTs

    Scientific Computing and Imaging Institute University of Utah · Princeton AI Lab Princeton University · National Institutes of Health

  16. ConDenseDrive: Knowledge-Dense, Token-Sparse Visual Encoder for End-to-End Autonomous Driving

    Oct 12, 2026Shengzhen Li, Yongkang Li, Hao Gao +5End-to-End Autonomous DrivingEfficient ViTs

    Huazhong University of Science and Technology · Yinwang Intelligent Technology Co. Ltd.

  17. Growing to See: Efficient Pretraining of Self-Supervised Vision Transformers

    Oct 12, 2026Amir Mohammad Karimi Mamaghan, Ferdinand Kapl, Andrea Dittadi +4Self-Supervised Visual Representation LearningSelf-Supervised Pre-Training

    KTH Royal Institute of Technology · Helmholtz AI, Munich · Technical University of Munich +3

  18. BARTER: Compact Dynamic Sparse Training with Standard Dense Kernels

    Oct 12, 2026Yueyuan Sui, Payal Mohapatra, Qi Zhu +1Dynamic Sparse TrainingNeural Network Pruning

    Northwestern University, USA · Arm Inc., USA

  19. Retraining-Free Depth Pruning of Vision Transformers: A Cross Domain Study of Layer Redundancy

    Oct 10, 2026Alessandro Viespoli, Loris NanniTransformer PruningEfficient ViTs

    University of Padua

  20. Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction

    Oct 8, 2026Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh +33D Foundation ModelsFeed-Forward 3D Reconstruction

    Advanced Micro Devices Inc. (AMD)

  21. One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts

    Oct 8, 2026Adrian Bulat, Yassine Ouali, Georgios TzimiropoulosEfficient ViTsMixture of Experts

    Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London

  22. SV-TAD: Native Sparse Convs for Efficient Temporal Action Detection

    Oct 8, 2026Ricardo Pizarro, Roberto Valle, José M. Buenaposada +2Temporal Action DetectionEfficient ViTs

    Universidad de Alcal´a, Alcal´a de Henares, Spain · Universidad Polit´ecnica de Madrid, Madrid, Spain · Universidad Rey Juan Carlos, M´ostoles, Spain

  23. Dissecting Representation Structure in Vision Transformers: A Rigorous Architectural Study

    Oct 8, 2026Kim-Cuc Nguyen, Ngai-Man CheungVision TransformerNeural Network Generalization

    Temasek Laboratories Singapore University of Technology and Design Singapore

  24. Hardware-aware Calibrated Clustered Attention for Efficient Visual Geometric Transformers

    Oct 7, 2026Weitian Wang, Shubham Rai, Cecilia De La Parra +1Neural Network Hardware AccelerationEfficient ViTs

    Robert Bosch GmbH, Germany · Ruhr University Bochum, Germany

  25. Later Is Better: Token Reduction for ViTs Under Distribution Shift

    Oct 6, 2026Hyeongheon Cha, Hyungjun Yoon, Sung-Ju LeeDistribution Shift RobustnessToken Pruning

    KAIST

  26. FLASHSWIN: Unlocking Large Windows and Dense Tokens in Swin Vision Transformers with Memory Efficient Attention

    Oct 3, 2026Tushar Kataria, Gerald Sabin, Ponnuswamy Sadayappan +1Vision TransformerEfficient ViTs

    Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, UT, USA · Kahlert School of Computing, University of Utah, Salt Lake City, UT, USA · RNET Technologies, Dayton, OH, USA

  27. RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers

    Oct 3, 2026Mengyuan Fan, Bokai Huang, JiaMing Pan +4Low-Bit QuantizationRotation-Based Quantization

    Peking University

  28. VASC: Value-Aware Sparse Attention with Cross-Layer Memory for Efficient 3D Reconstruction

    Oct 1, 2026Junyi Wu, Fanqing Kong, Leyang Chen +23D ViTs3D Reconstruction

    Shanghai Jiao Tong University

  29. Pixel-Level Transformers in Remote Sensing: A Canopy Height Case Study

    Sep 29, 2026Sven Ligensa, Jan Pauls, Karsten Schrödter +2Remote Sensing Image UnderstandingRemote Sensing

    University of Münster Münster, Germany · Laboratoire des Sciences du Climat et de l’Environnement Paris, France

  30. EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception

    Sep 29, 2026Yaoxin Niu, Zhangquan Chen, Yang Zhang +5VLM AdaptationEfficient ViTs

    Tsinghua University · Peng Cheng Laboratory · The Hong Kong University of Science and Technology +3