Efficient ViTs

ViT: Vision Transformer

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 300% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 152

All topics

Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
May 22, 2026Shuhong Zheng, Michael Oechsle, Erik Sandström +33D ViTs Multi-View 3D Reconstruction
University of Toronto & Vector Institute · Google · Technical University of Munich
Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models
May 22, 2026Maxim Henry, Adrien Deliège, Sébastien Piérard +1Deep Learning Optimization Efficient ViTs
Montefiore Institute, University of Liège, Liège, Belgium
ASAP: Attention Sink Anchored Pruning
May 21, 2026Jaehyuk Lee, Hanyoung Kim, Yanggee Kim +1Self-Attention Efficient ViTs
Department of Mathematics Korea University Seoul, Republic of Korea · Korea University Seoul, Republic of Korea
Accelerating Vision Foundation Models with Drop-in Depthwise Convolution
May 21, 2026Carmelo Scribano, Mohammad Mahdi, Nedyalko Prisadnikov +5Vision Foundation Models Vision Foundation Model Adaptation
University of Modena and Reggio Emilia, Modena, Italy. · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria
Machine Intelligence that Understands Visual and Linguistic Information and Interacts with Humans and Environments
May 20, 2026Van Quang NguyenImage Captioning Efficient ViTs
TOHOKU UNIVERSITY · Graduate School of Information Sciences · Department of System Information Sciences
Faster or Stronger: Towards Flexible Visual Place Recognition via Weighted Aggregation and Token Pruning
May 19, 2026Zichao Zeng, June Moh Goo, Junwei Zheng +4Visual Place Recognition Efficient ViTs
University College London, London, UK · Karlsruhe Institute of Technology, Karlsruhe, Germany · Shenzhen University, Shenzhen, China +1
LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift
May 18, 2026Haozhe Si, Yuxuan Wan, Yuqing Wang +2Representation Learning Masked Autoencoders
Department of Electrical and Computer Engineering · 2Siebel School of Computing and Data Science
Token-Space Mask Prediction for Efficient Vision Transformer Segmentation
May 18, 2026Calvin Galagain, Martyna Poreba, François GouletteVision Transformer Efficient ViTs
Universit´e Paris-Saclay, CEA List · U2IS, ENSTA Paris, Institut Polytechnique de Paris · F-91120 Palaiseau, France
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
May 17, 2026Hoai-Chau Tran, Chi H. Nguyen, Duy M. H. Nguyen +3Efficient Transformer Inference Image Segmentation
University of Illinois at Urbana-Champaign · VinUni-Illinois Smart Health Center, VinUniversity · College of Engineering & Computer Science, VinUniversity +3
Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability
May 17, 2026Nisreen Albzour, Sarah S. LamVision Transformer Efficient ViTs
School of Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY 13902, USA
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
May 15, 2026Yuxin Ren, Maxwell D Collins, Miao Hu +1Self-Attention Vision Transformer
University of Arizona, Tucson AZ, USA · TetraMem, Inc., San Jose CA, USA
Representative Attention For Vision Transformers
May 14, 2026Yuntong Li, Hainuo Wang, Hengxing Liu +2Vision Transformer Efficient ViTs
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
TurboVGGT: Fast Visual Geometry Reconstruction with Adaptive Alternating Attention
May 14, 2026David Huang, Guile Wu, Chengjie Huang +23D ViTs Multi-View 3D Reconstruction
Huawei Noah’s Ark Lab · University of Toronto · Foundation Model Department, Huawei
SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection
May 13, 2026Sandro Papais, Lezhou Feng, Charles Cossette +1Multi-View 3D Object Detection Token Pruning
University of Toronto · Zoox Inc.
Evolving Layer-Specific Scalar Functions for Hardware-Aware Transformer Adaptation
May 13, 2026Kieran Carrigg, Sigur de Vries, Amirhossein Sadough +1Efficient Neural Network Inference Efficient ViTs
Department of Machine Learning and Neural Computing Donders Institute for Brain, Cognition, and Behaviour Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Cores
May 12, 2026Alan Z. Song, Yinjie Chen, Mu Nan +3Visual Representation Learning Vision Transformer
Carnegie Mellon University · University of Hong Kong
Spectral Vision Transformer for Efficient Tokenization with Limited Data
May 12, 2026Alexandra G. Roberts, Maneesh John, Jinwei Zhang +10Medical Imaging Image Tokenization
DORA: Dynamic Online Reinforcement Agent for Token Merging in Vision Transformers
May 12, 2026Kaixuan He, Song Chen, Yi KangEfficient ViTs Visual Token Merging
University of Science and Technology of China, Hefei, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Breaking Global Self-Attention Bottlenecks in Transformer-based Spiking Neural Networks with Local Structure-Aware Self-Attention
May 12, 2026Lingdong Li, Hangming Zhang, Qiang YuSpiking Transformers Efficient ViTs
School of Future Technology, Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin 300354, China · School of Artificial Intelligence, Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin 300354, China
Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction
May 12, 2026Haoyu Zhang, Zeyu Zhang, Zedong Zhou +2Quantization-Aware Training 3D Reconstruction
1Peking University · 2La Trobe University
RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
May 11, 2026Byeongchan Kim, Arijit Sehanobish, Avinava Dubey +23D ViTs Self-Attention
1Seoul National University · 2Independent · 3Google Research +2
Investigating Single-Block Recurrence in Vision Transformers for Image Recognition
May 11, 2026Michal Byra, Pawel Olszowiec, Grzegorz Stefanski +2Vision Transformer Recurrent Transformers
Samsung AI Center, Warsaw, Poland · Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland
MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers
May 11, 2026Novendra Setyawan, Chi-Chia Sun, Mao-Hsiu Hsu +2Energy-Efficient ML Edge Inference
Department of Electro-Optics, National Formosa University, Taiwan · Department of Electrical Engineering, University of Muhammadiyah Malang, Indonesia · Department of Electrical Engineering, National Taipei University, Taiwan +1
PrePARE: Pre-AA Token Pruning for Frozen Multi-View Geometry Transformers
May 8, 2026Haotang Li, Zhenyu Qi, Shaohan Henry Wang +5Multi-View Geometry Efficient ViTs
University of Arizona · East Carolina University · Augusta University +1
An Efficient Token Compression Framework for Visual Object Tracking
May 8, 2026Weijing Wu, Qihua Liang, Bineng Zhong +3Visual Object Tracking Efficient ViTs
Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, 541004, China · Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China · University Engineering Research Center of Educational Intelligent Technology, Guangxi Normal University, Guilin, 541004, China +1
Amortized-Precision Quantization for Early-Exit Vision Transformers
May 8, 2026Rui Fang, Hsi-Wen Chen, Ming-Syan ChenMixed-Precision Quantization Dynamic Neural Networks
National Taiwan University
SAFformer:Improving Spiking Transformer via Active Predictive Filtering
May 8, 2026Zequan Xie, Weiming Zeng, Yunhua Chen +3Spiking Transformers Efficient ViTs
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China · Faculty of Science, Hong Kong Baptist University, China · School of Electronic Information, Wuhan University, China
LookWhen? Fast Video Recognition by Learning When, Where, and What to Compute
May 7, 2026Ali Salamatian, Anthony Fuller, Pritam Sarkar +3Efficient ViTs Video Action Recognition
University of British Columbia · Carleton University · Vector Institute
Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction
May 7, 2026Zecheng Tang, Jiaye Fu, Qiankun Gao +53D Reconstruction Efficient ViTs
School of Electronic and Computer Engineering, Peking University, Shenzhen 518055, China
RD-ViT: Recurrent-Depth Vision Transformer for Semantic Segmentation with Reduced Data Dependence Extending the Recurrent-Depth Transformer Architecture to Dense Prediction
May 5, 2026Renjie HeMedical Image Analysis Recurrent Transformers
Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

May 22, 2026, cs.CVJ/K move · Enter open · S save

Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers

Shuhong Zheng, Michael Oechsle, Erik Sandström, Marie-Julie Rakotosaona+2

University of Toronto & Vector Institute · Google · Technical University of Munich

3D ViTs Multi-View 3D Reconstruction Efficient ViTs Visual Token Pruning

Open paper AskPDF arXiv

Visual geometry transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. However, their computational cost grows quadratically with the input sequence length due to the global attention layers inside these models. This limits both their scalability and efficiency. In this work, we address this challenge with a simple yet general strategy: restricting the number of key/value tokens that each query interacts with during global attention. To achieve effective token selection, we introduce a two-stage framework. First, an inter-frame selection step operates at the frame level to identify frames that should be preserved. Second, an intra-frame selection step further discards more redundant tokens within the selected frames. Our analysis highlights the advantage of a diversity-based strategy for inter-frame selection, which ensures broad coverage of the scene. For intra-frame selection, we show that layer-aware sparsification is necessary, with the selection process guided by the entropy of the global attention pattern. Our approach offers a superior speed-accuracy trade-off compared to existing solutions. Extensive experiments show that it accelerates visual geometry transformers by over 85% for scenes with 500 images while maintaining, or even improving, baseline performance, which hints that how our token selection strategy can play a crucial role in future applications of visual geometry transformers. Our project website is available at https://zsh2000.github.io/good-token-hunting.github.io.

Momentum

Related topics

Latest papers 152