Efficient ViTs

ViT: Vision Transformer

Follow this topic Ask about this topic

Momentum

27 papers in the last four weeks, up 59% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 171

All topics

Spatial Priors via Space Filling Curves for Small and Limited Data Vision Transformers
Jun 8, 2026Leyla Naz Candogan, Arshia Afzal, Pol Puigdemont +1Vision Transformer Efficient ViTs
LIONS, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.
Beyond Self-Attention: Sub-Quadratic Vision Transformers for Fast Image Captioning
Jun 7, 2026Chiradeep Ghosh, Dakshina Ranjan KiskuImage Captioning Vision Transformer
Department of Computer Science and Engineering · National Institute of Technology Durgapur · Durgapur, India
NGram-MoSE: Efficient Remote Sensing Super-Resolution via N-Gram Context and Mixture-of-Experts
Jun 7, 2026Yun-Hsuan Huang, Trong-An Bui, Chih-Hung ChuangOOD Generalization Remote Sensing Image Super-Resolution
Institute of Aerospace and Systems Engineering, National Taipei University of Technology, Taipei City, Taiwan.
RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT
Jun 6, 2026Kyumin Choi, Ikbeom JangVision Transformer Efficient ViTs
Hankuk University of Foreign Studies Seoul, South Korea
ATT-CR: Adaptive Triangular Transformer for Cloud Removal
Jun 4, 2026Yang Wu, Ye Deng, Pengna Li +4Remote Sensing Transformer Attention
Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China · School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China · Ningbo University of Technology, Ningbo, Zhejiang 315211, China
UtVAA: Ultra-tiny Vision Transformer with Affix Attention for Mobile Image Classification
Jun 2, 2026Romiyal George, Sathiyamohan Nishankar, Selvarajah Thuseethan +1Vision Transformer Edge Inference
University of Peradeniya, Sri Lanka · Charles Darwin University, Australia
Efficient Transformer-Based Localized Patch Sampling for Choroid Plexus Segmentation in Multiple Sclerosis
Jun 2, 2026Po-Jui Lu, Alessandro Cagol, Mario Ocampo-Pineda +12Efficient ViTs Magnetic Resonance Imaging
Translational Imaging in Neurology (ThINk) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland · Department of Neurology, University Hospital Basel, Basel, Switzerland · Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland +4
PSViT: A Methodology for Structurally Pruning Spiking Vision Transformers
Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Efficient ViTs Spiking Neural Networks
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates (UAE)
LALE: Lightweight-Transformer Architecture for Land-Cover Estimation
Jun 1, 2026Ümit Mert Çağlar, Alptekin TemizelRemote Sensing Image Segmentation Hybrid CNN-Transformer Architectures
Graduate School of Informatics, METU, Ankara, Turkey
hZACH-ViT: Curved Latent Geometry for Compact Vision Transformers in Low-Data Medical Imaging
May 30, 2026Athanasios AngelakisGeometric Representation Learning Efficient ViTs
BioML Lab, Research Institute CODE, UniBw, Munich, Germany · Department of Epidemiology and Data Science, Amsterdam UMC, Amsterdam, Netherlands
Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders
May 30, 2026Yitong Jiang, Hongjun Wang, Collin McCarthy +15GPU Kernel Optimization Vision Foundation Models
NVIDIA · The Chinese University of Hong Kong · The University of Hong Kong +1
FSM-Net: An Efficient Frequency-Spatial Network for Real-World Deblurring
May 29, 2026Vinh-Thuan LyImage Deblurring Frequency-Domain Image Processing
University of Science, VNU-HCM, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam
SwInception -- Local Attention Meets Convolutions
May 28, 2026David Hagerman, Roman Naeem, Jakob Lindqvist +3Hybrid CNN-Transformer Architectures Efficient ViTs
Chalmers University of technology, 412 96 Gothenburg, Sweden · Zenseact, Lindholmspiren 2, 417 56 Gothenburg, Sweden
Accelerating HEVC Intra Partitioning via a CNN-Hierarchical Attention Transformer Hybrid
May 27, 2026Krishna Kumar Sharma, Somdyuti PaulEfficient Neural Network Inference Hybrid CNN-Transformer Architectures
Department of Artificial Intelligence, Indian Institute of Technology Kharagpur, Kharagpur, West Bengal 721302, India
Recursive Vision Transformer with Dynamic Depth and Width Adjustment for Resource-Efficient Image Semantic Communication
May 27, 2026Zhilong Zhang, Xinhui Zhang, Gongyu Jin +3Dynamic Neural Networks Semantic Communication
Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China
Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking
May 27, 2026Hongtao Yang, Bineng Zhong, Qihua Liang +4Visual Object Tracking Efficient ViTs
Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, 541004, China · Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China · Nanjing University of Science and Technology +1
JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
May 26, 2026Dongyun Zou, Zhuoyang Zhang, Junyu Chen +8Efficient Transformer Inference Vision Foundation Models
MIT · NVIDIA · University of Pennsylvania +1
AdaMerge: Salience-Aware Adaptive Token Merging for Training-Free Acceleration of Vision Transformers
May 26, 2026Semi Lee, Hyejin Go, Hyesong ChoiToken Merging Self-Attention
Electronic Engineering · Soongsil University · Seoul, South Korea
Event-to-Video Reconstruction using Spatio-Temporal and Frequency-Enhanced Deep Neural Networks
May 25, 2026Ramna Maqsood, Paulo Nunes, Luís Ducla Soares +1Event-Based Vision Vision Transformer
Instituto de Telecomunicações, Instituto Universitário de Lisboa (ISCTE-IUL) Lisbon, Portugal · Instituto de Telecomunicac¸˜oes, Instituto Universit´ario de Lisboa (ISCTE-IUL) Lisbon, Portugal
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
May 22, 2026Shuhong Zheng, Michael Oechsle, Erik Sandström +33D ViTs Multi-View 3D Reconstruction
University of Toronto & Vector Institute · Google · Technical University of Munich
Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models
May 22, 2026Maxim Henry, Adrien Deliège, Sébastien Piérard +1Deep Learning Optimization Efficient ViTs
Montefiore Institute, University of Liège, Liège, Belgium
ASAP: Attention Sink Anchored Pruning
May 21, 2026Jaehyuk Lee, Hanyoung Kim, Yanggee Kim +1Self-Attention Efficient ViTs
Department of Mathematics Korea University Seoul, Republic of Korea · Korea University Seoul, Republic of Korea
Accelerating Vision Foundation Models with Drop-in Depthwise Convolution
May 21, 2026Carmelo Scribano, Mohammad Mahdi, Nedyalko Prisadnikov +5Vision Foundation Models Vision Foundation Model Adaptation
University of Modena and Reggio Emilia, Modena, Italy. · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria
Machine Intelligence that Understands Visual and Linguistic Information and Interacts with Humans and Environments
May 20, 2026Van Quang NguyenImage Captioning Efficient ViTs
TOHOKU UNIVERSITY · Graduate School of Information Sciences · Department of System Information Sciences
Faster or Stronger: Towards Flexible Visual Place Recognition via Weighted Aggregation and Token Pruning
May 19, 2026Zichao Zeng, June Moh Goo, Junwei Zheng +4Visual Place Recognition Efficient ViTs
University College London, London, UK · Karlsruhe Institute of Technology, Karlsruhe, Germany · Shenzhen University, Shenzhen, China +1
LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shift
May 18, 2026Haozhe Si, Yuxuan Wan, Yuqing Wang +2Representation Learning Masked Autoencoders
Department of Electrical and Computer Engineering · 2Siebel School of Computing and Data Science
Token-Space Mask Prediction for Efficient Vision Transformer Segmentation
May 18, 2026Calvin Galagain, Martyna Poreba, François GouletteVision Transformer Efficient ViTs
Universit´e Paris-Saclay, CEA List · U2IS, ENSTA Paris, Institut Polytechnique de Paris · F-91120 Palaiseau, France
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
May 17, 2026Hoai-Chau Tran, Chi H. Nguyen, Duy M. H. Nguyen +3Efficient Transformer Inference Image Segmentation
University of Illinois at Urbana-Champaign · VinUni-Illinois Smart Health Center, VinUniversity · College of Engineering & Computer Science, VinUniversity +3
Systematic Evaluation of Vision Transformers for Automated Cervical Cancer Classification: Optimization, Statistical Validation, and Clinical Interpretability
May 17, 2026Nisreen Albzour, Sarah S. LamVision Transformer Efficient ViTs
School of Systems Science and Industrial Engineering, Binghamton University, Binghamton, NY 13902, USA
From Sparsity to Simplicity: Enabling Simpler Sequential Replacements via Sparse Attention Distillation
May 15, 2026Yuxin Ren, Maxwell D Collins, Miao Hu +1Self-Attention Vision Transformer
University of Arizona, Tucson AZ, USA · TetraMem, Inc., San Jose CA, USA

Jun 8, 2026, cs.CVJ/K move · Enter open · S save

Spatial Priors via Space Filling Curves for Small and Limited Data Vision Transformers

Leyla Naz Candogan, Arshia Afzal, Pol Puigdemont, Volkan Cevher

LIONS, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland.

Vision Transformer Efficient ViTs Attention Mechanisms

Open paper AskPDF arXiv

Though Vision Transformers (ViTs) have become the dominant backbone in many computer vision tasks, due to permutation equivariance, their attention mechanism lacks explicit spatial inductive biases. This become particularly important in two settings: when model capacity is small or training data is limited. Inspired by the attention masking strategies in Linear Transformers and the scanning patterns of Vision SSMs, we introduce VIOLIN, a lightweight masked attention mechanism that encodes spatial structure within attention via Space Filling Curves (SFCs) with less than 0.0015% extra parameters and negligible computational overhead. VIOLIN scans the image using multiple SFCs to construct curve-specific decay masks, which are then combined and multiplied with the attention matrix. Across a wide range of evaluations, VIOLIN consistently improves performance. In limited data regimes such as fine-tuning on VTAB-1K, it boosts accuracy across all task groups and by up to 8.7% on the tasks where spatial information is essential. It can be combined with parameter-efficient fine-tuning methods such as LoRA to further increase the performance. Beyond fine-tuning, VIOLIN improves various small scale ViT architectures (e.g., DeiT, DINO) during pretraining on ImageNet-1K. Additionally, on pixel-level CIFAR-100 training, a task that is highly dependent on location information, VIOLIN increases accuracy by up to 7.2%. Overall, VIOLIN provides a computationally efficient yet effective way to inject spatial inductive bias into ViTs, especially benefiting small models and limited data settings.

Momentum

Related topics

Latest papers 171