Vision Transformer

Also known as ViT

Vision Transformers divide images into patches and use self-attention to model relationships across them, supporting tasks from image classification to dense prediction and video understanding. Common approaches include supervised or self-supervised pretraining, hierarchical and local-global attention, and adaptation through fine-tuning, prompts, or lightweight adapters. Work also explores faster inference through token pruning, merging, quantization, and efficient attention, alongside interpretability and robustness tests on benchmarks such as ImageNet, COCO, and ADE20K. A central tension is that global context and flexible representations can come with high data and compute demands, while stronger spatial priors or compression may improve efficiency but affect transfer and fine detail.

Latest papers

373 papers33 papers in the last four weeks, down 25% on the four weeks before. 0.2% of all new papers.

Evaluating SAM-based and Convolutional Architectures in Thin Retinal Vessel Segmentation: Model Capacity, Computational Cost and Foundation Pre-training
Oct 30, 2026Carlos Eduardo Fontaneli, Cesar Henrique CominRetinal Vessel Segmentation Thin-Structure Segmentation
Department of Computer Science, Federal University of S˜ao Carlos, Rod. Washington Lu´ıs, km 235, 13565-905, S˜ao Carlos, SP, Brazil
A Patch is More Than a Token
Oct 30, 2026Zixiong Wang, Haiping MaVisual Representation Learning Vision Transformer
Shaoxing University
Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition
Oct 30, 2026Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira +2Face Recognition Multi-Head Attention
Fraunhofer Institute for Computer Graphics Research IGD, Germany · TU Darmstadt, Germany
ElasticFormer: Detecting Objects in HRW Shots via Elastic Computing Vision Transformer
Oct 29, 2026Wenxi Li, Jingchen Huang, Chenyang Lyu +4Object Detection Efficient Object Detection
KLATASDS-MOE, School of Statistics, East China Normal University, Shanghai, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China +3
ConvRoute-ViT: Convolutional Token Routing for Efficient Vision Transformers
Oct 28, 2026Manish Das, Shiv Ram Dubey, Satish Kumar SinghVision Transformer Efficient ViTs
IIIT Allahabad, Prayagraj, India
INTR++: INterpretable TRansformer with better Accuracy, Interpretability, and Adaptability
Oct 27, 2026Dipanjyoti Paul, Arpita Chowdhury, Maneet Chatterjee +4Transformer Interpretability Explainable Image Classification
IIT Kharagpur · IIEST Shibpur · University of Tsukuba +2
SAGE: Saliency-Annealed Gated Encoder for Universal Vision Recognition
Oct 27, 2026Le Yang, Yucheng Huang, Junyu Chen +4Vision Transformer Vision Foundation Model Adaptation
Huawei Technologies Co., Ltd.
SAP-ViT: A Self-Attributing Patch Vision Transformer for Medical Image Classification
Oct 26, 2026Wei Zhang, Wolfgang Nejdl, Ming TangExplainable Image Classification Vision Transformer
L3S Research Center, Leibniz University Hannover, 30167 Hannover, Germany
How Robust Are CNNs and Vision Transformers for Alzheimer's MRI Classification Across Datasets?
Oct 24, 2026Hafija Aktar, Molly Croes, Frank S. Menniti +1Alzheimer's Disease Classification Cross-Dataset Generalization
Department of Computer Science, University of Rhode Island, Kingston, RI 02881, USA · Department of Biomedical and Pharmaceutical Sciences, George & Anne Ryan Institute for Neuroscience, University of Rhode Island, Kingston, RI 02881, USA
TileShare: Training-Free Acceleration of Overlapping Tiled ViT Decoding
Oct 23, 2026Xiangzhong Ye, Zhongshen Li, Shen You +6Efficient ViTs Vision Transformer
NUViT: Vision Transformers for Fourier Space Data
Oct 23, 2026Jeffrey Gu, Robert C. Heeter, Ellen D. ZhongVision Transformer
Princeton University
GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images
Oct 22, 2026David Hamlyn, Sunny Chaudhary, Jonathon Hare +1Solar Irradiance Forecasting Vision Transformer
School of Electronics and Computer Science, University of Southampton, University Road, Southampton, SO17 1BJ, United Kingdom
Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images
Oct 22, 2026David Hamlyn, Sunny Chaudhary, Tasmiat RahmanVision Transformer Data-Efficient Learning
School of Electronics and Computer Science, University of Southampton, University Road, Southampton, SO17 1BJ, United Kingdom
CODI: Improving Self-Supervised Vision Transformers with Cross Distillation
Oct 22, 2026Siran Dai, Qianqian Xu, Peisong Wen +3Vision Transformer Self-Supervised Visual Representation Learning
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of AI Safety, Institute of Computing Technology, CAS +3
The Explainability Illusion: Must ViTs Exhibit Patch-Decoding Interpretability?
Oct 18, 2026Ronen Schaffer, Hila Manor, Tomer MichaeliTransformer Interpretability Explainable Image Classification
Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology
Dynamic self-ensemble on multi-exit residual vision transformer for small-data image classification
Oct 17, 2026Stephen Ekaputra Limantoro, Yan-Kai Wang, Ching-Chun Huang +4Vision Transformer Ensemble Learning
aDepartment of Electrical Engineering and Computer Science, National Yang Ming Chiao Tung University, Hsinchu, 30010, Taiwan · bDepartment of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, 30010, Taiwan · cDepartment of Computer Science and Information Engineering, National Taiwan University, Taipei, 10617, Taiwan +2
Spurious Attention Heads in Vision Transformers: Mechanistic Identification of Annotation-Driven Bias in Medical Chest X-Ray Analysis
Oct 15, 2026Joshua Muhumuza, Cornelius KalenziVision Transformer Attention Head Analysis
Makerere University
Deep Learning for Atmospheric Turbulence Mitigation: A Comparison Between GANs and Transformers
Oct 13, 2026Juan S. Carrillo, Jérôme GillesAtmospheric Turbulence Mitigation Image Restoration
San Diego State University, Computational Science Research Center, Department of Mathematics and Statistics, 5500 Campanile Drive, San Diego, CA 92182, USA · San Diego State University, Department of Mathematics and Statistics, 5500 Campanile Drive, San Diego, CA 92182, USA
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
Oct 13, 2026Vatsal, Soumil Malhotra, Priyanshu Chahal +1Vision Transformer Efficient ViTs
Department of Mathematics, Indian Institute of Technology Roorkee
Bottlenecked semantic compression by representation reconstruction
Oct 12, 2026Xiaoya Tang, Shilong Liu, Daniel Smolyak +3Visual Token Compression Efficient ViTs
Scientific Computing and Imaging Institute University of Utah · Princeton AI Lab Princeton University · National Institutes of Health
ChangeT $^3$ : Evaluating Vision Test-Time Training for Remote Sensing Change Detection
Oct 11, 2026Sharukshan Niranjan, Oshadha Samarakoon, Nichula Wasalathilaka +3Remote Sensing Change Detection Test-Time Training
University of Peradeniya, Peradeniya, Sri Lanka
Event-based Action Recognition by Pre-Training on Multi-Frame Representations
Oct 9, 2026Emma Norrby, Tomas Wilkinson, Per Nyblom +1Event-Based Vision Masked Autoencoders
Swedish Defence Research Agency (FOI), Linköping, Sweden
Decoding Register Tokens Preserves Global Scene Details
Oct 9, 2026David Jacob Drexlin, Julius Hense, Jonas DippelVision Transformer Image Reconstruction
Machine Learning Group, Technische Universität Berlin · BIFOLD – Berlin Institute for the Foundations of Learning and Data Franklinstraße 28/29, 10587 Berlin, Germany · Aignostics, Germany
Vision Transformers Learn Word Meaning from Images Alone
Oct 9, 2026Anders Gjølbye, Sebastian Ray Mason, Stella Frank +1Vision Transformer Semantic Clustering
Department of Applied Mathematics and Computer Science Technical University of Denmark
Learning Features, Not Layouts: Feature-Axis Attention as a Transfer Prior for Medical Imaging
Oct 9, 2026Aon Safdar, Mohamed SaadeldinMedical Image Representation Learning Vision Transformer
University College Dublin, Ireland
Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation
Oct 8, 2026Marco Stefani, Edoardo Parpaiola, Matteo Bino +5Neural Network Interpretability Saliency Map Evaluation
Department of Information Engineering (DEI), University of Padova, Padova, Italy
Dissecting Representation Structure in Vision Transformers: A Rigorous Architectural Study
Oct 8, 2026Kim-Cuc Nguyen, Ngai-Man CheungVision Transformer Neural Network Generalization
Temasek Laboratories Singapore University of Technology and Design Singapore
Region-Aware CLS Token Augmentation for Fine-Grained Image Retrieval
Oct 7, 2026Ian de Holanda Cavalcanti Bezerra, Vivek Trivedy, Lucas Pascotti Valem +1Fine-Grained Image Retrieval Image Retrieval
Institute of Mathematics and Computer Science (ICMC), University of S˜ao Paulo (USP), S˜ao Carlos, Brazil · Department of Computer and Information Science, Temple University, Philadelphia, PA, USA
On the Necessity of Attention-FFN Split in Vision Transformers
Oct 7, 2026Junhyeok Kim, Jinyeong Kim, Jae Wan Park +1Vision Transformer Attention Mechanisms
Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
Test-Time Adaptation of Quantized ViTs via Single-Pass Quantizer-Aligned Recalibration
Oct 6, 2026Hyeongheon Cha, Young D. Kwon, Sung-Ju LeeTest-Time Adaptation Vision Transformer
KAIST · Samsung AI Center-Cambridge · University of Cambridge

Oct 30, 2026J/K move · Enter open · S save

Evaluating SAM-based and Convolutional Architectures in Thin Retinal Vessel Segmentation: Model Capacity, Computational Cost and Foundation Pre-training

Carlos Eduardo Fontaneli, Cesar Henrique Comin

Department of Computer Science, Federal University of S˜ao Carlos, Rod. Washington Lu´ıs, km 235, 13565-905, S˜ao Carlos, SP, Brazil

Retinal Vessel Segmentation Thin-Structure Segmentation Vision Transformer Vision Foundation Models

Open paper AskPDF arXiv

In retinal vessel segmentation, plain Vision Transformers (ViTs) are commonly expected to perform worse than convolutional neural networks (CNNs) on thin vessels, for reasons such as their coarse patch tokenization, their weak locality bias and their need for large training sets. Although this expectation is widely shared in the literature, how plain ViTs actually compare with CNNs on thin vessels has not been systematically quantified. We address this question by comparing a model with a plain ViT encoder (MedSAM), a hybrid Transformer (LiteMedSAM) and three CNNs of different sizes on three retinal datasets (DRIVE, HRF, FIVES) and, as a second-domain check, a microscopy dataset (VessMap). We measured the centerline recall and precision of thin vessels and compared the models while also considering their computational cost and how many epochs they needed to train. We found that the pre-trained MedSAM had a centerline recall of thin vessels that was not significantly different from that of the best CNN on all four datasets (-0.6 to +2.5 percentage points). A compact plain ViT obtained through architecture search but trained from scratch did not retain this performance. For pre-trained plain ViTs, our results do not support the expectation that they miss thin vessels, although similar thin-vessel performance can be obtained with much smaller models.