Self-Supervised Vision Transformers

Follow this topic Ask about this topic

Recent momentum

-14%

19 papers in the last 28 days · 0.3% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

6 new papers

A weekly snapshot of new work published in Self-Supervised Vision Transformers.

Period ending 2026-09-14

5 new papers

A weekly snapshot of new work published in Self-Supervised Vision Transformers.

Period ending 2026-09-07

7 new papers

A weekly snapshot of new work published in Self-Supervised Vision Transformers.

223 papers

Latest in Self-Supervised Vision Transformers

Browse all topics →

ELSA: Exact Linear-Scan Attention for Fast and Memory-Light Vision Transformers
Apr 26, 2026Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao +1Quadratic Attention Real-Time Inference
From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers
Apr 25, 2026Jainum SanghaviSelf-Supervised Vision Transformers Vision Transformer
BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning
Apr 25, 2026Hongxiang Peng, Dewei Bai, Hong QuSelf-Supervised Vision Transformers Binary Spikes
Distilling Vision Transformers for Distortion-Robust Representation Learning
Apr 24, 2026Konstantinos Alexis, Giorgos Giannopoulos, Dimitrios GunopulosSelf-Supervised Vision Transformers Self-Supervised Representations
an interpretable vision transformer framework for automated brain tumor classification
Apr 23, 2026Chinedu Emmanuel Mbonu, Tochukwu Sunday Belonwu, Okwuchukwu Ejike Chukwuogo +1Brain Tumor Classification Self-Supervised Vision Transformers
Self-supervised pretraining for an iterative image size agnostic vision transformer
Apr 22, 2026Nedyalko Prisadnikov, Danda Pani Paudel, Yuqian Fu +1Self-Supervised Vision Transformers Vision Encoders
Cognitive Alignment At No Cost: Inducing Human Attention Biases For Interpretable Vision Transformers
Apr 21, 2026Ethan KnightsSelf-Supervised Vision Transformers Vision Transformer
Benign Overfitting in Adversarial Training for Vision Transformers
Apr 21, 2026Jiaming Zhang, Meng Ding, Shaopeng Fu +2Self-Supervised Vision Transformers Adversarial Training
Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding
Apr 21, 2026Kadir Yilmaz, Adrian Kruse, Tristan Höfer +23D Scene Understanding 3D Object Understanding
EX-FIQA: Leveraging Intermediate Early eXit Representations from Vision Transformers for Face Image Quality Assessment
Apr 21, 2026Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira +4Self-Supervised Vision Transformers Image Quality Assessment
ATTN-FIQA: Interpretable Attention-based Face Image Quality Assessment with Vision Transformers
Apr 21, 2026Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira +5Image Quality Assessment Facial Recognition
Benchmarking Vision Foundation Models for Domain-Generalizable Face Anti-Spoofing
Apr 21, 2026Mika Feng, Pierre Gallin-Martel, Koichi Ito +1Face Forgery Detection Spoofed Speech
Advancing Vision Transformer with Enhanced Spatial Priors
Apr 20, 2026Qihang Fan, Huaibo Huang, Mingrui Chen +2Self-Supervised Vision Transformers Vision Transformer
Spike-NVPT: Learning Robust Visual Prompts via Bio-Inspired Temporal Filtering and Discretization
Apr 20, 2026Qiugang Zhan, Anning Jiang, Ran Tao +4Visual In-Context Learning Self-Supervised Vision Transformers
MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition
Apr 20, 2026Chenxing Hu, Kun Xie, Qiguang Miao +3Facial Expression Recognition Motion-Focused Representations
PlankFormer: Robust Plankton Instance Segmentation via MAE-Pretrained Vision Transformers and Pseudo Community Image Generation
Apr 20, 2026Masaharu Miyazaki, Yurie Otake, Koichi Ito +3In-Situ Plankton Classifiers Self-Supervised Vision Transformers
Towards Generalizable Deepfake Image Detection with Vision Transformers
Apr 19, 2026Kaliki V Srinanda, M Manvith Prabhu, Hemanth K Mogilipalem +4Explainable Deepfake Detection Self-Supervised Vision Transformers
Hierarchical Vision Transformer Enhanced by Graph Convolutional Network for Image Classification
Apr 18, 2026Haibin JiaoVision Transformer Self-Supervised Vision Transformers
FairNVT: Improving Fairness via Noise Injection in Vision Transformers
Apr 18, 2026Qiaoyue Tang, Sepidehsadat Hosseini, Mengyao Zhai +2Self-Supervised Vision Transformers Vision Transformer
VADF: Vision-Adaptive Diffusion Policy Framework for Efficient Robotic Manipulation
Apr 17, 2026Xinglei Yu, Zhenyang Liu, Shufeng Nan +2Diffusion Policies Self-Supervised Vision Transformers
Dispatch-Aware Ragged Attention for Pruned Vision Transformers
Apr 16, 2026Seifeldin Abdellatif, Ahmad AlmasriVisual Token Pruning Self-Supervised Vision Transformers
Improved Multiscale Structural Mapping with Supervertex Vision Transformer for the Detection of Alzheimer's Disease Neurodegeneration
Apr 16, 2026Geonwoo Baek, David H. Salat, Ikbeom JangAlzheimer Self-Supervised Vision Transformers
HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNet
Apr 16, 2026Badri N. Patro, Vijay S. AgneeswaranSelf-Supervised Vision Transformers State Space Models
Revisiting Token Compression for Accelerating ViT-based Sparse Multi-View 3D Object Detectors
Apr 16, 2026Mingqian Ji, Shanshan Zhang, Jian Yang3D Object Detection Self-Supervised Vision Transformers
MegaFlow: Zero-Shot Large Displacement Optical Flow
Mar 26, 2026Dingxi Zhang, Fangjinhua Wang, Marc Pollefeys +1Optical Flow Motion Estimation
SIMPLER: Efficient Foundation Model Adaptation via Similarity-Guided Layer Pruning for Earth Observation
Mar 20, 2026Víctor Barreiro, Johannes Jakubik, Francisco Argüello +1Vision Foundation Models Earth Observation
HiAP: A Multi-Granular Stochastic Auto-Pruning Framework for Vision Transformers
Mar 12, 2026Andy Li, Aiden Durrant, Milan Markovic +1Self-Supervised Vision Transformers Vision Transformer
Separators in Enhancing Autoregressive Pretraining for Vision Mamba
Mar 4, 2026Hanpeng Liu, Zidan Wang, Shuoxi Zhang +2Self-Supervised Vision Transformers Bidirectional Mamba
Teacher-Guided Causal Interventions for Image Denoising: Orthogonal Content-Noise Disentanglement in Vision Transformers
Mar 1, 2026Kuai Jiang, Zhaoyan Ding, Guijuan Zhang +2Image Denoising Self-Supervised Vision Transformers
AFFMAE: Scalable Vision Pre-Training for High-Resolution Microscopy Segmentation on Desktop Hardware
Feb 18, 2026David Smerkous, Zian Wang, Behzad NajafianSelf-Supervised Vision Transformers Feature Pyramid Networks
Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding
Jan 28, 2026Haoyu Cao, Kun Yin, Yunfei Wu +16Visual Document Retrieval Self-Supervised Vision Transformers
VENI: Variational Encoder for Natural Illumination
Jan 20, 2026Paul Walker, James A. D. Gardner, Andreea Ardelean +2Inverse Rendering Variational Autoencoder
Attentive multilayer fusion for vision transformers
Jan 14, 2026Laure Ciernik, Marco Morik, Lukas Thede +4Self-Supervised Vision Transformers Vision Transformer
From Local Windows to Adaptive Candidates via Individualized Exploratory: Rethinking Attention for Image Super-Resolution
Jan 13, 2026Chunyu Meng, Wei Long, Shuhang GuGenerative Super-Resolution Self-Supervised Vision Transformers
Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers
Jul 21, 2025Ian Chuang, Jinyu Zou, Andrew Lee +2Robotic Perception Gaze Target Estimation
Post-Disaster Affected Area Segmentation with a Vision Transformer (ViT)-based EVAP Model using Sentinel-2 and Formosat-5 Imagery
Jul 21, 2025Yi-Shan Chu, Hsuan-Cheng WeiRemote Sensing Image Segmentation Self-Supervised Vision Transformers
GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers
Jun 13, 2025Guang Liang, Xinyao Liu, Jianxin WuSelf-Supervised Vision Transformers
Unlocking Pretrained Vision Transformers for Time Series Classification
Jun 10, 2025Simon Roschmann, Quentin Bouniot, Vasilii Feofanov +2Self-Supervised Vision Transformers
Fast and Accurate Image Restoration and Generation with Rank Enhanced Linear Attention
May 22, 2025Yuang AiImage Restoration Self-Supervised Vision Transformers
Quick ViTs: Speeding up Vision Transformers through Equivariance
May 21, 2025David Nordström, Johan Edstedt, Fredrik Kahl +1Self-Supervised Vision Transformers Vision Transformer
Efficient Concertormer for Image Deblurring and Beyond
Apr 9, 2024Pin-Hung Kuo, Jinshan Pan, Shao-Yi Chien +1Self-Supervised Vision Transformers Motion Blur
DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning
Date pendingBingchen Huang, Yifu Chen, Zhiling Wang +1Domain Incremental Learning Self-Supervised Vision Transformers
ActMap: Single-Pass Uncertainty Quantification from Generation-Time Activation Maps
Date pendingJacopo Dardini (University of Bologna), Roberta Calegari (University of Bologna)Large Language Model Uncertainty Etsi-Compliant Class Activation Map

Apr 26, 2026 · cs.LGJ/K move · Enter open · S save

ELSA: Exact Linear-Scan Attention for Fast and Memory-Light Vision Transformers

Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao, Chia-Ming Lee

Advanced Computer Vision Laboratory, National Yang Ming Chiao Tung University

Existing attention accelerators often trade exact softmax semantics, depend on fused Tensor Core kernels, or incur sequential depth that limits FP32 throughput on long sequences. We present \textbf{ELSA}, an algorithmic reformulation of online softmax attention that (i)~preserves exact softmax semantics in real arithmetic with a \emph{provable}

\mathcal{O}(u\log n)

FP32 relative error bound; (ii)~casts the online softmax update as a prefix scan over an associative monoid

(m,S,W)

, yielding

O(n)

extra memory and

O(\log n)

parallel depth; and (iii)~is Tensor-Core independent, implemented in Triton and CUDA C++, and deployable as a \emph{drop-in replacement} requiring no retraining or weight modification. Unlike FlashAttention-2/3, which rely on HMMA/GMMA Tensor Core instructions and provide no compatible FP32 path, ELSA operates identically on A100s and resource-constrained edge devices such as Jetson TX2 -- making it the only hardware-agnostic exact-attention kernel that reduces parallel depth to

O(\log n)

at full precision. On A100 FP32 benchmarks (1K--16K tokens), ELSA delivers

1.3

3.5\times

speedup over memory-efficient SDPA and

1.97

2.27\times

on BERT; on Jetson TX2, ELSA achieves

1.5

1.6\times

over Math (64--900 tokens), with

17.8

20.2\%

throughput gains under LLaMA-13B offloading at

\ge

32K. In FP16, ELSA approaches hardware-fused baselines at long sequences while retaining full FP32 capability, offering a unified kernel for high-precision inference across platforms. Our code and implementation are available at https://github.com/ming053l/ELSA.

Quadratic Attention Real-Time Inference Self-Supervised Vision Transformers Gumbel-Softmax Relaxation Transformer Architectures

Open paper Ask about it PDF