Transformers

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

Focused topic

Transformer Architectures

Within Transformers

684 papersExplore

Focused topic

Quadratic Attention

Within Transformers

144 papersExplore

Focused topic

Dynamic Sparse Attention

Within Transformers

110 papersExplore

Focused topic

Transformer Attention

Within Transformers

89 papersExplore

Focused topic

Transformer Encoder

Within Transformers

80 papersExplore

Focused topic

Transformer Layers

Within Transformers

80 papersExplore

Focused topic

Softmax Attention

Within Transformers

74 papersExplore

Focused topic

Unified Transformer

Within Transformers

40 papersExplore

Focused topic

Multi-Head Self-Attention

Within Transformers

35 papersExplore

Focused topic

Autoregressive Transformers

Within Transformers

33 papersExplore

Focused topic

Global Attention Mechanism

Within Transformers

20 papersExplore

1,209 papers

Latest in Transformers

Browse all topics →

SENTRY: Statistical Reliability Analysis of Vision Transformers Under Soft Errors
May 30, 2026Pramit Kumar Bhaduri, Mahdi Taheri, Samira Nazari +3Fault Injection Experiments Self-Supervised Vision Transformers
Looped Transformers with Layer Normalization Provably Learn the Power Method
May 30, 2026Lyumin Wu, Chenyang Zhang, Yuan CaoTransformer Layers Transformer Architectures
Spatiotemporal Multi-Task Graph Transformer for Trip-Level Transit Prediction
May 30, 2026Oluwaleke Yusuf, Adil Rasheed, Frank LindsethTransit Network Design Accurate Traffic Forecasting
Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters
May 29, 2026Xinjue Wang, Xiuheng Wang, Yejun Zhang +3Low-Rank Adaptation Adapters Low-Rank Adaptation
Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings
May 29, 2026Utsav Dutta, Gerardo Pastrana, Sina Khoshfetrat Pakazad +1Joint-Embedding Predictive Architectures Representation Learning
Reliable Multilingual Orthopedic Decision Support from Clinical Narratives: Language-Aware Adaptation and Verification-Guided Deferral
May 29, 2026Danish Ali, Li Xiaojian, Sundas Iqbal +1English Clinical Notes
Fixed Universal Transformers
May 29, 2026Jingwen Liu, Alexandr Andoni, Daniel HsuTransformer Architectures Unified Transformer
Agentic Transformers Provably Learn to Search via Reinforcement Learning
May 29, 2026Tong Yang, Yu Huang, Yingbin Liang +1Agentic Reinforcement-Learning Framework Tree Search
Vanilla ViT for Automotive Point Cloud Semantic Segmentation
May 29, 2026Gilles Puy, Nermin Samet, Alexandre Boulch +3Self-Supervised Vision Transformers Dense Semantic Segmentation
Light Interaction: Training-Free Inference Acceleration for Interactive Video World Models
May 29, 2026Jiacheng Lu, Haoyi Zhu, Sipei Yi +3Video World Models Training-Free Inference Framework
Contribution Weights: A Geometrical Analysis of Self-Attention Transformers
May 29, 2026Harry Jake Cunningham, Nicola Muca CironeTransformer Attention Importance
LVSA: Training-Free Sparse Attention for Long Video Diffusion
May 29, 2026Gael Glorian, Ioannis Lamprou, Zhen Zhang +2Dynamic Sparse Attention Bottlenecks
HADT: A Heterogeneous Multi-Agent Differential Transformer for Autonomous Earth Observation Satellite Cluster
May 29, 2026Mohamad A. Hady, Muhammad Anwar Masum, Siyi Hu +3Satellite Transformer Architectures
Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement
May 29, 2026Riju Marwah, Ritvik Garimella, Vishal Pallagani +3Autoregressive Language Models Large Language Model Performance
ImmersiveTTS: Environment-Aware Text-to-Speech with Multimodal Diffusion Transformer and Domain-Specific Representation Alignment
May 29, 2026Jun-Hak Yun, Seung-Bin Kim, Seong-Whan LeeZero-Shot Text-To-Speech Multimodal Diffusion Transformers
Reachability and asymptotics of Gaussian Transformer dynamics
May 29, 2026Albert Alcalde, Zhengping Ji, Enrique ZuazuaTransformer Architectures Probability Measures
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
May 29, 2026Ke Lei, Yu Zhang, Changhao Pan +4Spatial Audio Joint Audio-Video Generation
Cross-Layer Subspace Coupling for LLM Compression: A Unifying Framework and Its Empirical Limits
May 29, 2026Snigdha Chandan KhilarLarge Language Model Compression Cross-Layer Interactions
Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation
May 29, 2026Haozhou ZhangEfficient Chain-Of-Thought Reasoning Chain-of-Thought Reasoning
ConTrans: Learning Text-enhanced Local-global Temporal Representations for Zero-shot Temporal Action Localization
May 29, 2026Kanchan Keisham, Thenukan Pathmanathan, Thangarajah AkilanTemporal Action Localization Multiscale
Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't
May 28, 2026Anej Svete, William Merrill, Ryan Cotterell +1Transformer Attention Transformer Architectures
VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion
May 28, 2026Hidir Yesiltepe, Jiazhen Hu, Tuna Han Salih Meral +4Multi-Head Self-Attention Key-Value Cache
Veda: Scalable Video Diffusion via Distilled Sparse Attention
May 28, 2026Shihao Han, Hao Yang, Xinting Hu +3Pre-Trained Video Diffusion Models Dynamic Sparse Attention
Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables
May 28, 2026Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1Positional Encoding Transformer Architectures
A Dual-Path Architecture for Scaling Compute and Capacity in LLMs
May 28, 2026Markus Frey, Behzad Shomali, Joachim Koehler +1Transformer Layers Large Models
Towards Consistent Video Geometry Estimation
May 28, 2026Zhu Yu, Jingnan Gao, Runmin Zhang +9Feed-Forward Visual Geometry Estimation Geoquery
Robust and Generalizable Safety Steering for Text-to-Image Diffusion Transformers
May 28, 2026Zihao Xue, Yan Wang, Zhen Bi +7Multimodal Diffusion Transformers Text-To-Image
Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders
May 28, 2026Pierre-Antoine Lequeu, Camille Barboule, Benjamin PiwowarskiPositional Encoding Transformer Encoder
Improving Adversarial Robustness of Attribution via Implicit Regularization
May 28, 2026Amir Mehrpanah, Matteo Gamba, Hossein AzizpourTraining Data Attribution Explainability
Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames
May 28, 2026Mazen KobroslyTransformer Architectures
Brain-IT-VQA: From Brain Signals to Answers
May 28, 2026Roman Beliy, Matias Cosarinsky, Oliver Heinimann +2Brain Magnetic Resonance Imaging Textvqa
Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation
May 28, 2026Dayuan Chen, Kai Tang, Yukuan Zhang +2Motion Imitation Deformable Objects
Parallax: Parameterized Local Linear Attention for Language Modeling
May 27, 2026Yifei Zuo, Dhruv Pai, Zhichen Zeng +3Quadratic Attention
Accelerating HEVC Intra Partitioning via a CNN-Hierarchical Attention Transformer Hybrid
May 27, 2026Krishna Kumar Sharma, Somdyuti PaulSelf-Supervised Vision Transformers Neural Video Codecs
Multi-Mixer Models: Flexible Sequence Modeling with Shared Representations
May 27, 2026Kevin Y. Li, Asher Trockman, Ananda Theertha Suresh +1Sequence Modeling Softmax Attention
The Attentional White Bear Effect in Transformer Language Models
May 27, 2026Rebecca Ramnauth, Brian ScassellatiTransformer Architectures Hidden States
Transformers Provably Learn to Internalize Chain-of-Thought
May 27, 2026Yixiao Huang, Hanlin Zhu, Zixuan Wang +4Transformer Layers Transformer Architectures
PLS in the Mirror of Self-Attention
May 27, 2026Jiangsheng, YouSoftmax Attention Least Squares
Meta-Attention: Bayesian Per-Token Routing for Efficient Transformer Inference
May 27, 2026Alan FerrariSoftmax Attention Posterior
Sequential Neural Probabilistic Amplitude Shaping: Learning the Channel's Language
May 27, 2026Mohammad Taha Askari, Lutz Lampe, Amirhossein GhazisaeidiSource-Channel Coding Neural Audio Codecs
Measure-to-measure Regression with Transformers
May 27, 2026Matthew Vandergrift, Martha White, Yury Polyanskiy +2Representation Learning Transformer Architectures
ChWDTA: Channel-wise Wavelet-Domain Transformer Attention and Entropy Modeling for Learned Image Compression
May 27, 2026Haisheng Fu, Runyu Yang, Feng Ding +5Learned Image Compression Discrete Wavelet Transform
ReSAE: Residualized Sparse Autoencoders for Multi-Layer Transformer Interventions
May 27, 2026Prathyush Poduval, Calvin Yeung, Neel Desai +1Improving Sparse Autoencoders Transformer Residual Streams
Fine-Tuning Dynamics of In-Context Factual Recall in Transformers
May 26, 2026Ruomin Huang, Eshaan Nichani, Jason D. Lee +1In-Context Learning In-Context
Customer Churn Prediction on Structured Data Using FT-Transformer and Stacking Ensembles
May 26, 2026Joyjit Roy, Samaresh Kumar Singh, Laxmi ShawChurn Prediction Gradient Boosted Tree
UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training
May 26, 2026Keqi Deng, Shaoshi Ling, Ruchao Fan +1Dynamic Sparse Attention LLM Inference Efficiency
Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
May 26, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +3Key-Value Cache Quantization Quantization
Trinity: Unifying Class-Agnostic Terrain and Semantic Segmentation for Unstructured Outdoor Environments by Leveraging Synthetic Data
May 26, 2026Marcus G Müller, Wout Boerdijk, Maximilian Durner +5Traversability Terrain
SoftCap: Soft-Budget Control for Diffusion Transformer Acceleration
May 26, 2026Yuhang Zhang, Junxiang Qiu, Huixia Ben +3Multimodal Diffusion Transformers Unified Transformer
Balancing Fidelity and Diversity in Diffusion Models via Symmetric Attention Decomposition: Hopfield Perspective
May 26, 2026Hyunmin Cho, Woo Kyoung Han, Kyong Hwan JinSoftmax Attention Hopfield Networks
Particle-Lund Multimodality in Jet Taggers
May 26, 2026Loukas Gouskos, Benedikt MaierHigh Energy Physics Multimodal Transformers
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
May 26, 2026Zeyi Huang, Xuehai He, LiLiang Ren +8Transformer Architectures Recurrent Model
Transformers Can Learn Posterior Predictive Distributions In-Context
May 26, 2026Gyeonghun Kang, Changwoo J. Lee, Xiang ChengPosterior Transformer Architectures
NestedKV: Nested Memory Routing for Long-Context KV Cache Compression
May 26, 2026Hong Chen, Xiang Liu, Yubo Gao +5Key-Value Cache Compression Efficient Long-Context Inference
More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
May 26, 2026Mingze Wang, Jinbo Wang, Yikuan Xia +2Model Activations Feed-Forward Network
JetViT: Efficient High-Resolution Vision Transformer with Post-Training Attention Search
May 26, 2026Dongyun Zou, Zhuoyang Zhang, Junyu Chen +8Self-Supervised Vision Transformers Vision Foundation Models
Enabling KV Caching of Shared Prefix for Diffusion Language Models
May 26, 2026Younghun Go, Jaehoon Han, Changyong Shin +2Cache Diffusion Language Models
PRISM: Position-encoded Regressive Inverse Spectral Model for Multilayer Thin-Film Design
May 26, 2026Runtian Wang, Renhao Xue, Baige Chen +1Autoregressive Transformers Generative Inverse Design
Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
May 25, 2026Athanasios ZerisTransformer Attention Positional Encoding
CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection
May 25, 2026Durjoy Dey, Yuhong Yan, Hassan HajjdiabSkin Lesion Classification Hybrid Transformer

May 30, 2026 · cs.CVJ/K move · Enter open · S save

SENTRY: Statistical Reliability Analysis of Vision Transformers Under Soft Errors

Pramit Kumar Bhaduri, Mahdi Taheri, Samira Nazari, Maksim Jenihhin+2

Brandenburg University of Technology Cottbus-Senftenberg, Germany · Tallinn University of Technology, Tallinn, Estonia · Zanjan University, Iran

With the growth of Vision Transformers in safety-critical domains like autonomous systems and medical imaging, ensuring their reliability against soft errors is paramount. While ViTs offer state-of-the-art accuracy, their massive parameter counts render exhaustive fault injection campaigns infeasible. To bridge this gap, a statistical fault injection framework is presented, leveraging finite-population sampling theory to provide formal reliability guarantees. It is demonstrated that failure rates are bounded within a 1% margin at 99% confidence using only a few thousand samples, regardless of model scale. This methodology achieves up to a 10,700 times reduction in experimental cost compared to exhaustive approaches, while preserving the ability to localize vulnerabilities across architectural components. Through extensive evaluation of different architectures like ViT-Tiny and ViT-Small, a highly non-uniform reliability landscape is uncovered. It is shown that while only 3% of FP32 bit-flips result in failure, the vast majority of these events lead to catastrophic accuracy collapse. Specific vulnerabilities are localized to normalization layers and critical exponent bits within the IEEE-754 format, providing a mathematical foundation and actionable insights for the design of hardened, edge-deployed ViT architectures.

Fault Injection Experiments Self-Supervised Vision Transformers Vision Transformer Transformer Architectures

Open paper Ask about it PDF