Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization

    Aug 5, 2026Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay +1Vision TransformerVisual Token Pruning

  2. Training-Free Hashing-Based Attention via Binary Principal Components

    Aug 5, 2026Daohai Yu, Zhanpeng Zeng, Keyu Chen +6Dynamic Sparse AttentionQuadratic Attention

  3. Attention-based representations for multi-task computation

    Aug 4, 2026Daniel Hsu, Mingyue XuMulti-Head Self-AttentionAttention-Based Models

  4. Attention-Only White-Box Transformer via LeJEPA-Based Self-Supervised Pretraining

    Aug 4, 2026Yang Bai, Linyuan Wang, Haoyang Jiang +3Transformer AttentionSelf-Supervised

  5. Separating quantum circuits from classical LLMs

    Aug 4, 2026Srinivasan Arunachalam, Arkopal Dutt, Hari Krovi +1Quantum Circuit SimulationLarge Language Model Quantization

  6. Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson

    Aug 4, 2026Ekansh Singh, Eva Samuel, Alessandra Reneau +2Nvidia Jetson Orin NanoBimanual Manipulation

  7. A Physics-Flavored Transformer Network for Parametrizing Contraction Dynamics of Engineered Skeletal Muscle Tissues

    Aug 4, 2026Mattias Luber, Timo BetzConventional Physics-Informed Neural NetworksMuscle Anatomy

  8. The Transformer Revolution, Part 1: Dynamic Processing through Output-Weight Interconnections

    Aug 4, 2026Marco Giunti, Fabrizia Giulia GaravagliaTransformer Architectures

  9. Equivariant Music Transformer

    Aug 4, 2026Zixun Guo, Simon DixonMusic UnderstandingAudio Latent

  10. Sparse Weight Decomposition for Efficient Circuit Extraction

    Aug 4, 2026Chuanhao Yan, Xuhan Huang, Yawen Duan +4Model WeightsTransformer Architectures

  11. Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

    Aug 4, 2026Taekyung Heo, Rasoul Shafipour, Ritchie Zhao +6PrefillKey-Value Cache

  12. Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms

    Aug 4, 2026Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2Key-Value Cache QuantizationCalibration-Free Low-Bit Vector Quantization

  13. LoopMTP: A looped transformer guided by latent multi-token prediction

    Aug 4, 2026Behzad Shomali, Markus Frey, David Berghaus +2Transformer ArchitecturesNext-Token Prediction

  14. Probing Character-level Transformers for the Spanish L-shaped Morphome

    Aug 4, 2026Akhilesh Kakolu Ramarao, Kevin Tang, Wiebke Petersen +1MorphologyTransformer Architectures

  15. SLAMFormer-∞\infty: Infinite SLAM Transformer for Unbounded Frontend and Backend Processing

    Aug 4, 2026Zhijian Fang, Weicheng Zheng, Yijun Yuan +7Orb-Slam2Simultaneous Localization And Mapping

  16. State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking

    Aug 4, 2026Xiaohe Li, Yang LuState Space ModelsTransformer Architectures

  17. SGFormer: Structure-Guided Transformer for Robust Local Feature Matching

    Aug 4, 2026Runyu ZhuFeature MatchingTransformer Architectures

  18. SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference

    Aug 4, 2026Shanghao Liu, Renze Chen, Size Zheng +4Pre-Trained Video Diffusion ModelsDynamic Sparse Attention

  19. Provably Learning Multi-Head Attention with Queries

    Aug 4, 2026Sunyeop Kim, Insung Kim, Jian GuoMulti-Head Self-AttentionSharp Joint-Kl Oracle Theory

  20. SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

    Aug 4, 2026Lin ZhangKey-Value CacheKey-Value

  21. A Unified Resolution-Conditioned Framework for Orthogonal Line-Scanning Image Fusion

    Aug 4, 2026Yiming Gong, Kai WangVisible Image FusionMicroscopy

  22. Scaling an Autoregressive Transformer for Single-Cell Generation

    Aug 3, 2026Aleksandr Sharipov, Yusif Mukhtarov, Igor MolybogAutoregressive TransformersSingle-Cell Rna Sequencing

  23. ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

    Aug 3, 2026Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino +1Quadratic AttentionRotary Position Embedding

  24. Wiring Beats Blending: What Transfers Between Transformer Sizes -- and What Doesn't

    Aug 3, 2026Ravi Satya Durga Prasad YenugulaLarge ModelsTransformer Architectures

  25. Token Radius Attention for Efficient Video Generation

    Aug 3, 2026Jiayu Chen, Zhikun Jiang, Maoliang Li +6Dynamic Sparse AttentionNeighbors

  26. Geometry-Guided Layerwise FFN Width Allocation in Transformers

    Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateFeed-Forward NetworkTransformer Architectures

  27. TransSLR: A Lightweight Transformer for Sign Language Recognition

    Aug 3, 2026Lucia Yen Wanchi, Samuel Johnny, Victor Tolulope Olufemi +2Sign Language Recognition3D Encoder

  28. ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces

    Aug 3, 2026Kunal Kumar Pant, Nithin NagarajTransformer ArchitecturesChaos

  29. Output-Aware Rotation for INT2 KV-Cache Quantization

    Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4Key-Value Cache QuantizationKv-Cache Blocks

  30. TransNRank: Towards Accurate Neoantigen Ranking with Transformer

    Aug 3, 2026Zhiyin An, Yuenan Hou, Shumeng Duan +3Cancer Genome AtlasTransformer Architectures

  31. Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model

    Aug 3, 2026Ronglong BaoQwen3Quadratic Attention

  32. Divergent large language model predictions from convergent representations in ambiguous word pairs

    Aug 3, 2026K. Jack Scott, Narun Pat, Veronica LiesaputraTransformer EncoderVocabulary Expansion

  33. Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation

    Aug 3, 2026Xingyu Ren, Youran Sun, Chugang Yi +1Dynamic Sparse AttentionSparsity

  34. LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing

    Aug 3, 2026Wen Zan, Jiaqi Zhang, Jianchao Tan +11Dynamic Sparse AttentionEfficient Long-Context Inference

  35. Spike-HTR: Spiking Neural Transformer for Handwritten Text Recognition

    Aug 3, 2026Xiubo Liang, Jinxing Han, Yuke Li +3Handwritten Text RecognitionSpiking Neural Networks

  36. Linear Multi-Timescale Retention as a Memory-Efficient Vision-Language Bridge

    Aug 3, 2026Ashfak Yeafi, Mehedi Hasan, Md Khairul IslamLarge Vision Language ModelsCross-Modal Learning

  37. Probing the 3D Object-Level Understanding of Pre-Trained Detection Transformers

    Aug 2, 2026Robin Kim, Colin Samplawski, Benjamin M. Marlin3D Object Detection3D Scene Understanding

  38. DeVIT: Low-Power Vision Transformer Acceleration Using Delta Computation

    Aug 2, 2026Reyhaneh Hosseinzadeh, Parham Zilouchian Moghaddam, Mehdi ModarressiSelf-Supervised Vision TransformersSparse Deep Neural Network Accelerators

  39. UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction

    Aug 2, 2026Junno Yun, Yaşar Utku Alçalar, Mehmet AkçakayaMultimodal Diffusion TransformersU-Net

  40. Riemannian Attention Mechanisms for Transformers: A Theoretical Framework and Architecture Design

    Aug 2, 2026Sen SongTransformer AttentionSoftmax Attention

  41. Role-Decoupled Attention Residuals: Separating Matching and Content Retrieval Across Depth

    Aug 2, 2026Kehan WangTransformer Residual StreamsResidual Connections

  42. Mask-Based Priors Are More Persistent than Query-Key Initializations

    Aug 1, 2026Mingze Ma, Hemanth Saratchandran, Cameron Gordon +1Transformer ArchitecturesInductive Bias

  43. ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression

    Jul 31, 2026Yuhang Zhan, Lisi Chen, Shuo ShangKey-Value Cache CompressionCache

  44. Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets

    Jul 31, 2026Mark ShapiroLarge Language Model PretrainingRecurrent Model

  45. LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

    Jul 31, 2026Yirui Liu, Ruoling Qi, Longwen Wang +5Quadratic AttentionCache

  46. Chimera: Designing and Chinchilla-Scaling Hybrid Visual Diffusion Transformers

    Jul 30, 2026Chongjian Ge, Hanwen Jiang, Tianyu Wang +9Video PretrainingTransformer Architectures

  47. Causal Discovery with Inverted Self-attention for Multivariate Time Series

    Jul 30, 2026Yusen Liu, Yong Wang, Yifan Yin +3Causal Foundation ModelsTime Series

  48. SemPIC: Learning Semantic Position-Independent KV Caches

    Jul 30, 2026Hui Xie, Peng Xiao, Yutong Deng +3Key-Value CacheQuadratic Attention

  49. Generalization Bounds on Optimal Control for Transformer Training and Wasserstein Distributional Robustness

    Jul 30, 2026Kağan Akman, Naci Saldi, Serdar YükselGeneralization BoundsDistributional Learning

  50. VESTIGE: A Knowledge-Guided Masking Strategy for Corruption-Aware Fine-Tuning of Genomic Transformers, Validated on Ancient DNA Reconstruction

    Jul 30, 2026Angshuman Chakravertty, Rahul MaheshwariCorrupted ObservationsTransformer Architectures

  51. Looped Transformers with Source-Centered State Evolution

    Jul 30, 2026Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya +3Recurrent StateTransformer Architectures

  52. HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data

    Jul 30, 2026Xiaotong Yu, Joshua Y. Kim, HaeJin Lee +1Wearable DevicesTime-Series Classification

  53. Hierarchical Spatio-Temporal Transformer for Coherent Emergency Department Forecasting

    Jul 29, 2026Filipa Lino, Bárbara Tavares, Carlos Santiago +2Spatio-Temporal ForecastingEpidemic Forecasting

  54. A Compositional Theory of Causally Masked Transformers

    Jul 29, 2026Franz Nowak, Ryan Cotterell, Reda BoumasmoudTransformer ArchitecturesExpressivity

  55. Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding

    Jul 29, 2026Weiye Shi, Fanxu Meng, Muhan ZhangSpeculative DecodingLLM Inference Efficiency

  56. PlantBGC: Transformer for Plant BGC Discovery via Label-Free Domain Adaptation and Weak Supervision

    Jul 29, 2026Yuhan Zhao, Nidhi Grover, Zhishan Guo +1Genomics3D Plant Phenotyping

  57. MDTransformer: A Hardware-Software Co-Design of Mode-Division Photonic Transformer Accelerator with Inverse-Designed Coherent Crossbar

    Jul 28, 2026Solomon Micheal Serunjogi, Rachmad Vidya Wicaksana Putra, Ayat Taha +2Unified TransformerTransformer Architectures

  58. Penelope: Localized Latent Recurrence for Efficient Structured Reasoning

    Jul 28, 2026Yutong Chen, Shouqian Shi, Xinran Liu +5Efficient Latent ReasoningTransformer Encoder

  59. Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-design

    Jul 28, 2026Huy Ha, C. Karen Liu, Shuran SongEmbodimentRobot Systems

  60. Recursive transformers for semiconductor thermo-mechanical reliability

    Jul 28, 2026Kart-leong LimUnified TransformerSurrogate Models