Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. Localized Adaptation Reveals Distinct Learning Signatures in Transformers

    Jul 28, 2026Rebecca Ramnauth, Brian ScassellatiTransformer ArchitecturesDistributional Signatures

  2. Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance

    Jul 28, 2026Abraham ChachamovitsRotary Position EmbeddingTransformer Attention

  3. At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inference

    Jul 28, 2026Bowen Wang, Chi Zhang, Diyou Shen +3Tensor ProgramsSparse Deep Neural Network Accelerators

  4. CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention

    Jul 28, 2026Yufei Xue, Lin Niu, Hong Liu +6Dynamic Sparse AttentionEfficient Long-Context Inference

  5. Interpretable GOHR Agents via Sparse Autoencoders

    Jul 27, 2026Shiwei Tan, Yusong Zhao, Weiyi Qin +6Agent DecisionsAutoregressive Transformers

  6. Stable FP4 Training via Transposition-Invariant Block Quantization

    Jul 27, 2026Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi +6W4A4 QuantizationFp8

  7. PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention

    Jul 27, 2026Hong Liu, Yuan Cheng, Lin Niu +5Dynamic Sparse AttentionPer-Token Latency

  8. LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

    Jul 27, 2026Junsung HwangEfficient Long-Context InferenceKey-Value

  9. Physics Transformer: Tailoring Transformer for General PDE Prediction

    Jul 27, 2026Guoze Sun, Rui Zhang, Jiankai Tang +4Partial Differential EquationsUnified Transformer

  10. Self-Attention Dynamics with Rotary Position Embeddings: Twisted States and Explicit Consensus Rates on the Sphere

    Jul 27, 2026Hao YeRotary Position EmbeddingSoftmax Attention

  11. Grounding latent algorithm routing in transformer reasoning

    Jul 27, 2026Xiangbo Zhang, Xiaoxu MaTransformer ArchitecturesAdaptive Routing

  12. DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

    Jul 27, 2026Tan T. Nguyen, Quan V. DangKey-Value Cache CompressionCache

  13. Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

    Jul 27, 2026Haopeng Li, Yitong Li, Junsong Chen +8Dynamic Sparse AttentionHigh-Fidelity Video Generation

  14. SpecFormer: Mitigating Embedding and Attention Collapse via Spectral-Aware Transformer for Recommendation

    Jul 27, 2026Yu Cui, Yi Xu, Jiahao Wang +6Transformer AttentionRecommendation

  15. A Coulomb Particle Model for Learning Kernel Attention in Transformers

    Jul 26, 2026Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2Transformer AttentionFeature Learning

  16. Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers

    Jul 26, 2026Dongseong Hwang, Prasanth Yadla, Kaan Elgin +8Audio TokensZero-Shot Text-To-Speech

  17. The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning

    Jul 26, 2026Peng XieLow-Rank AdaptationCatastrophic Forgetting

  18. ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour

    Jul 26, 2026Jianhang Xie, Sicheng Tan, Vishnu Naresh Boddeti +1Homomorphic EncryptionTransformer Architectures

  19. CAPT: A Multi-task Continuous Autoregressive Transformer enabling Cross-dataset and Cross-species Transfer for Calcium Population Dynamics

    Jul 25, 2026Xinhong Xu, Yimeng Zhang, Yuanlong ZhangAutoregressive TransformersCausal Attention

  20. Structure over Depth: A Single-Block Spatio-Temporal Transformer for Multi-Entity Reasoning

    Jul 25, 2026Narthana Sivalingam, Santhirarajah Sivasthigan, Buddhi Wijenayake +3Spatio-Temporal TransformersTemporal Dynamics

  21. Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models

    Jul 25, 2026Dhruvil S, Fenil Sojitra, Ravirajsinh ChauhanMulti-Head Self-AttentionLatent Representation

  22. The Entropic Bound for Transformers: Why Static Rank Fails and Attention-Native Rank Recovers

    Jul 25, 2026Byeong Hoon YoonLow-Rank FactorizationTransformer Architectures

  23. Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

    Jul 25, 2026Bidhya Shrestha, Christos PapadopoulosAutonomous DrivingBidirectional Long Short-Term Memory

  24. Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates

    Jul 24, 2026Zhang Gongyue, Sheng Yixuan, Liu donghan +3AdamAffine

  25. Learning to Prepare Molecular Ground States with Transformer Models

    Jul 24, 2026Alex Koziell-Pipe, Jasmine Brewer, Jem Guhit +14Quantum Circuit SimulationN$-Qubit Stabilizer States

  26. LithoFormer: A Robust Framework for Stratigraphic Inference via Transformers

    Jul 24, 2026Shwetha Salimath, Francesca Bugiotti, Sylvain Wlodarczyk +1Lithology ClassificationGeological Semantic Understanding

  27. HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding

    Jul 24, 2026Chao Fang, Jun Yin, Man Shi +1Key-Value CacheGraphics Processing Unit Memory

  28. Interior interpretability with attention rollout: contraction and propagation profiles in Transformers

    Jul 24, 2026Umberto Biccari, Qian Huang, Enrique ZuazuaTransformer AttentionModel Interpretability

  29. Geometric 2D Scene Graph Generation

    Jul 24, 2026Christoph Jahn, Urs Waldmann, Bastian GoldlueckeHierarchical 3D Scene GraphsTransformer Architectures

  30. Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

    Jul 24, 2026Wan Zhang, Qinjie Lin, Chan Lee +3Time SeriesMixformer

  31. Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs

    Jul 24, 2026Ilia Sobakinskikh, Paul Alexander BilokonTime-Series Anomaly DetectionHybrid Transformer

  32. Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control

    Jul 24, 2026Federico Del Pup, Elisa Tentori, Manfredo AtzoriHand Gesture RecognitionMultimodal Physiological Signals

  33. RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

    Jul 24, 2026Anderson R. SantosDynamic Sparse AttentionLLM Inference Efficiency

  34. Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection

    Jul 23, 2026Debarshi Kundu, Swaroop Ghosh, Vasant HonavarDynamic Sparse AttentionEfficient Long-Context Inference

  35. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

    Jul 23, 2026Zahra Yousefijamarani, Alaa AlameldeenTransformer AttentionTransformer Architectures

  36. Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

    Jul 23, 2026Yidu Wu, Xiang Wang, Kejie Zhao +3LLM Inference EfficiencyLarge Language Model Pretraining

  37. Flash EQ-Linear: Accelerating Equivariant Linear Layers via Group-wise Discrete Fourier Transform

    Jul 23, 2026Zhongchen Zhao, Jixin Wang, Qi Xie +4Equivariant Neural NetworksExact Equivariance

  38. Agree on the Model, Verify the Inference: GKR Protocols for HND-Based Transformer Inference

    Jul 23, 2026Xiaolong Liang, Juanjuan Li, Rui Qin +1Homomorphic EncryptionTransformer Architectures

  39. Scaling Interpretable Transformers with Parity Bottleneck Layers

    Jul 22, 2026Andrew Mack, Kraig Yuheng Tou, Mark Henry +2Transformer ArchitecturesModel Interpretability

  40. The Giant Hippocampus: From Structural Monoculture to a System of Systems

    Jul 22, 2026Jaeho SeolHuman Visual CortexNeural Architectures

  41. On the Computational Complexity of Structural Generalization

    Jul 21, 2026Zichao WeiStructural GeneralizationCompositionality

  42. Appearance Pointers -- Multimodal Region Control of Diffusion Transformers

    Jul 21, 2026Rahul Sajnani, Yulia Gryaditskaya, Radomír Měch +2Multimodal Diffusion TransformersAppearance

  43. MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

    Jul 21, 2026Lenore Mulin, Gaetan HainsQuadratic AttentionTransformer Attention

  44. Breaking Feedback-Blindness: Utility-Augmented Transformer for Sequential Decision Making

    Jul 21, 2026Yuyang Shen, Shan Dai, Daimin ChenTransformer ArchitecturesSequential Decision Making

  45. Enhancing Transformer-based Routing by Encoding Distance via Relative Positional Encoding

    Jul 21, 2026Leyre Encío, Daniel Fuertes, Carlos R. del-Blanco +1Positional EncodingTransformer Encoder

  46. Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

    Jul 21, 2026Garrett Baker, Vinayak Pathak, Daniel Murfet +1PosteriorElicitation

  47. Beyond Transformers: Linear Attention Policy for Open-Vocabulary Object Goal Navigation

    Jul 21, 2026Jiahong Zhang, Yifan Lin, Yandong Zhang +6Object Goal NavigationNavigation

  48. Norm or Direction? Decoding Vision Mambas for High-Resolution Vision

    Jul 21, 2026Jin Yu, Juyoun ParkBidirectional MambaVisual Reasoning

  49. On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers

    Jul 20, 2026Sixu Li, Thomas Jacob Maranzatto, Jan Peszek +5Transformer ArchitecturesSoftmax Attention

  50. DuSPiT: Dual-Branch Sub-Patch Pixel Diffusion Transformer

    Jul 20, 2026Yunpeng Bai, Yossi Gandelsman, Michaël GharbiMultimodal Diffusion TransformersLarge-Scale Image Generation

  51. Adaptive Multi-Expert Graph Transformer for Interpretable EEG-Based Diagnostics

    Jul 20, 2026Maryam Rahimimovassagh, Md Elias Hossain, Ivan Garibay +1Electroencephalography Spatial Super-ResolutionFunctional Networks

  52. A Controlled Study of Attention-Only Transformers

    Jul 20, 2026Henry Ndubuaku, Karen Mosoyan, Jakub Mroz +5Transformer AttentionTransformer Encoder

  53. L1 Augmented Attention as an Improved Vector Similarity Metric

    Jul 20, 2026Kurt GoddenSoftmax AttentionCosine Similarity

  54. Mobius Learning: Cyclic Depth Folding in Transformers

    Jul 20, 2026Tongtian ZhuTransformer Architectures

  55. DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation

    Jul 20, 2026Yesol Park, Hye-Jung Yoon, Juno Kim +1Robotic PerceptionGeneralizable Robotic Manipulations

  56. Planning with Transformers: Chain of Computation and Structured Context Windows

    Jul 20, 2026Ehsan Futuhi, Nathan R. SturtevantLarge Language Model PlanningTransformer Architectures

  57. An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

    Jul 20, 2026Cheng Huan, Hongwei YuanTransformer Residual StreamsModel Sensitivity

  58. An Analysis of Residual-Stream Geometry Across Transformer Depth

    Jul 20, 2026Sunit Bhattacharya, Ravi Shankar KolliTransformer Residual StreamsResidual Stream

  59. Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

    Jul 20, 2026Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. DehaqaniVideo UnderstandingTransformer Architectures

  60. Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks

    Jul 20, 2026Damien Teney, Liangze Jiang, Hemanth Saratchandran +1Transformer ArchitecturesInductive Bias