Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,208 papers

Latest in Transformers

Browse all topics →
CardsList

  1. Training, Reading, and Editing Legible Transformers

    Jul 9, 2026Mark OskinTransformer ArchitecturesModel Activations

  2. Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks

    Jul 9, 2026Hong ZhaoBackpropagationNeural Network

  3. COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation

    Jul 9, 2026Yashal Shakti Kanungo, Gyanendra Das, Pooja A +1AdsClick-Through Rate Prediction

  4. Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing

    Jul 8, 2026Tommaso Cerruti, Tim Rieder, George Rowlands +2Quadratic AttentionGlobal Attention Mechanism

  5. The Key to Going Linear: Analysis-Driven Transformer Linearization

    Jul 8, 2026Anna Kuzina, Paul N. Whatmough, Babak Ehteshami BejnordiTransformer ArchitecturesCausal Attention

  6. PALS: Percentile-Aware Layerwise Sparsity for LLM Pruning

    Jul 8, 2026Yazdan Jamshidi, Alexey ShvetsHeterogeneous Pruning ScoresLarge Language Model Compression

  7. FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

    Jul 8, 2026Athanasios ZerisTransformer AttentionFourier

  8. DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

    Jul 7, 2026Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache CompressionEfficient Long-Context Inference

  9. At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

    Jul 7, 2026Truong Xuan KhanhGrokkingModular Addition

  10. Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models

    Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Speech EncoderOrthogonality

  11. STAGformer: A Spatio-temporal Agent Graph Transformer for Micro Mobility Demand Forecasting

    Jul 7, 2026Ye ZihaoTemporal Graph Neural NetworksMobility

  12. Umm... With Transformers? Insights from Filled Pause Use across Four Slavic Parliaments

    Jul 7, 2026Ivan Porupski, Branimir Dropuljić, Nikola LjubešićParliamentary DiscourseSpeaker Attributes

  13. Think Before You Grid-Search: Floor-First Triage for LLM Serving

    Jul 7, 2026Yihua LiuLarge Language Model ServingLow-Latency Systems

  14. TriRoute: Unified Learned Routing for Joint Adaptive Attention, Experts, and KV-Cache Allocation

    Jul 7, 2026Andrii Balashov, Olena PonomarovaAdaptive RoutingQuadratic Attention

  15. ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modeling

    Jul 6, 2026Archie ChaudhuryEfficient Long-Context InferenceTransformer Architectures

  16. KVpop -- Key-Value Cache Compression with Predictive Online Pruning

    Jul 6, 2026Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl +5Key-Value Cache EvictionKey-Value Cache Compression

  17. Input Pathways Shape Few-Shot, Not Zero-Shot, Binding in Tiny Transformers: A Fully-Enumerable Study

    Jul 6, 2026Yoshiyuki OotaniTransformer ArchitecturesFew-Shot Learning

  18. Pretraining Curricula Enable Selective Fine-tuning

    Jul 6, 2026Sebastian A. Bruijns, Jirko Rubruck, Mia H. Whitefield +3CurriculumBalanced Learning

  19. Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers

    Jul 6, 2026Ligong Han, Kai Xu, Hao Wang +3Homomorphic EncryptionTransformer Layers

  20. Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention

    Jul 5, 2026Siyu Ding, Mingchuan Ma, Jiabo Tong +3Fp8Large Language Model Pretraining

  21. Legible-by-Construction: Attention and End-to-End Transformers

    Jul 5, 2026Mark OskinTransformer ArchitecturesFeed-Forward Network

  22. HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inference

    Jul 5, 2026Hui Dong, Yanzhao Li, Jie Gao +5W4A4 QuantizationQuadratic Attention

  23. Full Glyph Images Beat Token Embeddings: A Controlled Study for Transformers

    Jul 4, 2026Shuyang Xiang, Hao GuanGraph Language ModelsTransformer Architectures

  24. FedACT: Federated Adaptive Coordinate Trust Modulation for Robust Transformer Training under Data Heterogeneity

    Jul 4, 2026Shuai Li, Qinglin Wang, Ping Luo +8FedavgTransformer Architectures

  25. ELiTeFormer: An Efficient Transformer for FPGAs

    Jul 4, 2026Victor Agostinelli, Nicolas Bohm Agostini, Antonino TumeoField-Programmable Gate ArraysLLM Inference Efficiency

  26. SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers

    Jul 3, 2026Jianing Deng, Yuanzhe Li, Jialu Wang +4Feed-Forward 3D ReconstructionDynamic Sparse Attention

  27. LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning

    Jul 3, 2026Licheng Zhang, Siew-Kei Lam, Naveed AkhtarRemote Sensing Image Change CaptioningRemote Sensing

  28. SeqGPT: A Constrained Transformer Agent for the Inverse Designof Multi-Panel Composite Structures

    Jul 3, 2026Driss Chraibi, Alejandro García Pis, Stéphane Grihon +1CompositesGenerative Inverse Design

  29. HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusion

    Jul 3, 2026Dongyeun Lee, Amir Zandieh, Vahab Mirrokni +2Dynamic Sparse AttentionHigh-Fidelity Video Generation

  30. Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

    Jul 3, 2026Xiang Hu, Xinyu Wei, Hao Gu +10Efficient Long-Context InferenceDynamic Sparse Attention

  31. Individual Parameters in Weight-Sparse Transformers Appear Interpretable

    Jul 3, 2026Arnau Marin-Llobet, Stefan HeimersheimMechanistic InterpretabilityModel Weights

  32. STAC: Selective Spatiotemporal Aggregation and Compression for Video Reasoning Segmentation

    Jul 3, 2026Syed Ariff Syed Hesham, Yun Liu, Guolei Sun +4Token CompressionQuadratic Attention

  33. Harmonic-Aware Transformer for Real-Time Catheter Localization in Interventional Procedures of Magnetic Particle Imaging

    Jul 3, 2026Abuobaida M. Khair, Wenjing Jiang, Xiaoli Yang +2Catheter Tip TrackingMagnetic Resonance Imaging Reconstruction

  34. Induction Heads Interpolate N-Grams

    Jul 2, 2026Francesco D'Angelo, Oguz Kaan Yuksel, Swathi Shree Narashiman +1Transformer ArchitecturesIn-Context Learning

  35. Self-Gating Attention for Efficient Time Series Forecasting

    Jul 2, 2026Dezheng Wang, Tong Chen, Wei Yuan +3Multivariate ForecastingTime Series

  36. A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

    Jul 2, 2026Wanyun CuiQuadratic AttentionDense Associative Memory

  37. Bridge-WA: Predicting Where and How the World Changes for Robotic Action

    Jul 2, 2026Yongjie Bai, Hanting Wang, Mingtong Dai +3Efficient World-Action ModelExecutable Robot Behavior

  38. X-LogSMask: Expand Transformer for Graph-Structured Data

    Jul 2, 2026Leyan Li, Rennong Yang, Zhenxing Zhang +1Heterogeneous Graph TransformersTransformer Attention

  39. Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale

    Jul 1, 2026Siddharth Gollapudi, Nilesh Gupta, Prasann Singhal +1In-ContextText Corpora

  40. The risk of KV cache compression

    Jul 1, 2026Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2Key-Value Cache CompressionContext Compression

  41. Precomputing the Future-Offset Average in TriAttention

    Jul 1, 2026Amarnath MukherjeeQuadratic AttentionKey-Value Cache

  42. How Should Transformers Encode Numeric Values in Electronic Health Records?

    Jul 1, 2026Maria Elkjær Montgomery, Christian Igel, Mikkel Odgaard +2Electronic Health RecordsTransformer Architectures

  43. Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training

    Jul 1, 2026Zijian Zhang, Rizhen Hu, Athanasios Glentis +4Transformer LayersReinforcement Learning Post-Training

  44. The State-Prediction Separation Hypothesis

    Jul 1, 2026Giovanni Monea, Nathan Godey, Kianté Brantley +1Transformer ArchitecturesNext Activity Prediction

  45. Fusion: A Framework for Unified Sequential Token AdaptatIon in VisiOn TraNsformers

    Jul 1, 2026Aravind Pradeep, Samira Nazari, Mahdi Taheri +1Self-Supervised Vision TransformersVision Transformer

  46. KnowledgeDebugger -- an Exploration Tool for Knowledge Localization and Editing in Transformers

    Jul 1, 2026Eric Benz, Lennart Stöpler, Nikolai Bolik +1Multimodal Knowledge EditingPer-Competency Bayesian Knowledge Tracing

  47. Post-Training Pruning for Diffusion Transformers

    Jul 1, 2026Chengzhi Hu, Xuewen Liu, Jing Zhang +3Multimodal Diffusion TransformersHeterogeneous Pruning Scores

  48. Soft Mixture-of-Recursions: Going Deeper with Recursive Vision Transformers

    Jul 1, 2026Sang In Lee, Jihun ParkSelf-Supervised Vision TransformersVision Transformer

  49. GKDT: General Keypoint Detection Transformer

    Jul 1, 2026Changsheng Lu, Yuxin Chen, Haokun Gui +5Keypoint DetectionTransformer Architectures

  50. ConRTF: Edge-Constrained Boundary Distribution Refinement for Realtime TransFormer Table Structure Recognition

    Jul 1, 2026Eliott Thomas, Tri-Cong Pham, Mickael Coustaty +5Rt-DetrTabular Data

  51. RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation

    Jul 1, 2026Yaofu Liu, Wanli Lan, Jinxi Li +2Quadratic AttentionRectification

  52. EVOTS: Evolutionary Transformer Search for Time Series Forecasting

    Jun 30, 2026AbdElRahman ElSaid, Damir PulatovMultivariate Time Series ForecastingEvolutionary Search

  53. Signed-Permutation Coordinate Transport for RMSNorm Transformers

    Jun 30, 2026John SweeneyTransformer Residual StreamsTransformer Architectures

  54. FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers

    Jun 30, 2026Hubert Dymarkowski, Xingjian Fu, Rappy Saha +2Self-Supervised Vision TransformersSparse Deep Neural Network Accelerators

  55. Review Residuals: Update-Conditioned Residual Gating for Transformers

    Jun 30, 2026Kyle KramerResidual ConnectionsBatch Normalization

  56. Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors

    Jun 30, 2026Mark OskinFeed-Forward NetworkTransformer Architectures

  57. Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior

    Jun 30, 2026Jiahui Fu, Zehao Huang, Han Li +2Lane TopologyGeometry-Aware Foundation Model

  58. SpikeLogBERT: Energy-Efficient Log Parsing Using Spiking Transformer Networks

    Jun 30, 2026Thuan Bui, Duong Do, Tung Vu +2Energy-EfficientBinary Spikes

  59. DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers

    Jun 30, 2026Shun Kenney, Teppei SuzukiPositional EncodingTransformer Architectures

  60. Temperature Field Reconstruction of Tungsten Monoblock Divertor on EAST using Physics-aware Neural Operator Transformer

    Jun 30, 2026Zikang Yan, Xiao Wang, Qingquan Yang +6Plasma Shape ControlNeural Operators