Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,201 papers

Latest in Transformers

Open your feed →
CardsList

  1. Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention

    Sep 21, 2026Julien Siems, Riccardo Grazzi, Korbinian Pöppel +8Quadratic AttentionExpressivity

  2. FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention

    Sep 21, 2026Anthony Givans, Michael Crawshaw, Mingrui LiuQuadratic AttentionGumbel-Softmax Relaxation

  3. Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models

    Sep 17, 2026Frank E. Bobe, Gregory D. Vetaw, Darshan W. Bryner +2Activation SteeringSteering

  4. FreqCondNorm: Towards Cross-domain Predictive Maintenance through a Frequency-Conditioned Transformer Foundation Model

    Sep 17, 2026Zaynab Raounak, Camille LHermine, Zhiguo ZengPredictive MaintenanceHybrid Transformer

  5. Relational Attention for Data-Efficient Language Modeling

    Sep 17, 2026Adrian Brasoveanu, Ece Takmaz, Jakub DotlačilTransformer EncoderRelational

  6. A Smaller Transformer in Your Transformer

    Sep 17, 2026Dhananjay Tomar, Marius Aasan, Andreas Kleppe +1Vision TransformerTransformer Architectures

  7. SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

    Sep 17, 2026Abraham Ezema, Chijioke Eze, Ferdinanda Ponci +1Multivariate Time Series ForecastingTransformer Architectures

  8. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

    Sep 17, 2026DeepSeek-AI, :, Anyi Xu +590Key-Value Cache CompressionKey-Value Cache

  9. Beyond Depth Truncation: Controlled Evaluation of Depth Utilization in Recursive Language Models

    Sep 17, 2026Ha Van Dau, Thanh Tung Khuat, Nguyen Thanh DungSearch DepthTruncation

  10. Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation

    Sep 17, 2026Daeun Kim, Junwha Hong, Changhun Oh +3Autoregressive Image GenerationDynamic Sparse Attention

  11. LSTM-UT and Recurrent-Depth Transformers on Cellular Automata

    Sep 17, 2026Aras KavuncuNeural Cellular AutomataUnified Transformer

  12. How Model Growth, Recursion, and Boundary Operators Influence Scaling Exponents

    Sep 16, 2026Zixi Chen, Akshay Vegesna, Samip Dahal +1Scaling LawsTransformer Layers

  13. Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence

    Sep 16, 2026Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui +1Transformer LayersTransformer Architectures

  14. Variational Quantum Transformer Architecture for Synthetic Language Generation

    Sep 16, 2026Julian Hager, Michael Kölle, Gerhard Stenzel +3Transformer ArchitecturesAutoregressive Language Models

  15. MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators

    Sep 16, 2026Dong Liu, Yanxuan YuKey-Value CacheAccelerator

  16. Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models

    Sep 16, 2026Shardul P. More, Tanuja S. PawarLarge Language Model UncertaintyHallucinations

  17. MoRE: Mixture of Reused Experts

    Sep 16, 2026Eric S. Qiu, Utku Umur Acikalin, Justin Lovelace +4Mixture-of-Experts ModelsExperts

  18. Long-Context Demonstration Selection Using State Space Models

    Sep 15, 2026Ziniu Zhang, Zhenshuo Zhang, Ruoxuan Xiong +2Efficient Long-Context InferenceIn-Context Learning

  19. QiT: Quantum-Inspired Transformer for Visual Recognition Task

    Sep 15, 2026Badri N. Patro, Vijay AgneeswaranSelf-Supervised Vision TransformersQuantum Machine Learning

  20. Self-Supervised Learning for Robust Resonance Mass Regression in Cascade Decays

    Sep 15, 2026Ho Fung Tsoi, Alex Yang, Luis Felipe Gutierrez Zagazeta +2Self-SupervisedMassive

  21. Persistent Recurrent Memory Between Transformer Layers - Improves Language Model Generalization

    Sep 15, 2026Eduardo Novaes HeringTransformer LayersTransformer Encoder

  22. Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

    Sep 15, 2026Dushyant RajputKey-Value CacheLow-Rank Adaptation Adapters

  23. TEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentation

    Sep 15, 2026Jiahong Yuan, Weiming Mi, Tao Zhang +1Medical Image SegmentationSpatio-Temporal Transformers

  24. The Latent That Never Was: A Forensic Re-run of the CVAE Ablation in Action Chunking Transformers

    Sep 15, 2026Bo KangLatent ActionsVariational Autoencoder

  25. Vision And Text Transformer For Predicting Answerability On Visual Question Answering

    Sep 15, 2026Tung Le, Huy Tien Nguyen, Le Minh NguyenKnowledge-Based Visual Question AnsweringSelf-Supervised Vision Transformers

  26. What Does Layer-Importance Reveal About Transformers and State-Space Models?

    Sep 15, 2026Istabrak Abbes, Nizar Islah, Irina Rish +1Transformer ArchitecturesTransformer Residual Streams

  27. Multimodal Emergency Vehicle Classification via Audio-Visual Transformers and Knowledge Distillation

    Sep 15, 2026Vijay John, Amar DabajaMultimodal TransformersCross-Modal Fusion

  28. Disentangling Representation Evolution in Transformers through Directional Decomposition

    Sep 14, 2026Shwai He, Haichao Zhang, Shen YanTransformer ArchitecturesRepresentation Space

  29. VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

    Sep 14, 2026Xingyang Li, Dongyun Zou, Shining Zhang +8Dynamic Sparse AttentionPre-Trained Video Diffusion Models

  30. Same path, different: a mechanistic comparison of looped and stacked transformer encoders on 12-lead ECG

    Sep 14, 2026Pawel Olszowiec, Michal Byra, Grzegorz Gruszczynski +2Transformer EncoderLoop

  31. On the role of the tokenizer in ECG transformer models

    Sep 14, 2026Jiawei Li, Fabio Bonassi, Johan Sundström +2TokenizationPhysiological Data

  32. Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders

    Sep 14, 2026Qingtao Xia, Jiahua Bao, Siyao Cheng +1Parameter-Efficient Fine-TuningFinetuning

  33. T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning With Dynamic Routing

    Sep 14, 2026Mingqian Yu, Wenpeng Zhang, Peilin ZhaoTransformer ArchitecturesToken-Level Uncertainty

  34. DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models

    Sep 14, 2026Pengyu Ji, Zichen Zhang, Xiang Hu +1Diffusion Language ModelsAutoregressive Transformers

  35. MoARa: Module-Aware Rank Allocation and Structure-Preserving Decomposition for Low-Rank LLM Pre-training

    Sep 14, 2026Keunyoung Kim, Nojun KwakGradient-Free Rank AllocatorLarge Language Model Pretraining

  36. Type Diversity Enables Transformers to Generalise Compositionally

    Sep 14, 2026Anssi Moisio, Mathias Creutz, Mikko KurimoCompositionalityGeneralisation

  37. Attention Quantization for Tabular Foundation Models

    Sep 14, 2026Jonas M. Kübler, Benjamin Jäger, Klemens Flöge +2Tabular Foundation ModelsKey-Value Cache Quantization

  38. SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

    Sep 14, 2026Zhiwei Li, Lei Zhu, Hao Gu +6Dynamic Sparse AttentionQuadratic Attention

  39. Learning Symbolic Constraint Representations from Examples: A Neuro-Symbolic Approach

    Sep 14, 2026Nassim Belmecheri, Arnaud Gotlieb, Nadjib Lazaar +1Neuro-Symbolic FrameworkTransformer Architectures

  40. ResoSeg: Resonance Tagger using Transformer and Segment Model

    Sep 14, 2026Chunkai Li, Junhao Yin, Ke Li +1High Energy PhysicsTagging

  41. OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation

    Sep 14, 2026Xiangrui Yang, Cheng Peng, Yunfeng Zhao +9Large-Scale Recommendation SystemsQuadratic Attention

  42. Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models

    Sep 14, 2026Kalyani Marathe, Artidoro Pagnoni, Tomasz Limisiewicz +4Chain-Of-Thought DistillationLarge Models

  43. RunningTensor: Generalizing Linear Attention to Higher-Order Recurrent States

    Sep 14, 2026Luca Herranz-Celotti, Vincent GuigueRecurrent StateQuadratic Attention

  44. Rethinking Heterogeneous System Disaggregation for Subquadratic Attention

    Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Quadratic AttentionGraphics Processing Unit Memory

  45. AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flow

    Sep 12, 2026Junran Wang, Zehao Jin, Tianyu Luan +1Text-To-Image Diffusion ModelsTransformer Architectures

  46. CEM-TUDASR: Computationally efficient multi-modality transformer based unsupervised domain adaptive super-resolution approach

    Sep 11, 2026Anjali Sarvaiya, Jay Kadel, Kishor Upla +1Video Capsule EndoscopyMultimodal Transformers

  47. Distance generalization in transformers: why bother with positional encoding?

    Sep 11, 2026Daniel Henrik Nevermann, Claudius GrosPositional EncodingTransformer Architectures

  48. Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models

    Sep 11, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer EncoderEncoders

  49. Quantifying Logical Consistency in Transformers via Query-Key Alignment

    Sep 10, 2026Eduard Tulchinskii, Anastasia Voznyuk, Laida Kushnareva +4Large Language Model ReasoningReasoning Benchmark

  50. Meta-Learning for Data-Efficient Plant Growth Estimation via Vision Transformers and Fuzzy Clustering

    Sep 9, 2026Sheikh Hasan Elahi, Rusith Chamara Hathurusinghe Dewage, Habib Ullah +3Apple Tree Leaf Disease Segmentation DatasetFew-Shot Learning

  51. Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search

    Sep 9, 2026Rui Liu, Tao Zhe, Yanyong Huang +5Tabular LearningSoftmax Attention

  52. Through the Looking Glass: Directly Reading and Writing Transformers

    Sep 9, 2026Mark OskinTransformer ArchitecturesActivation Functions

  53. TransGaze-Object: Transformer Based Driver Gaze Object Prediction Framework in Real Driving

    Sep 9, 2026Pavan Kumar Sharma, Ayush Pande, Pranamesh ChakrabortyGaze Target EstimationGaze

  54. ProbPlug: A Plugin Uncertainty Network for Reliable Confidence in LLM Binary Classification

    Sep 9, 2026Jianzong Wang, Chuhang Liu, Botao Zhao +6Confidence CalibrationBinary Classification

  55. Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses

    Sep 9, 2026Olli TuomiTransformer ArchitecturesHidden States

  56. Decision Transformer for UAV-Mounted RIS-Assisted Dynamic D2D Communications

    Sep 9, 2026Yaxuan LiuUnmanned Aerial Vehicle TrajectoriesReconfigurable Intelligent Surfaces

  57. Pretraining and Distillation Matter More Than Architecture Family for Label-Free Single-Cell Classification

    Sep 9, 2026Philip Graemer, Giuseppe Di CaprioModel PretrainingConvolutional Neural Networks

  58. Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

    Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanKey-Value CacheQuadratic Attention

  59. Length Generalization for Transformers via Compression

    Sep 8, 2026Georg Zetzsche, Hongjian Jiang, Andy Yang +4Transformer ArchitecturesGeneralization Bounds

  60. Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers

    Sep 8, 2026Amit Ben-Artzy, Roy SchwartzTransformer ArchitecturesRepresentational Capacity