Transformers

Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

42 new papers

A weekly snapshot of new work published in Transformers.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Transformers.

Inside this field

Focused directions

1,206 papers

Latest in Transformers

Browse all topics →
CardsList

  1. Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses

    Sep 9, 2026Olli TuomiTransformer ArchitecturesHidden States

  2. Decision Transformer for UAV-Mounted RIS-Assisted Dynamic D2D Communications

    Sep 9, 2026Yaxuan LiuUnmanned Aerial Vehicle TrajectoriesReconfigurable Intelligent Surfaces

  3. Pretraining and Distillation Matter More Than Architecture Family for Label-Free Single-Cell Classification

    Sep 9, 2026Philip Graemer, Giuseppe Di CaprioModel PretrainingConvolutional Neural Networks

  4. Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

    Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanKey-Value CacheQuadratic Attention

  5. Length Generalization for Transformers via Compression

    Sep 8, 2026Georg Zetzsche, Hongjian Jiang, Andy Yang +4Transformer ArchitecturesGeneralization Bounds

  6. Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers

    Sep 8, 2026Amit Ben-Artzy, Roy SchwartzTransformer ArchitecturesRepresentational Capacity

  7. Why shared attention vectors fail: a case for outcome-indexed tuning

    Sep 8, 2026Lenard DomeSoftmax AttentionGradient Descent

  8. A Transformer-Based Delta Expression Encoder for Psilocybin Transcriptional Response: Architecture, Representations, and Biological Validation

    Sep 8, 2026Sai JayakumarSpatial TranscriptomicsTransformer Encoder

  9. Jacap: Robust KV Cache Eviction via Jacobian-Based Nonlinear Information Capacity Preservation

    Sep 8, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +2Key-Value Cache EvictionQuadratic Attention

  10. Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction

    Sep 8, 2026Kunhan GuoMulti-Head Self-AttentionMarket

  11. LLM Layers Immediately Correct Each Other

    Sep 7, 2026Arjun Patrawala, Jiahai Feng, Erik Jones +1Transformer LayersModel Activations

  12. RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention

    Sep 7, 2026Bin Li, Sisi Liu, Chenyang Hu +3Dynamic Sparse AttentionAdaptive Routing

  13. Dense Structural Compression of Transformers via Gauge-Correct Channel Removal

    Sep 7, 2026Jed A. Duersch, Naïm Es-Sebbani, Nathanaël Haas +1Context CompressionTensor Programs

  14. CEDAR: Error-Bounded Residual Routing for Efficient Long-Context Attention

    Sep 7, 2026Siyu Li, Dong Wang, Jie Zhou +3Dynamic Sparse AttentionEfficient Long-Context Inference

  15. Encoded Early, Used Late: Where Transformers Begin to Act on an Inferred Partner's Expertise

    Sep 7, 2026Mika Okamoto, Gabriele SartiTransformer EncoderExpert Demonstrations

  16. Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer

    Sep 7, 2026Zhiwei Ning, Zhen Zhou, Puhua Jiang +73D Generative ModelsOn-Policy Distillation

  17. BioSync: Transformer-Based Cross-Modal Fusion for a Multimodal Physiological Digital Biomarker

    Sep 7, 2026Seyed Mahmoud Sajjadi MohammadabadiMultimodal Physiological SignalsBiomarker

  18. Hardware-Aware FP4 FlashAttention-4

    Sep 3, 2026Robert HuQuadratic AttentionTensor Programs

  19. Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

    Sep 3, 2026Sergii Kozyrev, Davyd MaiborodaFp8Quantization

  20. The Head Complexity of Boolean Functions in Single-Layer Attention

    Sep 3, 2026Rajmohan Rajaraman, Ravi Sundaram, Amanuel TesfayeMulti-Head Self-AttentionModel Complexity

  21. VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branch

    Sep 3, 2026WenJie FanKey-Value CacheCache

  22. What Matters for Aggressive Decoding-Time KV Eviction? Temporal Aggregation and Ranking Preservation

    Sep 3, 2026Bo Zeng, Yu Zhao, Yefeng Liu +3Key-Value Cache CompressionKey-Value Cache Eviction

  23. Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

    Sep 3, 2026Heng Wang, Jielin Qiu, Wenting Zhao +7Key-Value Cache EvictionEviction

  24. Modern Transformers Are Implicit Hybrids: From Functional Differentiation to Principled Hybrid Architecture Design

    Sep 2, 2026Runlin Shi, Bojian Yin, Guoqi LiTransformer AttentionRotary Position Embedding

  25. OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Items

    Sep 1, 2026Shuze Daniel Liu, David Simchi-Levi, Claire Chen +2Monotonic Inventory EstimationSupply Chain Optimization

  26. CRISP: Cliff-awaRe Input-adaptive Sparse Prefilling with Structural-Mass-Motivated Routing

    Sep 1, 2026Huu Huy Nguyen, Chien Van Nguyen, Franck Dernoncourt +4Dynamic Sparse AttentionQuadratic Attention

  27. Improved Automatic Target Recognition in Synthetic Aperture Sonar Imagery Using Large Deep Neural Networks

    Sep 1, 2026C. J. Moore, Alex Hurt, Jordan MalofModern Deep NetworksState-Of-The-Art

  28. CoViT: Instance-Correspondence Contrastive Learning for Vision Transformer

    Sep 1, 2026Yisen Wang, Zhirong Wu, Limin WangSelf-Supervised Vision TransformersVision Transformer

  29. One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

    Sep 1, 2026Skanda Athreya, Yutong WangTransformer LayersMulticlass Classification

  30. From Language to Behavior: Scaling Sequence Transformers for Industrial Recommendation Ranking with Rec-Native Designs

    Sep 1, 2026Jie Chen, Xiangqian Yu, Yanchao Lian +9RecommendationTransformer Architectures

  31. Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey

    Sep 1, 2026Arjan Blankestijn, Uraz Odyurt, Amirreza YousefzadehField-Programmable Gate ArraysAccelerator

  32. Scaled Idempotence in Transformer Attention: Paired OV Geometry and Shared-Value Algebras

    Sep 1, 2026Jiming Feng, Junliang LiTransformer AttentionTransformer Architectures

  33. Denoising Diffusion Generative Models Secretly Calculate Attentions

    Sep 1, 2026Farzan Haddadi, Leila Monfared, Ebrahim Rezaii +3Multimodal Diffusion TransformersImage Generation

  34. Measuring Optimal Transport in Transformer Depth

    Sep 1, 2026Alexandre QuemyTransformer LayersOptimal Transport

  35. Toppling the Hierarchy in Byte-level Language Modeling

    Aug 31, 2026Lukas Edman, Alexander FraserByte-Pair EncodingTransformer Layers

  36. Late Transformer Layers Recode Syntax Canonically: Evidence from Greek Scrambling and Cross-Layer Generalisation

    Aug 31, 2026Christos Nikolaos Zacharopoulos, Revekka Kyriakoglou, Chara Tsoukala +1Transformer LayersSyntactic Structure

  37. Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy

    Aug 31, 2026Shmuel Berman, Jia DengLong-Horizon TasksContext Compression

  38. Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers

    Aug 31, 2026Takuya Ito, Ruchir Puri, Murray Campbell +1Transformer ArchitecturesCircuits

  39. Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding

    Aug 31, 2026Zhigeng Liu, Zhiyuan Ning, Ruixiao Li +5Efficient Long-Context InferenceDynamic Sparse Attention

  40. Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper

    Aug 31, 2026Chanhee Cho, Junhyuk Choi, Bugeun KimAudio TokensSubsampling

  41. TrainSDC: Characterizing and Mitigating Silent Data Corruption in Large Language Model Training

    Aug 31, 2026Zhipeng Xia, Haotian Xu, Siyu Yun +4Corrupted ObservationsFault Injection Experiments

  42. Tracing distinguishability through transformer processing with stochastic LayerNorm

    Aug 31, 2026Kieran MurphyTransformer ArchitecturesBatch Normalization

  43. AQ3D: Adaptive Query Transformer for 3D Instance Segmentation

    Aug 31, 2026Keno Moenck, Thorsten Schüppstuhl3D Semantic Segmentation3D Object Understanding

  44. No Equivariant Architecture Covers All Equivariant Attention

    Aug 31, 2026Tīkun ÔngMulti-Head Self-AttentionEquivariance

  45. DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

    Aug 31, 2026Yanqi Yu, Pingwei Sun, Jianchao Tan +4Quadratic AttentionRecurrent State

  46. Beat-Synchronous Tokenization for ECG Transformers

    Aug 31, 2026Ahmed Sameh, Nolan Wilson, Max Enderlein +1TokenizationTransformer Architectures

  47. On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

    Aug 31, 2026Zihan Qiu, Zekun Wang, Xiao Li +33Qwen3N$-Grams Features

  48. Tail-Replay: Escaping the Curse of Linear Attention in Prefix Caching for Hybrid LLMs

    Aug 31, 2026Yirui Liu, Ruoling Qi, Xuaner Wu +2Quadratic AttentionEfficient Long-Context Inference

  49. CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration

    Aug 31, 2026Haoyun Jiang, Haolin Li, Jianwei Zhang +7Efficient Long-Context InferenceLLM Inference Efficiency

  50. INTERVenE: Temporal-Abstraction-Interval Based Transformers for Short-Horizon Medical Event Prediction

    Aug 30, 2026Shahar Oded, Yuval ShaharClinical PredictionMimic-Iv Datasets

  51. Higher-Dimensional Rotary Position Embedding

    Aug 30, 2026Yixing Li, Ruobing Xie, Yudong Zhang +3Rotary Position EmbeddingTransformer Architectures

  52. HyQuant: Hybrid-Precision Quantization for LLM Attention

    Aug 28, 2026Jiatong Ding, Bingxin Xing, Yu Zhang +9Large Language Model QuantizationMixed-Precision Quantization

  53. TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

    Aug 27, 2026Hong Chen, Yudong Zeng, Yongwei Huang +4Key-Value Cache EvictionCache

  54. PAGE: Partition-Aware Gated KV-Cache Eviction

    Aug 26, 2026Pankaj Kumar, Subhankar MishraKey-Value Cache EvictionPer-Token Latency

  55. TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

    Aug 25, 2026Xiaoda Yang, Yuxiang Liu, Kaiwen Zheng +12Joint Audio-Video GenerationDynamic Sparse Attention

  56. Toward a First-Principles Update Geometry for the Language-Model Head

    Aug 23, 2026Aditya Somasundaram, Charles Guille-Escuret, Alexander Moreno +2Spectral NormMulti-Head Self-Attention

  57. Ask Self, Ask Others: Relation Is All You Need

    Aug 20, 2026Yuting Ge, Pengju Yang, Mingkai NieQuadratic AttentionFlash

  58. Breaking the Compression Barrier: Cross-Architecture Compression Boundary Learning via Reverse Regrowth

    Aug 17, 2026Zhaocen Liu, Satvik Praveen, Yi ShengContext CompressionResource-Constrained Edge Devices

  59. Non-Parametric Spatiotemporal Trajectory Prediction via State-Conditioned Transition Sampling

    Aug 14, 2026Michael Fore, Akshay Jain, Justin Downes +2Trajectory PredictionTrajectory Prediction

  60. Algebraic Decomposition Theory for Transformer Length Generalization

    Aug 13, 2026Andy Yang, Blerta Veseli, Corentin Barloy +5Transformer ArchitecturesAlgebraic Structures