Overview

Efficient attention studies how to reduce the cost of comparing queries with keys, especially for long sequences and constrained hardware. Approaches include IO-aware fused kernels, sparse or hierarchical retrieval, linear and recurrent attention, and KV-cache compression; many combine algorithm design with GPU or accelerator-specific kernels. Evaluation spans language modeling and long-context retrieval as well as vision, video, and other sequence tasks, where quality, latency, memory, and energy can trade off differently. A central tension is that practical methods can work well on structured inputs, while preserving full-attention quality across arbitrary inputs—and providing strong guarantees—remains difficult.

Latest papers

CardsList

149 papers13 papers in the last four weeks, down 38% on the four weeks before. 0.1% of all new papers.

  1. The Kernel Reality Check: Benchmarking and Distilling Efficient Attention at Scale

    Nov 17, 2026Fırat Öncel, Cem Subakan, Mirco Ravanelli +1Efficient AttentionLinear Attention

  2. Attend at the Right Resolution: Head-Adaptive Sparse Attention via Query-Key Interaction Structure

    Nov 6, 2026Wajih Hassan Raza, Ahsan Bilal, Yidan Shen +3Sparse AttentionEfficient Attention

  3. ZYFA: Speed and Accuracy of FP8 FlashAttention on AMD CDNA4

    Nov 5, 2026Quentin Anthony, Ganesh Bikshandi, Praneeth Medepalli +1Efficient AttentionGPU Acceleration

  4. Softmax Attention with Any Temperature in Fixed Dimension: Matching the Best Known Maximum-Inner-Product Bounds

    Nov 4, 2026Zixuan Huang, Guozhang ChenEfficient AttentionSoftmax Attention

  5. Fast and Exact Relative-Position Attention via Structure-Aware Kernels

    Oct 31, 2026Delyan Boychev, Ivo Petrov, Dimitar I. DimitrovEfficient AttentionMemory-Efficient Attention

  6. Subquadratic Attention with a Hierarchical Tree-Structured Indexer

    Oct 30, 2026Leon Lufkin, Cutter Dawes, Alexander Speicher +3Efficient AttentionLong-Context Language Modeling

  7. Kernel-based Interpolatory Methods for Context Compression of Attention Mechanisms

    Oct 29, 2026Michel Fabrice Serret, Laura GrigoriLLM Context CompressionKV-Cache Compression

  8. BiHashFormer: Hash-Driven Dual-Branch Transformer for Efficient Object Detection in HRW Shots

    Oct 29, 2026Jingchen Huang, Wenxi Li, Chenyang Lyu +3Object DetectionEfficient Object Detection

  9. MixPT: Point Transformer with Interleaved Window and Global Attention

    Oct 22, 2026Minkyun Seo, Hyungtae Lim, Jaesik ParkPoint TransformerPoint Cloud Understanding

  10. OpenAttention: One Percent Is Enough with Tail Proxies

    Oct 22, 2026Hyunwoo Yu, Jongbeom Lee, Hyunbin Kim +8Efficient AttentionSparse Attention

  11. Flow-based Spectral Kernel Learning for Nonstationary Attention

    Oct 21, 2026Zi Yang, Ying Li, Alejandro Lancho +3Kernel MethodsEfficient Attention

  12. Transformers as Generic Deep Neural Networks: Exact Compilation, the Cost of Removing Activation-Activation Products, and Multi-Agent Attention Clustering

    Oct 20, 2026Sergey Salishev, Zeev Volkovich, Oleg GranichinEfficient Transformer InferenceTransformer Attention

  13. Not All Dimensions Are Equal: Outlier-Aware Attention Linearization with Grouped Quadratic Kernels

    Oct 9, 2026Weiqi Sun, Qianyue Wu, Wenjia Wang +1Attention ApproximationEfficient Attention

  14. High-Throughput Generative Video Super-Resolution on a Single Consumer GPU

    Oct 9, 2026Georgios Panaretou, Sotirios ChatzisVideo Super-ResolutionTemporal Consistency in Video Generation

  15. Sparse Attention Is Matrix Approximation, Not Choosing from a Bag of Values

    Oct 7, 2026Fang Wan, Xufeng Liu, Fan Li +1Sparse AttentionEfficient Attention

  16. Attention via Black-Box Vector Search

    Oct 7, 2026Stepan Zharkov, Krish Singal, Ashwin Padaki +1Attention ApproximationEfficient Attention

  17. LatentIndex: Cross-Layer Sharing with Layer-Specific Selection for Sparse Attention

    Oct 3, 2026Zhaohui Wang, Zhixin Pan, Fanxu Meng +1Cross-Layer AttentionLLM Inference Acceleration

  18. Retrieval-Centric Deep Learning in Growing Nonparametric Neural Networks

    Oct 2, 2026Maximilian Schlegel, Rajai Nasser, Seijin Kobayashi +6Deep Learning OptimizationMemory-Augmented Neural Networks

  19. Page-EntroKV: Hardware-Aligned, Entropy-Weighted KV-Cache Eviction under Grouped-Query Attention

    Oct 2, 2026Inbasekaran SKV-Cache EvictionMemory-Efficient Inference

  20. LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators

    Sep 30, 2026Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5AI Accelerator InferenceLLM Inference Acceleration

  21. Switching Linear Attention

    Sep 30, 2026Hyun Dong Lee, Xavier Gonzalez, Nicolas Zucchet +3Efficient AttentionLinear Attention

  22. Efficiently Approximating Attention Is Hard

    Sep 29, 2026Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2Attention ApproximationEfficient Attention

  23. SANTA++: Sampling Attention through Representative Keys

    Sep 28, 2026Kyle Lee, Christian Z. Pratt, Ruoyu Fang +4Memory-Efficient InferenceLLM Inference Acceleration

  24. When Can Attention Heads Be Statically Defined?

    Sep 28, 2026Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1Attention Head AnalysisEfficient Attention

  25. Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training

    Sep 28, 2026Junlin Chen, Daize Dong, Huanwei Di +9Softmax AttentionEfficient Attention

  26. MassAlloc Attention: Let Attention Allocate Its Own Compute

    Sep 26, 2026Jingze Shi, Zhangyang Peng, Xianduo Li +6Attention MechanismsEfficient Attention

  27. CoWindow Attention: Full Causal Coverage Is a Collective Property

    Sep 26, 2026Jingze Shi, Zhangyang Peng, Xianduo Li +6Efficient AttentionCausal Attention