Efficient Attention

Ask about this topic

Momentum

15 papers in the last four weeks, up 114% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 96

All topics
CardsList

  1. Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging

    May 20, 2026Aaron Wang, Zihan Zhao, Alan Xia +5Jet TaggingEfficient Attention

    University of Illinois Chicago Chicago, IL 60607, USA · University of California San Diego La Jolla, CA 92093, USA · California Institute of Technology Pasadena, CA 91125, USA +1

  2. DashAttention: Differentiable and Adaptive Sparse Hierarchical Attention

    May 18, 2026Yuxiang Huang, Nuno M. T. Gonçalves, Federico Alvetreti +5Self-AttentionLong-Context Modeling

    1Tsinghua University · 2Instituto Superior Técnico, Universidade de Lisboa. · 3Instituto de Telecomunicações. +6

  3. CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection

    May 16, 2026Jiwon Song, Dongwon Jo, Beomseok Kang +1Self-AttentionKV-Cache Management

    Seoul National University

  4. GQA-μP: The maximal parameterization update for grouped query attention

    May 14, 2026Kyle R. Chickering, Huijuan Wang, Mengxi Wu +7Grouped-Query AttentionRepresentation Learning

    UC Davis & MBZUAI IFM · USC & MBZUAI IFM · USC +4

  5. SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference

    May 13, 2026Anay Chauhan, Gurucharan Marthi Krishna Kumar, Arion Das +4KV CachingLong-Context Language Model Inference

    1Synopsys, India · 2McGill University, Canada · 3IIIT Ranchi, India +4

  6. ASAP: Amortized Doubly-Stochastic Attention via Sliced Dual Projection

    May 13, 2026Huy Tran, Max Milkert, David HydeTransformer InferenceSelf-Attention

    Department of Computer Science Vanderbilt University

  7. The Routing and Filtering Structure of Attention

    May 12, 2026Shafayeth Jamil, Rehan KapadiaAttention Head AnalysisSelf-Attention

    Ming Hsieh Department of Electrical and Computer Engineering University of Southern California Los Angeles, CA 90089

  8. Variational Linear Attention: Stable Associative Memory for Long-Context Transformers

    May 11, 2026Vishal Pandey, Gopal SinghEfficient AttentionLinear Attention

    Independent Researcher London, UK · Metriqual Athens, GR

  9. USEMA: a Scalable Efficient Mamba Like Attention for Medical Image Segmentation

    May 11, 2026Elisha Dayag, Nhat Thanh Tran, Jack XinImage SegmentationHybrid CNN-Transformer Architectures

    University of California Irvine, Irvine CA 92697

  10. Nectar: Neural Estimation of Cached-Token Attention via Regression

    May 10, 2026João Monteiro, Michal Klein, Pierre Ablin +1Softmax AttentionSelf-Attention

    Apple

  11. Attention Itself Could Retrieve.RetrieveVGGT: Training-Free Long Context Streaming 3D Reconstruction via Query-Key Similarity Retrieval

    May 10, 2026Zichen Zou, Xiaosong Jia, Zuxuan Wu +13D ViTsStreaming 3D Reconstruction

    Institute of Trustworthy Embodied AI (TEAI), Fudan University · 2Shanghai Key Laboratory of Multimodal Embodied AI

  12. Positional LSH: Binary Block Matrix Approximation for Attention with Linear Biases

    May 10, 2026Daniel Wolfson, Tal WagnerSelf-AttentionLong-Context Language Modeling

    Blavatnik School of Computer Science and AI, Tel Aviv University

  13. Long Context Pre-Training with Lighthouse Attention

    May 7, 2026Bowen Peng, Subho Ghosh, Jeffrey QuesnelleLanguage Model PretrainingSelf-Attention

    Nous Research

  14. Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers

    May 7, 2026Amrit NidhiEfficient Transformer InferenceSelf-Attention

    Independent Researcher

  15. Nearly Optimal Attention Coresets

    May 7, 2026Edo Liberty, Alexandr Andoni, Eldar KleinerSoftmax AttentionEfficient Attention

  16. Cascade Token Selection for Transformer Attention Acceleration

    May 4, 2026Stephen J. ThomasSelf-AttentionTransformer Attention

    Department of Computer Science and Engineering, Lehigh University, Bethlehem, PA 18015

  17. StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k

    May 4, 2026Jaber Jaber, Osama JaberGPU Kernel OptimizationSelf-Attention

    RightNow AI

  18. Better Models, Faster Training: Sigmoid Attention for single-cell Foundation Models

    Apr 29, 2026Vijay Sadashivaiah, Georgios Dasoulas, Judith Mueller +1Softmax AttentionSelf-Attention

    Merck & Co., Inc., Cambridge, MA, USA

  19. GateMOT: Q-Gated Attention for Dense Object Tracking

    Apr 29, 2026Mingjin Lv, Zelin Liu, Feifei Shao +4Visual Object TrackingGated Attention

    Huazhong University of Science and Technology, Wuhan, China · Zhejiang University, Hangzhou, China · La Trobe University, Melbourne, Australia

  20. MixerCA: An Efficient and Accurate Model for High-Performance Hyperspectral Image Classification

    Apr 28, 2026Mohammed Q. Alkhatib, Ali JamaliHyperspectral ImagingEfficient Attention

    aCollege of Engineering and IT, University of Dubai, Dubai, 14143, UAE · bDepartment of Geography, Simon Fraser University, 8888 University Dr, Burnaby, BC V5A 1S6, Canada

  21. LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs

    Apr 23, 2026Mohamed Ali Souibgui, Jan Fostier, Rodrigo Abadía-Heredia +3Transformer AttentionLLM Inference Acceleration

    Openchip & Software Technologies, S.L.

  22. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding

    Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Efficient Transformer InferenceSelf-Attention

    Harvard University

  23. LaplacianFormer:Rethinking Linear Attention with Laplacian Kernel

    Apr 22, 2026Zhe Feng, Sen Lian, Changwei Wang +5Efficient ViTsEfficient Attention

    1MAIS,Institute of Automation, Chinese Academy of Sciences · usr 3China Electronics Data Corporation · 5The Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology +4

  24. DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing

    Apr 21, 2026Jinyu Guo, Zhihan Zhang, Jiehui Xie +7Long-Context Language Model InferenceLLM Inference Acceleration

    School of Information and Software Engineering, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China · Bangladesh University of Engineering and Technology +2

  25. Open-TQ-Metal: Fused Compressed-Domain Attention for Long-Context LLM Inference on Apple Silicon

    Apr 18, 2026Sai VegasenaAI Accelerator InferenceSelf-Attention

    Ensue

  26. Efficient Video Diffusion Models: Advancements and Challenges

    Apr 17, 2026Shitong Shao, Lichen Bai, Pengfei Wan +2Model CompressionVideo Diffusion Models

    The Hong Kong University of Science and Technology (Guangzhou), China · Kling Team, Kuaishou Technology, China · The Hong Kong University of Science and Technology, Hong Kong

  27. Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU

    Apr 16, 2026Jevin Jiang, Ying Chen, Blake A. Hechtman +2Efficient Transformer InferenceAI Accelerator Inference

    Google

  28. AdaSplash-2: Faster Differentiable Sparse Attention

    Apr 16, 2026Nuno Gonçalves, Hugo Pitorro, Vlad Niculae +4GPU Kernel OptimizationSelf-Attention

    Instituto Superior Técnico, Universidade de Lisboa · Instituto de Telecomunicações · Carnegie Mellon University +4

  29. Attention-Weighted Value Projection for KV-Cache Compression

    Apr 13, 2026Samuel SalfatiGrouped-Query AttentionKV Caching

    fraQtl AI Research

  30. HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention

    Mar 30, 2026Yufei Xu, Fanxu Meng, Fan Jiang +11Self-AttentionLLM Inference Acceleration

    Xiaohongshu

  31. Beyond Pairwise Attention: Higher-Order Modular Attention for Efficient Sequence Learning

    Mar 11, 2026Shirin Amiraslani, Xin GaoSelf-AttentionTransformer Attention

    Department of Mathematics and Statistics York University Toronto, ON M3J 1P3, Canada

  32. SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routing

    Mar 9, 2026Xuanyi Zhou, Qiuyang Mang, Shuo Yang +7Video Diffusion ModelsEfficient Attention

    UC Berkeley, USA

  33. FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion

    Feb 5, 2026Zhuokun Chen, Jianfei Cai, Bohan ZhuangVideo Diffusion ModelsSelf-Attention

    1Monash University · 2Zhejiang University

  34. Transform Trained Transformer for Accelerating Native 4K Video Generation

    Dec 15, 2025Jiangning Zhang, Junwei Zhu, Teng Hu +9Efficient Transformer InferenceEfficient Attention

    Youtu Lab, Tencent · Zhejiang University

  35. Accelerating Attention with Basis Decomposition

    Oct 2, 2025Jialin ZhaoSelf-AttentionTransformer Attention

    Department of Computer Science Tsinghua University

  36. SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging

    Jun 10, 2025Nhat Thanh Tran, Fanghui Xue, Shuai Zhang +4Self-AttentionVision Transformer

    Department of Mathematics; University of California, Irvine; Irvine, CA 92697, USA · Qualcomm AI Research; San Diego, CA 92121, USA