Efficient Attention

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 114% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Attention via Black-Box Vector Search
Oct 7, 2026Stepan Zharkov, Krish Singal, Ashwin Padaki +1Efficient Attention Sparse Attention
University of Pennsylvania
LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators
Sep 30, 2026Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5AI Accelerator Inference LLM Inference Acceleration
Faculty of Mathematics University of Vienna, Austria · Huawei Heisenberg Research Center, Munich, Germany · Huawei Technologies Co. Ltd
Switching Linear Attention
Sep 30, 2026Hyun Dong Lee, Xavier Gonzalez, Nicolas Zucchet +3Efficient Attention Linear Attention
Stanford University · Unconventional AI. Work done at Stanford University.
SANTA++: Sampling Attention through Representative Keys
Sep 28, 2026Kyle Lee, Christian Z. Pratt, Ruoyu Fang +4Memory-Efficient Inference LLM Inference Acceleration
Department of Electrical and Computer Engineering University of California, Santa Barbara, CA, USA · Flucta, San Francisco, CA, USA
When Can Attention Heads Be Statically Defined?
Sep 28, 2026Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1Attention Head Analysis Efficient Attention
University of Edinburgh, United Kingdom · Chamber of Deputies, Brazil
Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training
Sep 28, 2026Junlin Chen, Daize Dong, Huanwei Di +9Softmax Attention Efficient Attention
Rutgers University · Carnegie Mellon University · Oracle +2
FoldAttention: Declared-Reference Softmax for Fast Decode and Deterministic Backward
Sep 27, 2026Sriman AchantaSoftmax Attention LLM Inference Acceleration
Virginia Commonwealth University
Benchmarking Attention for Tabular Foundation Models
Sep 25, 2026Maximilian Schambach, Clemens Biehl, Sam ThelinTabular Foundation Models Efficient Attention
SAP SE, Germany
Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models
Sep 23, 2026Ke Wan, Chen ChenLong-Context Language Model Inference Recurrent Transformers
Department of Computer Science University of Virginia Charlottesville, VA, USA
FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention
Sep 21, 2026Anthony Givans, Michael Crawshaw, Mingrui LiuSoftmax Attention Automatic Differentiation
Reaching Every Position Without Searching: Rotating Sparse Wiring on the Hypercube as a Substitute for Attention
Sep 16, 2026Yoshiaki TakashitaEfficient Attention Language Modeling
School of Law, Waseda University
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Disaggregated LLM Serving LLM Inference Acceleration
Harvard University Cambridge, Massachusetts, USA · NVIDIA Santa Clara, California, USA
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanEfficient Transformer Inference Grouped-Query Attention
FrontiersMind
Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction
Sep 8, 2026Kunhan GuoLow-Rank Attention Financial Forecasting
The University of Hong Kong
RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention
Sep 7, 2026Bin Li, Sisi Liu, Chenyang Hu +3Adaptive Model Routing Efficient Attention
Department of Computer Science, Xiamen University of Technology · School of Mathematics and Computer Science, Fuzhou University
CEDAR: Error-Bounded Residual Routing for Efficient Long-Context Attention
Sep 7, 2026Siyu Li, Dong Wang, Jie Zhou +3Long-Context Language Modeling Efficient Attention
School of Computer Science & Technology, Shanghai University · School of Computer Science & Technology, East China Normal University · School of Computer Engineering, Jiangsu Ocean University
Hardware-Aware FP4 FlashAttention-4
Sep 3, 2026Robert HuMixed-Precision Quantization GPU Kernel Optimization
GRAPHCORE RESEARCH
ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition
Aug 31, 2026Yu-Sheng Liu, Yu-Chen TungChannel Attention Fine-Grained Image Classification
Department of Physics, National Kaohsiung Normal University, Kaohsiung, 824, Taiwan
ClusterAttention: A training-free speedup of bidirectional attention
Aug 27, 2026Kasper Nordenram, Amelie DittmannEfficient Attention Attention Mechanisms
Independent researcher
SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
Aug 13, 2026Qi Zhao, Qirui Li, Hanlin Tang +10Video Diffusion Models Diffusion Transformer
Zhejiang University · Alibaba Group
Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage
Aug 12, 2026Kaiwen Tang, Jiaqi Zheng, Zixuan Zhu +3Spiking Transformers Efficient Attention
School of Computing, National University of Singapore · 2Sea AI Lab · 4Shanghai Advanced Research Institute, Chinese Academy of Sciences +2
Hybrid Gated Attention
Aug 12, 2026Zekun Zhou, Ruobing Xie, Lanrui Wang +1Low-Rank Matrix Decomposition Gated Attention
Tencent Hunyuan · Peking University
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Aug 11, 2026Zelei Cheng, Amritansh Mishra, Sambit Sahu +1Efficient Attention Long-Horizon Agent Tasks
AI Foundations, Capital One
Training-Free Hashing-Based Attention via Binary Principal Components
Aug 5, 2026Daohai Yu, Zhanpeng Zeng, Keyu Chen +6Long-Context Language Modeling LLM Inference Acceleration
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Tencent YouTu Lab, Shenzhen, China · Sino-Russian Research Center for Digital Economy.
LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing
Aug 3, 2026Wen Zan, Jiaqi Zhang, Jianchao Tan +11Long-Context Modeling LLM Inference Acceleration
Meituan LongCat Team
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Jul 30, 2026Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li +1Efficient Transformer Inference Electroencephalography
ETH Zurich, Zurich, Switzerland · Nanyang Technological University, Singapore · University of Bologna, Bologna, Italy
Recall Before You Rank: Similarity-Guided Top- $K$ Reuse for Efficient Long-Context Attention
Jul 30, 2026Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5Long-Context Language Model Inference Efficient Attention
School of Integrated Circuits, Peking University, Beijing, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China
PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
Jul 27, 2026Hong Liu, Yuan Cheng, Lin Niu +5LLM Inference Acceleration Efficient Attention
Tencent
A Coulomb Particle Model for Learning Kernel Attention in Transformers
Jul 26, 2026Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2Transformer Attention Efficient Attention
eBay Inc., San Jose, CA, USA
ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers
Jul 22, 2026Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun MoonLow-Rank Attention Transformer Attention
School of Electrical Engineering, KAIST, Daejeon 34141, Republic of Korea · Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
Jul 21, 2026Lenore Mulin, Gaetan HainsEfficient Transformer Inference Grouped-Query Attention
University at Albany, SUNY · LACL, Universit´e Paris-Est Cr´eteil
Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones
Jul 19, 2026Ayoub Ghriss, Sourav ChakrabortyEfficient Attention Linear Attention
Department of Computer Science, University of Colorado Boulder
DiTango: Cost-Effective Parallel Diffusion Generation with Selective Attention State Reuse
Jul 17, 2026Yuyang Chen, Runxin Zhong, Zan Zong +3Diffusion Transformer Efficient Attention
Shanghai Jiao Tong University, Shanghai AI Laboratory Shanghai, China · Tsinghua University Beijing, China · University of Science and Technology Beijing Beijing, China +1
VideoSEMA: a scalable and efficient Mamba-like attention for video understanding
Jul 16, 2026Nhat Thanh Tran, Fanghui Xue andShuai Zhang, Jiancheng Lyu +3Efficient ViTs Efficient Attention
Department of Mathematics University of California, Irvine Irvine, USA · Qualcomm AI Research San Diego, USA
AVQ-Attention: Adaptive Vector-Quantized Attention
Jul 14, 2026Winfried van den dool, Patrick Forré, Amir Habibian +2Transformer Attention Vector Quantization
QUVA Lab, University of Amsterdam, The Netherlands · AMLab, Informatics Institute, University of Amsterdam, The Netherlands · AI4Science Lab, University of Amsterdam, The Netherlands +3
FastTPS: An Optimized Method for LLM Token Phase for AI accelerators
Jul 13, 2026Wenzong Yang, Danyang Zhang, Kun Cao +17AI Accelerator Inference LLM Inference
Advanced Micro Devices, Inc., Santa Clara, California, USA.
STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
Jul 10, 2026Victor J. B. Jung, Gagandeep Singh, Joseph Melber +3Efficient Transformer Inference Energy-Efficient ML
Integrated Systems Laboratory (IIS), ETH Zürich, Switzerland. · AMD Research and Advanced Development (RAD). · Department of Electrical, Electronic and Information Engineering (DEI), University of Bologna, Italy.
Morphing into Hybrid Attention Models
Jun 29, 2026Disen Lan, Jianbin Zheng, Yuxi Ren +5Efficient Attention Neural Architecture Search
1Fudan University · 2ByteDance Seed · Work done at ByteDance Seed +1
Hierarchical Global Attention (HGA)
Jun 29, 2026Woernle Frank, Fedosov Vladimir, Grinenko ArtemiyKV-Cache Offloading Long-Context Modeling
BMW Group
Enhancing Layer Interaction Using Key-Correlated Layer Attention
Jun 24, 2026Jianlong Xiong, ChuanBo Xie, Le Yu +2Efficient Attention Linear Attention
College of Computer Science, Sichuan University, Chengdu, 610065, China · Department of Ultrasound, Zigong Maternal and Child Health Care Hospital, China
Scalable Peptide Design via Memory-Efficient Equivariant Transformer
Jun 23, 2026Rui Jiao, Xiangzhe Kong, Yinjun Jia +4Equivariant Neural Networks Efficient Attention
Department of Computer Science and Technology, Tsinghua University · Institute for AI Industry Research, Tsinghua University · Department of Electronic Engineering, Tsinghua University +1
Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers
Jun 20, 2026Xin GaoTransformer Inference KV Caching
Department of Mathematics and Statistics York University Toronto, M3J 1P3
Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention
Jun 18, 2026Vishesh Tripathi, Abhay KumarGrouped-Query Attention Transformer Attention
FrontiersMind
CSWinUNETR: Segmentation of Thin Anatomical Structures in Medical Images
Jun 18, 2026Junho Moon, Haejun Chung, Ikbeom JangHybrid CNN-Transformer Architectures Efficient Attention
Hanyang University, Seoul, Republic of Korea · Hankuk University of Foreign Studies, Yongin, Republic of Korea
Rethinking the Role of Efficient Attention in Hybrid Architectures
Jun 13, 2026Ziqing Qiao, Yinuo Xu, Chaojun Xiao +6Attention Head Analysis Long-Context Modeling
1Tsinghua University · 2OpenBMB
Semidirect Fourier Delta Attention: Phase-Controlled Delta Memory with Constructive Chunk-WY Kernels
Jun 12, 2026Tiantian ZhangEfficient Attention Linear Attention
Columbia University
MiniMax Sparse Attention
Jun 11, 2026Xunhao Lai, Weiqi Xu, Yufeng Yang +14GPU Kernel Optimization Long-Context Language Modeling
MiniMax · Peking University · NVIDIA +4
Gaussian Mixture Attention: Linear-Time Sequence Mixing via Probabilistic Latent Routing
Jun 9, 2026Yongchao Huang, Hassan RazaLong-Context Language Modeling Efficient Attention
Inside the Latent Flow: Causal Deciphering of Attention Dynamics in Audio Separation Foundation Models
Jun 8, 2026Yuxuan Chen, Haoyuan Yu, Peize HeCross-Attention Audio Diffusion Models
Jilin University · Hunan University · University of Electronic Science and Technology of China
Look Less, Reason More: Block-wise Attention Skipping for Efficient Multimodal LLMs
Jun 7, 2026Jie Ma, Zhike Qiu, Jiayi Ji +2Efficient Multimodal Inference Multimodal Large Language Models
Xiamen University
Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels
Jun 5, 2026Lenore Mullin, Gaetan HainsGPU Kernel Optimization Energy-Efficient ML
University at Albany, SUNY, Albany, NY, USA · LACL, Université Paris-Est Créteil, Créteil, France
Towards Tight Bounds for Streaming Attention
Jun 5, 2026Justin Y. Chen, Ying Feng, Piotr Indyk +3Efficient Attention KV-Cache Compression
MIT · EPFL
How Much Dense Attention is Necessary? Oracle-Guided Sparse Prefill for Full/GQA Layers in Hybrid Long-Context Models
Jun 5, 2026Hongxing Wang, Harenome Razanajato, Zhen Zhang +2Grouped-Query Attention Long-Context Language Modeling
Do Transformers Need Three Projections? Systematic Study of QKV Variants
Jun 1, 2026Ali Kayyam, Anusha Madan Gopal, M Anthony LewisEfficient Transformer Inference Transformer Attention
BrainChip Inc., Laguna Hills, CA, USA.
Post-Boundary Bridge: Must Local Attention Go Global Between Global Layers?
Jun 1, 2026Zhibo YangEfficient Attention Local Attention
Ocean University of China
Interdomain Attention: Beyond Token-Level Key-Value Memory
May 23, 2026Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4Self-Attention Long-Context Language Modeling
Approaching I/O-optimality for Approximate Attention
May 22, 2026Pál András Papp, Aleksandros Sobczyk, Anastasios ZouziasSelf-Attention Efficient Attention
Computing Systems Lab, Huawei Technologies, Zurich, Switzerland
Preisach Attention: A Hysteretic Model of Sequential Memory
May 22, 2026Piotr FrydrychTransformer Efficient Attention
The Metrology and Biomedical Engineering Institute · Faculty of Mechatronics, Warsaw University of Technology
ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention
May 21, 2026Joe SharrattMixed-Precision Quantization Self-Attention
Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
May 21, 2026Hangyue Zhao, Paul Caillon, Erwan Fagnou +1Self-Attention Structured Sparsity
ESPCI PSL, Paris, France · LAMSADE, Université Paris Dauphine - PSL, Paris, France

Oct 7, 2026, cs.DSJ/K move · Enter open · S save

Attention via Black-Box Vector Search

Stepan Zharkov, Krish Singal, Ashwin Padaki, Alexandr Andoni

University of Pennsylvania

Efficient Attention Sparse Attention LLM Inference Efficiency

Open paper AskPDF arXiv

Sparse attention mechanisms estimate attention over

n

tokens using a small subset of keys. Many existing approaches use maximum inner product search (MIPS) to retrieve the heaviest keys, which motivates the following question: given black-box access to a MIPS oracle, how many keys must be retrieved to output an

\varepsilon

-accurate attention estimate? We answer this question by unifying prior approaches through the framework of priority sampling. With a single MIPS index, we show that

Θ(\sqrt{n}/\varepsilon)

retrieved keys are both sufficient and necessary. With

Θ(\log n)

indices, we give an algorithm that retrieves only

O(\log n+1/\varepsilon^2)

keys and prove that this is near-optimal. More generally, we design algorithms that establish a smooth tradeoff between the number of MIPS indices and number of retrieved keys. We then show that if we allow augmentation of keys and queries, we can bypass the above lower bounds: there exists a simple priority-sampling estimator using a single MIPS index and

O(1/\varepsilon^2)

retrieved keys. When integrated into LLM inference, our algorithms outperform top-

k

and sampling approaches used in prior work and yield attention approximation that scales favorably to long contexts.

Momentum

Related topics

Latest papers 96