Efficient Attention

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 114% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 96

All topics

Attention via Black-Box Vector Search
Oct 7, 2026Stepan Zharkov, Krish Singal, Ashwin Padaki +1Efficient Attention Sparse Attention
University of Pennsylvania
LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators
Sep 30, 2026Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5AI Accelerator Inference LLM Inference Acceleration
Faculty of Mathematics University of Vienna, Austria · Huawei Heisenberg Research Center, Munich, Germany · Huawei Technologies Co. Ltd
Switching Linear Attention
Sep 30, 2026Hyun Dong Lee, Xavier Gonzalez, Nicolas Zucchet +3Efficient Attention Linear Attention
Stanford University · Unconventional AI. Work done at Stanford University.
SANTA++: Sampling Attention through Representative Keys
Sep 28, 2026Kyle Lee, Christian Z. Pratt, Ruoyu Fang +4Memory-Efficient Inference LLM Inference Acceleration
Department of Electrical and Computer Engineering University of California, Santa Barbara, CA, USA · Flucta, San Francisco, CA, USA
When Can Attention Heads Be Statically Defined?
Sep 28, 2026Weixian Waylon Li, Yintao Tai, Marcio Fonseca +1Attention Head Analysis Efficient Attention
University of Edinburgh, United Kingdom · Chamber of Deputies, Brazil
Broken Symmetry in BF16 Attention: Why FlashAttention Gradients Blow Up Late in Training
Sep 28, 2026Junlin Chen, Daize Dong, Huanwei Di +9Softmax Attention Efficient Attention
Rutgers University · Carnegie Mellon University · Oracle +2
FoldAttention: Declared-Reference Softmax for Fast Decode and Deterministic Backward
Sep 27, 2026Sriman AchantaSoftmax Attention LLM Inference Acceleration
Virginia Commonwealth University
Benchmarking Attention for Tabular Foundation Models
Sep 25, 2026Maximilian Schambach, Clemens Biehl, Sam ThelinTabular Foundation Models Efficient Attention
SAP SE, Germany
Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models
Sep 23, 2026Ke Wan, Chen ChenLong-Context Language Model Inference Recurrent Transformers
Department of Computer Science University of Virginia Charlottesville, VA, USA
FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention
Sep 21, 2026Anthony Givans, Michael Crawshaw, Mingrui LiuSoftmax Attention Automatic Differentiation
Reaching Every Position Without Searching: Rotating Sparse Wiring on the Hypercube as a Substitute for Attention
Sep 16, 2026Yoshiaki TakashitaEfficient Attention Language Modeling
School of Law, Waseda University
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Disaggregated LLM Serving LLM Inference Acceleration
Harvard University Cambridge, Massachusetts, USA · NVIDIA Santa Clara, California, USA
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Sep 8, 2026Vishesh Tripathi, Abhay Kumar, Ramsha KhanEfficient Transformer Inference Grouped-Query Attention
FrontiersMind
Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction
Sep 8, 2026Kunhan GuoLow-Rank Attention Financial Forecasting
The University of Hong Kong
RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention
Sep 7, 2026Bin Li, Sisi Liu, Chenyang Hu +3Adaptive Model Routing Efficient Attention
Department of Computer Science, Xiamen University of Technology · School of Mathematics and Computer Science, Fuzhou University
CEDAR: Error-Bounded Residual Routing for Efficient Long-Context Attention
Sep 7, 2026Siyu Li, Dong Wang, Jie Zhou +3Long-Context Language Modeling Efficient Attention
School of Computer Science & Technology, Shanghai University · School of Computer Science & Technology, East China Normal University · School of Computer Engineering, Jiangsu Ocean University
Hardware-Aware FP4 FlashAttention-4
Sep 3, 2026Robert HuMixed-Precision Quantization GPU Kernel Optimization
GRAPHCORE RESEARCH
ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition
Aug 31, 2026Yu-Sheng Liu, Yu-Chen TungChannel Attention Fine-Grained Image Classification
Department of Physics, National Kaohsiung Normal University, Kaohsiung, 824, Taiwan
ClusterAttention: A training-free speedup of bidirectional attention
Aug 27, 2026Kasper Nordenram, Amelie DittmannEfficient Attention Attention Mechanisms
Independent researcher
SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
Aug 13, 2026Qi Zhao, Qirui Li, Hanlin Tang +10Video Diffusion Models Diffusion Transformer
Zhejiang University · Alibaba Group
Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage
Aug 12, 2026Kaiwen Tang, Jiaqi Zheng, Zixuan Zhu +3Spiking Transformers Efficient Attention
School of Computing, National University of Singapore · 2Sea AI Lab · 4Shanghai Advanced Research Institute, Chinese Academy of Sciences +2
Hybrid Gated Attention
Aug 12, 2026Zekun Zhou, Ruobing Xie, Lanrui Wang +1Low-Rank Matrix Decomposition Gated Attention
Tencent Hunyuan · Peking University
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Aug 11, 2026Zelei Cheng, Amritansh Mishra, Sambit Sahu +1Efficient Attention Long-Horizon Agent Tasks
AI Foundations, Capital One
Training-Free Hashing-Based Attention via Binary Principal Components
Aug 5, 2026Daohai Yu, Zhanpeng Zeng, Keyu Chen +6Long-Context Language Modeling LLM Inference Acceleration
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Tencent YouTu Lab, Shenzhen, China · Sino-Russian Research Center for Digital Economy.
LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing
Aug 3, 2026Wen Zan, Jiaqi Zhang, Jianchao Tan +11Long-Context Modeling LLM Inference Acceleration
Meituan LongCat Team
S-CEReBrO: Breaking the Memory Barrier in Continuous EEG Monitoring
Jul 30, 2026Glenn Anta Bucagu, Thorir Mar Ingolfsson, Yawei Li +1Efficient Transformer Inference Electroencephalography
ETH Zurich, Zurich, Switzerland · Nanyang Technological University, Singapore · University of Bologna, Bologna, Italy
Recall Before You Rank: Similarity-Guided Top- $K$ Reuse for Efficient Long-Context Attention
Jul 30, 2026Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5Long-Context Language Model Inference Efficient Attention
School of Integrated Circuits, Peking University, Beijing, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China
PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention
Jul 27, 2026Hong Liu, Yuan Cheng, Lin Niu +5LLM Inference Acceleration Efficient Attention
Tencent
A Coulomb Particle Model for Learning Kernel Attention in Transformers
Jul 26, 2026Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2Transformer Attention Efficient Attention
eBay Inc., San Jose, CA, USA
ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers
Jul 22, 2026Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun MoonLow-Rank Attention Transformer Attention
School of Electrical Engineering, KAIST, Daejeon 34141, Republic of Korea · Department of Electrical Engineering and Computer Science, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea

Oct 7, 2026, cs.DSJ/K move · Enter open · S save

Attention via Black-Box Vector Search

Stepan Zharkov, Krish Singal, Ashwin Padaki, Alexandr Andoni

University of Pennsylvania

Efficient Attention Sparse Attention LLM Inference Efficiency

Open paper AskPDF arXiv

Sparse attention mechanisms estimate attention over

n

tokens using a small subset of keys. Many existing approaches use maximum inner product search (MIPS) to retrieve the heaviest keys, which motivates the following question: given black-box access to a MIPS oracle, how many keys must be retrieved to output an

\varepsilon

-accurate attention estimate? We answer this question by unifying prior approaches through the framework of priority sampling. With a single MIPS index, we show that

Θ(\sqrt{n}/\varepsilon)

retrieved keys are both sufficient and necessary. With

Θ(\log n)

indices, we give an algorithm that retrieves only

O(\log n+1/\varepsilon^2)

keys and prove that this is near-optimal. More generally, we design algorithms that establish a smooth tradeoff between the number of MIPS indices and number of retrieved keys. We then show that if we allow augmentation of keys and queries, we can bypass the above lower bounds: there exists a simple priority-sampling estimator using a single MIPS index and

O(1/\varepsilon^2)

retrieved keys. When integrated into LLM inference, our algorithms outperform top-

k

and sampling approaches used in prior work and yield attention approximation that scales favorably to long contexts.

Momentum

Related topics

Latest papers 96