Sparse Attention

Follow this topic Ask about this topic

Momentum

24 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 160

All topics

From Local Windows to Adaptive Candidates via Individualized Exploratory: Rethinking Attention for Image Super-Resolution
Jan 13, 2026Chunyu Meng, Wei Long, Shuhang GuEfficient ViTs Sparse Attention
University of Electronic Science and Technology of China
Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference
Dec 18, 2025Dhruv Deshmukh, Saurabh Goyal, Nipun Kwatra +1Self-Attention Long-Context Language Model Inference
Microsoft Research India.
Block Sparse Flash Attention
Dec 7, 2025Daniel Ohayon, Itay Lamprecht, Itay Hubara +3GPU Acceleration Self-Attention
Technion – Israel Institute of Technology, Haifa, Israel · Intel - Habana Labs · Intel – Habana Labs, Tel Aviv, Israel
NOSA: Native and Offloadable Sparse Attention
Oct 15, 2025Yuxiang Huang, Pengjie Wang, Jicheng Han +9KV-Cache Offloading GPU Acceleration
NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing, China. · OpenBMB, China. · BUPT, Beijing, China. +1
Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining
Sep 12, 2025Rupert Mitchell, Kristian KerstingLanguage Model Pretraining Softmax Attention
Department of Computer Science, TU Darmstadt, Darmstadt, Germany · Hessian Center for AI (hessian.AI), Darmstadt, Germany · German Research Center for Artificial Intelligence (DFKI), Darmstadt, Germany +1
FSA: An Alternative Efficient Implementation of Native Sparse Attention Kernel
Aug 25, 2025Ran Yan, Youhe Jiang, Zhuoming Chen +3GPU Kernel Optimization GPU Acceleration
1. The Hong Kong University of Science and Technology · 2. Carnegie Mellon University
NABLA: Neighborhood Adaptive Block-Level Attention
Jul 17, 2025Dmitrii Mikhailov, Aleksey Letunovskiy, Maria Kovaleva +6Video Diffusion Models Diffusion Transformer
Kandinsky Lab, Moscow, Russia
The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
Apr 24, 2025Piotr Nawrot, Robert Li, Renjie Huang +3Transformer Inference LLM Evaluation
University of Edinburgh · Cohere · Meta
Fewer yet critical: Reducing Redundant Token Dependencies for Transformer-based Time Series Forecasting
Mar 10, 2025Jianqi Zhang, Yuchan Liu, Zeen Song +2Time Series Forecasting Transformer Attention
National Key Laboratory of Space Integrated Information Systems, Institute of Software, Chinese Academy of Sciences, Beijing, China 1 · University of Chinese Academy of Sciences, Beijing, China 2 · Digital-Intelligent Operations Center, Guizhou Power Grid Co., Ltd., Guizhou, China 3 +1
SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis
Date pendingJongbeom Lee, Hyunwoo Yu, Jincheol Yang +2Image-to-Video Generation Self-Attention
Sogang University Seoul, Republic of Korea

Jan 13, 2026, cs.CVJ/K move · Enter open · S save

From Local Windows to Adaptive Candidates via Individualized Exploratory: Rethinking Attention for Image Super-Resolution

Chunyu Meng, Wei Long, Shuhang Gu

University of Electronic Science and Technology of China

Efficient ViTs Sparse Attention Image Super-Resolution

Open paper AskPDF arXiv

Single Image Super-Resolution (SISR) is a fundamental computer vision task that aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) input. Transformer-based methods have achieved remarkable performance by modeling long-range dependencies in degraded images. However, their feature-intensive attention computation incurs high computational cost. To improve efficiency, most existing approaches partition images into fixed groups and restrict attention within each group. Such group-wise attention overlooks the inherent asymmetry in token similarities, thereby failing to enable flexible and token-adaptive attention computation. To address this limitation, we propose the Individualized Exploratory Transformer (IET), which introduces a novel Individualized Exploratory Attention (IEA) mechanism that allows each token to adaptively select its own content-aware and independent attention candidates. This token-adaptive and asymmetric design enables more precise information aggregation while maintaining computational efficiency. Extensive experiments on standard SR benchmarks demonstrate that IET achieves state-of-the-art performance under comparable computational complexity.

Momentum

Related topics

Latest papers 160