Dynamic Sparse Attention

Ask about this topic

Recent momentum

-65%

14 papers in the last 28 days · 0.4% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Dynamic Sparse Attention

Neighbouring directions

Transformer AttentionModel AttentionSoftmax AttentionMulti-Head AttentionSelf-Attention Mechanism

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

3 new papers

A weekly snapshot of new work published in Dynamic Sparse Attention.

Period ending 2026-09-07

6 new papers

A weekly snapshot of new work published in Dynamic Sparse Attention.

201 papers

Latest in Dynamic Sparse Attention

Open your feed →
CardsList

  1. VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

    Sep 14, 2026Xingyang Li, Dongyun Zou, Shining Zhang +8Dynamic Sparse AttentionVideo Diffusion Models

  2. SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

    Sep 14, 2026Zhiwei Li, Lei Zhu, Hao Gu +6Dynamic Sparse AttentionEfficient Long-Context Inference

  3. Rethinking Heterogeneous System Disaggregation for Subquadratic Attention

    Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Dynamic Sparse AttentionGraphics Processing Unit Memory

  4. SequenceO1: End-to-End Ultra-Long (100K) Sequence Modeling in Recommendation with Low-Rank Caching

    Sep 8, 2026Lin Guan, Jia-Qi Yang, Zhishan Zhao +15Recommender SystemsSequence Modeling

  5. RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention

    Sep 7, 2026Bin Li, Sisi Liu, Chenyang Hu +3Dynamic Sparse Attention

  6. CEDAR: Error-Bounded Residual Routing for Efficient Long-Context Attention

    Sep 7, 2026Siyu Li, Dong Wang, Jie Zhou +3Dynamic Sparse AttentionEfficient Long-Context Inference

  7. RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers

    Sep 6, 2026Zekun Zhang, Yixiang Cai, Yuxi Liu +9Video Diffusion ModelsRotary Position Embedding

  8. VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branch

    Sep 3, 2026WenJie FanKey-Value CacheCache

  9. SGD-KV: Summarization Guided KV Cache Compression

    Sep 3, 2026Zeyu Liu, Woomin Song, Xuandi Fu +5Key-Value Cache CompressionEfficient Long-Context Inference

  10. BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

    Sep 2, 2026Bigyan Ghimire, Jon C. CalhounParallel Decoding and TrainingLarge Language Model Training

  11. CRISP: Cliff-awaRe Input-adaptive Sparse Prefilling with Structural-Mass-Motivated Routing

    Sep 1, 2026Huu Huy Nguyen, Chien Van Nguyen, Franck Dernoncourt +4Dynamic Sparse AttentionLLM Inference Efficiency

  12. Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding

    Aug 31, 2026Zhigeng Liu, Zhiyuan Ning, Ruixiao Li +5Efficient Long-Context InferenceDynamic Sparse Attention

  13. TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories

    Aug 31, 2026Daniel Agyei Asante, Yang LiLarge Language Model CompressionEfficient Long-Context Inference

  14. Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware

    Aug 31, 2026Simon Richter, Ruhai Lin, Jason Yik +4Dynamic Sparse AttentionNeuromorphic Hardware

  15. DASC: Decay-Aware State Compression for Hybrid Linear-Attention Serving

    Aug 31, 2026Yanqi Yu, Pingwei Sun, Jianchao Tan +4Dynamic Sparse AttentionRecurrent State

  16. On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

    Aug 31, 2026Zihan Qiu, Zekun Wang, Xiao Li +33Qwen3N$-Grams Features

  17. A.X K2 Technical Report

    Aug 31, 2026Cheolseung Baek, Dhammiko Arya, Eunki Kim +40Mixture-Of-Expert Language ModelsDynamic Sparse Attention

  18. vToken: Token-Level Virtualization for Reclaimable KV Caches

    Aug 13, 2026Yuanhang Gao, Xiangrui Yang, Yuanfeng Chen +4Key-Value CacheGraphics Processing Unit Memory

  19. SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

    Aug 13, 2026Qi Zhao, Qirui Li, Hanlin Tang +10Dynamic Sparse AttentionVideo Diffusion Models

  20. LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration

    Aug 12, 2026Enhuai Liu, Yunke Wang, Yutong Wang +2Video Diffusion ModelsDynamic Sparse Attention

  21. FLARE++: Low-rank attention with dynamic attention routing

    Aug 12, 2026Vedant Puri, Yongjie Jessica Zhang, Levent Burak KaraDynamic Sparse AttentionDifficulty-Adaptive Routing For Temporal Grounding

  22. Three Tokens Force Exponential Feature Rank in Nonnegative Kernel Attention

    Aug 11, 2026Vicente OpazoDynamic Sparse AttentionTop-K

  23. Sparse Attention to Emotion: Efficient Facial Emotion Recognition via Token Reduction

    Aug 9, 2026Aya Manel Zitouni, Aicha Zenakhri, Karim Haroun +1Facial Expression RecognitionDynamic Sparse Attention

  24. VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

    Aug 9, 2026Wenxu Jia, Dongjie Fu, Xize Cheng +6Audio TokensKey-Value Cache Compression

  25. Autonomy-of-Heads: Data-Free Sparse Attention from Frozen Query-Key Geometry

    Aug 7, 2026Yehan Yang, Junyuan Shang, Yang Li +3Dynamic Sparse AttentionLLM Inference Efficiency

  26. One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

    Aug 6, 2026Wang Chen, Yu Chen, Xiang Wang +3Matryoshka Representation LearningRelevance

  27. Training-Free Hashing-Based Attention via Binary Principal Components

    Aug 5, 2026Daohai Yu, Zhanpeng Zeng, Keyu Chen +6Dynamic Sparse AttentionEfficient Long-Context Inference

  28. Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms

    Aug 4, 2026Samuel Fernández-Menduiña, Amir Ziashahabi, Eduardo Pavez +2Key-Value Cache QuantizationCalibration-Free Low-Bit Vector Quantization

  29. SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference

    Aug 4, 2026Shanghao Liu, Renze Chen, Size Zheng +4Video Diffusion ModelsDynamic Sparse Attention

  30. TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

    Aug 4, 2026Wonpyo Park, Seung-won HwangKey-Value Cache CompressionKey-Value Cache Eviction

  31. SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels

    Aug 4, 2026Yongwan Jo, Jinyoung Park, Euihyun Lee +1Language Model Evasion AttacksSide-Channel Analysis

  32. When Should Graph Attention Be Sparse? Learning a Per-Edge Tsallis Index

    Aug 3, 2026Kleyton da Costa, Bernardo ModenesiMulti-View Gated Graph Attention NetworkText-Attributed Graph

  33. AnchorKV: Anchor-Residual KV Cache Compression

    Aug 3, 2026Malik Khalaf, Yara Shamshoum, Nitzan Hodos +2Key-Value Cache CompressionKey-Value Cache

  34. Token Radius Attention for Efficient Video Generation

    Aug 3, 2026Jiayu Chen, Zhikun Jiang, Maoliang Li +6Video Diffusion ModelsDynamic Sparse Attention

  35. Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model

    Aug 3, 2026Ronglong BaoDataset DistillationQwen3

  36. Energy-Efficient LLM Serving via Disaggregated Attention--FFN and Flexible Frequency Scaling

    Aug 3, 2026Cunchen Hu, Liangliang Xu, Tian Liu +9Energy Based ModelsArtificial Intelligence Infrastructure

  37. Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation

    Aug 3, 2026Xingyu Ren, Youran Sun, Chugang Yi +1Dynamic Sparse AttentionEfficient Long-Context Inference

  38. LongCat Sparse Attention: Taming the Lightning via Streaming-aware Hierarchical Cross-Layer Indexing

    Aug 3, 2026Wen Zan, Jiaqi Zhang, Jianchao Tan +11Dynamic Sparse AttentionEfficient Long-Context Inference

  39. Bole: Efficient Tree Speculation for Hybrid-Attention Language Models

    Aug 3, 2026Li Wang, Yi Su, Xiabao Wu +9Speculative DecodingLLM Inference Efficiency

  40. RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

    Aug 2, 2026Changwoo Baek, Seungjun Shin, Kyeongbo KongKey-Value Cache CompressionKey-Value Cache Eviction

  41. S^4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

    Aug 1, 2026Jialong Han, You Wu, Kewei TuKey-Value Cache CompressionKey-Value Cache

  42. EntropyMoE: Entropy-Aware Sparse Expert Routing for Tokenizer-Free LLMs

    Jul 31, 2026Bo Liu, Muxuab Yu, Yu Zhang +2Cold Mixture-Of-Expert ModelsMixture-Of-Expert Routing

  43. A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding

    Jul 30, 2026Yuesong Liu, Yuan Zeng, Min Lyu +3Speculative DecodingLLM Inference Efficiency

  44. Recall Before You Rank: Similarity-Guided Top-K Reuse for Efficient Long-Context Attention

    Jul 30, 2026Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5Dynamic Sparse AttentionTop-K

  45. CoSA: Accelerating Long-Context Inference via Proxy-Kernel Co-Designed Sparse Attention

    Jul 28, 2026Yufei Xue, Lin Niu, Hong Liu +6Dynamic Sparse AttentionEfficient Long-Context Inference

  46. PIVOT: Efficient Query-Group Indexing for Token-Level Sparse Attention

    Jul 27, 2026Hong Liu, Yuan Cheng, Lin Niu +5Dynamic Sparse AttentionIndex

  47. LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

    Jul 27, 2026Junsung HwangEfficient Long-Context InferenceKey-Value

  48. MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention

    Jul 27, 2026Jianlin Yu, Jing Lin, Linghui Kong +13Low-Bit QuantizationVideo Diffusion Models

  49. DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

    Jul 27, 2026Tan T. Nguyen, Quan V. DangKey-Value Cache CompressionLarge Language Model Compression

  50. Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification

    Jul 27, 2026Haopeng Li, Yitong Li, Junsong Chen +8Video Diffusion ModelsDynamic Sparse Attention

  51. A Coulomb Particle Model for Learning Kernel Attention in Transformers

    Jul 26, 2026Masoud Badiei Khuzani, Sharath Honnaiah, Atiq Islam +2Feature LearningTransformer Attention

  52. X-Stage: Modeling Post-Issue Backpressure in GPU Communication--Computation Fusion

    Jul 25, 2026Jianwen Xian, Zhiyuan Xu, Yuchen Li +9Tensor ProgramsGraphics Processing Unit Kernels

  53. HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding

    Jul 24, 2026Chao Fang, Jun Yin, Man Shi +1Key-Value CacheLarge Language Model Decoding

  54. RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

    Jul 24, 2026Anderson R. SantosDynamic Sparse AttentionLLM Inference Efficiency

  55. Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection

    Jul 23, 2026Debarshi Kundu, Swaroop Ghosh, Vasant HonavarDynamic Sparse AttentionEfficient Long-Context Inference

  56. Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

    Jul 23, 2026Yidu Wu, Xiang Wang, Kejie Zhao +3LLM Inference EfficiencyDynamic Sparse Attention

  57. C^2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference

    Jul 20, 2026Chuheng Du, Junyi Chen, Hanlin Tang +7Key-Value CacheEfficient Long-Context Inference

  58. Kernelized Linear Attention: Breaking the Capacity Wall with Symmetric Cones

    Jul 19, 2026Ayoub Ghriss, Sourav ChakrabortyDynamic Sparse AttentionAssociative Memory

  59. HyBDM: Multi-Scale Hybrid Experts for Time Series Forecasting with Bidirectional Dependency Modeling

    Jul 18, 2026Wenqiang Ma, Chen Cheng, Xue Cheng +1Time Series ForecastingTime Series

  60. FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation

    Jul 17, 2026Hao Liu, Chenghuan Huang, Ye Huang +7Dynamic Sparse AttentionVideo Diffusion Models