Efficient Long-Context Inference

Ask about this topic

Recent momentum

+10%

23 papers in the last 28 days · 0.4% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Efficient Long-Context Inference

Neighbouring directions

Context LengthLongbench

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

6 new papers

A weekly snapshot of new work published in Efficient Long-Context Inference.

Period ending 2026-09-14

10 new papers

A weekly snapshot of new work published in Efficient Long-Context Inference.

Period ending 2026-09-07

8 new papers

A weekly snapshot of new work published in Efficient Long-Context Inference.

229 papers

Latest in Efficient Long-Context Inference

Browse all topics →
CardsList

  1. Positional LSH: Binary Block Matrix Approximation for Attention with Linear Biases

    May 10, 2026Daniel Wolfson, Tal WagnerEfficient Long-Context InferenceLocality-Sensitive Hashing

  2. Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven

    May 10, 2026Jiwei Tang, Zhijing Huang, Xinyu Zhang +5Large Language Model CompressionEfficient Long-Context Inference

  3. ProxyKV: Cross-Model Proxy Pruning for Efficient Long-Context LLM Inference

    May 9, 2026Junjie Li, Jiong Lou, Jie LiLLM Inference EfficiencyEfficient Long-Context Inference

  4. ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing

    May 9, 2026Yongqi An, Chang Lu, Kuan Zhu +5Key-Value Cache EvictionEfficient Long-Context Inference

  5. An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference

    May 8, 2026Feiyu Yao, Zhixiong Niu, Xiaqing Li +3Dynamic Sparse AttentionEfficient Long-Context Inference

  6. MISA: Mixture of Indexer Sparse Attention for Long-Context LLM Inference

    May 8, 2026Ruijie Zhou, Fanxu Meng, Yufei Xu +4Dynamic Sparse AttentionDeepseek

  7. Reformulating KV Cache Eviction Problem for Long-Context LLM Inference

    May 8, 2026Tho Mai, Joo-Young KimKey-Value Cache EvictionKey-Value Cache

  8. Self-Consolidating Language Models: Continual Knowledge Incorporation from Context

    May 8, 2026Zekun Wang, Anant Gupta, Zihan Dong +1Efficient Long-Context InferenceConsolidation

  9. MiA-Signature: Approximating Global Activation for Long-Context Understanding

    May 7, 2026Yuqing Li, Jiangnan Li, Mo Yu +3SignaturesEfficient Long-Context Inference

  10. Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibility

    May 7, 2026Jungsuk Oh, Hyeseo Jeon, Hyunjune Ji +2Efficient Long-Context InferenceLLM Inference Efficiency

  11. Training Transformers for KV Cache Compressibility

    May 7, 2026Yoav Gelberg, Yam Eitan, Michael Bronstein +2Key-Value Cache CompressionLarge Language Model Compression

  12. On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility

    May 2, 2026Zhitao He, Haolin Yang, Rui Min +2Long-Form GenerationEfficient Long-Context Inference

  13. Context Recycling for Long-Horizon LLM Inference

    May 1, 2026Derek ThomasEfficient Long-Context InferenceConversational Benchmarks

  14. AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism

    Apr 29, 2026Ahan Gupta, Zhihao Wang, Neel Dani +3Efficient Long-Context InferenceParallel

  15. Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective

    Apr 28, 2026Jiaming Yang, Chenwei Tang, Liangli Zhen +1Key-Value Cache EvictionEviction

  16. Long-Context Aware Upcycling: A New Frontier for Hybrid LLM Scaling

    Apr 27, 2026Parsa Ashrafi Fashi, Utkarsh Saxena, Mehdi Rezagholizadeh +7Efficient Long-Context InferenceNeural Scaling Laws

  17. Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

    Apr 27, 2026Wang Fan, Wei Cao, Xi Zha +5Efficient Long-Context InferenceDynamic Sparse Attention

  18. Kwai Summary Attention Technical Report

    Apr 27, 2026Chenglong Chu, Guorui Zhou, Guowang Zhang +35Efficient Long-Context InferenceQuadratic Attention

  19. DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

    Apr 26, 2026DeepSeek-AI, Anyi Xu, Bangcai Lin +316Efficient Long-Context InferenceDeepseek

  20. From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors

    Apr 25, 2026Yitian Zhou, Chaoning Zhang, Jiaquan Zhang +6Large Language Model CompressionEfficient Long-Context Inference

  21. SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

    Apr 24, 2026Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16Dynamic Sparse AttentionEfficient Long-Context Inference

  22. Simplified Sparse Attention via Gist Tokens

    Apr 22, 2026Yuzhen Mao, Michael Y. Li, Emily B. FoxDynamic Sparse AttentionEfficient Long-Context Inference

  23. Absorber LLM: Harnessing Causal Synchronization for Test-Time Training

    Apr 22, 2026Zhixin Zhang, Shabo Zhang, Chengcan Wu +2Efficient Long-Context InferenceLarge Language

  24. 'The Order in the Horse's Heart': A Case Study in LLM-Assisted Stylometry for the Discovery of Biblical Allusion in Modern Literary Fiction

    Apr 21, 2026Ewan CameronLiteratureStylometric

  25. FG2^2-GDN: Enhancing Long-Context Gated Delta Networks with Doubly Fine-Grained Control

    Apr 21, 2026Pingwei Sun, Yuxuan Hu, Jianchao Tan +6Global Attention MechanismEfficient Long-Context Inference

  26. Sessa: Selective State Space Attention

    Apr 20, 2026Liubomyr HorbatkoEfficient Long-Context InferenceSequence Modeling

  27. Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling

    Apr 20, 2026Yujie Chen, Tailai Chen, Yifeng Gao +4PrefillEfficient Long-Context Inference

  28. OPSDL: On-Policy Self-Distillation for Long-Context Language Models

    Apr 19, 2026Xinsen Zhang, Zhenkai Ding, Tianjun Pan +4Efficient Long-Context InferenceUnsupervised On-Policy Self-Distillation

  29. Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation

    Apr 19, 2026Zhijiang Tang, Jiaxin Qi, Bing Zhao +1LongvideobenchLarge-Scale Video Generation Models

  30. SinkRouter: Sink-Aware Routing for Efficient Long-Context Decoding in Large Language and Multimodal Models

    Apr 18, 2026Junnan Liu, Xinyan Liu, Peifeng Gao +4Efficient Long-Context InferenceLarge Language Model Decoder

  31. AdaSplash-2: Faster Differentiable Sparse Attention

    Apr 16, 2026Nuno Gonçalves, Hugo Pitorro, Vlad Niculae +4Dynamic Sparse AttentionEfficient Long-Context Inference

  32. LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

    Apr 16, 2026Bowen Ping, Zijun Chen, Tingfeng Hui +4Efficient Long-Context InferenceContext Length

  33. KV Cache Offloading for Context-Intensive Tasks

    Apr 9, 2026Andrey Bocharnikov, Ivan Ermakov, Denis Kuznedelev +2Kv-Cache BlocksEfficient Long-Context Inference

  34. HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention

    Mar 30, 2026Yufei Xu, Fanxu Meng, Fan Jiang +11Dynamic Sparse AttentionEfficient Long-Context Inference

  35. StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigation

    Mar 24, 2026Zhiyuan Chen, Yuxuan Zhong, Fan Wang +4Persistent StateLong-Term Memory

  36. Learning When to Attend: Conditional Memory Access for Long-Context LLMs

    Mar 18, 2026Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato +4Efficient Long-Context InferenceQuadratic Attention

  37. How to Train Your Long-Context Visual Document Model

    Feb 16, 2026Austin VeselkaEfficient Long-Context InferenceLarge Vision Language Models

  38. FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion

    Feb 5, 2026Zhuokun Chen, Jianfei Cai, Bohan ZhuangDynamic Sparse AttentionDiffusion Language Models

  39. Threshold Differential Attention for Sink-Free, Ultra-Sparse, and Non-Dispersive Language Modeling

    Jan 17, 2026Xingyue Huang, Xueying Ding, Mingxuan Ju +3Dynamic Sparse AttentionGlobal Attention Mechanism

  40. Overcoming State Inertia: Minimally Invasive Temporal Alignment for Evolving Contexts

    Dec 3, 2025Yijun LiaoEfficient Long-Context InferenceMemory-Augmented Large Language Models

  41. Concertina: Data-Centric Adaptive Pipeline Parallelism for Efficient Heterogeneous Long-Context LLM Training

    Sep 25, 2025Shiju Wang, Yujie Wang, Fangcheng Fu +6Efficient Long-Context InferenceParallel

  42. CompLLM: Compression for Long Context Q&A

    Sep 23, 2025Gabriele Berton, Jayakrishnan Unnikrishnan, Son Tran +1Efficient Long-Context InferenceLarge Language Model Compression

  43. Multipole Semantic Attention: A Fast Approximation of Softmax Attention for Pretraining

    Sep 12, 2025Rupert Mitchell, Kristian KerstingSoftmax AttentionModel Pretraining

  44. PERK: Long-Context Reasoning as Test-Time Learning

    Jul 8, 2025Zeming Chen, Angelika Romanou, Gail Weiss +1Efficient Long-Context InferenceLarge Language Model Reasoning

  45. LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers

    Feb 20, 2025Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4Efficient Long-Context InferenceToken-Level Uncertainty

  46. LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning

    Feb 20, 2025Yansheng Mao, Yufei Xu, Jiaqi Li +5Efficient Long-Context InferenceContext Length

  47. A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation

    Date pendingMiguel Moura Ramos, Duarte M. Alves, André F. T. MartinsEfficient Long-Context InferenceToken-Level Supervision

  48. AdaRoPE: Not All Attention Heads Should Rotate and Scale Equally

    Date pendingShaowen Wang, Yuke Zheng, Tansheng Zhu +4Rotary Position EmbeddingEfficient Long-Context Inference

  49. SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering

    Date pendingMaryam Haghifam, Jason Cong, Yizhou SunEfficient Long-Context InferenceContext Length