Long-Context Language Model Inference

Ask about this topic

Momentum

30 papers in the last four weeks, up 275% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 166

All topics
CardsList

  1. KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

    Jun 15, 2026Mufei Li, Shikun Liu, Dongqi Fu +5KV CachingLong-Context Language Model Inference

    Georgia Institute of Technology · Meta

  2. Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

    Jun 12, 2026NVIDIA, :, Aaron Blakeman +571Mixture-of-Experts Language ModelsLong-Context Language Model Inference

    Allan

  3. Doc-to-Atom: Learning to Compile and Compose Memory Atoms

    Jun 10, 2026Xingjian Diao, Wenbo Li, Yashas Malur Saidutta +3Memory-Augmented Language ModelsLong-Context Language Model Inference

    AI Center-Mountain View, Samsung Electronics · Dartmouth College

  4. Express Language Modeling

    Jun 9, 2026Albert Gong, Annabelle Michael Carrell, Raaz Dwivedi +1Long-Context Language Model InferenceAttention Mechanisms

    Cornell Tech · University of Cambridge · Microsoft Research

  5. Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models

    Jun 9, 2026Jing Xiong, Qi Han, Shansan Gong +5Long-Context Language Model InferenceDiffusion Language Model Inference

    The University of Hong Kong · University of Michigan, Ann Arbor · LMSYS Org

  6. Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

    Jun 9, 2026Muhammad AhmedMemory-Augmented Language ModelsLong-Context Language Model Inference

    Independent Researcher

  7. End-to-End Context Compression at Scale

    Jun 8, 2026Ang Li, Sean McLeish, Haozhe Chen +12LLM CompressionLong-Context Language Model Inference

    New York University · University of Maryland · Princeton University +3

  8. From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

    Jun 8, 2026Zhanchao Xu, Haoyang Li, Qingfa Xiao +4Long-Context Language Model InferenceLLM Inference Acceleration

    Department of Computing, PolyU · DSA, HKUST(GZ) · CSE, HKUST

  9. FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

    Jun 8, 2026Yan Wang, Qifan Zhang, Jiachen Yu +12KV CachingLong-Context Language Model Inference

    Independent Researchers · Tencent · The Hong Kong University of Science and Technology (Guangzhou) +1

  10. Still: Amortized KV Cache Compaction in a Single Forward Pass

    Jun 5, 2026Charles O'Neill, Alex Sandomirsky, Harry Partridge +2KV CachingLong-Context Language Model Inference

    Baseten San Francisco · Baseten London

  11. You Only Index Once: Cross-Layer Sparse Attention with Shared Routing

    Jun 4, 2026Yutao Sun, Yanqi Zhang, Li Dong +2Long-Context Language Model InferenceKV-Cache Management

    Microsoft Research · Tsinghua University

  12. Dense Contexts Are Hard Contexts: Lexical Density Limits Effective Context in LLMs

    Jun 4, 2026Giovanni Dettori, Matteo Boffa, Danilo Giordano +2Long-Context RetrievalLLM Evaluation

    Department of Computer Science Politecnico di Torino Torino, IT 10129 · Department of Computer Science University of Turin Torino, IT 10124

  13. SparDA: Sparse Decoupled Attention for Efficient Long-Context LLM Inference

    Jun 3, 2026Yaosheng Fu, Guangxuan Xiao, Xin Dong +2KV-Cache OffloadingLong-Context Language Model Inference

    1NVIDIA · 2Thinking Machines Lab · 3ByteDance Seed +1

  14. LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

    Jun 3, 2026Haocheng Xia, Mihir Pamnani, Hanxi Fang +2Retrieval-Augmented GenerationKV Caching

    Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign · Nexla · Amazon +1

  15. MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference

    Jun 1, 2026Yu Li, Binxu Li, Tian LanKV CachingLong-Context Language Model Inference

    George Washington University · Princeton University

  16. Soft-NBCE: Entropy-Weighted Chunk Fusion for Long-Context

    May 31, 2026Shihao Ji, Mingyu Li, Zihui SongLong-Context Language Model InferenceLLM Inference Acceleration

    Chunjiang Intelligence · Beijing Normal University

  17. WaveFilter: Enhancing the Long-Context Capability of Diffusion LLMs via Wavelet-Guided KV Cache Filtering

    May 30, 2026Jinnan Yang, Yan Wang, Zhen Bi +5Long-Context Language Model InferenceDiffusion Language Model Inference

    Nanjing University of Science and Technology · Alibaba Group · Huzhou Normal University +1

  18. BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding

    May 29, 2026Liang He, Jingbo Wen, Qishi Zhan +4KV CachingLong-Context Language Model Inference

    Shanghai Institute of Optics and Fine Mechanics · The University of Sydney · Marquette University +4

  19. Augmenting Attention with Exponentially Decaying Memory Improves Query-Aware KV Sparsity

    May 27, 2026Xiuying Wei, Caglar GulcehreMemory-Augmented Language ModelsSelf-Attention

    EPFL, CLAIRE Lausanne, Switzerland

  20. UNIQUE: Universal Top-k Sparse Attention for Training-free Inference and Sparsity-aware Training

    May 26, 2026Keqi Deng, Shaoshi Ling, Ruchao Fan +1Self-AttentionLong-Context Language Model Inference

    Microsoft, USA

  21. Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference

    May 25, 2026Sangyun Lee, Sean McLeish, Tom Goldstein +1Memory-Augmented Language ModelsLong-Context Language Model Inference

    Carnegie Mellon University · University of Maryland

  22. IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

    May 25, 2026Xintong Yang, Hao Gu, Binxing Xu +6Memory-Augmented Language ModelsSelf-Attention

    The Hong Kong University of Science and Technology · Zhejiang University

  23. H2^{2}MT: Semantic Hierarchy-Aware Hierarchical Memory Transformer

    May 24, 2026Maryam Haghifam, Zifan He, Jason Cong +1Memory-Augmented Language ModelsLong-Context Language Model Inference

    University of California, Los Angeles USA

  24. CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

    May 24, 2026Yubo Li, Yidi MiaoKV CachingLong-Context Language Model Inference

    Carnegie Mellon University

  25. Adaptive Mass-Segmented KV Compression for Long-Context Reasoning

    May 22, 2026Junzhe Yang, Xiaoyu ShenKV CachingLong-Context Language Model Inference

    Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo

  26. ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention

    May 21, 2026Joe SharrattMixed-Precision QuantizationSelf-Attention

  27. EntmaxKV: Support-Aware Decoding for Entmax Attention

    May 20, 2026Gonçalo Duarte, Miguel Couceiro, Marcos V. TrevisoSelf-AttentionKV Caching

    Instituto Superior Técnico, Universidade de Lisboa. · ELLIS Unit Lisbon. · INESC-ID. +1

  28. Mix-Quant: Quantized Prefilling, Precise Decoding for Agentic LLMs

    May 19, 2026Haiquan Lu, Zigeng Chen, Gongfan Fang +2LLM QuantizationLong-Context Language Model Inference

    National University of Singapore

  29. Context Memorization for Efficient Long Context Generation

    May 18, 2026Yasuyuki Okoshi, Hao Mark Chen, Guanxi Lu +3Memory-Augmented Language ModelsLong-Context Language Model Inference

    Institute of Science Tokyo, Japan · Imperial College London, UK

  30. VeriCache: Turning Lossy KV Cache into Lossless LLM Inference

    May 17, 2026Jiayi Yao, Samuel Shen, Kuntai Du +7KV-Cache OffloadingKV Caching

    University of Chicago · Tensormesh Inc. · Samsung Semiconductor +1