Long-Context Language Model Inference

Ask about this topic

Momentum

30 papers in the last four weeks, up 275% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 166

All topics
CardsList

  1. Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog

    Oct 8, 2026Hadeel Al-Negheimish, Jasna Ilieva, Yoon KimLLM ReasoningLong-Context Language Model Inference

    King Saud University · Massachusetts Institute of Technology

  2. Evaluating Trajectory Features for Routing Final-Layer Attention

    Oct 7, 2026Yupeng YaoLLM RoutingLLM Inference Acceleration

  3. SPIN: Shadow Predictive Indexer for Sparse Attention

    Oct 6, 2026Yao Fu, Jiahan Chang, Ritchie Zhao +11LLM Inference AccelerationSparse Attention

  4. Hybrid Latent Attention for Looped Language Models

    Oct 6, 2026Yuhan Chen, Siyuan Zhang, Nan Wang +2Long-Context Language Model InferenceLLM Inference Acceleration

    Virginia Tech · Independent Researcher

  5. ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents

    Oct 6, 2026Yupeng Su, Jiayi Tian, Zheng Zhang +1LLM Inference EfficiencyLong-Context Language Model Inference

    University of California, Santa Barbara · Intel

  6. OVAL: Output-Aware Local Page Bases for KV Cache Retrieval

    Oct 5, 2026Ashkan Shahbazi, Chayne Thrash, Soheil KolouriLong-Context Language Model InferenceLLM Inference Acceleration

    College of Connected Computing Vanderbilt University Nashville, TN, USA

  7. DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

    Oct 5, 2026Zhe Wang, Jiakai Li, Yujia Sun +2Long-Context Language Model InferenceKV-Cache Eviction

    University of Electronic Science and Technology of China, Chengdu, China · Ubiquitous Intelligence and Trusted Services Key Laboratory of Sichuan Province, Chengdu, China

  8. SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scaling

    Oct 4, 2026Jeonghoon Park, Seongwoon Jo, Jongwon Lee +1Long-Context Language Model InferenceSpeculative Decoding

    UNIST

  9. HHR: Hierarchical Hash Retrieval for Efficient LLM Generation

    Oct 1, 2026Lianjun Liu, Tiantian Zheng, You Huang +5Long-Context Language Model InferenceLLM Inference Acceleration

    Hainan University, Haikou, China · Xiamen University, Xiamen, China · Zhejiang Normal University, Jinhua, China

  10. Role-aware Heuristic Episodic Attention for Conversational LLMs

    Oct 1, 2026Wanyang Hong, Zhaoning Zhang, Yi Chen +5Memory-Augmented Language ModelsLong-Context Language Model Inference

    National Key Laboratory of Parallel and Distributed Computing, College of Computer Science and Technology, National University of Defense Technology, Changsha, China.

  11. CommunityKV: Efficient Long-Context Decoding via Graph Partitioning

    Sep 30, 2026Joe McKenna, Anastasios Alexandridis, Nathan Susanj +1Long-Context Language Model InferenceKV-Cache Management

    Amazon AGI

  12. Working Around the Compute Ceiling: Byte-Exact Memory in Galahad Makes LLM Reading a One-Time Cost LLM Reading a One-Time Cost

    Sep 30, 2026Sietse SchelpeLLM ServingLong-Context Language Model Inference

    Corbenic AI

  13. PatchKV: Weight-Space Compensation of KV Cache

    Sep 30, 2026Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi +2Long-Context Language Model InferenceLLM Inference Acceleration

    KAIST

  14. SparseEngine: Sparse-First Inference Engine

    Sep 30, 2026Jitai Hao, Quansheng Gu, Qiang Huang +1LLM InferenceLong-Context Language Model Inference

  15. LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning

    Sep 29, 2026Quang Hieu Pham, Thuy Duong Nguyen, Jocelyn Qiaochu Chen +1Long-Context Language Model EvaluationLong-Context Language Model Inference

    University of Alberta · Alberta Machine Intelligence Institute (Amii)

  16. LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

    Sep 29, 2026Hao-Yuan He, Peng-Fei Liu, Si Shen +1Long-Context Language Model InferenceBlockwise Diffusion Language Models

  17. Cartridges++: KV Cache Compression without Off-Context Derailment

    Sep 28, 2026Sonia Laguna, Joao Monteiro, Marco Cuturi +2Memory-Augmented Language ModelsLong-Context Language Model Inference

    Apple

  18. The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading

    Sep 28, 2026Muath Alyobi, Mohamed Eltahir, Almoayyad Abuljdail +3Early StoppingLong-Context Language Model Evaluation

    King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England

  19. PulseInfer: I/O-Centric Sparse KV Cache Offloading for Efficient Long-Context LLM Decoding

    Sep 28, 2026Qiuyang Zhang, Kai Zhou, Kai Lu +6KV-Cache OffloadingLong-Context Language Model Inference

    Huazhong University of Science and Technology Wuhan, China · UCloud Technology Co., Ltd. Shanghai, China

  20. Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference

    Sep 28, 2026Timothy DeLise, Seth CromelinLong-Context Language Model InferenceKV-Cache Reuse

  21. DISCO: Distributed Long Context Scaling with Grounding-Reasoning Disaggregation

    Sep 27, 2026Guanzheng Chen, Viet Dac Lai, Subhojyoti Mukherjee +5LLM GroundingLong-Context Language Model Inference

    National University of Singapore · Adobe Research

  22. Just Let Linear States Forget the Distant Past: Prefix Caching via Suffix Replay for Hybrid LLMs

    Sep 27, 2026Yirui Liu, Ruoling Qi, Xuaner Wu +6Long-Context Language Model InferenceLLM Inference Acceleration

    Institute of Artificial Intelligence, China Telecom (TeleAI) China · Shanghai Jiao Tong University · Individual Researcher +1

  23. GSM: Efficient Language Modeling with Shared Global State

    Sep 27, 2026Yunao Zheng, Bin Wen, Xiaojie Wang +12Long-Context Language Model InferenceLong-Context Language Modeling

    Beijing University of Posts and Telecommunications. · Kuaishou Technology.

  24. Shared Global KV with Layer-Specific Local History

    Sep 23, 2026Xinglang XianKV CachingLong-Context Language Model Inference

  25. Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models

    Sep 23, 2026Ke Wan, Chen ChenLong-Context Language Model InferenceRecurrent Transformers

    Department of Computer Science University of Virginia Charlottesville, VA, USA

  26. Distilling Sequential Computation in Transformer Language Models

    Sep 23, 2026Zixuan Lan, Jessica Yang, Yanhong Li +2LLM CompressionLong-Context Language Model Inference

    The University of Chicago · Toyota Technological Institute at Chicago · Independent Researcher +1

  27. CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference

    Sep 22, 2026Zhen Huang, Ruizhe Yao, Danyi Liu +8Long-Context Language Model InferenceKV-Cache Management

    Tsinghua University · Harbin Institute of Technology, Shenzhen · Northeastern University

  28. Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

    Sep 22, 2026Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia +2LLM Inference EfficiencyLong-Context Language Model Inference

    Lucy Family Institute for Data & Society, University of Notre Dame, IN, USA · The University of Aizu, Aizuwakamatsu, Japan · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, IN, USA

  29. ARM: Attention with Routed-Memory for Learnable Sparse Control

    Sep 21, 2026Qiuhao Zeng, Jerry Huang, Peng Lu +7Memory-Augmented Language ModelsLong-Context Language Model Inference

    University of Toronto, Canada · Vector Institute, Canada · RIKEN AIP, Japan +3

  30. TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching

    Sep 18, 2026Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu +4On-Device Language Model InferenceLong-Context Language Model Inference

    University of Georgia Athens, GA, USA · Peking University Beijing, China · Western Digital Research San Jose, CA, USA