Long-Context Reasoning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

PatchKV: Weight-Space Compensation of KV Cache
Sep 30, 2026Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi +2Key-Value Cache Compression Key-Value Cache
KAIST
Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting
Sep 30, 2026Liang Twist Shan, Tianyu Hu, Hao Yan +1LLM Reasoning Strategies Long-Context Reasoning
Department of Statistics, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison
KV-Kaizen: Learning Context-Adaptive Cache Compression Choices
Sep 29, 2026Joao Monteiro, Louis Béthune, Anastasiia Filippova +3Key-Value Cache Compression Large Language Model Compression
Apple
LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
Sep 29, 2026Hao-Yuan He, Peng-Fei Liu, Si Shen +1Speculative Decoding Block-Diffusion Drafter
REALHOP: Rethinking Multi-Hop Reasoning Evaluation via Behavioral Auditing
Sep 29, 2026Jiawen Tao, Xiaokun Yuan, Yaoming Li +4Multi-Hop Reasoning Reasoning Benchmark
Hunyuan Team, Tencent · Peking University
CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory
Sep 29, 2026Jingguang Li, Yebo Wu, Zuyi Guo +7Long-Context Reasoning LLM Reasoning Strategies
Korea Advanced Institute of Science & Technology · University of Macau · Massachusetts Institute of Technology
The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading
Sep 28, 2026Muath Alyobi, Mohamed Eltahir, Almoayyad Abuljdail +3Efficient Long-Context Inference Long-Context Reasoning
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England
Learning to Learn from Context: Synthetic Training from Perturbed Public Documents
Sep 27, 2026Haoyi Wu, Yang Xiao, Yusong Sun +4LLM Reasoning Strategies Contextual Reasoning
AllSpark Team
When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA
Sep 22, 2026Yingrui Li, Han ChenLong-Context Reasoning Contextual Reasoning
Independent Researcher, United States
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
Sep 17, 2026DeepSeek-AI, :, Anyi Xu +590Key-Value Cache Compression Key-Value Cache
Long-Context Demonstration Selection Using State Space Models
Sep 15, 2026Ziniu Zhang, Zhenshuo Zhang, Ruoxuan Xiong +2Efficient Long-Context Inference In-Context Learning
Northeastern University · Emory University
ConvMem: Convolutional Memory for Long-Context Reasoning
Sep 9, 2026Hongming Zhang, Zhaozhen Gu, Fengshuo Bai +6Efficient Long-Context Inference Large Language Model Memory
Institute of Automation, Chinese Academy of Sciences
Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?
Sep 9, 2026Fumihiko Tachibana, Daisuke Miyashita, Jun DeguchiKey-Value Cache Cache
Kioxia Corporation
Positional task conditioning for scalable defect detection across product families in large product catalogs
Sep 9, 2026Soham Satyadharma, Gabriel Roccabruna, Suleiman A. KhanDefect Detection Large Language Model Pipelines
Amazon Catalog AI
Motif 3: Technical Report
Aug 10, 2026Junghwan Lim, Joon Son Chung, Sungmin Lee +24Mixture-Of-Experts Large Language Models Hybrid Mamba-Transformer Model
Motif Technologies
Chained Recursive Language Models for Multi-Iteration Reasoning
Aug 5, 2026Purbesh Mitra, Sennur UlukusLLM Reasoning Strategies Long-Context Reasoning
University of Maryland
K-EXAONE 2.0 Technical Report
Aug 5, 2026Eunbi Choi, Kibong Choi, Sehyun Chun +74Multilingual Language Models Technical Report
PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
Aug 4, 2026Dawei Liu, Haixu Song, Shuang Cheng +9Long-Context Reasoning Large Language Model Memory
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University +3
REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning
Jul 23, 2026Zhensheng Jin, Xin Dai, Zhenghao Liu +5LLM Reasoning Strategies Long-Context Reasoning
Northeastern University · Shenyang, China · Tsinghua University +1
Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
Jul 21, 2026Lizhe Fang, Weizhou Shen, Tianyi Tang +1Long-Context Reasoning LLM Reasoning Strategies
Peking University · Qwen Team of Alibaba Group · Alibaba Group ATH
SynthDocBench: Controlled Benchmark for Long-Context Visual Document Understanding
Jul 11, 2026Abhigya Verma, Khyati Mahajan, Amit Kumar Saha +4Visual Document Retrieval Long-Context Reasoning
ServiceNow AI · Mila · Université de Montréal
WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning
Jul 10, 2026Zixin Chen, Peng Liu, Haobo Li +7Question-Answering Benchmarks Reasoning Traces
Hong Kong University of Science and Technology · Qwen Team, Alibaba Group · *Work done during internship at Qwen Team
Understanding Axes of Difficulty For Long Context Tasks Via PredicateLongBench
Jul 9, 2026Siddhartha Jain, Ameya VelingkerLong-Context Reasoning Question-Answering Benchmarks
NVIDIA
FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference
Jul 7, 2026Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo +4Key-Value Cache Compression Depthweave-Kv
Instituto de Investigación en Visión Artificial
LongCrafter: Towards Diverse Long-Context Understanding via Evidence-Graph-Guided Instruction Synthesis
Jul 7, 2026Chenhao Yuan, Yinhao Xu, Shuwen Xu +8Long-Context Reasoning Instruction-Tuned Models
University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Jul 2, 2026Yanjun Zhao, Ruizhong Qiu, Tianxin Wei +6Long-Context Reasoning LLM Reasoning Strategies
University of Illinois Urbana-Champaign
Evidence-State Rewards for Long-Context Reasoning
Jul 2, 2026Ya Gao, Pekka MarttinenLong-Context Reasoning LLM Reasoning Strategies
1Aalto University
Morphing into Hybrid Attention Models
Jun 29, 2026Disen Lan, Jianbin Zheng, Yuxi Ren +5Hybrid Transformer Transformer Attention
1Fudan University · 2ByteDance Seed · Work done at ByteDance Seed +1
Randomized YaRN Improves Length Generalization for Long-Context Reasoning
Jun 22, 2026Manas Mehta, Fangcong Yin, Greg DurrettEfficient Long-Context Inference Long-Context Reasoning
New York University
EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory
Jun 19, 2026Chang Nie, Chaoyou Fu, Junlan Feng +1Efficient Long-Context Inference Text Embeddings
Nanjing University
Storyline Trees: Hierarchical Representations for Long-Form Narratives
Jun 18, 2026Litu Ou, Mirella LapataNarratives Long-Context Reasoning
School of Informatics University of Edinburgh
The Token Tax of Epistemic Accuracy: Comparing RAG and Long-Context Architectures for Document-Grounded Generative AI Applications
Jun 18, 2026Austin Hamilton, Ryan Singh, Michael Wise +6Long-Context Reasoning Efficient Long-Context Inference
Miami University · University at Buffalo
Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
Jun 17, 2026Xiaoyue Xu, Sikui Zhang, Xiaorong Wang +2Large Language Model Reinforcement Learning Long-Context Reasoning
OpenBMB · Tsinghua University
Recursive Agent Harnesses
Jun 11, 2026Elias Lumer, Sahil Sen, Kevin Paul +1Agent Harness Recursive Models
PricewaterhouseCoopers, U.S.
Multi-Turn Reasoning When Context Arrives in Pieces: Scalable Sharding and Memory-Augmented RL
Jun 11, 2026Shu Tong Luo, Wenqin Liu, Rui Liu +2Long-Context Reasoning Shard
The University of Melbourne · The University of Tokyo
EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering
Jun 5, 2026Xiaopeng Yuan, Zebin Wang, Suwen Wang +3Efficient Long-Context Inference Question-Answering Benchmarks
University of Illinois Urbana-Champaign · 2Harvard University · 3Brion, ASML US LP +1
HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning
Jun 3, 2026Zeju Li, Ziyang Zheng, Yizhou Zhou +1Long-Context Reasoning Contextual Reasoning
The Chinese University of Hong Kong · Independent Researcher
InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain
Jun 2, 2026Tiancheng Han, Yong Li, Wuzhou Yu +2Long-Term Agent Memory Long-Context Reasoning
1Tongji University · 2Shanghai Innovation Institute · 3Shanghai AI Laboratory
LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning
May 31, 2026Mengmeng Ji, Ravi Shanker Raju, Jonathan Lingjie Li +1Efficient Long-Context Inference Long-Context Reasoning
SambaNova Systems, Inc. San Jose, CA, USA
LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards
May 29, 2026Nianyi Lin, Jiajie Zhang, Lei Hou +1LLM Reasoning Strategies Long-Context Reasoning
Tsinghua University
CoMem: Context Management with A Decoupled Long-Context Model
May 29, 2026Yuwei Zhang, Chengyu Dong, Shuowei Jin +11Efficient Long-Context Inference Long-Context Reasoning
Halıcıoğlu Data Science Institute, University of California, San Diego · Amazon
Give it Space! Explicit Disentangling of Positional and Semantic Representations in Encoders
May 28, 2026Pierre-Antoine Lequeu, Camille Barboule, Benjamin PiwowarskiPositional Encoding Transformer Encoder
Sorbonne Université, CNRS, ISIR, Paris, France · Orange Innovation
Periodic RoPE for Infinite Context LLMs
May 27, 2026Simin HuoEfficient Long-Context Inference Long-Context Reasoning
Shanghai Jiao Tong University
Adaptive Mass-Segmented KV Compression for Long-Context Reasoning
May 22, 2026Junzhe Yang, Xiaoyu ShenKey-Value Cache Compression Key-Value Cache
Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo
ACC: Compiling Agent Trajectories for Long-Context Training
May 21, 2026Qisheng Su, Zhen Fang, Shiting Huang +8Efficient Long-Context Inference Long-Context Reasoning
MoE Key Lab of BIPC, University of Science and Technology of China · Shanghai Innovation Institute · Shanghai AI Laboratory
The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning
May 11, 2026Muhan Gao, Zih-Ching Chen, Kuan-Hao HuangLong-Context Reasoning Distractor
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA · NVIDIA AI Technology Center, NVIDIA Corporation, Santa Clara, CA, USA
Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition
May 11, 2026Haoren Xu, Guanhua FangLinear Attention In-Context Learning
Fudan University
MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
May 11, 2026Baibei Ji, Xiaoyang Weng, Juntao Li +3Long-Context Reasoning Precision Recall
Soochow University · Peking University
Route Before Retrieve: Activating Latent Routing Abilities of LLMs for RAG vs. Long-Context Selection
May 11, 2026Yiwen Chen, Kuan Li, Fuzhen Zhuang +6Large Language Model Routing Long-Context Reasoning
Beihang University · HKUST · Alibaba Group +1
Scaling Limits of Long-Context Transformers
May 8, 2026Giuseppe Bruno, Shi Chen, Zhengjiang Lin +2Linear Attention Efficient Long-Context Inference
MiA-Signature: Approximating Global Activation for Long-Context Understanding
May 7, 2026Yuqing Li, Jiangnan Li, Mo Yu +3Signatures Long-Context Reasoning
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Pattern Recognition Center, WeChat AI, Tencent +1
SCOUT: Active Information Foraging for Long-Text Understanding with Decoupled Epistemic States
May 6, 2026Zhenliang Zhang, Wenqing Wang, Yong Hu +4Long-Context Reasoning Epistemic State Replication
Peking University · WeChat, Tencent · Work done during an internship at WeChat, Tencent
Retrieval and Multi-Hop Reasoning in 1M-Token Context Windows: Evaluating LLMs on Classical Chinese Text
May 4, 2026Eric H. C. ChowLong-Context Reasoning Multi-Hop Reasoning
School of Humanities, The University of Hong Kong
On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility
May 2, 2026Zhitao He, Haolin Yang, Rui Min +2Long-Form Generation Large Language Model Generation
Hong Kong University of Science and Technology
NorBERTo: A ModernBERT Model Trained for Portuguese with 331 Billion Tokens Corpus
Apr 30, 2026Enzo S. N. Silva, Pablo B. Costa, Raphael C. Vlasman +8Portuguese Bert-Based Models
Itaú Unibanco, São Paulo, SP, Brasil · ICTi, São Paulo, SP, Brasil
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Apr 26, 2026DeepSeek-AI, Anyi Xu, Bangcai Lin +316Efficient Long-Context Inference Deepseek
SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference
Apr 24, 2026Yuqi Pan, Jinghao Zhuang, Yupeng Feng +16Dynamic Sparse Attention Efficient Long-Context Inference
Institute of Automation, Chinese Academy of Sciences · The Hong Kong Polytechnic University · Beijing Key Laboratory of Brain-Inspired General Intelligence Large Model +1
Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
Apr 22, 2026Neehal Tumma, Noel Loo, Daniela RusKimi Delta Attention Recurrent Model
1MIT · 2Liquid AI
CRISP: Compressing Redundancy in Chain-of-Thought via Intrinsic Saliency Pruning
Apr 19, 2026Yangsong Lan, Hongliang Dai, Piji LiReasoning Chain Chain-of-Thought Reasoning
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China.
LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
Apr 16, 2026Bowen Ping, Zijun Chen, Tingfeng Hui +4Efficient Long-Context Inference Large Language Model Reinforcement Learning
1Peking University · 2Shanghai Jiao Tong University · 3Beijing University of Posts and Telecommunications

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

PatchKV: Weight-Space Compensation of KV Cache

Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi, Donggyun Kim+1

KAIST

Key-Value Cache Compression Key-Value Cache Cache Large Language Model Compression Efficient Long-Context Inference Long-Context Reasoning+1

Open paper AskPDF arXiv

Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compression methods reduce the cache through token eviction or approximation, but degrade sharply at aggressive compression budgets. We propose PatchKV, a training-free framework that compensates KV cache compression methods by carrying part of the context in the model's weights. PatchKV pairs an off-the-shelf compressed KV cache with a context-specific weight patch, which is computed once at context-loading time and served for downstream queries for the context. The weight patch is derived in closed form via ridge regression, by aligning the block-wise activations of context-derived reference query tokens under the full cache and the compressed cache. Once merged into the model, the patch leaves the forward graph and per-query inference cost unchanged in the single-context, multi-query setting. Across long-context QA (SCBench with up to 170K tokens, SQuAD, NIAH) and math (GSM8K) benchmarks on three model architectures, PatchKV consistently improves cache compression methods, suggesting an alternative direction to compensate them at aggressive budgets.

Momentum

Related topics

Latest papers 70