Long-Context Reasoning in Language Models

Follow this topic Ask about this topic

Latest papers

40 papers10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning
Oct 15, 2026Shiwen Qin, Cameron Barker, Henry Gouk +4Long-Context Reasoning in Language Models Long-Context Language Model Evaluation
University of Edinburgh, Edinburgh, United Kingdom
Probing for Long-Horizon Deductive Reasoning Capabilities in Language Models with Prolog
Oct 8, 2026Hadeel Al-Negheimish, Jasna Ilieva, Yoon KimLLM Reasoning Long-Horizon LLM Reasoning
King Saud University · Massachusetts Institute of Technology
UNREAL: Unifying Retrieval and Long-Context with a Single Model
Oct 6, 2026Edan Kinderman, Elad Hoffer, Yochai Blau +4Retrieval-Augmented Generation Knowledge Augmentation for Language Models
NVIDIA · Technion
Periscope: Extending Frozen Language Models Beyond Their Context Window
Oct 2, 2026Mohamed Eltahir, Anas Obayd, Raed Rashid +5Long-Context Language Model Inference Long-Context Reasoning in Language Models
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England
Trained Agentic Context Management
Oct 1, 2026Bryce SandlundLong-Context Language Modeling Long-Context Reasoning in Language Models
LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillation
Sep 30, 2026Xiaoqiang Wang, Suyuchen Wang, Bang LiuLong-Context Reasoning in Language Models Efficient Language Model Reasoning
Université de Montréal · Mila – Quebec AI Institute
Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting
Sep 30, 2026Liang Twist Shan, Tianyu Hu, Hao Yan +1Numerical Reasoning in Language Models Long-Context Retrieval
Department of Statistics, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison
StateTree: Enhancing Long-Term Dialogue Reasoning via Reinforcement Learning
Sep 30, 2026Naen Xu, Wanqing Cui, Yibo Hu +5Temporal Reasoning in Language Models RL for Language Model Reasoning
Zhejiang University · Taobao & Tmall Group of Alibaba
LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning
Sep 29, 2026Quang Hieu Pham, Thuy Duong Nguyen, Jocelyn Qiaochu Chen +1Long-Context Language Model Evaluation Long-Context Language Model Inference
University of Alberta · Alberta Machine Intelligence Institute (Amii)
CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory
Sep 29, 2026Jingguang Li, Yebo Wu, Zuyi Guo +7Long-Context Reasoning in Language Models Memory-Augmented Language Models
Korea Advanced Institute of Science & Technology · University of Macau · Massachusetts Institute of Technology
Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
Sep 25, 2026Zhaoyuan Xia, Qinghongbing Xie, Yung Xiang Hue +7Long-Context Reasoning in Language Models Long-Context Modeling
Peking University · Baidu Inc · Baidu Inc. +1
ConvMem: Convolutional Memory for Long-Context Reasoning
Sep 9, 2026Hongming Zhang, Zhaozhen Gu, Fengshuo Bai +6Long-Context Reasoning in Language Models Memory-Augmented Language Models
Institute of Automation, Chinese Academy of Sciences
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Sep 4, 2026Janghyeon Kim, Minsoo Kim, Kyuhong Shim +1Long-Context Reasoning in Language Models KV-Cache Compression
Hanyang University · Sungkyunkwan University
Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
Aug 18, 2026Md. Faiyaz Abdullah SayeediLong-Context Reasoning in Language Models Hallucination in RAG
BRAC University, Bangladesh
SimpleOPD: Simple Tokenizer-Agnostic On-Policy Distillation for Long-Context Reasoning
Aug 14, 2026Haonan He, Haodi Lei, Yun Luo +13Long-Context Reasoning in Language Models On-Policy Distillation
SU-01 Team, Shanghai Artificial Intelligence Laboratory
Chained Recursive Language Models for Multi-Iteration Reasoning
Aug 5, 2026Purbesh Mitra, Sennur UlukusLong-Context Language Model Inference Language Modeling
University of Maryland
INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models
Jul 27, 2026Changyu Chen, Chenwei Lin, Xian XuNumerical Reasoning in Language Models Long-Context Reasoning in Language Models
Fudan University
ReContext: Recursive Evidence Replay as LLM Harness for Long-Context Reasoning
Jul 2, 2026Yanjun Zhao, Ruizhong Qiu, Tianxin Wei +6Long-Context Inference Long-Context Reasoning in Language Models
University of Illinois Urbana-Champaign
Randomized YaRN Improves Length Generalization for Long-Context Reasoning
Jun 22, 2026Manas Mehta, Fangcong Yin, Greg DurrettLong-Context Language Modeling LLM Fine-Tuning
New York University
Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
Jun 17, 2026Xiaoyue Xu, Sikui Zhang, Xiaorong Wang +2Training Data Curation RL for Language Model Reasoning
OpenBMB · Tsinghua University
LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning
Apr 15, 2026Sumeet Ramesh Motwani, Daniel Nichols, Charles London +17Long-Horizon LLM Reasoning CoT Reasoning
University of Oxford · Lawrence Livermore National Laboratory (LLNL) · MBZUAI +4
MEMENTO: Teaching LLMs to Manage Their Own Context
Apr 10, 2026Vasilis Kontonis, Yuchen Zeng, Shivam Garg +7LLM Context Management Efficient Language Model Reasoning
Microsoft Research
RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrieval
Apr 10, 2026Kyle Whitecross, Negin RahimiRetrieval-Augmented Reasoning Long-Context Reasoning in Language Models
University of Massachusetts Amherst
MemCoT: Test-Time Scaling through Memory-Driven Chain-of-Thought
Apr 9, 2026Haodong Lei, Junming Liu, Yirong Chen +2Long-Context Reasoning in Language Models Memory-Augmented Language Models
Southeast University Nanjing, Jiangsu, China · Shanghai Artificial Intelligence Laboratory Shanghai, China
A Decomposition Perspective to Long-context Reasoning for LLMs
Apr 9, 2026Yanling Xiao, Huaibing Xie, Guoliang Zhao +8Long-Context Reasoning in Language Models
Large Language Model Department, Tencent · Xi’an Jiaotong University, Xi’an, China · Fudan University, Shanghai, China +2
$π^2$ : Structure-Originated Reasoning Data Improves Long-Context Reasoning Ability of Large Language Models
Apr 6, 2026Quyet V. Do, Thinh Pham, Nguyen Nguyen +3Long-Context Reasoning in Language Models LLM Reasoning
Virginia Tech, VA 24061, USA
TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
Apr 6, 2026Weian Mao, Xi Lin, Wei Huang +5Long-Context Reasoning in Language Models KV-Cache Compression
MIT · ZJU · NVIDIA
Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
Apr 6, 2026Miao Li, Irina Saparina, Alexander Gurung +1Long-Context Reasoning in Language Models LLM Fine-Tuning
School of Informatics, University of Edinburgh
Parallel LLM Reasoning for Bias-Resilient, Robust Conceptual Abstraction
Apr 4, 2026Aisvarya Adeseye, Jouni Isoaho, Adeyemi AdeseyeLong-Context Reasoning in Language Models Parallel Reasoning in Language Models
University of Turku, Turku, Finland · Brilloconnetz Partners avoin yhtiö, Turku, Finland
Internalized Reasoning for Long-Context Visual Document Understanding
Mar 31, 2026Austin VeselkaMultimodal Understanding Document Image Understanding
LightOn

Oct 15, 2026J/K move · Enter open · S save

Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning

Shiwen Qin, Cameron Barker, Henry Gouk, Elliot J. Crowley+3

University of Edinburgh, Edinburgh, United Kingdom

Long-Context Reasoning in Language Models Long-Context Language Model Evaluation Long-Context Inference Long-Context QA

Open paper AskPDF arXiv

Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. However, context length alone cannot explain why models fail on long-context tasks that require reasoning over complex information. Contexts of equal length can pose very different challenges: a task may be solvable from a single local passage, or it may require integrating evidence distributed across the full context. We quantitatively characterise this difficulty through three properties: evidence scope, how much necessary information must be found; evidence dispersion, how widely that information is scattered within the context; and reasoning depth, how many dependent steps are needed to derive an answer from it. We construct LURCH, a multi-domain question-answering benchmark of 6,550 instances with supporting evidence and reasoning graphs, enabling all three properties to be quantified. Evaluations of proprietary and open-weight models show that: (1) at fixed context length, all our three properties are strongly negatively correlated with model performance; (2) as context length increases, performance degrades sharply on samples with dispersed evidence, whereas it remains relatively stable on samples with localised evidence; and (3) difficulty compounds across properties, as questions with moderately high values on all three are harder than those with an extreme value on just one. These findings establish evidence scope, dispersion and reasoning depth as measures of long-context reasoning difficulty that complement context length.