Long-Context QA

QA: Question Answering

Long-context QA asks language and vision-language models to answer questions using evidence distributed across books, reports, conversations, and multimodal documents. Approaches include long-context fine-tuning, retrieval and agentic document navigation, structured memories, evidence selection and compression, and efficient attention or KV-cache methods. Benchmarks such as LongBench, HotpotQA, LongMemEval, and long-document QA suites test multi-hop reasoning, answerability, citation support, and evidence spread across pages or narrative events. A central challenge is that larger context windows do not guarantee usable evidence: systems must balance accuracy and faithfulness against token, latency, and memory costs, while evaluation must distinguish retrieval failures from reasoning failures.

Latest papers

39 papers8 papers in the last four weeks, up 60% on the four weeks before. 0.0% of all new papers.

Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Oct 27, 2026Yue Wu, Yang Zhang, Weinan Gan +4Test-Time Training Inference-Time Adaptation of Language Models
University of Science and Technology of China China · National University of Singapore Singapore · Huawei Noah’s Ark Lab China
Mixing Matters: Evidence Position Bias Across Sequence Mixers in Long-Context Question Answering
Oct 22, 2026Aman Behera, Namit Solanki, Mehul AnandState Space Language Models Long-Context Language Modeling
IIT Roorkee · AISSMS IOIT, Pune · Independent Researcher
The KV-Surgeon: Compensation-Aware Offline KV-Cache Compaction
Oct 19, 2026Moritz Wagner, Christophe Roux, Max Zimmer +1KV-Cache Management KV-Cache Pruning
Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany · Institute of Mathematics, Technische Universität Berlin, Germany
The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering
Oct 18, 2026Debopriyo Banerjee, Owais Aijaz, Ahmer Tabassum +3RAG Evaluation Evidence-Grounded Generation
Inception42 · MBZUAI
Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning
Oct 15, 2026Shiwen Qin, Cameron Barker, Henry Gouk +4Long-Context Reasoning in Language Models Long-Context Language Model Evaluation
University of Edinburgh, Edinburgh, United Kingdom
Memorilla: Latent Semantic Memory for LLMs
Oct 15, 2026Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli +4Memory-Augmented Language Models Persistent Memory for Language Models
Stanford University · University of Toronto · Vector Institute +3
All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
Oct 15, 2026Bingbing Wang, Jingjie Lin, Weiming Qiao +2Persistent Memory for Language Models Memory-Augmented Language Model Agents
Harbin Institute of Technology, Shenzhen · The Hong Kong Polytechnic University · Fudan University +1
Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Oct 15, 2026Wanyang Hong, Zhaoning ZhangKV-Cache Management KV-Cache Compression
National Key Laboratory of Parallel and Distributed Computing College of Computer Science and Technology National University of Defense Technology, Changsha, China.
LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding
Oct 10, 2026Zeyu Shan, Yongxin Shi, Shihang Wu +4Document Understanding Historical Document Analysis
School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510641, China · College of Literature and Journalism, Sichuan University, Chengdu, 610065, China
Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
Sep 25, 2026Zhaoyuan Xia, Qinghongbing Xie, Yung Xiang Hue +7Long-Context Reasoning in Language Models Long-Context Modeling
Peking University · Baidu Inc · Baidu Inc. +1
When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA
Sep 22, 2026Yingrui Li, Han ChenLong-Context Retrieval LLM Planning
Independent Researcher, United States
Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model
Sep 19, 2026Wenji FuLong-Context QA LLM Reasoning with Graphs
Research Institute of Economics and Management Southwestern University of Finance and Economics
CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models
Sep 5, 2026Pengan Chen, Kaisheng Zheng, Liang Hong +10Efficient VLM Inference KV-Cache Reuse
The Chinese University of Hong Kong · Fudan University · LIGHTSPEED +1
TopoCompress: Long Context Compression via Graph-Wired Semantic Trajectories
Aug 31, 2026Daniel Agyei Asante, Yang LiLLM Compression Long-Context QA
1Iowa State University, United States
AWM: Answerable Working Memory for Long-Document VQA Agents
Aug 26, 2026Dongzhuoran Zhou, Yuqicheng Zhu, Yule Liu +5Long-Document VQA Long-Context QA
University of Oslo · Bosch Center for AI · University of Stuttgart +2
ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives
Aug 26, 2026Jihao Zhu, Zhiwei Yang, Wenxiao Zhang +7Long-Context QA Multi-Agent LLM Systems
University of Aberdeen, Aberdeen, United Kingdom · Institute of Information Engineering, CAS, Beijing, China · The University of Western Australia, Perth, Australia +1
PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memory
Aug 4, 2026Dawei Liu, Haixu Song, Shuang Cheng +9Memory-Augmented Language Models Long-Context QA
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University +3
SeDeM: Selective Decompression of Hidden-State Memories for Long-Context Question Answering
Jul 31, 2026Maryam Haghifam, Jason Cong, Yizhou SunMemory-Augmented Language Models Long-Context QA
University of California, Los Angeles, USA
Zero-Mem: Zero-Token Memory Operations for LLM Agents
Jul 31, 2026Yilin Xiao, Zhehan Zhu, Yujing Zhang +8Persistent Memory for Language Models Long-Context QA
The Hong Kong Polytechnic University, Hong Kong SAR · School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics · School of Artificial Intelligence, Jilin University
REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoning
Jul 23, 2026Zhensheng Jin, Xin Dai, Zhenghao Liu +5LLM Grounding Long-Context QA
Northeastern University · Shenyang, China · Tsinghua University +1
TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering
Jul 21, 2026Zhong Ji, Keqi Jin, Yan Zhang +1Multimodal RAG Retrieval-Augmented Generation
School of Electrical and Information Engineering, Tianjin University · The International Joint Institute of Tianjin University, Fuzhou, Tianjin University · Department of Automation, Tsinghua University
DocAtlas: Long-Document Understanding as Mutable-State Interaction
Jul 21, 2026Hongchen Wei, Yuanzhe Wang, Bei Liu +8Document Understanding Long-Context QA
Wuhan University · Microsoft
XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding
Jul 21, 2026Hongchen Wei, Yuanzhe Wang, Bei Liu +9Long-Context Retrieval LLM Evaluation
1Wuhan University · 2Microsoft
Is Progressive Disclosure All You Need for Long-Context Agents?
Jul 20, 2026Yifeng He, Yinzhe Zhao, Jicheng Wang +1Long-Context Retrieval Long-Context QA
University of California, Davis · Zhejiang University · The University of Hong Kong
WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning
Jul 10, 2026Zixin Chen, Peng Liu, Haobo Li +7Evidence-Grounded Reasoning Long-Context QA
Hong Kong University of Science and Technology · Qwen Team, Alibaba Group · *Work done during internship at Qwen Team
What Survives Into Context: A Diagnostic for Budget-Constrained Multi-Hop RAG and When Submodular Evidence Packing Improves It
Jul 1, 2026Ananto Nayan BalaMulti-Hop QA Long-Context QA
Ahsanullah University of Science and Technology
Storyline Trees: Hierarchical Representations for Long-Form Narratives
Jun 18, 2026Litu Ou, Mirella LapataLong-Context Retrieval Long-Context QA
School of Informatics University of Edinburgh
Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History
Jun 5, 2026Liuyin WangLong-Context QA LLM Agent Memory
Independent Researcher
EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering
Jun 5, 2026Xiaopeng Yuan, Zebin Wang, Suwen Wang +3Long-Context Retrieval Long-Context QA
University of Illinois Urbana-Champaign · 2Harvard University · 3Brion, ASML US LP +1
Narrative Knowledge Weaver: Narrative-Centric Retrieval-Augmented Reasoning for Long-Form Text Understanding
Jun 4, 2026Qiuyu Tian, Fengyi Chen, Yiding Li +9Retrieval-Augmented Generation Long-Context QA
Southeast University, Nanjing, China · Nanjing Normal University, Nanjing, China · ZhuiWen Technology Co., Ltd., Beijing, China +1

Oct 27, 2026J/K move · Enter open · S save

Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training

Yue Wu, Yang Zhang, Weinan Gan, Chongming Gao+3

University of Science and Technology of China China · National University of Singapore Singapore · Huawei Noah’s Ark Lab China

Test-Time Training Inference-Time Adaptation of Language Models Fast Weights Long-Context QA

Open paper AskPDF arXiv

Test-time training (TTT) enables a deployed language model to internalize incoming context on-the-fly into a small set of fast-weight parameters, rather than merely treating it as transient context for attention. In-Place TTT recently made this idea practical for large language models by repurposing the down-projection of standard MLP blocks as fast weights. However, we find that its benefit largely disappears on realistic web long-context question answering. We attribute this to In-Place TTT's uniform write schedule: every chunk is written with equal strength, so irrelevant content dilutes query-relevant information in fast-weight memory. Building on this insight, we propose Select-TTT, a model-agnostic, query-conditioned rule for In-Place TTT that scales each chunk's write strength by its relevance to the user query. This relevance signal requires no supervision and no new parameters: it is read directly from the model's frozen output head, as the probability mass that a chunk's hidden states assign to the query tokens. A temperature-scaled softmax turns these scores into learning-rate multipliers, leaving the chunk-wise, context-parallel computation of In-Place TTT intact. Select-TTT consistently improves long-context question answering on two benchmarks and two backbones. These findings highlight the value of query-conditioned fast weight writes for practical test-time training on long-context question answering, which is also potentially promising for long-horizon agent tasks involving multiple user queries over evolving contexts.