Long-Context Retrieval

Ask about this topic

Overview

Long-context retrieval studies how to locate and use relevant evidence when information is spread across very long documents, conversations, or model inputs. Common approaches include chunked dense or lexical search, hierarchical and structure-aware navigation, evidence aggregation, and model-side memories such as recurrent states or compressed KV caches. Benchmarks such as LongBench, LongMemEval, and needle-in-a-haystack tests measure retrieval, multi-hop reasoning, and evidence grounding across different context lengths and document types. A central challenge is that advertised context length does not guarantee reliable access to dispersed or competing evidence, while retrieval and compression methods must balance accuracy, cost, and faithful support for answers.

Latest papers

CardsList

45 papers12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Rethinking Retrieval in the Long-Context Era: Training-Free Semantic Trees for Adaptive Context Construction

    Oct 25, 2026Zhe Xu, Tianhao Liao, Zhenqing Ling +3Long-Context RetrievalInformation Retrieval

    Sun Yat-sen University · Alibaba Group

  2. Head Dimension Matters in Language Models for Long-Context Retrieval

    Oct 23, 2026Seiya Tokui, Shoichiro Yamaguchi, Sosuke KobayashiLong-Context RetrievalAttention Head Analysis

    Preferred Networks, Inc., Tokyo, Japan

  3. Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

    Oct 20, 2026Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González +5Long-Context RetrievalEvidence Retrieval

    CICATA Qro., Instituto Politécnico Nacional, México · Facultad de Informática, Universidad Autónoma de Querétaro, México · CIDETEC, Instituto Politécnico Nacional, México +3

  4. Weighted State Souping for Efficient Long-Context Retrieval

    Oct 19, 2026William L. Tong, Aryo Lotfi, Emmanuel Abbe +3State Space Language ModelsLong-Context Retrieval

    Harvard University · Apple

  5. Latent Compaction with Test-Time Training

    Oct 14, 2026Yingfa Chen, Xingyu Shen, Chaojun Xiao +1LLM Context CompressionKV-Cache Compression

    NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing, China

  6. VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

    Oct 12, 2026Simon Ott, Yufan Chen, Ruiping Liu +6Visual Document RetrievalLong-Context Retrieval

    Karlsruhe Institute of Technology · Hunan University

  7. NativeScope: Relation-Localized Retrieval over Native Topology with a Correct Anchor

    Oct 8, 2026Long WangInformation RetrievalLong-Context Retrieval

  8. Periscope: Extending Frozen Language Models Beyond Their Context Window

    Oct 2, 2026Mohamed Eltahir, Anas Obayd, Raed Rashid +5Long-Context Language Model InferenceLong-Context Reasoning in Language Models

    King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England

  9. Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA

    Sep 30, 2026Priyank Jayraj, Poonam Goyal, Navneet GoyalLong-Context RetrievalInformation Retrieval

    Birla Institute of Technology and Science, Pilani Pilani, Rajasthan, India

  10. Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting

    Sep 30, 2026Liang Twist Shan, Tianyu Hu, Hao Yan +1Numerical Reasoning in Language ModelsLong-Context Retrieval

    Department of Statistics, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison

  11. Auditing Long-Term Memory Evaluation: Repeated Judging, Reader Variation, and Negative Controls

    Sep 29, 2026Christopher J. ChanhnourackLong-Context RetrievalLLM Auditing

    Centennial Defense Systems (Archivist)

  12. Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression

    Sep 28, 2026Xingyu Zhu, Pu, Yi +6Long-Context RetrievalKV-Cache Compression

    Luke · ByteDance Seed · University of California, Berkeley

  13. On-Policy Attention Linearization

    Sep 25, 2026Arian Raje, Anupam Nayak, Anthony Fei +3Linear AttentionHybrid Transformer Architectures

    Carnegie Mellon University · Cornell University

  14. When Learned Context Planning Fails to Beat Strong Retrieval: A Controlled Study of Planning, Routing, and Reranking for Long-Context QA

    Sep 22, 2026Yingrui Li, Han ChenLong-Context RetrievalLLM Planning

    Independent Researcher, United States

  15. HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing

    Sep 22, 2026Jianyu Wei, Yizhao Gao, Qihao Zhang +12Long-Context RetrievalKV-Cache Management

    LLM-Core Xiaomi

  16. Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model

    Sep 19, 2026Wenji FuLong-Context QALLM Reasoning with Graphs

    Research Institute of Economics and Management Southwestern University of Finance and Economics

  17. REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling

    Aug 30, 2026Devrim Çavuşoğlu, Emre AkbaşLong-Context RetrievalLong-Context Modeling

    Middle East Technical University · OBSS AI

  18. PAGE: Partition-Aware Gated KV-Cache Eviction

    Aug 26, 2026Pankaj Kumar, Subhankar MishraLong-Context RetrievalKV Caching

    School of Computer Sciences National Institute of Science Education and Research An OCC of Homi Bhabha National Institute, India

  19. The Query Knows What to Forget: A Second Erase Direction for Linear Attention

    Aug 13, 2026Dhruman Gupta, Aritra Das, Debayan GuptaDelta-Rule Linear AttentionLong-Context Retrieval

    Truth Audit Labs

  20. MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

    Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Long-Context RetrievalIn-Context Retrieval

    Shanghai AI Laboratory · Tsinghua University · Fudan University

  21. CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

    Aug 7, 2026Gyuwan Kim, Cheoneum Park, Tao YangLong-Context RetrievalRetrieval-Augmented Generation

    University of California, Santa Barbara · Hanbat National University

  22. Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

    Aug 6, 2026Sagar Tamang, Ayush Vyas, Tabarakul HazarikaLong-Context RetrievalRetrieval-Augmented Generation

    1Indian Institute of Technology Patna, Bihta, India · 2TwoSpoon, India.

  23. When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings

    Aug 4, 2026Christopher Schröder, Lukas Gienapp, Ferdinand Schlatt +2Language Model PretrainingLong-Context Retrieval

    Institute for Applied Informatics at Leipzig University (InfAI) · ScaDS.AI Dresden/Leipzig · Seltz +1

  24. Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling

    Aug 3, 2026Qinwen Wang, Jieping Luo, Aoxiang Qin +5Long-Context RetrievalMemory-Augmented Neural Networks

    The Chinese University of Hong Kong, Shenzhen · Hainan Bielefeld University of Applied Sciences · BrainGalaxy

  25. XL-DocBench: Benchmarking Evidence-Grounded Extra-Long Document Understanding

    Jul 21, 2026Hongchen Wei, Yuanzhe Wang, Bei Liu +9Long-Context RetrievalLLM Evaluation

    1Wuhan University · 2Microsoft

  26. Is Progressive Disclosure All You Need for Long-Context Agents?

    Jul 20, 2026Yifeng He, Yinzhe Zhao, Jicheng Wang +1Long-Context RetrievalLong-Context QA

    University of California, Davis · Zhejiang University · The University of Hong Kong

  27. Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity

    Jul 8, 2026Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy +6Long-Context RetrievalMemory-Augmented Language Models

    Meta FAIR · Inria Paris & ENS-PSL University · University of Tübingen

  28. A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets

    Jul 2, 2026Wanyun CuiHybrid AttentionLong-Context Retrieval

    Shanghai University of Finance and Economics

  29. Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads

    Jul 1, 2026Aryo Pradipta Gema, Beatrice Alex, Pasquale MinerviniLong-Context RetrievalAttention Head Analysis

    QUniversity of Edinburgh · KHeriot-Watt University · V Miniml.AI

  30. EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory

    Jun 19, 2026Chang Nie, Chaoyou Fu, Junlan Feng +1Long-Context RetrievalText Embeddings

    Nanjing University