Long-Context Retrieval

Ask about this topic

Overview

Long-context retrieval studies how to locate and use relevant evidence when information is spread across very long documents, conversations, or model inputs. Common approaches include chunked dense or lexical search, hierarchical and structure-aware navigation, evidence aggregation, and model-side memories such as recurrent states or compressed KV caches. Benchmarks such as LongBench, LongMemEval, and needle-in-a-haystack tests measure retrieval, multi-hop reasoning, and evidence grounding across different context lengths and document types. A central challenge is that advertised context length does not guarantee reliable access to dispersed or competing evidence, while retrieval and compression methods must balance accuracy, cost, and faithful support for answers.

Latest papers

CardsList

45 papers12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Does RoPE Prevent or Degrade Retrieval Heads? A Mechanistic Analysis Across Model Families

    Jun 19, 2026Cengizhan BayramRotary Positional EmbeddingsLong-Context Retrieval

    Independent Researcher

  2. Storyline Trees: Hierarchical Representations for Long-Form Narratives

    Jun 18, 2026Litu Ou, Mirella LapataLong-Context RetrievalLong-Context QA

    School of Informatics University of Edinburgh

  3. Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation

    Jun 17, 2026Shanshan Lyu, Yiwei Wang, Yujun Cai +2Long-Context RetrievalDocument Chunking

    Chongqing University · State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences +2

  4. LEDGER: A Long-Context Benchmark of Corporate Annual Reports for Grounded Financial Retrieval and Extraction

    Jun 11, 2026Charles Moslonka, Amaury de Vitry, Arthur Garnier +2Long-Context RetrievalDocument Information Extraction

    Artefact Research Center Paris, France · MICS, CentraleSupélec, Université Paris-Saclay Gif-sur-Yvette, France · Ardian Paris, France

  5. Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It

    Jun 9, 2026Xinyu Zhou, Boyu Zhu, Yi Xu +4Long-Context RetrievalSupervised Fine-Tuning

    1LARK, HKUST(GZ) · 2UCL · 3Mistral AI +3

  6. EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering

    Jun 5, 2026Xiaopeng Yuan, Zebin Wang, Suwen Wang +3Long-Context RetrievalLong-Context QA

    University of Illinois Urbana-Champaign · 2Harvard University · 3Brion, ASML US LP +1

  7. Dense Contexts Are Hard Contexts: Lexical Density Limits Effective Context in LLMs

    Jun 4, 2026Giovanni Dettori, Matteo Boffa, Danilo Giordano +2Long-Context RetrievalLLM Evaluation

    Department of Computer Science Politecnico di Torino Torino, IT 10129 · Department of Computer Science University of Turin Torino, IT 10124

  8. Memory by Design: Bayesian Layers for Sequence Modeling

    May 29, 2026Matthew Dowling, Hyungju Jeon, Cristina Savin +1Long-Context RetrievalBayesian Filtering

    Champalimaud Research, Champalimaud Foundation, Portugal · Center for Neural Science, New York University, USA · Center for Data Science, New York University, USA

  9. Separating Semantic Competition from Context Length in RAG Reading

    May 26, 2026Vyzantinos Repantis, Ameya Gawde, Harshvardhan Singh +4Long-Context RetrievalRetrieval-Augmented Generation

    Meta Platforms

  10. WhenLoss: Diagnosing Write and Retrieval Bottlenecks in Long-Context Memory Systems

    May 23, 2026Jiangnan Yu, Kisson Songqi Lin, Jilong WuLong-Context RetrievalMemory-Augmented Language Models

  11. Retrieval and Multi-Hop Reasoning in 1M-Token Context Windows: Evaluating LLMs on Classical Chinese Text

    May 4, 2026Eric H. C. ChowLong-Context RetrievalIn-Context Retrieval

    School of Humanities, The University of Hong Kong

  12. Reproduction Beyond Benchmarks: ConstBERT and ColBERT-v2 Across Backends and Query Distributions

    Apr 11, 2026Utshab Kumar Ghosh, Ashish David, Shubham ChatterjeeMulti-Vector RetrievalLate Interaction Retrieval

    Missouri University of Science and Technology, Rolla, MO, USA

  13. Pooling and Semantic Shift: The Fundamental Challenges in Long Text Embedding and Retrieval

    Mar 22, 2026Hang Gao, Wujiang Xu, Kai Mei +1Text EmbeddingsText Embedding Models

    Rutgers University

  14. Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

    Feb 20, 2026Amine Kobeissi, Philippe LanglaisLong-Context RetrievalFinancial QA

    Department of Computer Science and Operations Research, Université de Montréal · RALI

  15. Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs

    Jan 5, 2026Amirali Ebrahimzadeh, Seyyed M. SaliliLong-Context RetrievalLLM Hallucination Mitigation

    Department of Electrical Engineering & Computer Science, University of Michigan, Ann Arbor, MI 48109, USA · Independent Researcher, Ann Arbor, MI 48106, USA