Overview

Long-context retrieval studies how to locate and use relevant evidence when information is spread across very long documents, conversations, or model inputs. Common approaches include chunked dense or lexical search, hierarchical and structure-aware navigation, evidence aggregation, and model-side memories such as recurrent states or compressed KV caches. Benchmarks such as LongBench, LongMemEval, and needle-in-a-haystack tests measure retrieval, multi-hop reasoning, and evidence grounding across different context lengths and document types. A central challenge is that advertised context length does not guarantee reliable access to dispersed or competing evidence, while retrieval and compression methods must balance accuracy, cost, and faithful support for answers.

Latest papers

CardsList

53 papers11 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Retrofitting Unbounded Context onto Stock Language Models

    Nov 6, 2026Mark OskinLong-Context Language Model InferenceLanguage Modeling

  2. Query-Guided RoPE Modulation for Lost-in-the-Middle Mitigation in Long-Context LLMs

    Nov 6, 2026Peiyu Zhang, Heng Ping, Shinibali Bhattacharyya +6Lost-in-the-Middle EffectLong-Context Language Model Inference

  3. Anchor: Augmented Cross-Layer Recurrent State for Long-Context Retrieval in Test-Time Training

    Nov 3, 2026Vimal William, Yuanzhe Li, Danella Zhao +1Memory-Augmented Language ModelsLong-Context Retrieval

  4. DAMI: Dual-Axis Memory Initialization for Enhanced Long-Context Retrieval in Gated Linear Attention

    Oct 30, 2026Zhangchi Feng, Harry Jiang, Yangyifei Luo +2Long-Context RetrievalLinear Attention

  5. Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

    Oct 29, 2026Daniel Strauß, Zhenshan Bing, Alois KnollLong-Context RetrievalMemory-Augmented Agents

  6. Head Dimension Matters in Language Models for Long-Context Retrieval

    Oct 23, 2026Seiya Tokui, Shoichiro Yamaguchi, Sosuke KobayashiLong-Context RetrievalAttention Head Analysis

  7. Scan or Index? large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG

    Oct 20, 2026Juan Terven, Diana-Margarita Córdova-Esparza, Julio-Alejandro Romero-González +5Long-Context RetrievalEvidence Retrieval

  8. Weighted State Souping for Efficient Long-Context Retrieval

    Oct 19, 2026William L. Tong, Aryo Lotfi, Emmanuel Abbe +3State Space Language ModelsLong-Context Retrieval

  9. Latent Compaction with Test-Time Training

    Oct 14, 2026Yingfa Chen, Xingyu Shen, Chaojun Xiao +1LLM Context CompressionKV-Cache Compression

  10. VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

    Oct 12, 2026Simon Ott, Yufan Chen, Ruiping Liu +6Visual Document RetrievalLong-Context Retrieval

  11. Periscope: Extending Frozen Language Models Beyond Their Context Window

    Oct 2, 2026Mohamed Eltahir, Anas Obayd, Raed Rashid +5Long-Context Language Model InferenceLong-Context Reasoning in Language Models

  12. Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA

    Sep 30, 2026Priyank Jayraj, Poonam Goyal, Navneet GoyalLong-Context RetrievalInformation Retrieval

  13. Targeted Retrieval, Compact Representations: How CoT Reasoning Improves Long-Context Counting

    Sep 30, 2026Liang Twist Shan, Tianyu Hu, Hao Yan +1Numerical Reasoning in Language ModelsLong-Context Retrieval

  14. Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression

    Sep 28, 2026Xingyu Zhu, Pu, Yi +6Long-Context RetrievalKV-Cache Compression

  15. On-Policy Attention Linearization

    Sep 25, 2026Arian Raje, Anupam Nayak, Anthony Fei +3Linear AttentionHybrid Transformer Architectures

  16. HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing

    Sep 22, 2026Jianyu Wei, Yizhao Gao, Qihao Zhang +12Long-Context RetrievalKV-Cache Management

  17. Beyond Linear Context: Graph-Guided Evidence Navigation for Long-Novel Reasoning with a Local 9B Language Model

    Sep 19, 2026Wenji FuLong-Context QALLM Reasoning with Graphs

  18. PAGE: Partition-Aware Gated KV-Cache Eviction

    Aug 26, 2026Pankaj Kumar, Subhankar MishraLong-Context RetrievalKV Caching

  19. MARCH: Scaling Recurrent Memory with Content-Routed State Anchors

    Aug 12, 2026Ming Zhang, Kaisen Yang, Shu Yu +6Long-Context RetrievalIn-Context Retrieval

  20. CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

    Aug 7, 2026Gyuwan Kim, Cheoneum Park, Tao YangLong-Context RetrievalRetrieval-Augmented Generation

  21. Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

    Aug 6, 2026Sagar Tamang, Ayush Vyas, Tabarakul HazarikaLong-Context RetrievalRetrieval-Augmented Generation