Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries

Anikait Singh; Teresa Zhang; Yoonho Lee; Roshen Sanjay Nair; Aviral Kumar; Chelsea Finn

cs.LGOct 15, 2026

Librarian Agents: Training LLMs to Curate Long-Context Workspaces that Anticipate Future Queries

Authors: Anikait Singh, Teresa Zhang, Yoonho Lee, Roshen Sanjay Nair, Aviral Kumar, Chelsea Finn

Organizations: Stanford University · Carnegie Mellon University

Abstract

Reasoning over long document contexts remains a challenge for large language models (LLMs). Standard retrieval methods are inherently reactive, bottlenecking agents with complex, cross-document reasoning at inference time, while precomputed structures like knowledge graphs often fall short because they rely on rigid heuristics. Instead, can we allow agents to perform anticipatory directed curation: learning to proactively reorganize unstructured corpora prior to inference to efficiently serve diverse future queries. We build on a key asymmetry in modern agentic harnesses (e.g., Claude Code): while LLM agents are adept at navigating existing filesystems to solve an immediate task, they are rarely trained to construct reusable workspaces for future use. To instantiate this capability, we introduce Librarian Agents, a framework in which a builder agent transforms raw documents into a hierarchical filesystem of synthetic files and reference links, which forms a workspace tailored for a downstream answering agent. We formulate workspace construction as a Markov decision process and optimize the builder policy via reinforcement learning to directly maximize downstream answering accuracy. Empirically, Librarian Agents outperforms the strongest retrieval and precomputation baselines by 33 percentage points on BrowseComp-Plus, amortizes upfront construction costs across repeated queries, and generalizes zero-shot to unseen domains on BRIGHT with a 31-point average accuracy improvement.

Figures & tables

Figure 1: Overview of Librarian Agents. (Left) We train a builder agent to construct a workspace consisting of synthetic files and directories that anticipates future queries for a downstream answering agent. (Right) This workspace is built once and repeatedly used to answer different queries.

Figure 2: MDP Definition of Librarian Agents. Starting from a raw list of documents, a builder agent can create a workspace with four actions: Create , Merge , Delete , and Stop . An answering agent navigates the resulting workspace through file-listing and file-reading actions, produces a response, and receives a task-specific correctness score. Here, we depict an augmented workspace from the BrowseComp-Plus dataset.

Figure 3: Dense Rewards via Probes. We train the builder agent with RL to directly maximize performance of answering agent. To densify the reward, the builder agent operates over workspace augmentations and is rewarded for change in accuracy. Probes to the answering agent are done at regular intervals (8 turns) up to the maximum horizon $H$ .

Figure 4: Results on BrowseComp-Plus. We compare Librarian Agents against RAG, Caching and Builder-Based Memory baselines using two answering agent models (Qwen 3.5 4B and Gemini 3.1 Flash Lite). We find that Librarian Agents significantly outperforms all methods for both answering agents.

Table 5

	AOPS		Biology		Stack Overflow		Average
Method	pass@1	pass@3	pass@1	pass@3	pass@1	pass@3	pass@1	pass@3
BM25	30.5 $\pm$ 5.9	37.0 $\pm$ 6.8	17.5 $\pm$ 4.6	26.0 $\pm$ 6.0	9.0 $\pm$ 3.3	15.5 $\pm$ 4.8	19.0 $\pm$ 2.7	26.2 $\pm$ 3.4
Dense	31.5 $\pm$ 6.0	38.0 $\pm$ 6.9	19.5 $\pm$ 4.8	29.0 $\pm$ 6.1	11.5 $\pm$ 3.9	17.0 $\pm$ 5.2	20.8 $\pm$ 2.9	28.0 $\pm$ 3.5
Hybrid	33.0 $\pm$ 6.4	37.0 $\pm$ 6.8	14.0 $\pm$ 4.3	20.5 $\pm$ 5.6	7.0 $\pm$ 2.4	15.5 $\pm$ 4.8	18.0 $\pm$ 2.7	24.3 $\pm$ 3.3
Rerank	32.0 $\pm$ 6.1	37.5 $\pm$ 6.9	14.0 $\pm$ 4.1	23.0 $\pm$ 5.6	6.0 $\pm$ 2.6	11.0 $\pm$ 4.3	17.3 $\pm$ 2.6	23.8 $\pm$ 3.3
Agentic	28.5 $\pm$ 5.6	37.0 $\pm$ 6.8	17.0 $\pm$ 4.5	25.5 $\pm$ 5.9	6.5 $\pm$ 2.5	14.5 $\pm$ 4.5	17.3 $\pm$ 2.5	25.7 $\pm$ 3.3

Table 3: Zero-Shot Transfer to BRIGHT. The builder agent, trained only on DeepResearch tasks (BrowseComp) is evaluated on BRIGHT without task-specific training, testing whether Librarian Agents learns a transferable capability of anticipation across different query and document sources. Librarian Agents outperforms baselines on 3 sub-tasks within BRIGHT (AOPS, Biology, and Stack Overflow) achieving an average a pass@1 accuracy of 52% and pass@3 accuracy of 67% with the Qwen3.5-4B model.

Figure 5: Qualitative comparison of RL and base building. On BrowseComp-Plus, the base model creates a fragmented filesystem that is difficult for the answering agent to navigate, whereas the RL-trained builder surfaces relevant information in dedicated files and produces a richer, more ergonomic organization.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Model	Syn. Files	Syn. Tok.	Tot. Tok.	Act. Files	Clusters	Merges
Bio.	Base	2.7	1064	1189	5.4	4.9	0.5
	RL	30.4	9883	10091	54.0	45.8	8.2
StackOv.	Base	3.3	1075	1250	4.5	4.0	0.5
	RL	32.0	9246	9511	49.5	40.6	8.8
AoPS	Base	3.9	879	1023	4.5	3.8	0.7
	RL	36.2	7309	7544	56.7	49.7	7.0

Appendix

Table 4: BRIGHT statistics by domain. We report answerer retrieval behavior and final filesystem-structure statistics for BRIGHT. Syn. Files, Syn. Tok., and Tot. Tok. measure files and tokens read by the answering agent, while Act. Files, Clusters, and Merges summarize the active synthetic filesystem produced by the builder.

Dataset	Model	Syn. Files	Syn. Tok.	Tot. Tok.	Act. Files	Clusters	Merges
BrowseComp+	Base	2.7	1069	1235	3.2	2.7	0.6
	RL	26.9	9040	9309	42.2	34.4	7.8

Appendix

Table 5: BrowseComp+ statistics. We report the same statistics on the amortized BrowseComp+ evaluation. Syn. Files, Syn. Tok., and Tot. Tok. again measure answering-agent retrieval behavior, while Act. Files, Clusters, and Merges summarize the active synthetic filesystem produced by the builder.

Method	Earth Science	Economics	Psychology	Robotics	Sustainable Living	LeetCode	Pony	TheoremQA Questions	TheoremQA Theorems
Base Model	26.3%	16.3%	32.5%	23.7%	12.5%	28.7%	48.7%	35.0%	38.8%
Librarian	73.8%	72.5%	72.5%	55.0%	43.8%	56.2%	72.5%	63.7%	56.2%

Appendix

Table 6: Comparison with the Base Model on Additional BRIGHT Domains. Librarian Agents outperforms the base model across all nine additional domains. Results are reported as percentages.

Panel of Figure 6 — Figure 6: BrowseComp-Plus reuse curves. Left: recorded construction and navigation tokens per query. Right: recorded tokens per expected correct answer after normalization by observed full-denominator accuracy. Shaded regions show paired corpus-bootstrap 95% confidence intervals.

Memory organization	Accuracy (%)	Navigation tokens
Original	$55.86\pm 2.58$	$3{,}635\pm 216$
Flattened	$57.70\pm 2.81$	$5{,}497\pm 356$
Shuffled	$51.72\pm 2.87$	$3{,}612\pm 221$

Appendix

Table 7: Filesystem-structure ablation on BrowseComp+. Accuracy and answering-agent navigation-token cost are evaluated on 435 expanded amortization questions across the 49 corpora with completed filesystem snapshots. All three conditions use the same matched answering protocol. Standard errors use a corpus-cluster bootstrap.

Figure 7: Recovery on BrowseComp-Plus using provenance information. When the synthetic filesystem omitted a required detail, the answering agent followed provenance links to the relevant raw source and recovered the correct answer.

Figure 8: Training curve for the BrowseComp-Plus builder. Aggregate rollout reward during GRPO training, smoothed with an exponential moving average.

Agentic RAG

Agentic retrieval-augmented generation lets language models plan searches, use tools, and refine evidence before producing grounded answers.

See all 246 papers

May 25, 2026

Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki

LLM agents require retrieval to behave less like one-shot context fetching and more like reasoning: searching, reading, traversing, and deciding when evidence is sufficient. Yet current Retrieval-Augmented Generation (RAG) systems organize external knowledge as flat chunks retrieved by embedding similarity, exposing a retrieval-as-lookup interface ill-suited to iterative reasoning agents. We propose LLM-Wiki, an agent-native retrieval system that operationalizes the Retrieval-as-Reasoning paradigm by treating external knowledge as a compilable, composable, and self-evolving structure rather than a static retrieval index. LLM-Wiki compiles documents into structured Wiki pages with bidirectional links, exposes search, read, and link-following operations through standard tool-calling interfaces, and introduces an Error Book for persistent structural and semantic self-correction. LLM-Wiki achieves state-of-the-art results on HotpotQA, MuSiQue, and 2WikiMultiHopQA, outperforming HippoRAG 2, LightRAG, and GraphRAG by 2.0-8.1 F1 points. On AuthTrace, LLM-Wiki achieves the best overall accuracy, with especially strong gains on multi-document structured queries, confirming that compilation-based retrieval generalizes beyond chain-style multi-hop reasoning.

Haoliang Ming, Feifei Li, Xiaoqing Wu +1

WeChat, Tencent Inc., Beijing, China

Multi-Hop QA Retrieval-Augmented Generation Tool-Augmented Language Model Agents Agentic Retrieval LLM Agents

Ask PDF

Mar 9, 2026

SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation

Answering complex, real-world queries often requires synthesizing facts scattered across vast document corpora. In these settings, standard retrieval-augmented generation (RAG) pipelines suffer from incomplete evidence coverage, while long-context large language models (LLMs) struggle to reason reliably over massive inputs. We introduce SPD-RAG, a hierarchical multi-agent framework for exhaustive cross-document question answering that decomposes the problem along the document axis. Each document is processed by a dedicated document-level agent operating only on its own content, enabling focused retrieval, while a coordinator dispatches tasks to relevant agents and aggregates their partial answers. Agent outputs are synthesized by merging partial answers through a token-bounded synthesis layer (which supports recursive map-reduce for massive corpora). This document-level specialization with centralized fusion improves scalability and answer quality in heterogeneous multidocument settings while yielding a modular, extensible retrieval pipeline. On the LOONG benchmark (EMNLP 2024) for long-context multi-document QA, SPD-RAG achieves an Avg Score of 58.1 (GPT-5 evaluation), outperforming Normal RAG (33.0) and Agentic RAG (32.8) while using only 38% of the API cost of a full-context baseline (68.0).

Yagiz Can Akay, Muhammed Yusuf Kartal, Esra Alparslan +2

TOBB University of Economics and Technology · OSTIM Technical University

Multi-Document QA Retrieval-Augmented Generation Agentic RAG Document QA Multi-Agent LLM Systems

Ask PDF

Sep 6, 2026

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

Sequential memory agents process long documents by reading chunks one after another while maintaining a compact memory state, coupling document traversal to reasoning depth. This coupling introduces sensitivity to evidence placement and ties inference latency linearly to document length. We introduce PARSER, which decouples reading from reasoning. A bank of lightweight subagents each bound to a single chunk read the entire document in parallel, while a lead agent reasons in depth through iterative scatter--gather rounds: at each round it broadcasts a query to all subagents, aggregates the returned evidence, and formulates a deeper follow-up query conditioned on what has been found so far. This decoupled design concentrates all learnable behavior in the lead agent, which is optimized with reinforcement learning, while the subagents remain frozen off-the-shelf models. On multi-hop QA with contexts ranging from 7K to 896K tokens, PARSER with a 4B backbone outperforms the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens. Scaling to a 9B backbone, PARSER surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments confirm that PARSER is robust to perturbations in evidence position, order, and distance, conditions that cause large accuracy swings in sequential methods, while reducing inference latency by up to 11x.

Kun Li, Zexuan Qiu, Tianhua Zhang +2

The Chinese University of Hong Kong

Long-Context Inference Multi-Agent LLMs Multi-Hop QA

Ask PDF

Abstract

Agentic RAG

Explore similar work

Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki

SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents