From Knowledge Access to Source Learning: Developing Source-Specific Competence

Lucheng Fu; Kejing Xia; Yiyang Wang; Yiqiao Jin; Jinjin He; Xiyuan Yang; Haoxin Liu; Ye Yu

cs.CLOct 1, 2026

From Knowledge Access to Source Learning: Developing Source-Specific Competence

Authors: Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, +5 more

Organizations: Georgia Institute of Technology · University of Illinois at Urbana-Champaign · University of California, Los Angeles

Abstract

Large language model (LLM) agents increasingly rely on persistent external sources to solve sequences of knowledge-intensive tasks. Existing methods improve how source content is accessed and organized, while agent-memory systems preserve reusable knowledge from prior interactions, but repeated use of the same source is still largely treated as repeated access rather than an opportunity to progressively improve understanding of that source. We study source learning: developing reusable source-specific competence over a persistent authoritative source. We represent this competence with a persistent source model that captures reusable understanding of the source, including how its knowledge is structured, interpreted, and applied. To construct and progressively refine such models, we propose SourceLearn, which combines two complementary learning mechanisms. Self-Directed Source Learning identifies what remains incompletely understood and adaptively revisits the source, while Task-Guided Source Learning uses downstream experience to reveal local representational gaps and recurring needs in how source knowledge should be organized. In both cases, learning signals determine what should be reconsidered, while persistent updates are reconstructed from the authoritative source. Across five benchmarks and three LLM backends, SourceLearn achieves the best performance in 13 of 15 settings, with gains of up to 22.6 points over Hybrid RAG and substantial overall improvements over static source representations and experience-based memory baselines.

Figures & tables

Figure 1 : Problem Illustration. I) Knowledge access retrieves task-relevant evidence from the source, but does not necessarily accumulate reusable source understanding across tasks, requiring the model to repeatedly reconstruct how evidence should be interpreted or combined. II) Our goal is source learning : progressively developing reusable source understanding through continued study and use, while retaining direct access to the authoritative source.

Figure 2 : Overview of SourceLearn . Initial Construction builds a provisional source model $M_{0}$ from the authoritative source $D$ . Self-Directed Source Learning deepens and connects source understanding through adaptive study, producing $M_{S}$ . Task-Guided Source Learning uses guidance-task failures and recurring representation needs to further refine the model into $M_{T}$ . At inference time, Source-Model Activation combines relevant source-model context with directly retrieved evidence for downstream tasks.

		Document QA		Code QA	Tool Use	Interactive Env.
Model	Method	MultiDoc2Dial	NarrativeQA	SWE-QA	APIBench	AppWorld
GPT-5.6 -Luna	Hybrid RAG	$64.9$	$66.0$	$56.9$	$47.8$	$72.0$
	RAPTOR	$\underline{71.2}_{\color[rgb]{0,0.65,0.31}\uparrow 6.3}$	$\underline{68.1}_{\color[rgb]{0,0.65,0.31}\uparrow 2.1}$	$50.8_{\color[rgb]{0.91,0.59,0.48}\downarrow 6.1}$	$61.9_{\color[rgb]{0,0.65,0.31}\uparrow 14.1}$	$75.6_{\color[rgb]{0,0.65,0.31}\uparrow 3.6}$
	HippoRAG 2	$66.3_{\color[rgb]{0,0.65,0.31}\uparrow 1.4}$	$66.6_{\color[rgb]{0,0.65,0.31}\uparrow 0.6}$	$\underline{62.7}_{\color[rgb]{0,0.65,0.31}\uparrow 5.8}$	$55.2_{\color[rgb]{0,0.65,0.31}\uparrow 7.4}$	$\underline{77.4}_{\color[rgb]{0,0.65,0.31}\uparrow 5.4}$
	AWM	$65.1_{\color[rgb]{0,0.65,0.31}\uparrow 0.2}$	$62.1_{\color[rgb]{0.91,0.59,0.48}\downarrow 3.9}$	$56.7_{\color[rgb]{0.91,0.59,0.48}\downarrow 0.2}$	$\underline{62.8}_{\color[rgb]{0,0.65,0.31}\uparrow 15.0}$	$66.7_{\color[rgb]{0.91,0.59,0.48}\downarrow 5.3}$
	SourceLearn	$\mathbf{77.6_{\color[rgb]{0,0.65,0.31}\uparrow 12.7}}$	$\mathbf{80.2_{\color[rgb]{0,0.65,0.31}\uparrow 14.2}}$	$\mathbf{69.5_{\color[rgb]{0,0.65,0.31}\uparrow 12.6}}$	$\mathbf{70.4_{\color[rgb]{0,0.65,0.31}\uparrow 22.6}}$	$\mathbf{81.5_{\color[rgb]{0,0.65,0.31}\uparrow 9.5}}$

Table 1 : Main results across document QA, code QA, tool use, and interactive environments. We report task performance (%) with changes relative to Hybrid RAG shown as subscripts. The best and second-best results are highlighted with bold and underline , respectively.

Figure 3 : Ablation of SourceLearn with GPT-5.6-Luna: the source learning stages (left) and the two pathways of Task-Guided Source Learning (right). Each bar reports answer accuracy (%) on two sources of a QA benchmark.

Figure 4 : Source learning beyond compression. From $M_{0}$ to $M_{S}$ to $M_{T}$ , source learning (a) changes what the model represents, (b) makes conditions more explicit, (c) increases coverage of source knowledge required by future tasks, and (d) higher coverage is associated with higher answer accuracy.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5 : Robustness to missing retrieved evidence on MultiDoc2Dial with GPT-5.6-Luna. (a) Accuracy as the number of retrieved elements decreases; at

k=0

, SourceLearn uses only the activated source model. (b) Accuracy when the gold document is removed from retrieval at

k=8

	MultiDoc2Dial		SWE-QA	APIBench
Method	DMV	VA	Sphinx	HuggingFace
Hybrid RAG	$\underline{76.2}$	$70.2$	$62.7$	$48.6$
RAPTOR	$70.2$	$\underline{73.8}$	$61.8$	$\underline{52.9}$
HippoRAG 2	$71.4$	$71.5$	$\mathbf{73.5}$	$51.4$
SourceLearn	$\mathbf{77.0}$	$\mathbf{77.8}$	$\underline{72.5}$	$\mathbf{56.7}$
Evidence tokens	16.5k	18.2k	23.4k	17.1k

Appendix

Table 2 : Retrieval counterparts at matched context budgets (GPT-5.6-Luna; accuracy %). Each counterpart receives the same per-question context budget as SourceLearn by reading further from its own ranking. Best and second-best results are in bold and underlined .

Source	Model	Supported	Partial	Unsupported	Contradicted
MultiDoc2Dial (StudentAid)	$M_{0}$	79.2	19.2	1.7	0.0
	$M_{S}$	75.0	20.8	4.2	0.0
	$M_{T}$	63.3	22.5	13.3	0.8
SWE-QA (pytest)	$M_{0}$	89.2	10.8	0.0	0.0
	$M_{S}$	86.7	13.3	0.0	0.0
	$M_{T}$	65.0	31.7	3.3	0.0

Appendix

Table 3 : Grounding audit of source-model units (%). Each unit is judged against its cited source elements (120 units per model): supported if all claims are entailed, partial if the main claim is supported but some detail is not, unsupported if the cited evidence does not establish the claim, and contradicted if it states otherwise.

Source	Dropped by	Preserved	Lost	Contradicted
MultiDoc2Dial (StudentAid)	Self-Directed ( $M_{0}\!\to\!M_{S}$ )	98.3	1.7	0.0
MultiDoc2Dial (StudentAid)	Task-Guided ( $M_{S}\!\to\!M_{T}$ )	98.3	0.0	1.7
SWE-QA (pytest)	Self-Directed ( $M_{0}\!\to\!M_{S}$ )	93.3	5.0	1.7
SWE-QA (pytest)	Task-Guided ( $M_{S}\!\to\!M_{T}$ )	100.0	0.0	0.0
APIBench (HuggingFace)	Self-Directed ( $M_{0}\!\to\!M_{S}$ )	96.7	3.3	0.0
APIBench (HuggingFace)	Task-Guided ( $M_{S}\!\to\!M_{T}$ )	100.0	0.0	0.0

Appendix

Table 4 : Preservation of dropped units (%). Units removed by a learning stage are compared with the new model on the same entity (60 units per step). A claim is preserved if it remains represented after rewording, merging, or condensation.

	Hybrid RAG		$\bm{M_{0}}$		$\bm{M_{S}}$		$\bm{M_{T}}$ ( SourceLearn )
Benchmark	Seen	Unseen	Seen	Unseen	Seen	Unseen	Seen	Unseen
MultiDoc2Dial	$64.2$	$65.3$	$69.0_{\color[rgb]{0,0.65,0.31}\uparrow 4.8}$	$72.3_{\color[rgb]{0,0.65,0.31}\uparrow 7.0}$	$77.4_{\color[rgb]{0,0.65,0.31}\uparrow 13.2}$	$72.0_{\color[rgb]{0,0.65,0.31}\uparrow 6.7}$	$\mathbf{78.9_{\color[rgb]{0,0.65,0.31}\uparrow 14.7}}$	$\mathbf{76.7_{\color[rgb]{0,0.65,0.31}\uparrow 11.4}}$
SWE-QA	$57.3$	$56.4$	$70.3_{\color[rgb]{0,0.65,0.31}\uparrow 13.0}$	$61.1_{\color[rgb]{0,0.65,0.31}\uparrow 4.7}$	$66.2_{\color[rgb]{0,0.65,0.31}\uparrow 8.9}$	$64.6_{\color[rgb]{0,0.65,0.31}\uparrow 8.2}$	$\mathbf{72.9_{\color[rgb]{0,0.65,0.31}\uparrow 15.6}}$	$\mathbf{66.3_{\color[rgb]{0,0.65,0.31}\uparrow 9.9}}$
NarrativeQA	$65.8$	$66.2$	$74.2_{\color[rgb]{0,0.65,0.31}\uparrow 8.4}$	$74.0_{\color[rgb]{0,0.65,0.31}\uparrow 7.8}$	$77.2_{\color[rgb]{0,0.65,0.31}\uparrow 11.4}$	$78.4_{\color[rgb]{0,0.65,0.31}\uparrow 12.2}$	$\mathbf{80.1_{\color[rgb]{0,0.65,0.31}\uparrow 14.3}}$	$\mathbf{79.9_{\color[rgb]{0,0.65,0.31}\uparrow 13.7}}$
All QA	$62.4$	$62.9$	$71.6_{\color[rgb]{0,0.65,0.31}\uparrow 9.2}$	$69.6_{\color[rgb]{0,0.65,0.31}\uparrow 6.7}$	$73.3_{\color[rgb]{0,0.65,0.31}\uparrow 10.9}$	$72.2_{\color[rgb]{0,0.65,0.31}\uparrow 9.3}$	$\mathbf{77.3_{\color[rgb]{0,0.65,0.31}\uparrow 14.9}}$	$\mathbf{75.0_{\color[rgb]{0,0.65,0.31}\uparrow 12.1}}$

Appendix

Table 5 : Accuracy (%) by overlap with guidance-task regions (GPT-5.6-Luna). Seen regions overlap with at least one guidance task; unseen regions do not. Subscripts show changes relative to Hybrid RAG within each stratum.

Source	Cycle	Units	Observations	Accuracy (%)
SSA	0 ( $M_{0}$ )	571	–	73.3
	1 ( $M_{S}$ )	615	95	75.0
	2	660	53	80.0
	3	683	54	75.0
	4	688	45	73.3
StudentAid	0 ( $M_{0}$ )	478	–	73.3

Appendix

Table 6 : Self-Directed Source Learning over one to four cycles on two MultiDoc2Dial sources (GPT-5.6-Luna). Cycle 0 is

M_{0}

and cycle 1 is

M_{S}

. Observations are inspection notes produced in each cycle; accuracy is measured over the 60 questions of each source.

Explore similar work

MeMo: Memory as a Model
May 14, 2026Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6Large Language Model Memory
Institute of Data Science, National University of Singapore, Singapore · Integrative Sciences and Engineering Programme, NUSGS, Singapore · Agency for Science, Technology, Research (A*STAR), Singapore +6
When Does Continual Learning Require Learning
Jul 8, 2026Anne Harrington, Nayan Saxena, Michael Murphy +7Continual Learning In-Context Learning
1UC Berkeley · 2Independent · 3Capital Fund Management
Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
Jul 15, 2026Eric Hanchen Jiang, Zhi Zhang, Yuchen Wu +11Large Language Model Agents Graph-Structured Memories
University of California Los Angeles · University of Washington · Northwestern University

May 14, 2026, cs.CLJ/K move · Enter open · S save

MeMo: Memory as a Model

Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong, Arun Verma+5

Institute of Data Science, National University of Singapore, Singapore · Integrative Sciences and Engineering Programme, NUSGS, Singapore · Agency for Science, Technology, Research (A*STAR), Singapore+6

Large Language Model Memory

Open paper AskPDF arXiv

Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications require timely, domain-specific information, motivating the need for efficient mechanisms to incorporate new knowledge. In this paper, we introduce MeMo (Memory as a Model), a modular framework that encodes new knowledge into a dedicated memory model while keeping the LLM parameters unchanged. Compared to existing methods, MeMo offers several advantages: (a) it captures complex cross-document relationships, (b) it is robust to retrieval noise, (c) it avoids catastrophic forgetting in the LLM, (d) it does not require access to the LLM's weights or output logits, enabling plug-and-play integration with both open and proprietary closed-source LLMs, and (e) its retrieval cost is independent of corpus size at inference time. Our experimental results on three benchmarks, BrowseComp-Plus, NarrativeQA, and MuSiQue, show that MeMo achieves strong performance compared to existing methods across diverse settings.

Abstract

Explore similar work

MeMo: Memory as a Model

When Does Continual Learning Require Learning

Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents