cs.CLOct 7, 2026

ExperienceIndex: Artifact-Grounded Memory

Authors: Peter Baile Chen, Geoffrey X. Yu, Xinming Liu, Samuel Madden, Dan Roth, Jacob Andreas, Doug Downey, Michael Cafarella

Organizations: MIT · McKinsey & Company · Oracle AI & UPenn · AI2

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Knowledge-intensive tasks require answering many questions by reasoning about a shared corpus of artifacts (e.g., court cases, or scientific literature). As humans interact with these corpora, they naturally accumulate experiential knowledge about artifacts, enabling them to quickly identify the complete set of relevant artifacts for each new task. However, existing AI agents lack appropriate memory solutions to build or reuse such artifact-grounded experience, leading to lower answer quality and higher online cost. Existing memory solutions extract and reuse information from prior task-solving traces, but they primarily focus on user preferences, factual attributes, or abstract reasoning patterns rather than persistent artifact-specific knowledge. We introduce ExperienceIndex, a novel experience layer for AI agents that captures and reuses knowledge about artifacts based on prior reasoning traces. ExperienceIndex stores two complementary forms of experience: (i) single-artifact experiences that summarize an artifact's contribution to prior tasks and (ii) artifact-pair experiences that encode structural relationships discovered during past reasoning. Integrated as lightweight middleware, ExperienceIndex uses an experience retrieval mechanism to guide agents toward the complete set of relevant artifacts for new tasks, improving both answer quality and efficiency. Across diverse corpora and agentic solutions with different search frameworks, ExperienceIndex delivers consistent gains, raising answer quality by up to 11.0 points and reducing online dollar cost by up to 50.5%. We further demonstrate two benefits: (i) cross-task generalization, where experiences accumulated from text-to-SQL tasks transfer to factoid QA tasks over the same artifact corpus, and (ii) teacher-student learning, where experiences from a stronger model enable a weaker model to reach comparable performance.

Figures & tables

Figure 1: ExpIdx (green) and its integration into an agentic solution as middleware. Without ExpIdx , an agentic solution iteratively explores the artifact corpus for each user task using search tools, possibly in many rounds, to gather the complete information needed to solve the task (an error - prone and inefficient process). ExpIdx optimizes this process by giving the agent the full set of relevant artifacts derived from past experiences, improving answer quality while reducing execution cost.
Algorithm 1 ExpIdx integrated into a standard agentic solution with a solver model M\mathcal{M} , and a search framework R\mathcal{R} (e.g., vector_search) over a corpus C\mathcal{C} of artifacts A\mathcal{A} ( A\textsubscriptID\mathcal{A}\textsubscript{ID} refers to the artifact_id s). Green text shows additions introduced by ExpIdx .
DatasetDomainModalityTask typeBaseline solutionSearch frameworks
Locomo ( Maharana et al., 2024 )GeneralFilesConversational QAReAct ( Yao et al., 2022 )File-system search (e.g., bash )
SWE-bench-ver.     
( Jimenez et al., 2023 )TechFilesBug fixesReActFile-system search
SQA ( Bragg et al., 2025 )ScienceDocumentsLong-form generationScholarQA ( Singh et al., 2025 )Custom retrieve-and-rerank to Semantic Scholar
Freshstack ( Thakur et al., 2025 )Technical doc.DocumentsLong-form generationReActVector search tool ( Qwen3-0.6B-Embedding )
Musique ( Trivedi et al., 2022 )GeneralDocumentsMulti-hop QAReActVector search tool ( Qwen3-8B-Embedding )
Table 1: Details on datasets and baseline solutions.
Figure 4
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Table 7: Prompt for analyzing artifact relevance.
Table 8: System prompt for injecting retrieved experiences.
Table 9: Prompt for generating contribution summary and relationships.
Figure 3: An example Locomo task where ExpIdx surfaces files and snippets from relevant prior single-artifact experiences, which helps the agent efficiently reach the correct answer.
Figure 4: An example MMQA task where ExpIdx surfaces a table the vector search missed ( Products_eafb ), which ultimately helps the agent reach the correct answer.
Table 10: Standard agentic soluton on an example Locomo task.
Table 11: ExpIdx on the same Locomo example as Table 10 . Green text refers to changes due to ExpIdx .
Table 12: On MMQA, the result becomes correct once the new tables introduced through edge connections by ExpIdx are incorporated. Green text refers to changes due to ExpIdx .

Explore similar work

CardsList
May 29, 2026cs.CL

Eywa: Provenance-Grounded Long-Term Memory for AI Agents

AI agents that persist across sessions need memory they can retrieve, audit, update, and erase. Existing memory systems often collapse source evidence, extracted facts, retrieved context, and answer policy into one opaque prompt path, making failures difficult to diagnose: a wrong answer may come from missing evidence, unsupported extraction, stale state, retrieval loss, or answer-model behavior. We present Eywa, a provenance-grounded memory architecture built around evidence before belief. Eywa stores immutable source evidence before deriving canonical facts, validates extracted memories against typed signals and source support, and retrieves bounded memory context through a deterministic multi-route read path with zero LLM calls inside retrieval. Retrieved context is returned separately from answer instructions, allowing the same memory substrate to be evaluated across frontier, budget, and local answer models. Under a frozen, artifact-recorded retrieval configuration, Eywa reaches 90.19% judge accuracy on the LoCoMo C1-C4 split with Claude Sonnet 4.6 write and QA roles. On LongMemEval-S, it reaches 88.2% retrieval-sufficiency accuracy. On BEAM, a 700-question technical-memory stress benchmark, it reaches 81.45% mean nugget score and 85.29% pass@score >= 0.5. Full per-question artifacts, including questions, gold answers, model answers, retrieved context, and labels, are published at https://eywa.to/research.

Resham Joshi

Eywa

Long-Term MemoryLong-TermLoss Function
AskPDF
Sep 29, 2026cs.AI

KUPAS MASTER: Distilling the Tacit Expertise of Master Practitioners into Agent-Ready Experience Corpora

Experienced professionals know more than just facts and conclusions. They know which cues matter, why a judgment is reasonable, and which action to take. Routine work records often leave out this tacit knowledge, making it difficult for Large Language Model (LLM) agents to use professional experience effectively. We introduce KUPAS MASTER, an experience engineering platform built around nine-layer cognitive corpus construction. It turns heterogeneous work records and practitioner interviews into traceable, reusable experience corpora for agents. Six case elements preserve the task process: context, cues, judgment, action, boundaries, and outcomes. Nine-layer cognitive corpus construction organizes tacit experience along nine extraction dimensions and stores the resulting assets in six libraries: rules, constraints, best practices, negative examples, corner cases, and skills. Semantic alignment, individual experience distillation, organizational consolidation, and cross-review preserve source evidence, conditions of use, and unresolved disagreements. The platform packages these assets into callable skills with explicit inputs, steps, dependencies, and stopping conditions, connecting experience collection to task execution and evaluation feedback. Using authorized samples from 20 randomly selected practitioners, the platform processed 1,576 source files into 23,024 individual experience records and 13,113 organizational assets. The evaluation spans multiple professional domains. Under common task inputs and scoring criteria, the base model, raw corpus retrieval-augmented generation (RAG), and KUPAS MASTER agent scored 70.63, 79.75, and 89.58, respectively. The KUPAS MASTER agent improved on raw-corpus RAG in all seven scoring dimensions. The platform provides a practical path from individual tacit experience to organizational knowledge and agent capabilities.

Changmian Wang, Yuchao Ma, Xuchao Lu +13

Shanghai Kupas Technology Co., Ltd. and Tongji University

Expert DemonstrationsText Corpora
AskPDF
Jun 2, 2026cs.AI

DELTAMEM: Incremental Experience Memory for LLM Agents via Residual Trees

Large Language Model (LLM)-based agents increasingly rely on memory to learn from experiences over continual interactions. However, storing experiences as independent, flat units leads to substantial redundancy and retrieval conflicts, as similar episodes repeat overlapping content and subtle scene variations cause retrieved memories to offer contradictory guidance. To address this, we introduce residual experience, positing that newly acquired experience is often an incremental variation of existing knowledge. We propose DeltaMem, a framework that organizes experience memory into two independent residual trees, one storing goal-conditioned task experience as reusable skills and another for scene-level environment knowledge. Each tree uses a root node for generalized base experiences and incremental delta nodes for subsequent variations, allowing related experiences to share a common foundation without duplication. For retrieval, a failure-penalized similarity scan locates the best match, reconstructing the full experience via root-to-match chain composition. An autonomous consolidation mechanism distills high-frequency paths into new root nodes, enabling the trees to self-organize from general heuristics to specialized variants. Experiments across diverse interactive environments show that DeltaMem consistently outperforms existing baselines. To facilitate future research, we release the code at https://github.com/import-myself/DeltaMem.

Haoran Tan, Zeyu Zhang, Zhicheng Cao +2

Gaoling School of Artificial, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +1

Large Language Model AgentsGated DeltanetScene Reasoning
AskPDF