Designing high-performance system heuristics is a creative, iterative process requiring experts to form hypotheses and execute multi-step conceptual shifts. While Large Language Models (LLMs) show promise in automating this loop, they struggle with complex system problems due to two critical failure modes: evolutionary neighborhood bias and the coherence ceiling. Evolutionary methods often remain trapped in local optima by relying on scalar benchmark scores, failing when coordinated multi-step changes are required. Conversely, existing agentic frameworks suffer from context degradation over long horizons or fail to accumulate knowledge across independent runs. We present Engram, an agentic researcher architecture that addresses these limitations by decoupling long-horizon exploration from the constraints of a single context window. Engram organizes exploration into a sequence of agents that iteratively design, test, and analyze mechanisms. At the conclusion of each run, an agent stores code snapshots, logs, and results in a persistent Archive and distills high-level modeling insights into a compact, persistent Research Digest. Subsequent agents then begin with a fresh context window, reading the Research Digest to build on prior discoveries. We find that Engram exhibits superior performance across diverse domains including multi-cloud multicast, LLM inference request routing, and optimizing KV cache reuse in databases with natural language queries.
Figures & tables
Figure 1. The three paradigms for LLM-based heuristic design. Evolutionary approaches with code mutation invoke an LLM with a predefined context format, mutating and selecting candidates based on scalar scores. Iterative design with flexible tool access (e.g., Glia) performs coherent experiment-guided exploration, but each exploration is restricted to a bounded LLM context window. Engram combines agent explorations with a shared research digest that persists insights across explorations ( §3 ), imporving persistence while preserving long-horizon coherence and flexibility.
Approach
Coherence
Flexibility
Persistence
Evolutionary code mutation
Low
Low
High
Iterative design via flexible tools
High
High
Low
Engram
High
High
High
Table 1. Trade-offs among LLM-based heuristic design. Code-mutation evolution is persistent is limited is coherence and flexibility; tool-based agents enable coherent and flexible exploration but have limited persistence. Engram achieves all three.
Figure 2. Engram’s design is based on a sequence of reasoning-based agent explorations that produce and evaluate ideas based on hypotheses driven from experimental data analysis. Each agent begins by analyzing the problem and reviewing the research digest summarizing the findings of the previous agents, using that information to formulate its own exploration and experimentation plan. The agent executes this plan through design, experimentation, and analysis. Upon completion, it writes a summary of its findings to the research digest, stores all the details in the Archive, and hands of the research process to the next agent. The process typically terminates when the research budget is exhausted.
Figure 3. Workspace of a single agent in Engram.
System
Parameter Name
Value
EoH
Initial population size (N)
20
FunSearch
# Prompt programs (k)
3
OpenEvolve (Vidur)
Number of islands
6
Exploration probability
0.6
Exploitation probability
0.4
Initial program
LLQ router
Table 2. Experimental settings and hyperparameters.
Figure 4. Multi-cloud data replication from a source (purple) to destinations (blue) across the world, either directly or via a waypoint (yellow) to avoid expensive or slow links. 4 4 4 Image reproduced by the authors inspired from Cloudcast ( Wooders et al., 2024 ) .
Figure 5. comparison for multi-cloud multicast with “Direction” prompt and o3 (lower is better). Engram achieves the strongest average best cost, outperforming both evolutionary approaches (EoH, FunSearch, OpenEvolve) and Glia. The whiskers show 90% confidence intervals.
Figure 6. Best cost-so-far versus number of simulations on multi-cloud multicast. OpenEvolve plateaus on simple heuristics; Glia makes progress but eventually exhausts its context/budget; Engram continues improving and finds solver-based solutions. The shades are 90% confidence intervals of best costs so far across runs.
Figure 7. Average and 90% confidence interval of best-cost 10 runs for multi-cloud multicast (OE: OpenEvolve, lower is better). Engram outperforms OpenEvolve in all settings.
Figure 8. Engram tolerates non-monotonic intermediate outcomes while staying within an optimization-based approach. Early attempts temporarily degrade cost ( 772→1104 ) before Engram recovers and achieves a large improvement ( 1104→644 ).
Figure 9. Best-response time comparison on the LLM request routing problem (lower is better). Engram achieves the strongest average best response time, outperforming evolutionary approaches (EoH, FunSearch, OpenEvolve), Glia, and Expert. The whiskers show 90% confidence intervals.
Figure 10. Average best score so far vs. number of simulations for all methods seeded with a simple baseline program for the LLM-SQL task. Engram, Glia, and OpenEvolve converge to comparable final scores with Engram being faster in convergence, while FunSearch and EoH lag behind. The GGR ( Liu et al., 2025a ) line shows the score of the state-of-the-art algorithm.
Strategy
CBL ↓
CBL-Multi ↓
EPLB ↑
Prism ↑
Telemetry ↑
TXN ↑
Human SOTA
101.7
92.3
0.251
21.89
0.822
2724.8
Engram
103.6 ± 1.1
79.9 ± 0.8
0.273 ± 0.00
27.94 ± 1.70
0.954 ± 0.00
3918.6 ± 56.6
OE ( Sharma, 2025 )
103.4 ± 0.9
79.9 ± 0.4
0.214 ± 0.06
26.21 ± 0.03
0.953 ± 0.00
3713.7 ± 77.9
Table 3. Average best scores across 10 runs with 90% confidence intervals (higher is better ↑ ; lower is better ↓ ). Engram exceeds Human SOTA on five of six tasks and improves over OpenEvolve on four.
Figure 11. Ablation on multi-cloud multicast (average best cost; lower is better). Single-agent variants (Single Agent and Summarization) are constrained by context growth and perform worst, while multi-agent variants improve upon them. Engram performs better than mere Sequential code transfer, highlighting the benefit of persistent knowledge. Removing either the Archive or Digest slightly worsen the performance.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12. Average score vs. number of simulations for all methods seeded with the GGR baseline program for the LLM-SQL task using the same prompt as the Simple case in Figure 10 .
Figure 13. Average score vs. number of simulations for all methods seeded with the GGR baseline program for the LLM-SQL task using the GGR-centered prompt used by ADRS ( Cheng et al., 2025a ) .
Strategy
Telemetry
EPLB
Prism
TXN
CBL
CBL-Multi
Multi-Cloud Multicast
Vidur
Engram
0.965
0.870
0.903
0.830
0.870
0.900
0.835
0.720
OE ( Sharma, 2025 )
0.822
0.650
0.795
0.760
0.820
0.940
0.934
0.680
Appendix
Table 4. Median normalized AUC (area under the normalized best-so-far progress curve) across tasks (higher is better; larger values indicate faster improvement toward the final best score).
Figure 14. Cost distribution for 100 programs generated via single-shot LLM prompting (lower is better). We compare against Direct (independent cost-shortest paths), MinST (Steiner-tree heuristic), and the human SOTA (Cloudcast ( Wooders et al., 2024 ) ).
Figure 15. System-prompt ablation on multi-cloud multicast. Removing the Struggle protocol worsens performance, shifting the cost distribution upward relative to the full prompt.
Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows. Despite rapid architectural development, the empirical foundations of these systems remain fragile: existing benchmarks are often underscaled, evaluation metrics are misaligned with semantic utility, performance varies significantly across backbone models, and system-level costs are frequently overlooked. This survey presents a structured analysis of agentic memory from both architectural and system perspectives. We first introduce a concise taxonomy of MAG systems based on four memory structures. Then, we analyze key pain points limiting current systems, including benchmark saturation effects, metric validity and judge sensitivity, backbone-dependent accuracy, and the latency and throughput overhead introduced by memory maintenance. By connecting the memory structure to empirical limitations, this survey clarifies why current agentic memory systems often underperform their theoretical promise and outlines directions for more reliable evaluation and scalable system design.
Dongming Jiang, Yi Li, Songtao Wei +8
University of Texas at Dallas · University of California, Davis · Texas A&M University
The rapid evolution of Large Language Model (LLM) agents has necessitated robust memory systems to support cohesive long-term interaction and complex reasoning. Benefiting from the strong capabilities of LLMs, recent research focus has shifted from simple context extension to the development of dedicated agentic memory systems. However, existing approaches typically rely on rigid retrieval granularity, accumulation-heavy maintenance strategies, and coarse-grained update mechanisms. These design choices create a persistent mismatch between stored information and task-specific reasoning demands, while leading to the unchecked accumulation of logical inconsistencies over time. To address these challenges, we propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities. AMA employs a hierarchical memory design that dynamically aligns retrieval granularity with task complexity. Specifically, the Constructor and Retriever jointly enable multi-granularity memory construction and adaptive query routing. The Judge verifies the relevance and consistency of retrieved content, triggering iterative retrieval when evidence is insufficient or invoking the Refresher upon detecting logical conflicts. The Refresher then enforces memory consistency by performing targeted updates or removing outdated entries. Extensive experiments on challenging long-context benchmarks show that AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods, demonstrating its effectiveness in maintaining retrieval precision and long-term memory consistency.
Weiquan Huang, Zixuan Wang, Hehai Lin +6
The Hong Kong University of Science and Technology (Guangzhou) · Shandong University · Nanyang Technological University +1
As autonomous LLM agents are deployed across multi-session environments, conventional memory architectures suffer from Associative Blindness (inability to traverse multi-hop relational dependencies), Scaffolding Amnesia (temporal decay evicting core persona invariants), and Static Topology Stagnation (immutable graphs ignoring usage dynamics). Grounded in Complementary Learning Systems (CLS) principles, we propose EngramRAG, an adaptive memory architecture coupling a low-latency Waking State reflex with an asynchronous background Dreaming State consolidation cycle. EngramRAG introduces: (1) Usage-Modulated Personalized PageRank (U-PPR), where transition probabilities adapt via Hebbian plasticity to promote persistent entities into high-centrality Epistemic Macro-Hubs; (2) Consolidation-Activated Topology Decay (CATD), which scales retention half-life by topological load-bearing weight rather than wall-clock recency, protected by a cold-start grace period (N_grace >= 4); (3) Directed SUPERSEDES DAG filtering to suppress obsolete state during fact mutations; and (4) Triple-source hybrid retrieval fusing dense vectors, BM25, and U-PPR via dynamic Reciprocal Rank Fusion (RRF). Evaluating on all 1,982 QA pairs across 10 long-term conversations in the LoCoMo benchmark, EngramRAG achieves +38.9% relative improvement in Recall@5 (53.21% vs. 38.29%, p < 0.001) and +43.1% in MRR (0.4203 vs. 0.2937) over dense vector RAG, significantly outperforming Okapi BM25 (48.66%) and isolated static graph retrieval (8.50%). On temporal reasoning, EngramRAG reaches 62.33% Recall@5 (+16.67 points over dense vectors). In controlled mutation tests, SUPERSEDES suppresses split-brain hallucinations from 70.0% to 0.0%, while 90-day simulations show 100.0% scaffolding retention under a 26.21ms interactive retrieval reflex.