Enterprise LLM agents operate in dynamic environments where foundation models evolve rapidly, workflows update frequently, and deployments need to satisfy accuracy and cost requirements. We propose ARISE (Agent Refinement through Iterative Self-Evaluation), a practical memory architecture in which the agent's memory evolves from its own failures without updating model weights, tools, or workflow logic. ARISE uses the model itself or a teacher model to attribute the failures and propose corrections. Then, ARISE keeps the corrections as retrievable micro memory, consolidates them into macro rules, and accepts a rule update only if held-out accuracy rises. Across 14 enterprise-workflow domains from SOP-Bench and τ2-bench using 5 models, ARISE-Teacher, which uses Claude Opus 4.6 for attribution and consolidation, gains 10.9 pp over the baseline on SOP-Bench and 5.8 pp on τ2-bench. ARISE-Teacher also outperforms GEPA by 4.5 pp on τ2-bench and 1.4 pp on SOP-Bench. With the deployment model as its own attributor, ARISE-Self still gains 10.0 pp on SOP-Bench. ARISE-Teacher uses 4.0% fewer model calls and 5.2% fewer output tokens at comparable per-task cost (-1.0%).
Figures & tables
Figure 1: Overall accuracy over the 14 domains plotted against SOP-Bench cost per task, per model. The overall accuracy is the 11:3 blend of the suite averages in Table 2 .
Figure 2: ARISE architecture. The deployed model weights, tools, and workflow logic stay fixed. An offline loop attributes failed tasks using the trace, the output, the post-task outcome and diagnostic feedback, stores micro corrections, consolidates macro rules, and accepts a rule update only if held-out validation accuracy improves.
Method
Attribution
Instance-level
Generalized
Summarization
Validation
Injection
ExpeL
Self
✓
✓
Insight update
×
Insights
GEPA
Teacher
×
✓
Evolution
Pareto
Optimized prompt
ARISE-Self
Self
✓
✓
Bounded rewrite
✓
Top- k + ≤K rules
ARISE-Teacher
Teacher
✓
✓
Bounded rewrite
✓
Top- k + ≤K rules
Table 1: Design comparison of ARISE with ExpeL and GEPA. Instance-level : task-specific experience retrieval. Generalized : rules applied across tasks.
Figure 3: Best SOP-Bench validation accuracy so far after each training iteration, averaged over training runs and models. Iteration 0 is the original prompt on the validation set.
Model
Accuracy (%)
ARISE-Self over Baseline
ARISE-Teacher over Baseline
Base
ExpeL
Refl.
GEPA
Self
Teacher
Δ Acc.
Δ Tok.
Δ Calls
Δ Cost
Δ Acc.
Δ Tok.
Δ Calls
Δ Cost
SOP-Bench (11 domains; one test pass per trained configuration)
Qwen3-32B
65.9
68.2
67.0
72.8
78.4
78.1
+12.5
− 7.4
− 8.9
− 7.6
+12.2
− 6.2
− 8.6
− 6.6
DeepSeek v3.2
76.6
78.3
76.6
89.2
84.3
88.6
+7.7
+0.8
− 2.3
+0.8
+12.0
+6.7
− 0.9
+6.4
Haiku 4.5
79.2
74.7
78.8
88.3
87.2
88.0
+8.0
+5.7
+0.7
+3.6
+8.8
+0.8
− 2.6
− 0.7
Sonnet 4.6
79.2
75.0
80.2
87.3
89.4
89.2
+10.2
+2.2
− 2.2
+1.1
+10.0
+3.6
− 2.1
+2.4
Table 2: Accuracy (%) and changes from baseline (equal-domain averages, overall weights domains 11:3). ARISE values average three runs. On Opus 4.6 the two settings coincide, so the ARISE-Self values repeat the ARISE-Teacher values. GEPA SOP-Bench: per-task re-evaluation. τ2 -bench evaluation trials are described in § 4.1 . Δ Acc. is in pp from unrounded values. Δ Tok. (input+output), Δ Calls and Δ Cost are in %. Bold: best. Underline: second.
Figure 4: Prompts that write and serve the memory. The reference notes appear only when the micro gate passed. On τ , {expected} holds the failed action checks, and the two memory sections are added to the policy prompt.
Domain
Metric
Base
GEPA
Self
Teacher
Airline
pass^1
55.2
55.2
56.2
58.8
pass^4
41.0
38.0
40.3
44.3
Retail
pass^1
48.1
46.1
49.1
48.5
pass^4
34.0
28.0
39.8
38.0
Telecom
pass^1
66.1
71.8
72.2
79.5
pass^4
44.5
52.0
55.8
60.7
Table 3: Reliability on τ . ARISE-Self values use four trials from its first training run. ARISE-Teacher values average metrics from four trials computed separately within three training runs.
Figure 8
Model
Run ($)
1M tasks ($)
Tasks
Share
Qwen3-32B
1.49
3,142
476
0.048%
DeepSeek v3.2
6.98
19,157
364
0.036%
Haiku 4.5
6.76
21,052
321
0.032%
Sonnet 4.6
29.85
94,176
317
0.032%
Opus 4.6
42.20
133,655
316
0.032%
Mean
17.46
54,236
322
0.032%
Table 4: One-time adaptation cost of one ARISE-Teacher run and inference cost of serving one million SOP-Bench tasks with the adapted agent. Tasks : served tasks whose inference cost equals one run. Share : run cost as a percentage of serving 1M tasks. Values are means over 33 runs per model (11 domains, three runs). The mean row uses the 5-model mean costs.
Figure 7: Case studies with Opus 4.6. ARISE outputs come from run 3. Before is the same run with retrieval switched off in Case 1 and the baseline in Case 2. Red marks the wrong answer and green the correct one. Excerpts are verbatim, and […] marks cuts.
Hyperparameter
Low
Def.
High
Retrieval policy (k,θ)
85.6
87.4
88.2
Batch size B (10 / 30 / 50)
86.3
87.4
87.6
Train/val split (60/40 / 70/30 / 80/20)
87.0
87.4
88.1
Budget kmax×B (3 × 50 / 5 × 30 / 15 × 10)
87.3
87.4
87.6
Macro threshold ϵ0 (1 / 2 / 5)
88.5
87.4
87.7
Table 5: SOP-Bench accuracy (%) under one-at-a-time changes (low / default / high). Retrieval uses θ=0.8 at k=3 and θ=0.6 otherwise. Bold marks the default configuration rather than the best result.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Models
Opus 4.6, Sonnet 4.6, Haiku 4.5, DeepSeek v3.2 and Qwen3-32B via Amazon Bedrock
Temperature
0 for the agent and ARISE, 1.0 for GEPA
Simulator
Opus 4.6, temperature 1.0
Data splits
SOP-Bench train and test split by row at 70/30. τ2 official splits 30/20, 74/40 and 74/40. Training sets shuffled with seed 42, then split 70/30 for iteration and validation
Validation size
SOP-Bench 14 to 58, airline 9, retail 23, telecom 23
Table 7: Accuracy differences of the averages in Table 2 (A / B: A minus B in pp), with p values in parentheses.
pass^k (all k )
pass@ k ( ≥ 1 of k )
Domain
Method
k =1
2
3
4
2
3
4
Airline
Baseline
55.2
46.8
43.0
41.0
63.7
68.2
71.0
GEPA
55.2
46.8
41.8
38.0
63.7
67.0
69.0
Macro
56.8
48.0
42.2
38.0
65.5
68.5
70.0
ARISE-T
58.8
51.4
47.2
44.3
66.1
69.2
71.0
Retail
Baseline
48.1
40.8
36.8
34.0
55.5
58.9
61.0
Appendix
Table 8: Full τ consistency (%), averaged over 5 models. pass^k: solved in all k trials; pass@ k : solved in at least one of k trials. Best per domain in bold .
Baseline
ARISE-Teacher
Model
Domain
p^1
p^4
p@4
p^1
p^4
p@4
Δ p^4
Opus 4.6
Airline
78.8
60.0
90.0
77.5
61.7
88.3
+ 1.7
Retail
67.5
62.5
75.0
66.9
64.2
69.2
+ 1.7
Telecom
83.1
67.5
100.0
98.3
94.2
100.0
+ 26.7
Sonnet 4.6
Airline
81.2
75.0
90.0
77.5
68.3
86.7
− 6.7
Retail
58.8
45.0
70.0
60.2
51.7
69.2
+ 6.7
Appendix
Table 9: Per-model τ consistency (%): Baseline vs. ARISE-Teacher on pass^1, pass^4 and pass@4. In the column headings, p abbreviates pass; the last column is the change in pass^4.
SOP-Bench
τ
Method
Model
InTok
OutTok
Calls
Cost
InTok
OutTok
Calls
Cost
Baseline
Qwen3-32B
21,105
333
5.28
0.0034
135,231
1,260
16.74
0.0210
DeepSeek v3.2
25,695
1,121
6.06
0.0180
119,780
2,267
14.25
0.0785
Haiku 4.5
13,570
1,141
2.89
0.0212
113,758
2,106
12.06
0.1367
Sonnet 4.6
20,461
1,479
3.96
0.0919
103,453
1,529
11.69
0.3666
Opus 4.6
19,595
1,157
3.79
0.1396
103,803
1,651
11.36
0.6163
Appendix
Table 10: Per-task efficiency: average input tokens, output tokens, model calls and cost ()permodelandmethod.SOP−Benchisaveragedover11domainsand\tau$ over 3 domains. GEPA’s SOP-Bench values come from the test pass of its own run. On Opus 4.6 the ARISE-Self row repeats ARISE-Teacher.
Domain
Model
Baseline
ExpeL
Reflexion
GEPA
Macro
Self
ARISE-T
Aircraft Insp.
Opus 4.6
97.1
88.2
97.1
97.1
97.1
96.9
96.9
Sonnet 4.6
67.6
97.1
67.6
94.1
97.1
97.1
97.0
Haiku 4.5
97.1
97.1
97.1
97.1
97.1
97.1
97.1
DeepSeek v3.2
79.4
73.5
85.3
94.1
91.2
85.3
86.3
Qwen3-32B
97.1
97.1
97.1
94.1
97.1
96.1
95.1
Content Flag.
Opus 4.6
37.3
62.7
43.2
92.2
94.1
92.8
92.8
Appendix
Table 11: Full SOP-Bench accuracy (%) per model, domain and method. ARISE-Teacher (ARISE-T) and ARISE-Self (Self) average three training runs, and on Opus 4.6 the Self cells repeat ARISE-T. Baseline, ExpeL, Reflexion, GEPA and Macro use one run. GEPA is the per-task re-evaluation of its best prompt (§ 4.1 ).
Domain
Model
Baseline
ExpeL
Reflexion
GEPA
Macro
Self
ARISE-T
Airline
Opus 4.6
78.8
78.8
75.0
75.0
73.8
77.5
77.5
Sonnet 4.6
81.2
78.8
75.0
77.5
72.5
76.7
77.5
Haiku 4.5
55.0
61.2
62.5
56.2
62.5
51.7
64.6
DeepSeek v3.2
41.2
48.8
52.5
50.0
48.8
46.7
50.4
Qwen3-32B
20.0
23.8
20.0
17.5
26.2
18.3
23.8
Retail
Opus 4.6
67.5
65.0
63.8
62.5
64.4
66.9
66.9
Appendix
Table 12: Full τ accuracy (%) per model, domain and method. ARISE-Self uses one pass from each of three runs, except Opus 4.6, which reuses the identical Teacher evaluations. Other methods use four trials, with ARISE-Teacher averaged over three runs.
Agent memory architectures define how LLM agents store and reuse experience across tasks, but they are typically fixed after deployment, limiting their adaptability across task types. Recent self-evolving approaches instead adapt these memory architectures using execution feedback, typically represented as evaluation scores or compressed summaries. However, such coarse-grained feedback is insufficient for precise failure localization in current agent systems, where long-horizon task execution involves multiple interacting components. Without a precise failure location, existing approaches lack a clear code-level target for modification, leaving the update scope weakly constrained and often leading to large-scale rewrites of an otherwise capable memory architecture. We propose MemScope, a framework for evidence-grounded, localized evolution of an existing memory architecture. Its tool-using Patch Agent jointly inspects raw execution traces, runtime memory state, and current architecture code to identify repair targets and apply bounded, in-place edits under provider-identity and memory-compatibility constraints. On GAIA, the trace-and-memory inspection variant achieves 86.67% accuracy, compared with 70.00% for trace-only and 63.33% for summary-only feedback. Under xBench tasks, the evolved memory architecture improves over its original version from 49 to 55 correct answers with DeepSeek V3.2 and from 50 to 53 with Kimi K2.7 Code. On disjoint ALFWorld unseen tasks, success rises from 50.75% to 71.64%. Across archived architecture-snapshot transitions, the median code-change ratio is 4.4%. These results support fine-grained execution evidence as a basis for localized, in-place memory-architecture evolution.
Miao Su, Yucan Guo, Zixuan Li +4
State Key Laboratory of AI Safety · Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +1
Long-term memory is essential for LLM agents that operate across multiple sessions, yet existing memory systems treat retrieval infrastructure as fixed: stored content evolves while scoring functions, fusion strategies, and answer-generation policies remain frozen at deployment. We argue that truly adaptive memory requires co-evolution at two levels: the stored knowledge and the retrieval mechanism that queries it. We present EvolveMem, a self-evolving memory architecture that exposes its full retrieval configuration as a structured action space optimized by an LLM-powered diagnosis module. In each evolution round, the module reads per-question failure logs, identifies root causes, and proposes targeted configuration adjustments; a guarded meta-analyzer applies them with automatic revert-on-regression and explore-on-stagnation safeguards. This closed-loop self-evolution realizes an AutoResearch process: the system autonomously conducts iterative research cycles on its own architecture, replacing manual configuration tuning. Starting from a minimal baseline, the process converges autonomously, discovering effective retrieval strategies including entirely new configuration dimensions not present in the original action space. On LoCoMo, EvolveMem outperforms the strongest baseline by 25.7% relative and achieves a 78.0% relative improvement over the minimal baseline. On MemBench, EvolveMem exceeds the strongest baseline by 18.9% relative. Evolved configurations transfer across benchmarks with positive rather than catastrophic transfer, indicating that the self-evolution process captures universal retrieval principles rather than benchmark-specific heuristics. Code is available at https://github.com/aiming-lab/SimpleMem.
Since large language model (LLM) agents rely on long-term memory, what they store now shapes what they do later. A memory manager decides how each input changes this memory and which records the agent acts on. It can fail in two ways: keeping a fact that has since changed (outdated memory), or storing a planted passage or injected instruction (memory poisoning). Most prior work studies them separately, yet they call for opposite write decisions, so no fixed write rule handles both. We therefore formalize memory robustness as handling both without doing worse on other inputs. Since accuracy checks only the answer, we measure robustness with evidence-conditioned accuracy (ECA), which also checks that the records the manager selects include all the evidence the answer needs, updated and free of attacks. However, existing memory systems and defenses are not more robust than an untuned manager on our test sets. We improve robustness with a simple one-shot recipe that fine-tunes the manager on episodes synthesized around a single conversation with inserted changes and attacks. On three backbones, it improves every category score on our test sets and more than doubles the untuned manager's overall ECA, though robustness remains incomplete.
Runze Zhao, Dongruo Zhou, Sumit Kumar Jha +2
Indiana University Bloomington · University of Florida · United States Military Academy