ARISE: Practical Self-Evolving Memory for Accurate and Cost-Efficient Enterprise Agents
Organizations: Amazon
Abstract
Enterprise LLM agents operate in dynamic environments where foundation models evolve rapidly, workflows update frequently, and deployments need to satisfy accuracy and cost requirements. We propose ARISE (Agent Refinement through Iterative Self-Evaluation), a practical memory architecture in which the agent's memory evolves from its own failures without updating model weights, tools, or workflow logic. ARISE uses the model itself or a teacher model to attribute the failures and propose corrections. Then, ARISE keeps the corrections as retrievable micro memory, consolidates them into macro rules, and accepts a rule update only if held-out accuracy rises. Across 14 enterprise-workflow domains from SOP-Bench and -bench using 5 models, ARISE-Teacher, which uses Claude Opus 4.6 for attribution and consolidation, gains 10.9 pp over the baseline on SOP-Bench and 5.8 pp on -bench. ARISE-Teacher also outperforms GEPA by 4.5 pp on -bench and 1.4 pp on SOP-Bench. With the deployment model as its own attributor, ARISE-Self still gains 10.0 pp on SOP-Bench. ARISE-Teacher uses 4.0% fewer model calls and 5.2% fewer output tokens at comparable per-task cost (-1.0%).
Figures & tables
| Method | Attribution | Instance-level | Generalized | Summarization | Validation | Injection |
|---|---|---|---|---|---|---|
| ExpeL | Self | Insight update | Insights | |||
| GEPA | Teacher | Evolution | Pareto | Optimized prompt | ||
| ARISE-Self | Self | Bounded rewrite | Top- + rules | |||
| ARISE-Teacher | Teacher | Bounded rewrite | Top- + rules |
| Model | Accuracy (%) | ARISE-Self over Baseline | ARISE-Teacher over Baseline | |||||||||||
| Base | ExpeL | Refl. | GEPA | Self | Teacher | Acc. | Tok. | Calls | Cost | Acc. | Tok. | Calls | Cost | |
| SOP-Bench (11 domains; one test pass per trained configuration) | ||||||||||||||
| Qwen3-32B | 65.9 | 68.2 | 67.0 | 72.8 | 78.4 | 78.1 | +12.5 | 7.4 | 8.9 | 7.6 | +12.2 | 6.2 | 8.6 | 6.6 |
| DeepSeek v3.2 | 76.6 | 78.3 | 76.6 | 89.2 | 84.3 | 88.6 | +7.7 | +0.8 | 2.3 | +0.8 | +12.0 | +6.7 | 0.9 | +6.4 |
| Haiku 4.5 | 79.2 | 74.7 | 78.8 | 88.3 | 87.2 | 88.0 | +8.0 | +5.7 | +0.7 | +3.6 | +8.8 | +0.8 | 2.6 | 0.7 |
| Sonnet 4.6 | 79.2 | 75.0 | 80.2 | 87.3 | 89.4 | 89.2 | +10.2 | +2.2 | 2.2 | +1.1 | +10.0 | +3.6 | 2.1 | +2.4 |
| Domain | Metric | Base | GEPA | Self | Teacher |
|---|---|---|---|---|---|
| Airline | pass^1 | 55.2 | 55.2 | 56.2 | 58.8 |
| pass^4 | 41.0 | 38.0 | 40.3 | 44.3 | |
| Retail | pass^1 | 48.1 | 46.1 | 49.1 | 48.5 |
| pass^4 | 34.0 | 28.0 | 39.8 | 38.0 | |
| Telecom | pass^1 | 66.1 | 71.8 | 72.2 | 79.5 |
| pass^4 | 44.5 | 52.0 | 55.8 | 60.7 |
| Model | Run ($) | 1M tasks ($) | Tasks | Share |
|---|---|---|---|---|
| Qwen3-32B | 1.49 | 3,142 | 476 | 0.048% |
| DeepSeek v3.2 | 6.98 | 19,157 | 364 | 0.036% |
| Haiku 4.5 | 6.76 | 21,052 | 321 | 0.032% |
| Sonnet 4.6 | 29.85 | 94,176 | 317 | 0.032% |
| Opus 4.6 | 42.20 | 133,655 | 316 | 0.032% |
| Mean | 17.46 | 54,236 | 322 | 0.032% |
| Hyperparameter | Low | Def. | High |
|---|---|---|---|
| Retrieval policy | 85.6 | 87.4 | 88.2 |
| Batch size (10 / 30 / 50) | 86.3 | 87.4 | 87.6 |
| Train/val split (60/40 / 70/30 / 80/20) | 87.0 | 87.4 | 88.1 |
| Budget (3 50 / 5 30 / 15 10) | 87.3 | 87.4 | 87.6 |
| Macro threshold (1 / 2 / 5) | 88.5 | 87.4 | 87.7 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Models | Opus 4.6, Sonnet 4.6, Haiku 4.5, DeepSeek v3.2 and Qwen3-32B via Amazon Bedrock |
| Temperature | 0 for the agent and ARISE, 1.0 for GEPA |
| Simulator | Opus 4.6, temperature 1.0 |
| Data splits | SOP-Bench train and test split by row at 70/30. official splits 30/20, 74/40 and 74/40. Training sets shuffled with seed 42, then split 70/30 for iteration and validation |
| Validation size | SOP-Bench 14 to 58, airline 9, retail 23, telecom 23 |
| Memory | 5 macro rules, 5 retrieved corrections, cosine threshold |
| Comparison | SOP | Overall | |
|---|---|---|---|
| Teacher / Base | +10.9 ( .01) | +5.8 ( .01) | +9.8 ( .01) |
| Self / Base | +10.0 ( .01) | +2.6 (.05) | +8.4 ( .01) |
| Teacher / GEPA | +1.4 (.02) | +4.5 ( .01) | +2.0 ( .01) |
| pass^k (all ) | pass@ ( 1 of ) | |||||||
|---|---|---|---|---|---|---|---|---|
| Domain | Method | =1 | 2 | 3 | 4 | 2 | 3 | 4 |
| Airline | Baseline | 55.2 | 46.8 | 43.0 | 41.0 | 63.7 | 68.2 | 71.0 |
| GEPA | 55.2 | 46.8 | 41.8 | 38.0 | 63.7 | 67.0 | 69.0 | |
| Macro | 56.8 | 48.0 | 42.2 | 38.0 | 65.5 | 68.5 | 70.0 | |
| ARISE-T | 58.8 | 51.4 | 47.2 | 44.3 | 66.1 | 69.2 | 71.0 | |
| Retail | Baseline | 48.1 | 40.8 | 36.8 | 34.0 | 55.5 | 58.9 | 61.0 |
| Baseline | ARISE-Teacher | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Domain | p^1 | p^4 | p@4 | p^1 | p^4 | p@4 | p^4 |
| Opus 4.6 | Airline | 78.8 | 60.0 | 90.0 | 77.5 | 61.7 | 88.3 | 1.7 |
| Retail | 67.5 | 62.5 | 75.0 | 66.9 | 64.2 | 69.2 | 1.7 | |
| Telecom | 83.1 | 67.5 | 100.0 | 98.3 | 94.2 | 100.0 | 26.7 | |
| Sonnet 4.6 | Airline | 81.2 | 75.0 | 90.0 | 77.5 | 68.3 | 86.7 | 6.7 |
| Retail | 58.8 | 45.0 | 70.0 | 60.2 | 51.7 | 69.2 | 6.7 | |
| SOP-Bench | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Model | InTok | OutTok | Calls | Cost | InTok | OutTok | Calls | Cost |
| Baseline | Qwen3-32B | 21,105 | 333 | 5.28 | 0.0034 | 135,231 | 1,260 | 16.74 | 0.0210 |
| DeepSeek v3.2 | 25,695 | 1,121 | 6.06 | 0.0180 | 119,780 | 2,267 | 14.25 | 0.0785 | |
| Haiku 4.5 | 13,570 | 1,141 | 2.89 | 0.0212 | 113,758 | 2,106 | 12.06 | 0.1367 | |
| Sonnet 4.6 | 20,461 | 1,479 | 3.96 | 0.0919 | 103,453 | 1,529 | 11.69 | 0.3666 | |
| Opus 4.6 | 19,595 | 1,157 | 3.79 | 0.1396 | 103,803 | 1,651 | 11.36 | 0.6163 | |
| Domain | Model | Baseline | ExpeL | Reflexion | GEPA | Macro | Self | ARISE-T |
|---|---|---|---|---|---|---|---|---|
| Aircraft Insp. | Opus 4.6 | 97.1 | 88.2 | 97.1 | 97.1 | 97.1 | 96.9 | 96.9 |
| Sonnet 4.6 | 67.6 | 97.1 | 67.6 | 94.1 | 97.1 | 97.1 | 97.0 | |
| Haiku 4.5 | 97.1 | 97.1 | 97.1 | 97.1 | 97.1 | 97.1 | 97.1 | |
| DeepSeek v3.2 | 79.4 | 73.5 | 85.3 | 94.1 | 91.2 | 85.3 | 86.3 | |
| Qwen3-32B | 97.1 | 97.1 | 97.1 | 94.1 | 97.1 | 96.1 | 95.1 | |
| Content Flag. | Opus 4.6 | 37.3 | 62.7 | 43.2 | 92.2 | 94.1 | 92.8 | 92.8 |
| Domain | Model | Baseline | ExpeL | Reflexion | GEPA | Macro | Self | ARISE-T |
|---|---|---|---|---|---|---|---|---|
| Airline | Opus 4.6 | 78.8 | 78.8 | 75.0 | 75.0 | 73.8 | 77.5 | 77.5 |
| Sonnet 4.6 | 81.2 | 78.8 | 75.0 | 77.5 | 72.5 | 76.7 | 77.5 | |
| Haiku 4.5 | 55.0 | 61.2 | 62.5 | 56.2 | 62.5 | 51.7 | 64.6 | |
| DeepSeek v3.2 | 41.2 | 48.8 | 52.5 | 50.0 | 48.8 | 46.7 | 50.4 | |
| Qwen3-32B | 20.0 | 23.8 | 20.0 | 17.5 | 26.2 | 18.3 | 23.8 | |
| Retail | Opus 4.6 | 67.5 | 65.0 | 63.8 | 62.5 | 64.4 | 66.9 | 66.9 |
LLM Agent Self-Improvement
Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.
See all 298 papers