On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification

Qinyuan Ye; Yu Li; Yada Pruksachatkun; Jiaxin Zhang; Chien-Sheng Wu

cs.AIAug 18, 2026

On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification

Authors: Qinyuan Ye, Yu Li, Yada Pruksachatkun, Jiaxin Zhang, Chien-Sheng Wu

Organizations: Salesforce AI Research

Abstract

Memory-based self-improving agents--those that learn from an online stream of tasks and improve over time by maintaining a textual memory bank--have shown great promise in recent literature. However, the reliability aspects of these methods have been critically overlooked. In this work, we conduct a comprehensive re-evaluation of two memory-based methods, broadening the scope of evaluation along two axes: (1) including multiple self-improving runs to quantify variance, and (2) shuffling the tasks to investigate the effect of task order. Through these experiments, we make two observations that expose the fragility of current methods: First, agent evaluation is inherently noisy in complex environments and on multi-step tasks, and stacking a self-improving loop on top can further amplify this noise. Second, the agent's improvement is highly dependent on task order. Prior works often adopt default orderings that impose an implicit curriculum, acting as a hidden prerequisite for success. To better understand this fragility, we manually examine the agents' memory and hypothesize that task and environment underspecification contribute to this fragility. We validate this hypothesis by incorporating information that enables better specification, such as detailed rubrics and environment feedback, into the memory construction process. While this added information partially closes the performance degradation in previous experiments, significant gaps still remain, suggesting that other uncharacterized factors contribute to this fragility. Looking ahead, our work advocates for more rigorous evaluation protocols for self-improving agents by reporting results across multiple runs and stress-testing them under challenging conditions. Moreover, our findings on underspecification call for systems and interfaces that enable effective human oversight, preventing agents from failing in unforeseeable ways.

Figures & tables

Figure 1: (a) Overview of Memory-based Self-improving Methods. In this setting, agents will perform a sequence of tasks and generate memories at the end of each tasks. The memories will be saved to a memory bank and will be retrieved for performing future tasks. (b)(c) Example memories. We consider two established methods (AWM [ 11 ] and RBank [ 12 ] ) in our evaluation.

Standard deviation of pass@1 across 3 runs (%)
Method	WebArena						VisualWebArena			SCUBA
	Shopping	Admin	GitLab	Map	Reddit	Multisite	Classifieds	Shopping	Reddit	Admin	Sales	Service
	(187)	(182)	(180)	(109)	(106)	(48)	(234)	(466)	(210)	(158)	(64)	(45)
Baseline	1.53	1.19	1.98	1.30	1.78	0.98	1.01	0.83	1.03	0.30	1.95	2.77
AWM	1.26 (-18%)	2.99 (+152%)	1.20 (-39%)	1.98 (+53%)	2.48 (+39%)	0.98 (0%)	1.79 (+78%)	1.01 (+22%)	2.38 (+131%)	2.09 (+600%)	1.28 (-35%)	2.77 (0%)
RBank	1.33 (-13%)	2.12 (+79%)	3.34 (+69%)	3.89 (+200%)	2.22 (+25%)	4.28 (+336%)	1.23 (+22%)	1.58 (+91%)	1.47 (+43%)	1.58 (+429%)	1.47 (-24%)	3.78 (+36%)

Table 1: Variance-related metrics across 3 runs. Memory-based self-improving agent methods introduce increased variance in 17 out of 24 (

\approx

71%) cases, among which 11 cases show a relative increase of more than 50%.

Table 3

Figure 2: Implicit curriculum in the default task order. We visualize the moving average (window=30) of task pass rates as the no-memory baseline agent performs tasks in the default order. In (a)(b), success rates drop on tasks with larger task IDs. In (c), we observe a more complex oscillating pattern. In general, tasks of different difficulties are not distributed evenly in the default order.

Figure 3: Performance using different task orders. We use the default task order and two shuffled orders. Model performance degrade significantly in 6 out of 8 cases when shuffled orders are used.

Figure 4: Mentions of Selected Keywords in Retrieved Memory. In (a)(b), we show that while the environment does not support API-based actions, agents increasingly save and retrieve memories about completing tasks with APIs. In (c), we show that in travel time related tasks in the map domain, the agent proposes to use Haversine Estimation instead of relying on the websites route engine, and this unintended strategy is amplified throughout the course of self-improvement.

Rubrics and Scores (+Rub):
Evaluator: string_match \| Score: 0.0 (FAIL)
Prediction: "Top-2 best-selling products in 2022: 1) Quest Lumaflex™ Band, 5 units; 2) Cruise Dual Analog Watch, 4 units."
must_include: "Quest Lumaflex™ Band" → PASS
must_include: "Sprite Stasis Ball 65 cm" → FAIL
Environment Feedback (+Env):
Actions: click(31), input_text(41, “05/01/2022”), input_text(44, “12/31/2022”), click(59) # Preceding Context

Table 4: Three types of additional information used in § 4.2 . We hypothesize such information can reduce underspecification when constructing memories. Examples are truncated due to space limit.

Figure 5: Addressing Underspecification with Additional Information. (a) We apply the three types of information individually and altogether. (b) We experiment with +All using three task orders.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Shopping	Admin	GitLab	Map	Reddit	Multisite	All
	(187)	(182)	(180)	(109)	(106)	(48)	(812)
Avg. Pass	45.8	62.6	53.9	53.2	69.5	34.0	54.8
Pass@3	55.1	72.0	63.3	67.0	82.1	47.9	65.4
Passˆ3	34.8	52.2	45.0	40.4	53.8	20.8	43.3

Appendix

Table 5: Evaluation of the no-memory baseline agent on WebArena.

Metric	VisualWebArena				SCUBA
	Classifieds	Shopping	Reddit	All	Admin	Sales	Service	All
	(234)	(466)	(210)	(910)	(158)	(64)	(45)	(267)
Avg. Milestone	-	-	-	-	46.7	61.8	42.3	49.6
Avg. Pass	60.1	57.6	43.3	54.9	26.2	39.6	28.1	29.7
Pass@3	73.9	69.1	51.4	66.3	39.2	57.8	40.0	43.8
Passˆ3	44.9	45.1	33.8	42.4	13.9	20.3	13.3	15.4

Appendix

Table 6: Evaluation of the no-memory baseline agent on VisualWebArena and SCUBA.

Task Order: Ordinal
Method	Shopping	Admin	GitLab	Map	Reddit	Multisite	All
	(187)	(182)	(180)	(109)	(106)	(48)	(812)
No Memory	45.8/1.53/3.74	62.6/1.19/2.75	53.9/1.98/4.44	53.2/1.30/2.75	69.5/1.78/3.77	34.0/0.98/2.08	54.8
AWM	48.3/1.26/2.67	59.7/2.99/6.59	51.7/1.20/2.78	50.5/1.98/4.59	72.3/2.48/5.66	32.6/0.98/2.08	54.1
RBank	52.8/1.33/3.21	60.6/2.12/4.95	59.3/3.34/7.78	48.6/3.89/8.26	71.1/2.22/4.72	27.8/4.28/10.42	56.3
Task Order: Shuffle-1

Appendix

Table 7: Per-domain pass rate (%) on WebArena. Each cell reports mean/std/best–worst gap, based on three runs of memory construction. The right-most column (“All”) reports the overall pass rate.

Task Order: Ordinal
Method	Classifieds	Shopping	Reddit	All
	(234)	(466)	(210)	(910)
No Memory	60.1/1.01/2.14	57.6/0.83/1.93	43.3/1.03/2.38	54.9
AWM	61.3/1.79/4.27	55.3/1.01/2.15	45.1/2.38/5.71	54.5
ReasoningBank	59.5/1.23/2.99	58.3/1.58/3.86	45.4/1.47/3.33	55.6
Task Order: Shuffle 1

Appendix

Table 8: Per-domain pass rate (%) on VisualWebArena. Each cell reports mean/std/best–worst gap, based on three runs of memory construction. The right-most column (“All”) reports the overall pass rate.

Method	Admin	Sales	Service	All
	(158)	(64)	(45)	(267)
Baseline	46.7/0.25/0.59	61.8/1.57/3.36	42.3/1.40/3.02	49.6
Synapse [ 68 ]	51.3/1.39/3.40	64.9/2.64/5.94	48.6/5.24/12.51	54.1
AWM	45.8/1.40/3.25	60.8/2.98/7.19	49.6/4.15/9.13	50.1
ReasoningBank	50.7/0.19/0.47	62.4/2.70/6.59	36.3/2.85/6.11	51.1

Appendix

Table 9: Per-domain milestone score (%) on SCUBA. Each cell reports mean/std/best–worst gap, based on three runs of memory construction. The right-most column (“All”) reports the overall pass rate.

Method	Admin	Sales	Service	All
	(158)	(64)	(45)	(267)
Baseline	26.2/0.30/0.63	39.6/1.95/4.69	28.1/2.77/6.67	29.7
Synapse [ 68 ]	30.4/1.37/3.16	43.2/2.66/6.25	28.9/1.81/4.44	33.2
AWM	22.4/2.09/4.43	37.5/1.28/3.12	34.1/2.77/6.67	28.0
ReasoningBank	30.2/1.58/3.80	40.1/1.47/3.12	21.5/3.78/8.89	31.1

Appendix

Table 10: Per-domain pass rate (%) on SCUBA (binary success: milestone score

=1.0

). Each cell reports mean/std/best–worst gap, based on three runs of memory construction. The right-most column (“All”) reports the overall pass rate.

$n=3$ runs
Method	Shopping	Admin	GitLab	Map	Reddit	Multisite	All
	(187)	(182)	(180)	(109)	(106)	(48)	(812)
No Memory	45.8/1.5/3.7	62.6/1.2/2.8	53.9/2.0/4.4	53.2/1.3/2.8	69.5/1.8/3.8	34.0/1.0/2.1	54.8
RBank (Ordinal)	52.8/1.3/3.2	60.6/2.1/5.0	59.3/3.3/7.8	48.6/3.9/8.3	71.1/2.2/4.7	27.8/4.3/10.4	56.3
$n=6$ runs
No Memory	45.8/1.7/4.3	61.7/1.8/5.5	53.7/1.5/4.4	52.0/2.2/6.4	68.1/2.7/7.6	34.0/1.0/2.1	54.2

Appendix

Table 11: WebArena Results, Three Additional Runs using GPT-5-mini . Each cell reports avg/std/best-worst gap across

n

runs, and the right-most column (“All”) reports the overall pass rate.

GPT-OSS-120B
Method	Shopping	Admin	GitLab	Map	Reddit	Multisite	All
	(187)	(182)	(180)	(109)	(106)	(48)	(812)
No Memory	35.5/1.1/2.7	46.7/0.4/0.8	45.4/0.9/2.2	33.0/1.5/3.7	45.9/2.5/5.7	24.3/3.9/8.3	40.6
RBank (Ordinal)	38.0/0.9/2.1	49.0/1.7/4.1	42.4/0.3/0.6	30.0/2.2/4.6	48.1/5.4/13.2	18.8/1.7/4.2	40.5
RBank (Shuffle-1)	34.3/1.5/3.2	40.4/2.6/5.8	34.5/1.7/3.9	33.9/2.2/5.5	49.4/1.2/2.8	22.9/3.4/8.3	37.0
Gemini-2.5-Pro

Appendix

Table 12: WebArena Results using Additional Models. Each cell reports avg/std/best-worst gap across three runs, and the right-most column (“All”) reports the overall pass rate.

GPT-OSS-120B
Method	Calendar	CSM	Drive	Email	HR	Hybrid	ITSM	Teams	All
No memory	21.3/2.7/6.6	21.4/0.8/1.9	41.2/2.0/4.7	54.2/2.5/6.0	12.4/0.9/2.0	23.6/1.5/3.7	5.8/0.8/1.9	35.5/4.3/9.8	24.5
With memory	26.2/2.7/6.6	19.7/2.8/6.8	33.9/6.3/14.1	52.7/1.4/3.0	5.9/1.4/2.9	25.6/2.0/4.9	1.3/0.5/1.0	36.1/3.5/8.2	22.3
GPT-5.4-mini
No memory	38.3/0.8/1.6	21.4/1.6/3.9	47.9/2.7/6.3	58.7/2.5/6.0	20.6/1.4/2.9	24.8/4.0/9.8	25.6/3.7/7.8	21.3/2.3/4.9	30.5
With memory	36.1/1.3/3.3	25.9/1.7/3.9	51.6/3.4/7.8	56.7/1.2/3.0	16.7/2.1/4.9	23.6/5.8/13.4	24.6/1.2/2.9	15.3/4.7/11.5	29.7

Appendix

Table 13: Enterprise-Ops Gym Results. Each cell reports avg/std/best-worst gap across three runs, and the right-most column (“All”) reports the overall pass rate.

Task Query
I have jaw bruxism problem, show me something that could alleviate the problem.
Ground-Truth Evaluator
“Jaw bruxism” and “mouth guard” must be included in the content of the final page.
Agent’s Final Message
(The agent didn’t browse the shopping website and ends the task after 1 step.) Here are practical things that can help reduce jaw bruxism (teeth grinding/clenching). This is general information — consult your dentist or doctor for personalized advice: 1) Night guard / oral splint - Best: a custom-fitted occlusal guard from your dentist (protects teeth and reduces muscle stress). [text omitted for brevity…]
Agent’s Memory

Appendix

Table 14: Example of Ambiguous Task Query: WebArena Task 118. The task is intended to be a web navigation task but is interpreted as a question answering task by the agent.

Task Query
How much I spend on 4/19/2023 on shopping at One Stop Market?
Ground-Truth Evaluator
The answer string must include “0”.
Agent’s Final Message
I could not find any orders dated 4/19/2023 in your One Stop Market account. I scanned the full order history (37 orders shown after setting per-page to 50) and the extraction returned no matches for 4/19/2023. Therefore, according to this account, the amount spent on 4/19/2023 at One Stop Market is $0.00.
Author Comment

Appendix

Table 15: Example of Evaluator Bug: WebArena Task 329. A correct answer may fail strict string-matching rubric, and further lead to the creations of irrelevant memories.

Task
Check if Duquesne University in Pittsburgh can be reached in one hour by car from Pittsburgh Airport.
Rubric
Answer string must include "yes".
Agent’s Answer
Result and conclusion:
What I could not do: The OpenStreetMap directions page did not show a numeric route or duration because it displayed messages indicating JavaScript was not available, so a live OSRM route summary could not be extracted.

Appendix

Table 16: Example of Using Haversine Formula: WebArena Task 38. The agent receives an reward for including “yes” in its final answer, while the strategy to reach the answer is unintended.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 171 papers

Agent Plasticity: Measuring Self-Improvement Through Experience
Oct 6, 2026Harman Singh, Anton Bakhtin, Rulin Shao +8Self-Improving Agents LLM Agent Self-Improvement
UC Berkeley · Meta Superintelligence Labs · University of Washington +1
Self-Improvements in Modern Agentic Systems: A Survey
Jul 14, 2026Zhe Ren, Yimeng Chen, Dandan Guo +9LLM Agent Self-Improvement Self-Improving Agents
School of Artificial Intelligence, Jilin University · King Abdullah University of Science and Technology (KAUST) · Independent Researcher +2
RSI-Forge: From Research Papers to Environments for Recursive Self-Improvement
Oct 7, 2026Renxiong Wang, Darvin Yi, Abril Herrlein +16Self-Improving Agents Recursive Self-Improvement
Scale AI · University of California, Santa Cruz · University of North Carolina at Chapel Hill

Oct 6, 2026J/K move · Enter open · S save

Agent Plasticity: Measuring Self-Improvement Through Experience

Harman Singh, Anton Bakhtin, Rulin Shao, Gabriel Synnaeve+7

UC Berkeley · Meta Superintelligence Labs · University of Washington+1

Self-Improving Agents LLM Agent Self-Improvement AI Agent Evaluation

Open paper AskPDF arXiv

AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.

Abstract

Self-Improving Agents

Explore similar work

Agent Plasticity: Measuring Self-Improvement Through Experience

Self-Improvements in Modern Agentic Systems: A Survey

RSI-Forge: From Research Papers to Environments for Recursive Self-Improvement