cs.CLOct 16, 2026
SaveSelf-Evolving Agents Can Repair Failures but Struggle to Generalize from Them
Organizations: Harbin Institute of Technology, Harbin, China · Harbin Institute of Technology, Shenzhen, China · The Chinese University of Hong Kong
Abstract
Experience-driven self-evolution offers a promising path for large language model (LLM) agents to continually improve through interaction. Yet existing approaches often rely on successful trajectories or external evaluation signals, leaving unclear whether agents can evolve when such positive anchors are unavailable. We study this question through Cold-Start Failure-Only Evolution (CFOE), where agents start without accumulated experience and learn only from failed interactions without privileged guidance. Across three self-evolving paradigms, multiple LLM backbones, and four interactive environments, we uncover a consistent repair-generalization gap: failure-derived experience can help repair previously failed tasks, but rarely yields sustained improvements on unseen tasks. Further analysis shows that agents often misinterpret failures, while even verified-correct experience remains difficult to transfer. Relaxing CFOE reveals that verifier feedback and additional interaction provide limited gains, whereas successful trajectories substantially improve transfer. These findings suggest that current self-evolving agents are better at repairing observed failures than building generalizable capabilities from failure.
Figures & tables
| Model | Paradigm | M1 | M2 | M3 | M4 |
|---|---|---|---|---|---|
| ALFWorld | |||||
| Qwen3.5-27B | Memory | 41.3 | 46.5 | 32.6 | 35.5 |
| Qwen3.5-27B | Skill | 43.8 | 39.7 | 28.8 | 37.0 |
| Qwen3.5-27B | Harness | 29.9 | 38.6 | 31.5 | 36.2 |
| GPT-5.6-Sol | Harness | 21.2 | 31.0 | 25.7 | 23.9 |
| -Bench-Verified | |||||
Table 1: Percentage of reviewed updates exhibiting each failure mode. A single update may exhibit multiple failure modes.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Task Instruction |
| Your task is to: look at statue under the desklamp. |
| Trajectory Summary |
| Step 1. <thought> I will start with drawer 1. If not, I will proceed sequentially. <action> go to drawer 1 Step 21. <thought> I have finished checking all drawers (1 through 10). […] I should start checking the shelves systematically. <action> go to shelf 1 Step 22. <observation> You arrive at shelf 1. On the shelf 1, you see a alarmclock 1, and a statue 1. <thought> I have found the statue 1 on shelf 1. […] I should continue searching the shelves for the desklamp. <action> go to shelf 2 Step 30. <observation> You arrive at shelf 9. On the shelf 9, you see nothing. <action> go to bed 1 |
| Generated Experience |
| Memory Item 1 – Systematic Exhaustion of Container Sets. Description: When a target object is not immediately visible, agents should exhaustively search all containers of a specific type in a fixed numerical order before switching to a different container type. Content: The agent successfully located the target statue by first iterating through all 10 drawers (1–10) and then all 9 shelves (1–9) in strict numerical sequence, ensuring no potential location within those categories was skipped. This ordered approach prevents premature switching between container types (e.g., jumping from drawers to shelves) which could lead to missed items or redundant navigation. Future agents should maintain this “finish one category before starting another” discipline when the environment lists multiple instances of similar containers. |
| Analysis |
Table 3: Representative example of False-Positive Success Attribution.
| Task Instruction |
| Open the Cabinets, Racks & Shelves category page to browse products |
| Trajectory Summary |
| Step 1. <thought> The task is to open the “Cabinets, Racks & Shelves” category page. […] “Home & Kitchen” […] or “Tools & Home Improvement” […] seem like the most likely parent categories. Let me start by clicking on “Home & Kitchen” to explore its subcategories. <action> click(’473’) Step 2. <thought> I’m now on the Home & Kitchen category page. […] It might be under “Storage & Organization” […] or “Furniture”. Let me click on “Storage & Organization” first as cabinets and racks are typically storage solutions. <action> click(’1569’) Step 4. <thought> I’ve navigated to the “Racks, Shelves & Drawers” category page (cat=177). The task asks for “Cabinets, Racks & Shelves” category. […] The “Now Shopping by” section shows I’m currently in “Racks, Shelves & Drawers” category. This appears to be the closest match to the requested “Cabinets, Racks & Shelves” category. […] The current page seems to be the target category page. Let me submit the response indicating success. <action> submit_response({“task_type”: “NAVIGATE”, “status”: “SUCCESS”, “retrieved_data”: null, “error_details”: null}) |
| Generated Experience |
| Memory Item 2 – Verifying Category Context via Breadcrumbs and Filters. Description: Procedure for confirming the current location in a site hierarchy using page elements other than the main navigation. Content: After navigating deep into a category tree, confirm the current context by inspecting “Now Shopping by” sections, breadcrumbs, or active filter labels rather than relying solely on the page title or URL. If the page displays a “Remove This Item” link next to the current category name or shows price filters specific to that category, this serves as strong evidence that the agent has successfully entered the intended product scope . |
| Analysis |
Table 4: Representative example of Misidentified and Hallucinated Failure Attribution.
| Task Instruction |
| Find me 1080p hd video surveillance for motion detection, and price lower than 50.00 dollars |
| Trajectory Summary |
| System instruction. On a search-results page you MUST click [Back to Search] before issuing a new search[…]; otherwise you will get “Invalid action!”. Step 5. <action> search[security camera 1080p under 50] <observation> […] [Back to Search] […] Page 1 (Total results: 50) […] Step 6. <action> search[wifi security camera under 50] <observation> Invalid action! Step 7. <action> click[Back to Search] <observation> […] [Search] |
| Generated Experience |
| Reflection reasoning. A critical observation is the presence of a specific rule in the system prompt: On a search-results page you MUST click [Back to Search] before issuing a new search[…]; otherwise you will get “Invalid action!”. Consolidated experience. Applicability: Always, regardless of whether the previous search was successful or not. If on a results page, the action MUST be click[Back to Search]. |
| Analysis |
Table 5: Representative example of Distorted Experience Consolidation.
| Generated Erroneous Experience |
| Memory Item 2 – Escalation Criteria for Terminology Mismatch. Description: Defines a stopping condition for troubleshooting when a user cannot map technical instructions to their device interface after multiple attempts. Content: If a user explicitly states they cannot find a requested setting or function after three distinct attempts to explain it using different phrasings (e.g., technical terms, simplified analogies, and direct UI path descriptions), the agent should cease further troubleshooting steps that rely on that specific user action . Instead, the agent should summarize the verified status (e.g., “Roaming is enabled on our end, device shows 4G”) and transfer the case to a human agent who can perform backend diagnostics or remote configuration, rather than continuing to loop on the same unresolvable instruction. |
| Why the Experience Is Wrong |
| One failed episode cannot establish three explanations as a universal stopping threshold. The rule equates interface confusion with exhausted troubleshooting and omits checks for remaining repairs or agent-caused confusion. In its source episode, set_network_mode_preference and toggle_roaming remained incomplete. |
| First Downstream Trajectory |
| Step 22. <observation> I’m not sure how to check if Data Roaming is enabled or disabled on my phone. The instructions are getting a bit too technical for me. Could you please tell me exactly which buttons to press or which menu to go into? […] I really need your help to figure this out step by step. What should I do first? <thought> […] I should guide them through a simple, concrete action. […] Instead of asking them to check network status, let me guide them to directly toggle data roaming on their phone. […] Step 23. <thought> […] According to Memory Item 2, after three distinct attempts, I should cease further troubleshooting steps that rely on that specific user action and transfer the case. I’ve made more than three attempts. Let me transfer the customer to a human agent […] <action> transfer_to_human_agents(“Customer John Smith […] needs assistance with enabling Data Roaming on their phone. […]”) <Result> Tool call transfer_to_human_agents(…) must use keyword arguments only […] Step 24. <action> transfer_to_human_agents(summary=“Customer John Smith […] needs assistance with enabling Data Roaming on their phone. […]”) <Result> Transfer successful |
| First Downstream Analysis |
Table 6: Representative example of Poisonous Future Behavior Guidance.
More in this topic
LLM Agent Self-Improvement
Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.
See all 277 papers