Trustworthy Recursive Self-Improvement with Statistical Guarantees
Organizations: Zhejiang University · George Washington University
Abstract
Recursive self-improvement offers a promising path toward AI agents that improve through experience by refining their own harnesses without retraining their underlying large language models (LLMs). Making this self-improvement process reliable, however, requires more than generating new updates: agents must decide which changes are worth keeping. A greedy gate that accepts updates based solely on higher validation scores can mistake an observed gain for genuine progress and lead to performance degeneration, whereas overly conservative statistical gates may reject beneficial updates and hinder performance improvement. We introduce Trustworthy Recursive Updates with Statistical Testing (TRUST), a framework that adapts an online multiple hypothesis testing procedure to control the rate of false evolution across rounds, with an LLM analyzing historical context to help determine how much evidence is required to update the incumbent. Under interpretable assumptions, we prove that TRUST limits the proportion of accepted updates that result in unacceptable performance loss. We further derive a lower bound on the final performance gain. Across the benchmarks HotpotQA, DROP, and APPS with the models DeepSeek-V4-Flash and Qwen-3.8-Flash, TRUST reduces the average ratio of accepted inferior updates from the greedy gate's 45.59% to 2.78%, while achieving higher final scores than the state-of-the-art statistical gate baselines for recursive self-improvement. These findings demonstrate that TRUST strikes a balance between statistical reliability and performance gain in recursive self-improvement.
Figures & tables
| DeepSeek-V4-Flash | Qwen-3.8-Flash | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Dataset | Gate | Gain (pp) | FER 0 (%) | TER (%) | MDD (pp) | Gain (pp) | FER 0 (%) | TER (%) | MDD (pp) |
| HotpotQA | Greedy | 7.87 0.55 | 64.67 3.28 | 40.03 5.06 | 4.96 0.93 | 20.66 2.45 | 44.67 5.23 | 58.40 9.39 | 4.01 0.49 |
| PACE | 10.15 0.75 | 0.00 0.00 | 12.63 3.65 | 0.00 0.00 | 26.42 1.60 | 6.67 6.67 | 16.31 2.94 | 0.06 0.06 | |
| SGM-B | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | |
| SGM-H | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 0.00 0.00 | 5.77 5.77 | 0.00 0.00 | 1.25 1.25 | 0.00 0.00 | |
| TRUST | 11.58 1.30 | 10.00 10.00 | 18.11 8.39 | 0.02 0.02 | 27.72 0.81 | 0.00 0.00 | 20.82 3.69 | 0.00 0.00 | |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Operator | Function |
|---|---|
| ADD_SPECIALIST_RULE | Add a local, cluster-level, or global rule with a trigger, exclusions, and executable steps. |
| REFINE_RULE | Rewrite only the procedure of an existing rule. |
| SET_DECOMPOSITION_POLICY | Set a general decomposition policy, such as identifying comparison entities before extracting their attributes. |
| GENERALIZE_RULE_SCOPE | Broaden a rule by relaxing its trigger, removing exclusions, or expanding scope. |
| PROMOTE_RULE_TO_DEFAULT | Raise a rule’s scope or priority toward default behavior. |
| DELETE_RULE | Remove an obsolete or harmful specialist rule. |
| Operator | Function |
|---|---|
| SET_ROUTE_MODE | Select full context, lexical reordering, lexical top- , bridge reordering, or bridge top- . |
| REWEIGHT_RELEVANCE_SCORER | Adjust title, body, position, and minimum-overlap relevance weights. |
| ENABLE_BRIDGE_AWARE_EXPANSION | Toggle expansion in which a first-hop entity raises the rank of linked second-hop evidence. |
| SET_FIXED_TOP_K | Cap the number of ordinary evidence passages retained by sparse routing. |
| SET_FULL_FALLBACK | Configure fallback to full context when sparse evidence is insufficient. |
| Operator | Function |
|---|---|
| ADD_LOCAL | Atomically add one to three closed cases to active memory for immediate retrieval. |
| REFINE_PROCEDURE | Revise the steps, evidence requirement, or safe fallback used after retrieving a memory. |
| MODIFY_TRIGGER | Change the similarity threshold for cluster retrieval. |
| MODIFY_PRIORITY | Change retrieval and rendering priority. |
| DELETE | Move a harmful memory from the active bank to the archive. |
| Operator | Function |
|---|---|
| SET_VERIFY_MODE | Set the strength of internal checks before finalization. |
| SET_EVIDENCE_ALIGNMENT | Specify whether the answer and supporting fact identifiers must align with evidence. |
| SET_STOP_RULE | Specify when evidence checking stops and output begins. |
| SET_FALLBACK_MODE | Specify behavior under uncertainty or insufficient evidence. |
| Method | Setting |
|---|---|
| Greedy | Positive empirical paired gain. |
| PACE ( Shawn, 2026 ) | Bet 0.5, candidate-level . |
| SGM-B ( Wu et al., 2025 ) | Bernstein lower confidence bound , , , . |
| SGM-H ( Wu et al., 2025 ) | Hoeffding lower confidence bound , same , , . |
| TRUST | , , , betting mixture with (0.10,0.25,0.50,0.75) |
| Gain (pp) | FER 0 (%) | TER (%) | MDD (pp) | |
|---|---|---|---|---|
| Gain (pp) | FER 0 (%) | TER (%) | MDD (pp) | |
|---|---|---|---|---|
| Gain (pp) | FER 0 (%) | TER (%) | MDD (pp) | |
|---|---|---|---|---|
Recursive Self-Improvement
Recursive self-improvement studies AI systems that use feedback from their own work to modify and evaluate future versions.
See all 107 papers