stat.MLNov 7, 2026

Trustworthy Recursive Self-Improvement with Statistical Guarantees

Authors: Zihe Gao, Zhengling Qi, Yifan Cui

Organizations: Zhejiang University · George Washington University

Abstract

Recursive self-improvement offers a promising path toward AI agents that improve through experience by refining their own harnesses without retraining their underlying large language models (LLMs). Making this self-improvement process reliable, however, requires more than generating new updates: agents must decide which changes are worth keeping. A greedy gate that accepts updates based solely on higher validation scores can mistake an observed gain for genuine progress and lead to performance degeneration, whereas overly conservative statistical gates may reject beneficial updates and hinder performance improvement. We introduce Trustworthy Recursive Updates with Statistical Testing (TRUST), a framework that adapts an online multiple hypothesis testing procedure to control the rate of false evolution across rounds, with an LLM analyzing historical context to help determine how much evidence is required to update the incumbent. Under interpretable assumptions, we prove that TRUST limits the proportion of accepted updates that result in unacceptable performance loss. We further derive a lower bound on the final performance gain. Across the benchmarks HotpotQA, DROP, and APPS with the models DeepSeek-V4-Flash and Qwen-3.8-Flash, TRUST reduces the average ratio of accepted inferior updates from the greedy gate's 45.59% to 2.78%, while achieving higher final scores than the state-of-the-art statistical gate baselines for recursive self-improvement. These findings demonstrate that TRUST strikes a balance between statistical reliability and performance gain in recursive self-improvement.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges

    Oct 26, 2026Jiayue Pu, Ang Gao, Jinyi Bai +9Recursive Self-ImprovementLanguage Model Self-Improvement

  2. The Self-Improvement Problem

    Oct 15, 2026Brian B. Moser, Stanislav Frolov, Tobias C. Nauen +2Self-Improving AgentsRecursive Self-Improvement

  3. RSI-Env: An Open-Ended Environment for Recursive Self-Improvement of Agents

    Nov 1, 2026Wanying Qu, Zhiyuan Fan, Jiyao Liu +4LLM Agent Self-ImprovementRecursive Self-Improvement