When Is Enough Enough in Self-Evolving LLM Systems?

Enoch Yin; Bin Liu; Zhengling Qi

cs.AIOct 3, 2026

When Is Enough Enough in Self-Evolving LLM Systems?

Authors: Enoch Yin, Bin Liu, Zhengling Qi

Organizations: Department of Statistics, The George Washington University · Department of Statistics and Data Science, School of Management, Fudan University · Department of Decision Sciences, School of Business, The George Washington University

Abstract

Self-evolving large language model (LLM) systems repeatedly propose, evaluate, and incorporate updates to prompts, skills, or other persistent artifacts. Despite their growing effectiveness, these systems typically operate under a predetermined iteration or compute budget, without a principled criterion to determine when further evolution is no longer worthwhile. This can lead to two undesirable consequences: unnecessary computation after performance has saturated and the risk of returning late updates that overfit or exploit the evaluation signal. These issues motivate us to study two fundamental questions: when should a self-evolving system stop, and what should it output once it stops? We address the first by formulating an online sequential testing problem and constructing an anytime-valid restart detector using the per-item paired evaluation outcomes already produced by self-evolving LLM systems. We address the second by formulating a change-point estimation problem and using the estimated transition to select an earlier artifact for output. The resulting procedure is plug-and-play and requires no modification of the underlying self-evolving algorithms. Across two self-evolving frameworks, three LLM model families, and five benchmarks, our method substantially reduces computation costs while maintaining comparable unseen-test performance. For example, on SearchQA with SkillOpt and DeepSeek V4 Flash, our method stops at round 4 rather than the full budget of 40, reducing token usage by 91.6% while achieving 82.43% unseen-test accuracy versus 82.00% under the full-budget run.

Figures & tables

Figure 1: A self-evolving run on SearchQA with DeepSeek V4 Flash. Performance improves during the first few rounds and then plateaus, yet the fixed-budget loop continues to round 40. Our procedure stops at round 4 and selects round 1 as the output, achieving comparable unseen-test performance while avoiding most of the subsequent computation.

Figure 2: Our stopping and output-selection Plug-and-Play module monitors paired validation outcomes from a self-evolving loop.

Figure 3: SpreadsheetBench trajectory under SkillOpt with DeepSeek V4 Flash.

Figure 4: LiveMath trajectory under SkillOpt with DeepSeek V4 Flash.

Stopping behavior
Benchmarks	SearchQA	GSM8K	OfficeQA
Validation size $n$	200	200	24
Full-run budget $N$	40	40	16
Alarm round $T_{\mathrm{alarm}}$	4	4	8
Change point $\hat{\nu}$	2	2	5
Computation cost

Table 1: Main results for DeepSeek V4 Flash + SkillOpt with a shared detector configuration (

\varepsilon=0.01

\delta=0.05

, aGRAPA).

Stopping behavior
	SearchQA		GSM8K
	SkillOpt	GEPA	SkillOpt	GEPA
Alarm round $T_{\mathrm{alarm}}$	4	5	4	6
Change point $\hat{\nu}$	2	2	2	3
Computation cost
Tokens to alarm (M)	2.18	2.36	1.21	1.79

Table 2: Robustness across self-evolving systems using DeepSeek V4 Flash and the same detector configuration. We compare SkillOpt and GEPA on the same benchmarks, reporting stopping behavior, computation cost, and unseen-test performance. GEPA token counts are estimated from metric-call counts.

Stopping behavior
	DeepSeek V4 Flash	Qwen3-32B	GPT-5.6 Luna
Validation size $n$	200	200	200
Full-run budget $N$	40	40	40
Alarm round $T_{\mathrm{alarm}}$	4	12	10
Change point $\hat{\nu}$	2	9	3
Computation cost

Table 3: Robustness across underlying models on SearchQA using SkillOpt with the same detector configuration. The detector adapts its stopping time to different evolution dynamics while reducing computation and retaining comparable unseen-test performance.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Stopping behavior
	DeepSeek	Qwen	Luna
Validation size $n$	200	200	200
Full-run budget $N$	40	40	40
Alarm round $T_{\mathrm{alarm}}$	4	8	4
Change point $\hat{\nu}$	2	2	2
Computation cost

Appendix

Table 4: Robustness across underlying models on GSM8K using SkillOpt with the same detector configuration.

System	Benchmark	Model	mix	aGRAPA
SkillOpt	SearchQA ( $n{=}200$ )	DeepSeek V4 Flash	5/2	4/2
	SearchQA ( $n{=}200$ )	Qwen3-32B	12/9	12/9
	SearchQA ( $n{=}200$ )	GPT-5.6 Luna	11/3	10/3
	SearchQA ( $n{=}100$ )	DeepSeek V4 Flash	10/4	6/4
	GSM8K ( $n{=}200$ )	DeepSeek V4 Flash	5/2	4/2
	GSM8K ( $n{=}200$ )	Qwen3-32B	9/2	8/2

Appendix

Table 5: Alarm /

\hat{\nu}

for a single configuration (

\varepsilon=0.01

\delta=0.05

) across SkillOpt and GEPA settings. “mix” denotes the fixed mixture with

\Lambda=\{0.1,0.2,0.4\}

, and “aGRAPA” denotes the adaptive growth-optimal plug-in.

Stopping behavior
	SearchQA		GSM8K
	$n=200$	$n=100$	$n=200$	$n=100$
Full-run budget $N$	40	40	40	40
Alarm round $T_{\mathrm{alarm}}$	4	6	4	5
Change point $\hat{\nu}$	2	4	2	2
Computation cost

Appendix

Table 6: Sensitivity to validation-set size using SkillOpt with DeepSeek V4 Flash and the same detector configuration. Smaller validation sets provide less evidence per round and therefore lead to slightly later stopping, while substantial computation is still saved and unseen-test performance remains comparable.

$\varepsilon\backslash\delta$	0.01	0.05	0.10
0.002	10/2	4/2	4/2
0.005	5/2	4/2	4/2
0.01	4/2	4/2	3/2
0.02	4/2	3/2	3/2
0.05	3/2	3/2	3/2

Appendix

Table 7: Sensitivity of the alarm round

T_{\mathrm{alarm}}

and estimated change point

\hat{\nu}

(\varepsilon,\delta)

. Each entry reports

T_{\mathrm{alarm}}/\hat{\nu}

; “–” indicates that no alarm is raised within the evolution horizon.

Figure 5: Evolution trajectories under SkillOpt with DeepSeek V4 Flash on SearchQA, GSM8K, and OfficeQA. The red dashed line marks $T_{\mathrm{alarm}}$ , the purple star marks $\hat{\nu}$ , and the gray dotted line denotes the full evolution budget.

Explore similar work

Sep 29, 2026

When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain

Self-evolution lets large language models (LLMs) improve iteratively using their own generated data, but often suffers from self-evolution degeneration: performance improves, plateaus, then declines. Existing methods address this issue at the component level, targeting either the Questioner or the Solver, and overlook that self-evolution is a tightly coupled system. We propose a holistic framework based on learnable information gain, which measures how much novel, parameterizable information a round provides relative to the previous round. Theoretically, this gain equals the Kullback-Leibler divergence between the two rounds' data distributions plus their entropy change. Practically, it is estimated by fitting a small language model to the previous round and scoring new data via negative log-likelihood. Based on this diagnostic, we propose ATRI (Adaptive Training Regulation via Information-gain), which reweights samples within a round and halts training across rounds when information gain remains low. Experiments on popular datasets demonstrate the superiority of our proposal.

Chenxu Wang, Chaozhuo Li, Xinze Shi +6

Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence · Central South University +2

Language Model Self-Improvement LLM Training Self-Training for Language Models

Ask PDF

Mar 19, 2026

Learning to Self-Evolve

We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time. We situate LSE in the setting of test-time self-evolution, where a model iteratively refines its context from feedback on seen problems to perform better on new ones. Existing approaches rely entirely on the inherent reasoning ability of the model and never explicitly train it for this task. LSE reduces the multi-step evolution problem to a single-step RL objective, where each context edit is rewarded by the improvement in downstream performance. We pair this objective with a tree-guided evolution loop. On Text-to-SQL generation (BIRD) and general question answering (MMLU-Redux), a 4B-parameter model trained with LSE outperforms self-evolving policies powered by GPT-5 and Claude Sonnet 4.5, as well as prompt optimization methods including GEPA and TextGrad, and transfers to guide other models without additional training. Our results highlight the effectiveness of treating self-evolution as a learnable skill.

Xiaoyin Chen, Canwen Xu, Yite Wang +3

Mila – Quebec AI Institute · University of Montreal · Snowflake

Language Model Self-Improvement RL for Language Models

Ask PDF

May 31, 2026

On the Generalization Gap in Self-Evolving Language Model Reasoning

Recent work suggests that large language models (LLMs) can improve through self-evolution (SE), using supervision signals generated by the model itself. In this work, we ask: under a strict closed-loop setup, where the self-evolution algorithm has access only to an unlabeled prompt set and a base model, how close can internally generated supervision come to oracle-supervised training? We analyze four representative strategies in a unified offline self-evolution framework: single-round verification, multi-turn revision with feedback, iterative training, and curriculum learning. Our primary experiments use Knights and Knaves (KK) logical reasoning tasks, which provide deterministic solutions, controlled difficulty levels, and a clean testbed for easy-to-hard generalization. We first show that self-evolution consistently improves over the base model, but plateaus after excessive training compute is invested, and eventually still leaves a non-trivial gap to oracle supervision. We find that multi-turn critic-revision with large models can reach strong self-evolution performance, with Gemma 12B nearly matching oracle-supervised training. Beyond Knights and Knaves, we also evaluate self-evolution on real-world reasoning benchmarks, where gains are also modest. Overall, our results characterize when closed-loop self-evolution can help and show how internally generated supervision remains insufficient under this minimal formulation.

Zhenting Qi, Susanna Maria Baby, Stefanie Anna Baby +5

Google Research · Harvard University · Google +1

LLM Self-Refinement Language Modeling Language Model Self-Improvement LLM Reasoning

Ask PDF