When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
Authors: Chenxu Wang, Chaozhuo Li, Xinze Shi, Songyang Liu, Kyrie You Wu, Ziluowen Luo, Shun Zhang, Chenxi Li, +1 more
Organizations: Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence · Central South University · Graduate School of China Academy of Engineering Physics · The Chinese University of Hong Kong, Shenzhen
Self-evolution lets large language models (LLMs) improve iteratively using their own generated data, but often suffers from self-evolution degeneration: performance improves, plateaus, then declines. Existing methods address this issue at the component level, targeting either the Questioner or the Solver, and overlook that self-evolution is a tightly coupled system. We propose a holistic framework based on learnable information gain, which measures how much novel, parameterizable information a round provides relative to the previous round. Theoretically, this gain equals the Kullback-Leibler divergence between the two rounds' data distributions plus their entropy change. Practically, it is estimated by fitting a small language model to the previous round and scoring new data via negative log-likelihood. Based on this diagnostic, we propose ATRI (Adaptive Training Regulation via Information-gain), which reweights samples within a round and halts training across rounds when information gain remains low. Experiments on popular datasets demonstrate the superiority of our proposal.
Figures & tables
Figure 1: Previous self-evolution loops monitor each component separately and drift into decline (left). Our ATRI model tracks system-level diagnostic (right).
Figure 2: Analysis on Qwen3-4B-Base with MATH. (a) Partial signals cannot reflect performance degradation. (b) Lower learnable information gain can signal later performance degradation.
Figure 3: The overview of the proposed ATRI framework.
Method
Math AVG
Overall AVG
Mathematical Reasoning Benchmarks
General Reasoning
GSM8K
MATH
AMC
Minerva
Olymp.
AIME24
AIME25
MMLU-Pro
SuperGPQA
BBEH
Qwen3-4B-Base
Base Model
42.58
36.34
87.86
67.96
45.59
38.10
41.16
11.03
6.35
37.17
20.84
7.33
STaR ( Zelikman et al., 2022 )
43.96
37.74
88.36
71.20
47.96
39.65
41.92
11.47
7.16
39.54
22.55
7.60
SPIN ( Chen et al., 2024 )
45.26
39.45
89.35
73.60
50.02
41.03
42.53
11.97
8.35
44.60
24.82
8.25
AZR ( Zhao et al., 2025 )
46.51
41.38
89.49
76.31
52.47
42.20
42.50
12.23
10.36
52.66
27.28
8.34
Table 1: Main results: ATRI versus existing self-evolution methods on Qwen3-4B-Base and Qwen3-8B-Base across seven mathematical reasoning benchmarks and three general reasoning benchmarks. Bold : best per column. Underline : second best.
Figure 5
Figure 5: Three-phase lifecycle across self-evolution methods (Qwen3-4B-Base / MATH). Ct falls below the alarm threshold τ one to four rounds before the acc peak and then fluctuates around zero.
Figure 7Table 8
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Task
Base model
Peak round
Alarm round
Peak acc.
Round-15 acc.
Reference
MATH
Qwen3-4B
4
3
42.99
29.22
New model family
MATH
Llama-3.1-8B
5
4
45.16
37.84
New task type
MBPP
Qwen3-4B
6
5
59.10
51.70
Appendix
Table 6: Lifecycle and alarm behaviour across base models and tasks. Accuracy is held-out performance at the listed round.
Method
Round 15
Validation stopping
Oracle checkpointing
Passive Ct stopping
Vanilla
28.93 (15)
42.99 (4)
42.99 (4)
42.99 (4)
STaR
36.37 (15)
45.18 (8)
45.43 (9)
44.87 (6)
SPIN
37.41 (15)
47.55 (7)
47.55 (7)
46.61 (6)
AZR
41.58 (15)
48.11 (8)
48.27 (9)
47.22 (7)
R-Zero
44.32 (15)
48.62 (10)
48.74 (9)
47.86 (8)
R-Diverse
45.47 (15)
49.58 (9)
49.91 (8)
48.36 (7)
Appendix
Table 7: MATH accuracy of the checkpoint returned by four stopping rules on the six baseline runs of Section 4.7 . The selected round is shown in parentheses.
Samples per round
Trials matching full-data round
Maximum shift
100%
94.2%
1 round
75%
91.3%
1 round
50%
85.8%
1 round
25%
77.6%
2 rounds
Appendix
Table 8: Bootstrap stability of the stopping round at four per-round data sizes. Each row summarises 1,000 trials.
Setting
Correlation
Alarm shift
Stopping shift
Same stopping round
Original
1.000
0
0
100%
Length matched
0.982±0.007
0
0
95.6%
Format standardised
0.969
0
0
100%
Both controls
0.958±0.010
0
0
93.1%
Appendix
Table 9: Stopping behaviour under surface-level controls. Correlation is Spearman against the original Ct trajectory. Ranges are standard deviations over the 100 length-matching repetitions.
Setting
Proxy data
Alarm round
Peak round
Rounds before peak
MATH
Previous MATH round
3
4
1
MBPP
Previous MBPP round
5
6
1
MATH → MBPP
Final MATH round
8
6
−2
MATH → MBPP
First MBPP round
5
6
1
Appendix
Table 10: Alarm timing under task shifts. The last column counts the rounds by which the alarm precedes the performance peak.
Base model
Main loop
Proxy +Ct
Relative overhead
Qwen3-4B-Base
8.2
0.35
4.1%
Qwen3-8B-Base
15.5
0.35
2.3%
Qwen3-14B-Base
26.8
0.35
1.3%
Appendix
Table 11: Measured per-round cost in GPU-hours at three base-model scales.
Department of Statistics, The George Washington University · Department of Statistics and Data Science, School of Management, Fudan University · Department of Decision Sciences, School of Business, The George Washington University