Richness Is Not Evenness: Ordered Renyi Crossings under Self-Training

Pranjal Srivastava

cs.CLNov 6, 2026

Richness Is Not Evenness: Ordered Renyi Crossings under Self-Training

Authors: Pranjal Srivastava

Organizations: University of Michigan

Abstract

Recursive self-training of language models, a setting studied in model collapse, can reduce vocabulary while making frequent words, mostly function words, less dominant. We fine-tune English GPT-2 on its own output for nine generations, vary the training-data sampling temperature, and measure every model with one fixed sampler. As training temperature rises, Renyi entropy changes turn from loss to gain at distinct temperatures: min-entropy first, then collision entropy, Shannon entropy and observed richness. Between the outer crossings, word types fall while the most-common word becomes less dominant. Untruncated training at T = 1 lies in this interval: at 13,000 words, generation 9 has 24% fewer types than generation 0 yet collision entropy is 0.91 bits higher. The ordering persists with a finer grid, different measurement samplers, a fixed word budget and five fresh training seeds. Min-entropy again crosses before richness in four further settings (in Hindi only at interpolated roots) and in 12 of 13 distinct measurements under eight text representations. Next-token identities and a categorical simulation isolate decoding and sampling effects; the crossing order remains empirical. Fine-tuning on human text can produce the same split while likelihood improves, and no single order's sign reliably identifies worse likelihood across the tested settings. We recommend reporting entropy profiles at stated budgets with an independent quality measure.

Figures & tables

Figure 1: English GPT-2 after nine rounds of recursive full fine-tuning, five seeds per cell. Changes compare generation 9 with generation 0 under the fixed measurement sampler (

T=1

, top-

p=0.95

) at common-length budgets. Left: measured means and seed standard deviations near the crossing region. Open circles mark linearly interpolated roots; inside the shaded interval richness falls while min-entropy rises, as measured at four temperatures. Right: collision-entropy changes at three training top-

p

values. Labels give the cross-perplexity ratio interpolated at each root; no model was trained at exactly those temperatures.

top- $p$	$T$	$\Delta H_{0}$	$\Delta H_{1}$	$\Delta H_{2}$	$\Delta H_{\infty}$	types	cross-ppl
$1.00$	$0.70$	$-4.35$	$-3.39$	$-1.50$	$-0.59$	$-95\%$	$11.6\times$
$1.00$	$0.90$	$-2.91$	$-2.26$	$-0.89$	$-0.23$	$-87\%$	$7.8\times$
$1.00$	$1.00$	$\mathbf{-0.81}$	$+0.08$	$\mathbf{+0.94}$	$+0.93$	$\mathbf{-43\%}$	$3.8\times$
$1.00$	$1.10$	$+0.88$	$+3.14$	$+4.87$	$+4.16$	$+85\%$	$8.1\times$
$1.00$	$1.25$	$+1.44$	$+4.36$	$+6.62$	$+6.66$	$+172\%$	$45.0\times$
$1.00$	$1.50$	$+1.80$	$+4.87$	$+7.25$	$+7.37$	$+248\%$	$109.8\times$

Table 1: Coarse 15-cell grid for English GPT-2 under full fine-tuning: five-seed means of the change from generation 0 to generation 9 at common-length budgets (

48{,}125

–

70{,}367

words). “Types” is the seed mean of the relative change in the number of word types, and cross-perplexity is the generation-9 value relative to generation 0. The largest seed standard deviation in the four entropy columns is

0.733

bits (

H_{0}

at top-

p=0.40

T=1

); per-seed values are released. Bold marks cells where richness and collision entropy move in opposite directions, or where

\Delta H_{2}

moves toward zero while cross-perplexity worsens. Figure 1 adds the five refinement temperatures.

Setting	script / unit	$T^{\ast}(H_{0})$	$T^{\ast}(H_{1})$	$T^{\ast}(H_{2})$	$T^{\ast}(H_{\infty})$	interval
GPT-2 124M, full-FT	Latin / word	$1.035$	$0.997$	$0.962$	$0.927$	$[0.93,1.03]$
GPT-2 124M, LoRA	Latin / word	$0.961$	$0.950$	$0.932$	$0.884$	$[0.88,0.96]$
OLMo-2-1B ( $1.48$ B), LoRA	Latin / word	$1.020$	$1.011$	$1.000$	$0.940$	$[0.94,1.02]$
german-gpt2, full-FT	Latin / word	$1.055$	$1.030$	$1.004$	$0.984$	$[0.98,1.05]$
gpt2-hindi, full-FT	Devanagari / word	$1.044$	$1.035$	$1.020$	$1.000$	$[1.00,1.04]$
Wenzhong, full-FT	Han / char	—	—	—	$1.007$	unit fails

Table 2: First upward crossings at top-

p=1

. Rows 1, 3 and 5 use five seeds and the others three. Row 1 includes both refinements (crossing brackets of 0.02–0.05 in temperature); the other grids are spaced 0.10–0.15 apart, so their third decimals are not resolved. The last column gives the interval between the outer crossings, rounded from the unrounded roots. In Hindi the

H_{\infty}

root is fragile (see text). The Han-only Chinese unit loses coverage before most roots can be bracketed.

	$\Delta$ at top- $p{=}1.0$				crossing $T^{\ast}$ at top- $p$
metric	$T{=}0.90$	$T{=}1.00$	$T{=}1.10$	$T{=}1.50$	$1.00$	$0.80$	$0.60$
$H_{\infty}$	$-0.233$	$+0.930$	$+4.160$	$+7.374$	$0.920$	$1.072$	$1.213$
$H_{2}$	$-0.891$	$+0.942$	$+4.867$	$+7.251$	$0.949$	$1.129$	$1.285$
self-BLEU- $4$	$+0.197$	$\mathbf{-0.019}$	$-0.060$	$-0.067$	$0.991$	$1.231$	—
distinct- $2$	$-0.485$	$\mathbf{+0.024}$	$+0.335$	$+0.337$	$0.995$	$1.207$	$1.416$
distinct- $3$	$-0.572$	$\mathbf{+0.008}$	$+0.052$	$+0.051$	$0.999$	$1.241$	$1.485$

Table 3: Coarse-grid comparison of generation 9 with generation 0 at each pair’s common-length budget, five-seed means, ordered by the top-

p=1

crossing. Entropies are in bits; the other scores lie in

[0,1]

, so their changes are not on a common scale. Bold marks untruncated training at

T=1

, where four comparison metrics barely move and do not agree in sign across seeds despite

3.8

times worse cross-perplexity. Rep-4 and self-BLEU cross downward because they rise with repetition. A dash marks too few valid segments for a complete five-seed self-BLEU bracket; cell-level coverage is in the release.

Setting	unit	gen. 0	truncated	untruncated	types (trunc./untrunc.)
			generation-9 $H_{2}$
English full-FT, GPT-2	word	$6.85$	$5.62\downarrow$	$7.48\uparrow$	$-89\%$ / $-29\%$
German full-FT, GPT-2	word	$7.51$	$5.67\downarrow$	$7.65\uparrow$	$-92\%$ / $-32\%$
Spanish full-FT, GPT-2	word	$6.32$	$4.80\downarrow$	$6.39\uparrow$	$-90\%$ / $-10\%$
English LoRA, GPT-2	word	$6.97$	$5.02\downarrow$	$8.07\uparrow$	$-92\%$ / $+46\%$
English LoRA, Qwen-0.5B	word	$6.61$	$5.50\downarrow$	$6.71\uparrow$	$-83\%$ / $-22\%$

Table 4: Ten self-training settings: three-seed means under the measurement sampler, with a shared generation 0. The

H_{2}

columns use the logged measurement samples at full length; type changes use the first 13,000 units of separate dumps (Table 6 ), because richness depends strongly on length. Arrows give the sign of the mean change from generation 0. The English row is an independent rerun of the four-arm study’s truncated and untruncated arms; its separate dumps give

5.65

and

7.46

bits in Table 8 , reflecting different measurement samples. On an English generation-0 text (the seed-0 baseline of the main grid), i.i.d. resampling at

N=13{,}000

gives a

0.013

-bit

H_{2}

bias (Appendix E.4 ); this is not a general correction. Character and word values have different scales. ∗ Sampled with the checkpoint’s default repetition penalty of 1.1; without it the truncated arm falls to

5.54

bits (§ 6.6 ).

$\alpha$	$0$	$0.5$	$1$	$1.5$	$2$	$3$	$5$	$10$	$\infty$
settings meeting the criterion	2	2	3	3	3	6	6	7	7

Table 5: Settings, out of ten, in which all truncated seeds move one way and all untruncated seeds the other, by Rényi order. Changes below

0.02

bits decide some entries, so individual classifications may not be stable across reruns. This sign criterion is stricter than the magnitude separation of § 6.4 . The counts include the penalized Qwen2.5-0.5B-Instruct setting (§ 6.6 ), which meets the criterion at

\alpha\leq 1.5

; with its penalty-free rerun instead, they are 1, 1, 2, 2, 3, 7, 7, 8 and 7.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Setting (truncated arm)	$\Delta H_{0}$	$\Delta H_{1}$	$\Delta H_{2}$	$\Delta H_{\infty}$
English full-FT, GPT-2	$-3.25$	$-2.49$	$-1.23$	$-0.45$
German full-FT, GPT-2	$-3.59$	$-3.46$	$-1.97$	$-1.14$
Spanish full-FT, GPT-2	$-3.30$	$-3.44$	$-1.46$	$-0.27$
English LoRA, GPT-2	$-3.60$	$-3.59$	$-1.96$	$-0.55$
English LoRA, Qwen-0.5B	$-2.55$	$-2.45$	$-1.25$	$-0.32$
English LoRA, SmolLM2-Instruct	$-3.04$	$-3.12$	$-1.67$	$-0.40$

Appendix

Table 6: Change in

H_{\alpha}

from generation 0 to generation 9 on the truncated arm: three-seed means at the fixed 13,000-unit budget, which every setting meets. Richness (

H_{0}

) falls everywhere. On the instruction-tuned Qwen checkpoint, sampled with its default repetition penalty of 1.1, the higher orders move the other way: the model restricts its vocabulary while using the remaining words more evenly. ∗ Without the penalty all four orders fall (Table 7 ).

penalty	sampler	$H_{2}$ , gen. 0	$H_{2}$ , gen. 9	$\Delta H_{0}$	$\Delta H_{1}$	$\Delta H_{2}$	$\Delta H_{\infty}$	types	cross-ppl
1.1	top- $p$ 0.60	$7.62$	$7.70$	$-1.58$	$-1.32$	$+0.07$	$+1.12$	$-66\%$	$16.4\times$
1.1	top- $p$ 0.80	$7.62$	$8.13$	$-1.30$	$-0.95$	$+0.47$	$+1.62$	$-59\%$	$21.8\times$
1.1	top- $p$ 0.95	$7.62$	$8.85$	$-0.45$	$+0.06$	$+1.29$	$+2.42$	$-27\%$	$5.5\times$
1.1	top- $p$ 1.0	$7.62$	$9.82$	$+0.44$	$+1.34$	$+2.35$	$+2.90$	$+36\%$	$3.4\times$
1.1	$T=1.5^{\dagger}$	$7.62$	$11.32$	$+1.60$	$+3.44$	$+5.48$	$+5.68$	$+203\%$	$3.0\times$
1.0	top- $p$ 0.60	$6.57$	$5.08$	$-3.20$	$-2.96$	$-1.53$	$-0.40$	$-89\%$	$2.2\times$

Appendix

Table 7: Qwen2.5-0.5B-Instruct under LoRA with the repetition penalty of 1.1 inherited from the checkpoint’s generation configuration, and with the penalty set to 1.0, for each training sampler (top-

p

T=1

, or

T=1.5

at top-

p=0.95

). Everything else is matched: initial adapters, random streams, corpus and budgets. Three-seed means. The two

H_{2}

columns are the logged two-batch measurement samples; the changes

\Delta

and “types” (the change in the number of word types) use the separate eight-batch dumps and compare generation 9 with generation 0 at 13,000 words; cross-perplexity is the generation-9 ratio to generation 0. The penalized top-

p

0.80 and 1.0 rows are the truncated and untruncated arms of Table 4 . † Generation-9 dumps are shorter than 13,000 words, so each seed’s changes use the common length of its generation-0 and generation-9 dumps (10,369–11,096 words). About 94% of their words are distinct, so the logged value, from a sample about a quarter as long, sits near

\log_{2}

of that sample’s length and differs from the dump-based endpoint.

Metric	truncated (top- $p$ $0.80$ )	untruncated (top- $p$ $1.0$ )	flags untruncated arm?
$H_{2}$	$6.86\!\to\!5.65\,\downarrow$	$6.86\!\to\!7.46\,\uparrow$	no
Shannon $H_{1}$	$9.14\!\to\!6.50\,\downarrow$	$9.14\!\to\!9.12\,\approx$	no
distinct-1	$0.086\!\to\!0.006\,\downarrow$	$0.086\!\to\!0.048\,\downarrow$	yes (gen. 2)
distinct-2	$0.55\!\to\!0.04\,\downarrow$	$0.55\!\to\!0.53\,\approx$	no
self-BLEU	$0.44\!\to\!0.93\,\uparrow$	$0.44\!\to\!0.39\,\downarrow$	no
MAUVE	$0.64\!\to\!0.10\,\downarrow$	$0.64\!\to\!0.46\,\downarrow$	yes (gen. 2)

Appendix

Table 8: Standard diversity metrics on the rerun of the four-arm study’s truncated and untruncated arms (the English row of Table 4 ), generation 0 to 9, three-seed means over full-length texts (70,106–77,122 words at generations 0 and 9). A metric flags the untruncated arm when it changes by at least 10% in the conventional collapse direction, an exploratory rule. Self-BLEU here is sentence BLEU-2 with NLTK’s method-1 smoothing on the first 80 eligible sentences, a different protocol from the grid’s BLEU-4. At 13,000 words (Table 4 ), the untruncated arm loses 29% of its types, while its rising

H_{2}

reflects more even use of a smaller observed vocabulary.

Figure 2: The four-arm English study with common random numbers, three seeds. Truncated training samplers lower word-level

H_{2}

from generation to generation, while untruncated training (top-

p=1

) drifts up . Held-out cross-perplexity rises fastest for the tightest nucleus, and mixing 10% real text into each training corpus limits the rise to

1.6\times

. The shaded band in the two

H_{2}

panels marks the generation-0 value read with the arms’ own samplers, averaged over arms and seeds (

6.81

bits); its

\pm 0.06

-bit width is for display only.

Figure 3: English GPT-2, generation 0 to 9. On the truncated arms, predictive perplexity

2^{\overline{H_{1}(q_{t})}}

falls while cross-perplexity on held-out real text rises. The first two panels average entropy over human contexts, using the arm’s own sampler or the measurement sampler; they do not score generated sequences. The rightmost group in each panel is the top-

p=0.95

arm with 10% real text.

Figure 4: High-temperature self-training (English GPT-2, training top-

p

fixed at

0.95

T=1

–

5

;

H_{2}

read with the measurement sampler; three seeds). Left: high training temperatures raise

H_{2}

. Centre: predictive perplexity

2^{\overline{H_{1}(q_{t})}}

on human contexts rises at high training temperatures; the

T=1

series has endpoint measurements only and is dashed. Right: every endpoint is worse than generation 0, with larger increases at high

T

; the dotted line is the generation-9 cross-perplexity of the top-

p=0.95

arm with 10% real text (about 82).

budget	reader $p$	$T^{\ast}(H_{\infty})$	$T^{\ast}(H_{2})$	$T^{\ast}(H_{1})$	$T^{\ast}(H_{0})$	$\Delta H_{0}$	$\Delta H_{\infty}$	split
common	$0.9$	$0.924$	$0.959$	$0.993$	$1.029$	$-0.746$	$0.932$	5/5
common	$0.95$	$0.927$	$0.962$	$0.997$	$1.035$	$-0.814$	$0.930$	5/5
common	$1$	$0.929$	$0.964$	$1.003$	$1.041$	$-0.882$	$0.951$	5/5
$13{,}000$	$0.9$	$0.925$	$0.959$	$0.984$	$1.016$	$-0.317$	$0.911$	5/5
$13{,}000$	$0.95$	$0.936$	$0.963$	$0.990$	$1.022$	$-0.400$	$0.929$	5/5
$13{,}000$	$1$	$0.939$	$0.964$	$0.996$	$1.029$	$-0.499$	$0.943$	5/5

Appendix

Table 9: Reader sensitivity on the eleven-temperature English grid (five seeds). All readers use

T=1

and top-

k=0

. The

p=0.95

rows repeat the primary measurement;

p=0.9

and

1

are matched readings of the same checkpoints. Roots are first upward interpolations of the seed mean. Changes at training

T=1

are in bits; “split” counts seeds with

\Delta H_{0}<0<\Delta H_{\infty}

measurement	$T^{\ast}(H_{0})$	$T^{\ast}(H_{1})$	$T^{\ast}(H_{2})$	$T^{\ast}(H_{\infty})$	ordered in $\alpha$
top- $p=0.95$ , top- $k=50$ , $T=1$	$1.010$	$0.955$	$0.937$	$0.928$	yes
top- $p=1$ , top- $k=50$ , $T=1$	$1.009$	$0.955$	$0.934$	$0.913$	yes
teacher-forced (no sampling)	—	$0.979$	$0.967$	$0.964$	yes

Appendix

Table 10: Crossing temperatures on the top-

p=1

axis under alternative measurements of the same endpoint checkpoints. Teacher-forced

H_{0}

is constant at

\log_{2}|\mathcal{V}|

for a dense softmax, so it has no crossing.

Setting	gen. 0	truncated	untruncated	hot
		(top- $p$ $0.80$ )	(top- $p$ $1.0$ )	( $T{=}1.5$ )
Spanish full-FT, GPT-2	$6.32$	$4.80$	$6.39$	$12.81$
English LoRA, Qwen-Instruct ∗	$7.62$	$8.13$	$9.82$	$11.32$
English LoRA, Qwen-Instruct, no penalty	$6.57$	$5.54$	$6.68$	$11.31$
Chinese full-FT, Wenzhong	$8.20$	$6.28$	$7.02$	n/a †

Appendix

Table 11: Generation-9

H_{2}

under three training samplers, three-seed means of the logged two-batch measurement samples. Their budgets are unequal: the matching eight-batch generation-9 dumps shrink to 10.4–11.1k words for the instruction-tuned

T=1.5

arm and grow to about 41k for the Spanish one, and the logged samples are about a quarter as long, so the ordering is descriptive and unequal-budget bias is not controlled. Spanish spans

8.01

bits. With the penalty, the instruction-tuned setting keeps the arm ordering but places every arm above generation 0; without it, the truncated arm falls below generation 0. ∗ Sampled with the checkpoint’s default repetition penalty of 1.1. † About 145 Han characters per corpus: the entropy is defined, but coverage is insufficient for this comparison.

Setting	script	bearing tokens	vocabulary %	gen. 0	gen. 9	error
		tokenizer		observed
English, GPT-2	a–z	$44{,}170$	$79.5$	$75.7$	$\mathbf{79.3}$	$0.2$
Hindi, gpt2-hindi	Devanagari	$19{,}171$	$28.0$	$77.1$	$\mathbf{23.9}$	$4.1$
Chinese, Wenzhong	Han	$51$	$0.0$	$83.8$	$\mathbf{0.05}$	$0.0$

Appendix

Table 12: Tokenizer composition and observed script share under high-temperature self-training. “Bearing tokens” counts entries of each tokenizer’s

50{,}257

-entry base vocabulary, whose separate decoding contains a character of the target script; “vocabulary %” is that script’s share of characters in the decoded vocabulary. Observed shares are of all characters, including whitespace, at training

T=2

(English, Hindi) or

T=1.5

(Chinese); they are three-seed means, and errors are percentage points rounded to one decimal. Vocabulary composition serves only as a descriptive reference.

rule	above cutoff	no training	human adaptation
magnitude: $M_{g}>1$ bit	$92/95$	$0/6$	$0/3$
profile: $M_{g}>1$ or $D_{g}$	$95/95$	$0/6$	$3/3$
profile with no sign floor	$95/95$	$6/6$	$3/3$

Appendix

Table 13: Alarm rules evaluated over generations 1–9: number of trajectories on which each rule ever fires. “Above cutoff” trajectories reach at least twice their generation-0 cross-perplexity; the no-training and human-adaptation controls are described below.

	$H_{0}$	$H_{1}$	$H_{2}$	$H_{\infty}$	distinct-1	distinct-2	$H_{0}{+}H_{2}$	$H_{2}{+}$ distinct-2
Spearman $\rho$ , log cross-ppl	$-0.52$	$-0.39$	$-0.22$	$-0.04$	$-0.48$	$-0.42$	—	—
mean held-out AUROC	$0.818$	$0.779$	$0.650$	$0.575$	$0.817$	$0.860$	$0.961$	$0.963$
EN $T{=}2.0$ ( $n_{+}{=}24$ )	$0.000$	$0.000$	$0.000$	$0.000$	$0.000$	$0.340$	$0.972$	$1.000$
Qwen-Instruct ∗ ( $n_{+}{=}29$ )	$0.666$	$0.606$	$0.412$	$0.303$	$0.662$	$0.648$	$0.906$	$0.909$
ES high- $T$ ( $n_{+}{=}1$ )	$0.038$	$0.038$	$0.231$	$0.192$	$0.038$	$0.423$	$0.692$	$0.808$

Appendix

Table 14: Pooled correlations and fixed-feature leave-one-setting-out AUROCs on 660 corpora from 16 settings. Each feature set is held fixed across folds, and the mean AUROC excludes the two single-class folds; the last three rows are individual held-out settings, with

n_{+}

positives. Nested selection instead chooses

(H_{2},\text{distinct-}2)

in 11 of 14 folds and has mean AUROC

0.897

. Shared model families limit this to transfer across conditions. ∗ Sampled with the checkpoint’s repetition penalty of 1.1.

rule (fires when…)	fires on	median lead	range	never fires	reverses
$\|\Delta H_{0}\|>1$ or $\|\Delta H_{\infty}\|>1$ or signs disagree †	$\mathbf{75/75}$	$\mathbf{+1}$	$-1$ to $+3$	$\mathbf{0}$	$\mathbf{0}$
$\|\Delta H_{2}\|>1$	$65/75$	$+1$	$-6$ to $+2$	$10$	$1$
$\Delta H_{2}<-1$ (one-sided)	$33/75$	$-1$	$-6$ to $+1$	$42$	$1$
$\Delta H_{0}<-1$ (one-sided)	$40/75$	$+1$	$+0$ to $+1$	$35$	$0$
distinct-2 movement	$40/75$	$+0$	$-2$ to $+1$	$35$	$0$
rep-4 movement	$40/75$	$+1$	$+0$ to $+1$	$35$	$0$

Appendix

Table 15: Lead time of alarm rules on the 75 coarse-grid trajectories. Positive lead means the rule fires before the quality threshold is reached. A trajectory counts as reversing if, at some generation, a rule that fired at the previous generation stops firing. The distinct-2 and rep-4 rules fire when the absolute change exceeds 20 times the largest difference between generation-0 texts (

0.309

and

0.029

). † Both orders must exceed the

0.20

-bit floor for the sign clause to fire.

arm (same training regime)	median ppl ratio	median $\Delta H_{0}$	median $\Delta H_{2}$
untruncated, $T=1$	$1.94\times$	$-1.21$	$-0.26$
truncated, top- $p{=}0.80$	$6.30\times$	$-4.75$	$-2.38$
hot, $T{=}2.0$ , top- $p{=}0.95$	$3.83\times$	$\mathbf{+1.53}$	$\mathbf{+5.80}$

Appendix

Table 16: Regime-controlled Pythia-160m endpoints (five seeds per arm).

inject at generation $b$	$1$	$3$	$5$	$8$
final cross-perplexity, injected	$18.1\times$	$14.1\times$	$7.5\times$	$\mathbf{2.3\times}$
ratio to its own placebo	$0.63$	$0.46$	$0.27$	$\mathbf{0.07}$
same, at matched lag $0$	$0.89$	$0.58$	$0.24$	$0.06$
seeds improved (of 5)	$4$	$5$	$5$	$5$
$H_{0}$ recovered (bits)	$-0.07$	$+0.23$	$+0.30$	$\mathbf{+0.89}$

Appendix

Table 17: Checkpoint-branch intervention: medians over five seeds. The unbranched no-injection endpoint is

30.7\times

its generation-0 cross-perplexity. The third row gives the same ratio at the injection generation itself (lag 0) instead of at generation 9. The richness row compares each branch with its placebo on the first 13,000 words of their generation-9 measurement texts.

inject at generation $b$	sequential trainer	matched trainer	disjoint pool
$1$	$0.63$ (4/5)	$0.73$ (5/5)	$\mathbf{1.15}$ (1/5)
$3$	$0.46$ (5/5)	$0.41$ (5/5)	$\mathbf{1.10}$ (2/5)
$5$	$0.27$ (5/5)	$0.17$ (5/5)	$0.55$ (5/5)
$8$	$0.07$ (5/5)	$0.03$ (5/5)	$0.19$ (5/5)
$H_{0}$ recovered at $b{=}8$ (bits)	$+0.89$	$+0.92$	$+0.21$
ordered in $\alpha$ at $b{=}8$ ?	yes	yes	no

Appendix

Table 18: Branch ratios of final cross-perplexity to the branch’s own placebo (seeds improved of 5), for the sequential trainer (raw text), a trainer matched to the main harness, and the matched trainer with a disjoint injection pool (both using space-joined NLTK word lists). Richness rows use the first 13,000 words of generation 9. “Ordered in

\alpha

” means the median recovery is non-increasing over

\alpha=0,1,2,\infty

reference	GPT-2 $T=1$	Pythia above $2\times$	Pythia AUROC
Persuasion	$3.83\times$	$12/15$	$0.944$
Sense and Sensibility	$3.43\times$	$11/15$	$0.977$
Moby Dick	$4.48\times$	$15/15$	—
The Adventures of Buster Bear	$4.47\times$	$15/15$	—
three additional works, geometric mean	$4.10\times$	$15/15$	—

Appendix

Table 19: Dependence of the quality label on the reference text, at generation 9. GPT-2 values are mean cross-perplexity ratios over five seeds at top-

p=1

T=1

. Pythia keeps the same

\max(|\Delta H_{0}|,|\Delta H_{2}|)

score and

2\times

cutoff. A dash marks a single label class. The final row first averages log ratios over the three additional works within each endpoint; no score or threshold is refitted.

	English grid	Hindi	total
distinct measured curves	$6$	$7$	$13$
$\Delta H_{2}<0$ at the lowest and $>0$ at the highest swept temperature	$6$	$6$	$12/13$
outer ordering $T^{\ast}(H_{\infty})<T^{\ast}(H_{0})$	$6$	$6$	$12/13$
full ordering	$5$	$4$	$9/13$

Appendix

Table 20: Crossing behavior across representations of the same stored generations, counting each distinct curve once.

$K$	$N$	$\min_{\mathrm{heavy}}Nq_{i}/\ln K$	missing $/K$	$\log_{2}\frac{K}{\mathbb{E}S_{N}}$	$\log_{2}K-\mathbb{E}\log_{2}S_{N}$
$500$	$19{,}310$	$12.3$	$0.183935$	$0.293244$	$0.293494$
$2{,}000$	$115{,}547$	$15.1$	$0.183939$	$0.293251$	$0.293314$
$8{,}000$	$646{,}157$	$17.9$	$0.183940$	$0.293252$	$0.293268$
$32{,}000$	$3{,}443{,}498$	$20.7$	$0.183940$	$0.293252$	$0.293256$
$128{,}000$	$17{,}701{,}447$	$23.4$	$0.183940$	$0.293252$	$0.293253$

Appendix

Table 21: Occupancy under a law with

K/2

light types of mass

1/N

. Values are exact to the displayed precision: the missing fraction and

\log_{2}(K/\mathbb{E}S_{N})

follow from the closed form in Proposition 6 , and the plug-in deficit from an occupancy recursion over the light types (each heavy type is absent with probability below

10^{-30}

$N$	$\gamma=1$	$\gamma=2$	$\log_{2}N$
$300$	$0.6609$	$8.2312$	$8.2288$
$1{,}000$	$0.6615$	$9.9665$	$9.9658$
$3{,}000$	$0.6616$	$11.5510$	$11.5507$

Appendix

Table 22: Exact richness deficit

\log_{2}K-\mathbb{E}\log_{2}S_{N}

for uniform laws on

K=N^{\gamma}

types, computed by the occupancy recursion over draws.

$T$	$\bar{C}$	$V$	$C(\bar{p})$	$H_{2}(\bar{p})$
0.1	0.90471	0.88476	0.01995	5.647
0.5	0.58531	0.56781	0.01749	5.837
1.0	0.26548	0.25539	0.01008	6.632
2.0	0.02557	0.02499	0.00058	10.751

Appendix

Table 23:

V

-bridge values on 200 fixed human-text contexts (Qwen2.5-3B-Instruct, per-token BPE distributions over the full vocabulary).

Figure 5: The

V

-bridge on Qwen2.5-3B-Instruct, over 200 fixed human-text contexts and the full

151{,}936

-token vocabulary. Left: mean conditional collision decomposes as

\bar{C}=C(\bar{p})+V

. Right: the entropy of the mean conditional lies above

-\log_{2}\bar{C}

. All plotted entropies are computed from conditional softmax probabilities.

	Common endpoint length		13,000 words
Order	root	95% interval	root	95% interval
$H_{\infty}$	$0.906$	$[0.871,0.937]$	$0.903$	$[0.856,0.944]$
$H_{2}$	$0.962$	$[0.955,0.969]$	$0.963$	$[0.951,0.971]$
$H_{1}$	$0.999$	$[0.996,1.002]$	$0.993$	$[0.990,0.996]$
$H_{0}$	$1.032$	$[1.026,1.038]$	$1.020$	$[1.016,1.026]$

Appendix

Table 24: Fresh-seed replication: first upward crossing temperatures and 95% paired-seed bootstrap intervals. The same five resampled seed indices are used across all temperatures. Intervals condition on the selected grid and exclude interpolation error.

Rule	Recursive	Human adaptation	Frozen
Magnitude only, $M_{g}>1$ bit	$3/5$	$0/5$	$0/5$
Profile, $0.20$ -bit sign floor	$5/5$	$5/5$	$0/5$
Profile, $0.02$ -bit sign floor	$5/5$	$5/5$	$5/5$
Profile without a sign floor	$5/5$	$5/5$	$5/5$

Appendix

Table 25: Trajectories ever flagged in generations 1–9 under the primary reader, five seeds per arm. Recursive runs cross the reference-likelihood cutoff; human and frozen runs stay below it. All 150 measurement texts reach the 13,000-word budget.

Held-out family	Nested	$H_{0}+H_{2}$	$H_{2}+\text{distinct-}2$	$\|\Delta H_{0}\|$	Generation
GPT-2	$0.871$	$0.852$	$0.840$	$0.888$	$0.778$
Qwen2.5	$0.936$	$0.941$	$0.895$	$0.838$	$0.885$
SmolLM2	$0.999$	$0.999$	$1.000$	$0.999$	$0.900$
Equal-family mean	$0.935$	$0.931$	$0.912$	$0.908$	$0.855$

Appendix

Table 26: AUROC when a whole model family is withheld. The last row weights the three families equally and averages unrounded values. All predeclared methods are shown.

Self-Training for Language Models

Self-training for language models uses their own generations, feedback, and interaction traces as supervision to improve capabilities with less labeled data.

See all 90 papers

Oct 31, 2026

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence

Recursive self-training can collapse language models, and many proposed remedies target overconfidence, yet their effects on confidence are rarely measured. We study a five-round self-training loop on Qwen3-1.7B, using three seeds per compared method, and distinguish confidence temperature (drift from a calibrated starting point) from discrimination (ranking correct answers above incorrect ones). Accuracy and overall discrimination remain nearly unchanged, but confidence sharpens and the most confident answers become less reliable. Matching predictive entropy to the calibrated starting model each round recalibrates nearly as well as label-based temperature fitting, at both 1.7B and 8B. Sharpening is not inherent to self-training: exact samples from a model's distribution have zero expected gradient. Instead, most drift comes from fitting each generated answer twice, with top-p truncation contributing the rest; removing both keeps entropy on generated outputs flat. Remedies affect the two properties differently: truncated cross-entropy, an entropy bonus, and single-pass training preserve temperature without improving discrimination, while focal loss and multi-sample targets improve discrimination but cause underconfidence. Reference-data mixing also overshoots. Pairing a ranking-improving loss with label-free recalibration achieves both.

Dipto Sumit, Syeda Fabiayee Jahan, Farig Sadeque

BRAC University, Dhaka, Bangladesh

Self-Training for Language Models Confidence Estimation in Language Models Language Model Calibration Data Repetition in Language Model Training

Ask PDF

May 29, 2026

Fine-Tuned Language Models Exhibit Improved Information Conveyance

Fine-tuning is often believed to reduce uncertainty and diversity in large language models, but existing analyses overlook output length, a key confounder, and therefore fail to capture how uncertainty is distributed across an entire generation rollout. To address this, we propose Canopy Entropy (

\mathrm{CE}^\star

), a measure that views language generation from a tree perspective, where "canopy" represents the space of all possible rollouts, making

\mathrm{CE}^\star

naturally quantify the effective size of the generation space.

\mathrm{CE}^\star

jointly captures uncertainty in both the output length

N

and the generated sequence

Y_{1:N}

-- indeed, we show that it equals total Shannon entropy

H(N, Y_{1:N}\mid X)

, where

X

denotes the prompt. This formulation yields interpretable metrics, including a length-entropy correlation term

ρ(N, r_N)

, where

r_N

is the entropy rate, quantifying information conveyance efficiency by indicating whether longer outputs are more or less informative per token. Empirically, across tasks and model families, we find that fine-tuned models predominantly exhibit more positive correlation

ρ(N, r_N)

, even when total entropy decreases. Furthermore, after controlling for model family, task, prompt, and output-length effects, we find that fine-tuning nearly triples the correlation strength between entropy rate and semantic diversity, suggesting that aligned models convert token uncertainty into semantic diversity more efficiently. Overall, these results demonstrate that fine-tuning does not simply reduce uncertainty, but fundamentally reorganizes it into more informative and semantically meaningful generations. Our code is available at https://github.com/WeiyiTian/canopy-entropy.

Yuwei Cheng, Weiyi Tian, Haifeng Xu

Department of Statistics, University of Chicago, Chicago, IL 60637 · Department of Data Science, University of Chicago, Chicago, IL 60637 · Department of Computer Science, University of Chicago, Chicago, IL 60637

Language Model Generation Evaluation LLM Fine-Tuning Language Modeling LLM Uncertainty Estimation Diverse Text Generation Fine-Tuning Language Models+2

Ask PDF

May 20, 2026

Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies

Successive self-training on a language model's own outputs is widely characterized as a process of flattening: diversity drops, distributions narrow, and the text becomes "more like itself." We provide evidence that this characterization is incomplete. Across eleven generations of self-training on five models (GPT-2 124M, Pythia-410M, Pythia-1.4B, OPT-1.3B, Pythia-2.8B), language is not flattened uniformly -- it is restructured. Surface markers (discourse connectives, hedges, em-dashes) rise, while mid- and deep-syntactic structures (questions, parentheticals, passives, subjunctives) collapse. We formalize this asymmetric collapse as the Structural Depth Hypothesis (SDH): the per-generation decay rate of a linguistic feature is predicted primarily by its structural depth -- the number of nested syntactic dependencies it requires -- and only secondarily by its generation-zero output frequency. Pooling 17-feature panels from five models spanning three architecture families (N=85), the pooled Spearman correlation is rho=0.540 (p < 10^{-6}; cluster-bootstrap 95% CI [0.434, 0.634]), while frequency is a substantially weaker predictor (rho=0.225). A matched human-text fine-tuning control yields rho=0.039 (p=0.88), confirming the gradient is self-training-specific. We further document a Superficial Complexity Paradox: aggregate complexity proxies (dep-tree depth, TTR, word length) all rise as the underlying clause structure dies, with direct implications for training-data curation and LLM-text detection.

Ming Liu

Amazon

Self-Training for Language Models Syntactic Generalization in Language Models Model Collapse Language Modeling Language Model Self-Improvement Recursive Language Models

Ask PDF

Abstract

Self-Training for Language Models

Explore similar work

Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence

Fine-Tuned Language Models Exhibit Improved Information Conveyance

Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies