We propose Autolearn, a framework that enables language models to learn from documents they read, with no external supervision. Passages that produce anomalously high per-token loss are flagged, verified through a self-generated Q&A chain, and trained on with conviction-proportional β2 adjustment. We introduce the perturbation gap (paraphrase-to-original perplexity ratio) as a metric that distinguishes memorization from understanding. The key mechanism is the training data format: Q&A-format training drives the perturbation gap below the pre-trained baseline (2.098 vs. 2.204, Δ=−0.106, >10σ), suppressing token-sequence memorization, while standard fine-tuning's best attempt remains within noise (Δ=−0.010, <1σ). Across four models spanning Qwen3 and Phi-4 families, Autolearn is the only method that enters this regime. Stochastic evaluation reveals passage-specific knowledge acquisition: the probability of generating a correct novel fact rises from 6% to 54% after training (p<10−4), and Q&A format outperforms standard fine-tuning on genuinely novel facts. The system is self-extinguishing: learned content reduces surprisal below threshold and is skipped on re-encounter.
Figures & tables
Figure 1: Autolearn pipeline overview. Stage 1 detects surprising passages via passage-level surprisal. Stage 2 generates Q&A pairs, checks consistency against existing knowledge, grades each passage by conviction depth k , and digests epistemic boundaries. Stage 3 trains on verified content with conviction-proportional β2 adjustment.
Category
k>0
Full chain ( k=N )
Mean k/N
Novel (20)
20 (100%)
14 (70%)
0.82
Corrupt (19)
19 (100%)
13 (68%)
0.79
Table 1: Stage 2 conviction depth distribution under graduated accept (39 testable passages; the topic-overlap-excluded corrupt passage is omitted from this analysis but contributes to the training corpus). All testable passages produce k>0 and enter training. Discrimination occurs through conviction depth: corrupt passages that fail early receive weak learning ( β2 near 0.999), while those that pass the full chain are indistinguishable from novel, the structural blind spot of single-model verification.
Figure 2: (a) Each point represents a (method, lr) condition at its sweet-spot epoch. Error bars: ±1 std ( n=5 ); SFT is deterministic. Pre-trained perturbation gap ( 2.204 ). Below = pure semantic learning. Autolearn is the only method entering the sub-baseline region. (b) Per-question change in stochastic D.Nov hit rate (temp 0.7 , n=50 ). Only the 8 questions with significant change ( p<0.05 , per-question binomial test) are shown; 13 questions show no significant change. Five questions show knowledge acquisition (right), three show forgetting (left). ∗∗p<0.01 ; ∗p<0.05 .
Condition
lr
Sweet spot
Target PPL
Pert. Gap
D.Cor
Pre-trained
—
—
8.52
2.204
42.1%
SFT
5×10−6
3 ep
7.04 ( −17% )
2.194
36.8%
SFT
10−5
2 ep
6.71 ( −21% )
2.201
42.1%
SFT
5×10−5
1 ep
5.58 ( −35% )
2.38
47.4%
SFT
10−4
1 ep
4.29 ( −50% )
2.94
47.4%
Autolearn ( r=1.0 )
5×10−6
1 ep
7.70 ( −10% )
2.10
42.1%
Table 2: Stage 3 results across learning rates ( n=5 for lr∈{10−5,5×10−5} ; single run otherwise). r=1.0 = Q&A format with standard AdamW (no β2 gating). r=0.999 = full Autolearn. The two produce nearly identical perturbation gaps, confirming that Q&A format is the mechanism driving memorization suppression. D.Cor = corrupt-direct accuracy (higher is better).
lr=5×10−6 , 3 ep
Topic
Pre
SFT
r=1.0
r=0.999
shingles vaccine (Wales)
100%
100%
94%
92%
recombinant vaccine (AS01)
18%
6%
0%
0%
lightning (mesoscale)
6%
8%
12%
24%
keratin (dentinal tubule)
100%
90%
68%
72%
graphene (SiC)
6%
28%
54%
48%
Table 3: Knowledge acquisition via stochastic evaluation (temperature 0.7 , n=50 , Qwen3-14B). All trained conditions use identical lr and epoch. Q&A format ( r=1.0 , r=0.999 ) outperforms SFT on genuinely novel facts. Bold = best trained condition. Questions with 0% across all conditions omitted.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Condition
Pert gap
Gap Δ
Fac. D.Cor
Reas. D.Nov
Reas. D.Cor
Reas. A.Cor
Qwen3-14B
Pre-trained
2.204
—
42%
19%
95%
92%
SFT
2.201
−0.003
42%
29%
100%
92%
Autolearn
2.114
−0.090
43%
29%
100%
92%
Qwen3-8B
Pre-trained
2.235
—
53%
33%
95%
92%
SFT
2.191
−0.044
53%
33%
95%
92%
Autolearn
2.146
−0.089
58%
33%
90%
92%
Appendix
Table 4: Cross-model results at sweet-spot epoch (auto-stop, r=0.999 ). Gap Δ = pert gap − baseline. Autolearn achieves lower perturbation gap than SFT in all four models and is the only method that enters the sub-baseline region. Bold = best in group.
Condition
PPL Δ %
Pert gap
Fac. D.Nov
Fac. D.Cor
Reas. D.Nov
Reas. D.Cor
Pre-trained
—
2.204
24%
42%
19%
95%
With passage, 5×10−6
−10%
2.100
24%
42%
29%
100%
No passage, 5×10−6
−6%
2.106
24%
42%
29%
100%
With passage, 10−5
−12%
2.108
19%
43%
29%
100%
No passage, 10−5
−6%
2.078
19%
42%
24%
100%
Appendix
Table 5: No-passage ablation. Q&A pairs alone produce identical factual D.Cor and reasoning D.Cor as with-passage training. Source windows contribute PPL reduction (token-level familiarity) but not mechanistic understanding. The lowest perturbation gap observed in all experiments ( 2.078 ) occurs in the no-passage condition.
Factual scoring
Reasoning scoring
Condition
D.Nov
D.Cor
A.Cor
D.Nov
D.Cor
A.Cor
Pre-trained
23.8%
42.1%
90%
19%
100%
92%
SFT 5×10−6 (3ep)
23.8%
36.8%
90%
29%
100%
92%
SFT 10−5 (2ep)
23.8%
42.1%
90%
29%
100%
92%
SFT 5×10−5 (1ep)
23.8%
47.4%
90%
33%
100%
92%
Autolearn 5×10−6 (1ep)
23.8%
42.1%
90%
29%
100%
92%
Appendix
Table 6: Dual evaluation (greedy decoding, Qwen3-14B). D.Nov = novel-direct (higher = more learned), D.Cor = corrupt-direct (higher = better resistance), A.Cor = adjacent-corrupt (constant ≥90% , no corruption spread). Autolearn achieves the highest D.Cor ( 50.5% ) and D.Nov ( 38% ) simultaneously. Factual D.Nov drops from 23.8% to 19.1% because Q&A training shifts the model’s default response from fact-oriented to mechanism-oriented; stochastic sampling confirms that the factual knowledge is retained in the probability distribution.