Recursive self-training can collapse language models, and many proposed remedies target overconfidence, yet their effects on confidence are rarely measured. We study a five-round self-training loop on Qwen3-1.7B, using three seeds per compared method, and distinguish confidence temperature (drift from a calibrated starting point) from discrimination (ranking correct answers above incorrect ones). Accuracy and overall discrimination remain nearly unchanged, but confidence sharpens and the most confident answers become less reliable. Matching predictive entropy to the calibrated starting model each round recalibrates nearly as well as label-based temperature fitting, at both 1.7B and 8B. Sharpening is not inherent to self-training: exact samples from a model's distribution have zero expected gradient. Instead, most drift comes from fitting each generated answer twice, with top-p truncation contributing the rest; removing both keeps entropy on generated outputs flat. Remedies affect the two properties differently: truncated cross-entropy, an entropy bonus, and single-pass training preserve temperature without improving discrimination, while focal loss and multi-sample targets improve discrimination but cause underconfidence. Reference-data mixing also overshoots. Pairing a ranking-improving loss with label-free recalibration achieves both.
Figures & tables
Figure 1: Each replicated protocol on the two properties of confidence (mean ± sd over three seeds). Protocols in green keep a calibrated model calibrated (shaded band, ∣ drift ∣≤0.02 ); those in orange improve discrimination; violet is 50% reference data. The dotted line marks the starting model g0 . No protocol reaches the upper centre, where both properties would be good.
Figure 2: All protocols, sorted by calibration drift. (a) Drift at g5 from a calibrated g0 . Circles are seeds, bars are means, and the shaded band marks ∣ drift ∣≤0.02 . Colours follow Figure 1 , and grey labels mark arms with a single seed. (b) ECE at g5 under three temperatures: one fitted at g0 and never updated (stale), one chosen every round without labels by entropy matching (§ 4.1 ), and one refitted every round on labels (oracle). The label-free temperature comes within 0.010 of the oracle for every protocol except label smoothing.
Entropy kept
Accuracy
AUROC
Calibration drift
ECE at g5
Protocol (3 seeds each)
per round
at g5
at g5
from calibrated g0
stale T0
label-free
Standard self-training (CE)
0.795 ± 0.005
0.567 ± 0.004
0.704 ± 0.010
+0.049 ± 0.006
0.090 ± 0.006
0.090 ± 0.012
Hold the calibration fixed point
Half data, 2 epochs
0.881 ± 0.003
0.564 ± 0.010
0.721 ± 0.016
− 0.003 ± 0.021
0.083 ± 0.011
0.082 ± 0.007
One epoch per round
0.944 ± 0.002
0.569 ± 0.008
0.727 ± 0.020
− 0.017 ± 0.004
0.091 ± 0.011
0.080 ± 0.008
TCE, τ=0.9 ( Zibakhsh Shabgahi et al., 2026 )
0.833 ± 0.006
0.568 ± 0.005
0.714 ± 0.008
− 0.019 ± 0.019
0.092 ± 0.001
0.088 ± 0.009
Table 1: Standard self-training and the seven protocols run with three seeds (mean ± sd). Entropy kept per round is κ from § 3 ; a value of 1 means no sharpening. Calibration drift is mean confidence minus accuracy at g5 under a temperature fitted once at g0 (cross-fitted); positive values are overconfidence. The last two columns give ECE at g5 with that stale temperature and with label-free recalibration. At g0 , accuracy is 0.573 and AUROC 0.711. Qwen3-1.7B, all 600 Global-MMLU items.
Figure 3: The model’s mean entropy on the answers it generates for training, relative to round 0. Standard self-training (two epochs, top- p 0.95) sharpens the model steadily. Exact sampling (top- p off) slows the sharpening, a single pass per round slows it more, and the two together keep the entropy flat. Thin lines are individual seeds.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Protocol
Seeds
Entropy kept/round
Acc. g5
AUROC g5
Calibration drift
Raw Δ ECE
Standard self-training (CE)
3
0.795
0.567
0.704
+0.049
+0.015
Half data, 2 epochs
3
0.881
0.564
0.721
− 0.003
+0.007
One epoch per round
3
0.944
0.569
0.727
− 0.017
− 0.001
TCE, τ=0.9 ( Zibakhsh Shabgahi et al., 2026 )
3
0.833
0.568
0.714
− 0.019
+0.005
Entropy bonus ( Pereyra et al., 2017 )
3
0.858
0.565
0.715
+0.000
+0.014
Focal loss (CAL family) ( Zibakhsh Shabgahi et al., 2026 )
3
0.969
0.565
0.742
− 0.132
− 0.034
Appendix
Table 2: Every protocol we ran, including those with a single seed. Raw Δ ECE is the change in uncalibrated ECE from g0 to g5 ; it favours any protocol that softens the overconfident base model.
Protocol ( g5 ; 3 seeds each)
AUROC
Margin AUROC
Average precision
Acc. of top 20%
g0 (shared start)
0.711
0.710
0.782
0.867
Standard self-training (CE)
0.704 ± 0.010
0.704 ± 0.011
0.741 ± 0.011
0.800 ± 0.030
Half data, 2 epochs
0.721 ± 0.016
0.720 ± 0.016
0.757 ± 0.008
0.817 ± 0.008
One epoch per round
0.727 ± 0.020
0.726 ± 0.020
0.781 ± 0.011
0.844 ± 0.013
TCE, τ=0.9 ( Zibakhsh Shabgahi et al., 2026 )
0.714 ± 0.008
0.713 ± 0.007
0.767 ± 0.004
0.831 ± 0.024
Entropy bonus ( Pereyra et al., 2017 )
0.715 ± 0.006
0.714 ± 0.005
0.752 ± 0.010
0.831 ± 0.005
Appendix
Table 3: Discrimination at g5 under four measures (mean ± sd over three seeds). All are computed from exact debiased log-probabilities, which contain no ties.
Model
Protocol
Stale T0
Label-free
Oracle
1.7B
Standard self-training (CE)
0.090
0.090
0.089
1.7B
Half data, 2 epochs
0.083
0.082
0.083
1.7B
One epoch per round
0.091
0.080
0.084
1.7B
TCE, τ=0.9 ( Zibakhsh Shabgahi et al., 2026 )
0.092
0.088
0.085
1.7B
Entropy bonus ( Pereyra et al., 2017 )
0.080
0.083
0.086
1.7B
Focal loss (CAL family) ( Zibakhsh Shabgahi et al., 2026 )
0.147
0.094
0.091
Appendix
Table 4: ECE at g5 with a temperature fitted once at g0 (stale), chosen without labels by entropy matching (§ 4.1 ), and refitted on labels every round (oracle), with two-fold cross-fitting. At 8B the probabilities were not stored at full precision, so the label-free and oracle temperatures there use the same saturated scores.
Self-training for language models uses their own generations, feedback, and interaction traces as supervision to improve capabilities with less labeled data.
We introduce Self-Anchoring Calibration Drift (SACD), a hypothesized tendency for large language models (LLMs) to show systematic changes in expressed confidence when building iteratively on their own prior outputs across multi-turn conversations. We report an empirical study comparing three frontier models -- Claude Sonnet 4.6, Gemini 3.1 Pro, and GPT-5.2 -- across 150 questions spanning factual, technical, and open-ended domains, using three conditions: single-turn baseline (A), multi-turn self-anchoring (B), and independent repetition control (C). Results reveal a complex, model-heterogeneous pattern that partially diverges from pre-registered hypotheses. Claude Sonnet 4.6 exhibited significant decreasing confidence under self-anchoring (mean CDS = -0.032, t(14) = -2.43, p = .029, d = -0.627), while also showing significant calibration error drift (F(4,56) = 22.77, p < .001, eta^2 = .791). GPT-5.2 showed the opposite pattern in open-ended domains (mean CDS = +0.026) with significant ECE escalation by Turn 5. Gemini 3.1 Pro showed no significant CDS (t(14) = 0.38, p = .710), but its Condition C data reveals a striking ECE pattern: without self-anchoring, Gemini's calibration error drops from .327 to near zero across repetitions, whereas self-anchoring holds ECE flat at approximately .333 -- indicating that SACD can manifest as suppression of natural calibration improvement rather than ac
Calibration is central to reliable semantic uncertainty quantification, yet prior work has largely focused on discrimination, neglecting calibration. As calibration and discrimination capture distinct aspects of uncertainty, focusing on discrimination alone yields an incomplete picture. We address this gap by systematically evaluating both aspects across a broad set of confidence measures. We show that current approaches, particularly fixed-temperature heuristics, produce systematically miscalibrated and poorly discriminative semantic confidence distributions. We demonstrate that optimising a single scalar temperature, which, we argue, provides a suitable inductive bias, is a surprisingly simple yet effective solution. Our exhaustive evaluation confirms that temperature scaling consistently improves semantic calibration, discrimination, and downstream entropy, outperforming both heuristic baselines and more expressive token-level recalibration methods on question-answering tasks.
Tom A. Lamb, Desi R. Ivanova, Philip H. S. Torr +1
University of Oxford · University of Toronto & Vijil
Large language models can report a numerical confidence together with generated content, but it is unclear whether this report is more than calibrated rhetoric. We analyze three training-free signals: confidence verbalized with the answer, post-hoc P(True), and agreement with three additional generations on the same 100 TriviaQA questions for two model families. Direct verbalization is a surprisingly strong baseline: after auditing benchmark errors, it reaches AUROC 0.956 and 0.937 for correctness prediction. Three-sample agreement is substantially weaker (0.765 and 0.790), and a fixed interpolation with verbalized confidence has no statistically reliable benefit. Four of nine errors from one model and two of eight from the other receive unanimous sample support, showing that self-consistency can amplify shared misconceptions. Re-eliciting confidence for the same fixed answers with equivalent prompts changes scores by 0.043 to 0.084 on average and flips 4% to 9% of decisions at a 0.8 threshold. An exploratory audit of 100 confidence-tagged biography claims further finds only a modest confidence gap between supported and contradicted claims. These results argue that useful self-reports remain sensitive to elicitation, correlated errors, and benchmark noise.