From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning

Yupeng Chang; Wenxuan Zhang; Yuan Wu

cs.LGSep 29, 2026

From Checkpoint Variation to Selection Gains in Supervised Fine-Tuning

Authors: Yupeng Chang, Wenxuan Zhang, Yuan Wu

Organizations: School of Artificial Intelligence, Jilin University · Singapore University of Technology and Design · Key Laboratory of Symbolic Computation and Knowledge Engineering, Jilin University

Abstract

Checkpoint selection is a routine decision in supervised fine-tuning (SFT): training produces multiple checkpoints, but only one is retained. Yet fixed-budget comparisons do not by themselves distinguish three empirical claims: whether more validation data improve checkpoint selection, whether a selection rule outperforms validation-loss selection, and whether it improves over simply retaining the final checkpoint. We therefore treat checkpoint selection as a finite-information decision problem. Holding completed training trajectories, candidate checkpoints, and independent test items fixed, we vary the validation budget and separately measure improvement from additional validation data, gain over negative log-likelihood (NLL) selection, and gain over the final checkpoint. Across 60 mathematical SFT trajectories and 19 configurations, increasing the validation budget from 32 to 305-313 examples raises independent-test accuracy by 0.32 percentage points (pp) for generated-accuracy selection and 0.29 pp for checkpoint agreement, with 95% configuration-bootstrap CIs of [0.10, 0.56] and [0.11, 0.50], respectively. At the full validation budget, the two generation-based rules outperform matched NLL selection by 0.71 and 0.85 pp, respectively, while their gains over the final checkpoint remain unresolved. A cross-domain replication on 12 newly trained Commonsense trajectories shows the same qualitative separation: increasing the validation budget from 32 to 1,024 questions improves generated-accuracy and checkpoint-agreement selection by 0.87 and 0.27 pp, while gains over the final checkpoint again remain unresolved. Together, these results show that benefiting from more validation data, outperforming NLL selection, and outperforming the final checkpoint are distinct empirical claims that require separate evidence.

Figures & tables

Figure 1: Checkpoint variation and attainable selection gain are distinct. (a) For a completed SFT trajectory, a validation-based rule selects checkpoint

c_{s}

, while the fixed reference retains the final checkpoint

c_{T}

. Both decisions are evaluated on the same independent items

\mathcal{I}

, so a positive gain means that the selected checkpoint outperforms the final checkpoint. (b) Validation NLL and task accuracy can evolve differently during training. In the original 75-trajectory grid, NLL-based selection yields a paired accuracy difference of

-0.76

pp relative to the final checkpoint (95% configuration-bootstrap CI

[-1.32,-0.22]

). This motivates our central question: how much checkpoint variation can finite validation data reliably convert into better performance on independent test examples?

A. Original mathematical pools 60 trajectories / 19 configurations
Rule	Budget gain	vs. matched NLL	vs. final
	$D_{r}$ : Full $-$ 32	$H_{r}$ : Full	$G_{r}$ : Full
GSM8K + MATH; full = 305/312/313 examples from 228–244 source groups
Generated accuracy	$+0.32$ $[+0.10,\,+0.56]$	$+0.71$ $[+0.25,\,+1.22]$	$+0.01$ $[-0.33,\,+0.35]$
Checkpoint agreement	$+0.29$ $[+0.11,\,+0.50]$	$+0.85$ $[+0.36,\,+1.40]$	$+0.15$ $[-0.08,\,+0.37]$
Matched NLL	$+0.05$ $[-0.05,\,+0.14]$	$0$ (reference)	$-0.70$ $[-1.37,\,-0.09]$

Table 1: Validation-budget effects across primary, cross-domain, and larger-pool analyses. Entries are paired independent-test accuracy differences (pp) with marginal 95% configuration-bootstrap CIs.

D_{r}

measures improvement from 32 examples to the full validation budget,

H_{r}

measures recovery over same-budget matched NLL, and

G_{r}

measures gain over the final checkpoint. Panel A reports the original mathematical analysis; Panel B is a cross-domain Commonsense replication on newly trained trajectories; Panel C uses a larger, separately constructed GSM8K source-question validation pool. The populations are analyzed separately and are not pooled.

Figure 2: Validation-budget improvement and gain over final checkpoint are distinct. Post-hoc reanalysis of 60 mathematical trajectories across 19 configurations. All rules use identical validation subsets and test items. Shading shows pointwise 95% configuration-bootstrap intervals. Full denotes 305, 312, or 313 examples by dataset. Positive means higher independent-test accuracy.

A. Independent validation selection		81 trajectories, 26 configurations
Rule	Gain vs. matched NLL	Gain vs. final
Generated accuracy	$+0.74$ $[+0.38,\,+1.14]$	$-0.01$ $[-0.27,\,+0.24]$
Checkpoint agreement	$+0.90$ $[+0.50,\,+1.33]$	$+0.14$ $[-0.05,\,+0.32]$
B. Matched-half selection		75 trajectories, 24 configurations
Rule	Selection-half gain	Complementary-half gain	Paired difference
Empirical accuracy maximum	$+1.33$ $[+0.92,\,+1.80]$	$+0.05$ $[-0.22,\,+0.34]$	$+1.29$ $[+1.01,\,+1.59]$

Table 2: Reference gains and same-item optimism. (A) Independent-validation selection compares gains over matched NLL and final checkpoint. (B) Matched-half selection compares selection-half and complementary-half gains at fixed size. Entries are paired differences (pp) with marginal 95% configuration-bootstrap intervals; panels use overlapping populations and are analyzed separately.

Figure 3: Full-pool gains vary across mathematical configurations. Markers show seed-averaged gains across 60 trajectories in 19 configurations. These means are descriptive; no subgroup-specific selector is fitted. Panels show independent-test gains over (a) the final checkpoint and (b) same-budget matched NLL. Positive values favor the named selector. Both panels share a horizontal scale; the separator divides GSM8K and MATH configurations.

Sensitivity	Rule	Budget gain $D_{r}$	vs. matched NLL $H_{r}$	vs. final $G_{r}$
One variant / source	Generated accuracy	$+0.22$	$+0.61$	$-0.09$
	Agreement	$+0.22$	$+0.78$	$+0.08$
Test-question resampling	Generated accuracy	$+0.32$	$+0.71$	$+0.01$
	Agreement	$+0.29$	$+0.85$	$+0.15$

Table 3: Core-claim sensitivities (pp). Point estimates for the two generation-based rules. Full protocols, 95% intervals, and detailed tables are reported in Appendices A.7 and A.8 .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Question	Traj. / config.	Selection and scoring	Direct contrast
Recovery over NLL vs. gain over final	81 / 26	Select on original validation; score on independent test items.	Gain over matched NLL and gain over the final checkpoint.
Same-item vs. unseen-item gain	75 / 24	Select once on half the evaluation items; score that choice on both halves.	Selection-half minus complementary-half gain; fixed selection size.
Validation-budget response	60 / 19	Select on shared nested validation subsets; score on a fixed independent test.	Full-minus-32 gain ( $D_{r}$ ); same-budget comparisons.
Frozen GSM8K new-pool extension	12 / 4	Select on 32–1,024 newly constructed source questions; reuse existing training trajectories and test predictions.	1,024-minus-32 gain ( $D_{r}$ ); gains over the final checkpoint and matched NLL.
Cross-domain Commonsense replication	12 / 4	Train 12 new trajectories; select on 32 or 1,024 questions across eight Commonsense tasks and score frozen choices on 4,000 independent test questions.	1,024-minus-32 gain ( $D_{r}$ ); gains over the final checkpoint and matched NLL.

Appendix

Table 4: Analyses of selection gain. The populations overlap, so rows are not independent replications. The first three analyses are post-hoc reanalyses. The GSM8K new-pool extension freezes its protocol before inference on the new validation pool; the Commonsense replication uses newly trained trajectories, freezes its protocol before training, and freezes checkpoint choices before independent-test evaluation.

Rule	GSM8K (48 traj.)	MATH (15)	Commonsense (18)
Original NLL	$-0.59$ $[-1.28,\,+0.05]$	$-1.01$ $[-2.30,\,+0.25]$	$-0.97$ $[-1.80,\,-0.29]$
Generated accuracy	$-0.01$ $[-0.42,\,+0.41]$	$+0.04$ $[-0.37,\,+0.28]$	$-0.06$ $[-0.21,\,+0.10]$
Agreement	$+0.25$ $[+0.03,\,+0.47]$	$-0.01$ $[-0.73,\,+0.47]$	$-0.02$ $[-0.13,\,+0.13]$

Appendix

Table 5: Independent selection gains by task group (pp). Results use the 81-trajectory, 26-configuration population of Table 2 A; positive values indicate gain over the final checkpoint. Task strata and marginal 95% configuration-bootstrap intervals are descriptive, not independent confirmations.

Pooled (60 trajectories, 19 configurations)
Rule	$G_{r}(32)$	$G_{r}(\mathrm{full})$	$D_{r}$
Generated accuracy	$-0.31$ $[-0.71,\,+0.06]$	$+0.01$ $[-0.33,\,+0.35]$	$+0.32$ $[+0.10,\,+0.56]$
Agreement	$-0.14$ $[-0.40,\,+0.12]$	$+0.15$ $[-0.08,\,+0.37]$	$+0.29$ $[+0.11,\,+0.50]$
Matched NLL	$-0.75$ $[-1.42,\,-0.14]$	$-0.70$ $[-1.37,\,-0.09]$	$+0.05$ $[-0.05,\,+0.14]$
GSM8K (45 trajectories, 15 configurations)
Generated accuracy	$-0.29$ $[-0.73,\,+0.14]$	$0.00$ $[-0.43,\,+0.44]$	$+0.29$ $[+0.06,\,+0.52]$

Appendix

Table 6: Paired effects of increasing the validation budget (pp).

G_{r}(n)

is independent-test gain over the final checkpoint;

D_{r}=G_{r}(\mathrm{full})-G_{r}(32)

. Brackets are marginal 95% configuration-bootstrap intervals; repetitions are averaged within each trajectory before resampling. Full pools contain 305–313 examples. Task strata are descriptive, especially MATH with four configurations.

Rule	Budget	$G_{r}(n)$	$H_{r}(n)$
Generated accuracy	32	$-0.31$ $[-0.71,\,+0.06]$	$+0.44$ $[+0.15,\,+0.75]$
	64	$-0.26$ $[-0.62,\,+0.09]$	$+0.48$ $[+0.16,\,+0.85]$
	128	$-0.17$ $[-0.50,\,+0.15]$	$+0.55$ $[+0.18,\,+0.96]$
	256	$-0.07$ $[-0.39,\,+0.26]$	$+0.62$ $[+0.21,\,+1.08]$
	Full	$+0.01$ $[-0.33,\,+0.35]$	$+0.71$ $[+0.25,\,+1.22]$
Agreement	32	$-0.14$ $[-0.40,\,+0.12]$	$+0.61$ $[+0.21,\,+1.05]$

Appendix

Table 7: Complete pooled validation-budget curves (pp). Same common-pool subsets and exact uniform checkpoint-score tie expectation for all rules; 60 trajectories across 19 configurations.

G_{r}(n)

is independent-test gain over the final checkpoint;

H_{r}(n)

is gain over matched token-mean NLL selected on the same subset. Brackets are pointwise, unadjusted 95% configuration-bootstrap intervals, conditional on the existing pools and grid. Full is a categorical endpoint (305–313 examples).

Rule	Reference	$n=32$	$128$	$512$	$1024$
Generated accuracy	$G_{r}(n)$	-0.06	+0.06	+0.27	+0.35
Generated accuracy	$H_{r}(n)$	+0.05	+0.17	+0.38	+0.45
Agreement	$G_{r}(n)$	-0.06	-0.02	+0.02	+0.03
Agreement	$H_{r}(n)$	+0.04	+0.08	+0.13	+0.14
Matched NLL	$G_{r}(n)$	-0.10	-0.10	-0.11	-0.10
Matched NLL	$H_{r}(n)$	0.00	0.00	0.00	0.00

Appendix

Table 8: All frozen budget points in the larger-pool GSM8K extension (pp). Pooled means over the same 12 trajectories.

G_{r}(n)

is gain over the final checkpoint and

H_{r}(n)

is gain over same-budget matched NLL. Endpoints and their descriptive uncertainty are reported in the main text; complete marginal intervals are provided in the accompanying artifact. Lines between these points would not identify a sufficient budget or saturation.

Rule	Contrast	Mean	Config. 95% interval	Test-question 95% interval
Generated accuracy	$D_{r}$	$+0.403$	$[+0.063,+0.664]$	$[+0.125,+0.686]$
	$G_{r}(1{,}024)$	$+0.347$	$[-0.360,+1.339]$	$[-0.060,+0.761]$
	$H_{r}(1{,}024)$	$+0.449$	$[-0.101,+0.998]$	$[-0.028,+0.932]$
Agreement	$D_{r}$	$+0.095$	$[-0.292,+0.309]$	$[-0.155,+0.350]$
	$G_{r}(1{,}024)$	$+0.035$	$[-0.278,+0.392]$	$[-0.250,+0.316]$
	$H_{r}(1{,}024)$	$+0.136$	$[-0.941,+0.802]$	$[-0.360,+0.635]$

Appendix

Table 9: New-pool endpoints under separate uncertainty sources. Twelve trajectories across four configurations; all effects are in pp. Configuration intervals are descriptive and condition on the test set; test-question intervals condition on configurations and frozen choices. Neither is a joint interval or an independent replication.

Rule	Full $-$ 32	Full vs. final	Full vs. matched NLL
Generated accuracy	$+0.22$ $[+0.07,\,+0.39]$	$-0.09$ $[-0.47,\,+0.28]$	$+0.61$ $[+0.23,\,+1.03]$
Agreement	$+0.22$ $[+0.08,\,+0.37]$	$+0.08$ $[-0.13,\,+0.29]$	$+0.78$ $[+0.29,\,+1.32]$
Matched NLL	$+0.07$ $[-0.01,\,+0.16]$	$-0.71$ $[-1.37,\,-0.09]$	$0$

Appendix

Table 10: Budget gains with one variant per original question (pp). All 60 trajectories across 19 configurations; full is 228–244 unique questions. Brackets are marginal 95% configuration-bootstrap intervals conditional on the existing pools. This is a multiplicity sensitivity analysis, not an independent validation-pool replication.

Rule	Paired contrast	Mean	Configuration CI	Test-question CI
Generated accuracy	Full versus final checkpoint	$+0.01$	$[-0.33,+0.35]$	$[-0.27,+0.30]$
	Full versus matched NLL	$+0.71$	$[+0.25,+1.22]$	$[+0.40,+1.02]$
	Full minus 32	$+0.32$	$[+0.10,+0.56]$	$[+0.17,+0.48]$
Agreement	Full versus final checkpoint	$+0.15$	$[-0.08,+0.37]$	$[-0.04,+0.35]$
	Full versus matched NLL	$+0.85$	$[+0.36,+1.40]$	$[+0.49,+1.22]$
	Full minus 32	$+0.29$	$[+0.11,+0.50]$	$[+0.15,+0.44]$

Appendix

Table 11: Separate configuration and test-question uncertainty (pp). The same fixed-pool budget contrasts on 60 trajectories across 19 configurations. Configuration intervals condition on the saved test questions; test-question intervals hold configurations and validation choices fixed, using paired draws shared across all trajectories within each task. Both use 20,000 draws and are marginal 95% intervals. They are not combined into a joint interval.

Task group	Model / adaptation	Steps	Seeds
GSM8K	Llama-3-8B	252	3
GSM8K	Mistral-7B	252	3
GSM8K	Qwen2.5-7B	252	3
GSM8K	Qwen2.5-7B-Instruct	252	3
GSM8K	Qwen3-1.7B	252	3
GSM8K	Qwen3-1.7B (full fine-tuning)	252	3

Appendix

Table 12: Primary SFT configuration grid. Unless noted, adaptation is rank-8 LoRA with three seeds. The 24 original-grid configurations plus two direct-contrast completed-decay variants (marked ∗ ) are shown; the nonfactorial grid is reported for coverage, not causal comparison.

Source pool	Before split	Training	Validation	Intersection
GSM8K	3,000 / 2,421	2,695 / 2,177	305 / 244	0 / 0
MATH	2,990 / 2,083	2,677 / 1,854	313 / 229	0 / 0
Commonsense	3,000 / 2,992	2,683 / 2,676	317 / 316	0 / 0

Appendix

Table 13: Available source-pool reconstruction. Entries give records / distinct original-question groups. The final column counts shared groups / exact queries between reconstructed training and validation. This is not a historical per-run consumed-record manifest.

Explore similar work

May 13, 2026cs.LG

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning

Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top-

k

subset. However, effective SFT training subsets are often produced through ordered curation recipes, where filtering, mixing, and deduplication operators jointly shape the final data distribution. We formulate this problem as fixed-pool data recipe search: given a raw instruction pool and a library of grounded operators, the goal is to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations, without generating, rewriting, or augmenting training samples. We introduce AutoSelection, a two-layer solver that decouples fixed-pool materialization based on cached task-, data-, and model-side signals from expensive full evaluation, using warmup probes, realized subset states, local recipe edits, Gaussian-process-assisted ranking, and stagnation-triggered reseeding. Experiments on a 90K instruction pool show that AutoSelection achieves the strongest in-distribution reasoning average across three base models, outperforming full-data training, random recipe search, random top-

k

, and single-operator selectors. Additional Out-of-distribution graph-reasoning results, search-stability analyses, structural ablations, and 1.5B-to-7B transfer checks further show that recipe structure matters beyond individual selection operators. Code is available at https://github.com/w253/AutoSelection.

Haodong Wu, Jiahao Zhang, Lijie Hu +1

The Hong Kong University of Science and Technology (Guangzhou) · Mohamed bin Zayed University of Artificial Intelligence

Supervised Finetuning Model Fine-Tuning Recipe Risc-V

Ask PDF

May 13, 2026cs.LG

Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning

Data selection during supervised fine-tuning (SFT) can critically change the behavior of large language models (LLMs). Although existing work has studied the effect of selecting data based on heuristics such as perplexity, difficulty, or length, the reported findings are often inconsistent or context-dependent. In this work, we systematically study the role of data difficulty in fine-tuning from both empirical and theoretical perspectives, and find that there is no universally optimal difficulty level; rather, its effectiveness depends on the dataset size. We show that for a fixed data budget, there exists an optimal data difficulty for SFT, and that this optimal difficulty shifts toward harder data as the data budget increases. To explain this phenomenon, we conduct controlled synthetic experiments that reveal a simple underlying mechanism: the interplay between the (in-distribution) generalization gap and the extrapolation gap. We further support this mechanism through a theoretical analysis using PAC-Bayesian generalization bounds. Overall, our results clarify how data size and difficulty jointly affect the trade-off between generalization and extrapolation in SFT, providing guidance for difficulty-based data selection under certain model and data conditions.

Siyuan Liu, Tinghong Chen, Xinghan Li +2

IIIS, Tsinghua University · College of AI, Tsinghua University · Shanghai Qi Zhi Institute +1

Large Language Model Fine-Tuning Model Fine-Tuning Alignment-Aware Selection Extrapolation

Ask PDF

Apr 30, 2026cs.LG

Diversity in Large Language Models under Supervised Fine-Tuning

Supervised Fine-Tuning (SFT) is essential for aligning Large Language Models (LLMs) with user intent, yet it is believed to suppress generative diversity. Although this reduction is frequently referenced, formal empirical testing of the phenomenon remains limited. The expressiveness of LLMs by itself was addressed by multiple prior methods. Their varying perspectives suggest that deeper investigation could yield further improvements. In this study, we attribute the decline to two primary drivers: the neglect of low-frequency patterns within fine-tuning datasets and the forgetting of preexisting knowledge. Motivated by our theoretical analysis, we develop Tempered Focal (TOFU) loss, a novel objective that addresses both stated challenges simultaneously. Our extensive evaluation confirms at scale that generation breadth narrows after SFT and strengthens the hypothesis explaining this effect. Across multiple models and benchmarks, we demonstrate that TOFU enhances output diversity while preserving high response quality, offering a principled approach to SFT.

Roman Klypa, Oleksandr Cherednichenko

Univ. Grenoble Alpes, CNRS, Grenoble INP, LJK, 38000 Grenoble, France · Department of Mathematics and Mathematical Statistics, Integrated Science Lab, Umeå University, Sweden.

Diversity Taco

Ask PDF