Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage

Eric Fithian; Kirill Skobelev; X. Y. Han

cs.CLSep 16, 2026

Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage

Authors: Eric Fithian, Kirill Skobelev, X. Y. Han

Organizations: University of Chicago · Northwestern University

Abstract

In verifiable domains such as math and coding, finding one correct solution among many attempts can matter more than the pass rate of each attempt. Post-training can concentrate large language model outputs around a few modes, while increasing sampling temperature has limited effectiveness. We introduce Don't Repeat Yourself Supervised Fine-Tuning (DRY-SFT), a post-training method that increases output diversity and coverage: the probability of at least one correct solution among many attempts. DRY-SFT has two stages. First, for each problem, sequentially generate K solutions, showing the model all prior attempts and asking for a different solution. Second, fine-tune on each attempt independently, removing prior attempts from the context. The process uses no reward, verifier, or correctness filter. On HumanEval+, MBPP+, and DS-1000, DRY-SFT raises pass@100 by 10.8, 12.5, and 12.4 percentage points, respectively, at a small cost to pass@1. Structural diversity, measured by abstract syntax tree edit distance among passing solutions, rises significantly on all three benchmarks. DRY-SFT also solves 244 of 600 problems that the base model did not solve in the same 200 attempts. Across nine open-weight models, lower structural diversity of the base model significantly predicts larger DRY-SFT gains, indicating that the method is especially effective on more mode-collapsed models.

Figures & tables

Figure 1: DRY-SFT . Generate: a chain of solutions per problem, each conditioned on those before it. Fine-tune: the conditioning is discarded and each solution is paired with the problem for SFT: no reward, no verifier, no filter. Evaluate: solutions are drawn independently and in parallel from the prompt. Right: the illustrative change in the distribution over solution types. The distributions illustrate the mechanism and are not measured. The measured structural change is in Figure 2 .

Figure 2: DRY-SFT increases coverage and structural diversity. Top: pass@k+ from $n{=}200$ independent samples per problem at $T{=}0.8$ and $p{=}0.95$ . The i.i.d. SFT control and P-SFT use the same $5$ independently self-generated attempts; P-SFT adds resampled random-state prefixes during training but not at inference. DRY-SFT uses $5$ sequentially generated targets conditioned on prior solutions. Bands are pointwise $95\%$ problem bootstrap. Bottom: structural diversity, the mean pairwise tree edit distance between the abstract syntax trees (ASTs) of a problem’s passing solutions, normalized to $[0,1]$ ( Zhang and Shasha, 1989 ) , over the first $10$ samples (Section 4 , Structural diversity). Bars have $95\%$ bootstrapped confidence intervals over problems with at least two passing solutions. DRY-SFT Training Data is the set of Stage 1 solutions DRY-SFT was fine-tuned on, not a model arm.

Figure 3: Temperature recovers coverage but not structure. Base-model pass@100+ (left) and structural diversity (normalized AST edit distance among passing solutions, defined in Section 4 ) (right) against temperature on HumanEval+, over a grid of $12$ temperatures from $0.6$ to $3.0$ . Coverage climbs to a peak at $T{=}2.6$ and falls away beyond it, while structural diversity reaches DRY-SFT ’s level at no temperature. Shaded regions are pointwise $95\%$ problem-bootstrap confidence bands; dashed lines and bands mark DRY-SFT at $T{=}0.8$ .

Figure 4: The gain is not tied to the problems the targets came from. Left: pass@100+ on the held-out half of a 50/50 split, with DRY-SFT trained only on the other half. Right: DRY-SFT trained on the opposite EvalPlus benchmark against same-domain training, evaluated on the benchmark named. Error bars are $95\%$ problem bootstrap.

Figure 5: The gain tracks how concentrated the base model already is. Each point is one fine-tuned model: its base structural diversity — the normalized AST edit distance among its passing solutions — against the change in pass@100+ after DRY-SFT . The line is an ordinary least-squares fit over nine runs with a $95\%$ confidence band; $R^{2}$ and $p$ are printed in the panel. The fitted slope is $-6.14$ points per $0.1$ of base diversity, with $95\%$ CI $[-10.18,-2.10]$ . Models below the dashed line lose coverage. Every model is measured on HumanEval+.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	Setting	Value
Generation	model	Qwen3-4B-Instruct-2507
	solutions per problem, $K$	$5$
	temperature / nucleus $p$	$0.8$ / $0.95$
	correctness filtering	none
Fine-tuning	method	LoRA
	LoRA rank / scaling $\alpha$	$64$ / $128$

Appendix

Table A.1: Every hyperparameter of the reported runs. Appendix B covers how the epoch is selected and Appendix C the choice of

K

Benchmark	$K$	$\Delta$ pass@1+	$\Delta$ pass@100+
HumanEval+	2	$-0.8$ $[-3.6,+2.0]$	$+7.9$ † $[+4.2,+12.2]$
	3	$-1.3$ $[-4.2,+1.6]$	$+10.2$ † $[+6.0,+14.8]$
	5	$-1.7$ $[-4.9,+1.6]$	$+10.4$ † $[+6.1,+15.2]$
	7	$-1.5$ $[-5.1,+2.2]$	$+10.5$ † $[+6.0,+15.1]$
	10	$-3.1$ $[-7.4,+1.1]$	$+9.3$ † $[+4.8,+14.3]$
MBPP+	2	$+0.9$ $[-1.3,+3.3]$	$+10.2$ † $[+7.4,+13.3]$

Appendix

Table C.1: Change in pass@1+ and pass@100+ compared to the base model for each

K

with

95\%

paired bootstrap confidence intervals over problems. † indicates a statistically significant difference. The sweep holds training at three epochs rather than the validation-selected protocol of the reported runs (Appendix B ), so its

K{=}5

row differs from Section 4.1 .

Figure C.1: pass@100+ ( top ) and pass@1+ ( bottom ) for different $K$ . Shaded regions are $95\%$ bootstrap confidence intervals over problems; dashed lines show the base model. We choose $K{=}5$ for fine-tuning. Appendix C reports the change compared to the base model.

Figure D.1: Pass rate at each position in a chain of $K{=}20$ solutions, evaluated on the extended test suite. Shaded regions show $95\%$ confidence intervals.

Figure E.1: Held-out pass@k+ with and without correctness filtering before Stage 2. Both arms use the same generated targets before filtering and the same dataset split. Shaded regions are pointwise

95\%

bootstrap confidence intervals over problems. Appendix E reports paired differences.

Benchmark	$k$	Unfiltered	Filtered	$\Delta$ [ $95\%$ CI]
HumanEval+	1	$75.7$	$76.7$	$+1.1$ [ $+0.2$ , $+2.0$ ]
	100	$95.6$	$95.7$	$+0.1$ [ $-1.7$ , $+1.9$ ]
MBPP+	1	$65.1$	$67.1$	$+2.0$ [ $+0.1$ , $+4.1$ ]
	100	$88.1$	$87.4$	$-0.6$ [ $-2.4$ , $+1.0$ ]
DS-1000	1	$32.0$	$33.6$	$+1.6$ [ $+0.1$ , $+3.2$ ]
	100	$69.6$	$72.8$	$+3.2$ [ $+0.8$ , $+5.8$ ]

Appendix

Table E.1: Correctness-filtering ablation. Coverage is reported as a percentage;

\Delta

is filtered minus unfiltered in percentage points, with pointwise

95\%

paired problem-bootstrap intervals from

2{,}000

resamples. Intervals quantify held-out task uncertainty for one training seed per arm. Values are rounded to one decimal place.

Figure F.1: HumanEval+ pass@k+ for each model, comparing DRY-SFT with its base model and matched i.i.d. SFT control. Models with lower base structural diversity are shown first. Before evaluation, we remove output that is not code.

Explore similar work

Selecting Diverse SFT Traces Improves Post-RL Generalization
Sep 27, 2026Dylan Zhang, Mingyuan Wu, Jinning LiReasoning Diversity RL for Language Model Reasoning
University of Illinois Urbana-Champaign, work done at Google · Google
Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
May 20, 2026Tony Lee, Percy LiangSelf-Training for Language Models Language Model Self-Assessment
Stanford University
Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies
Aug 13, 2026Conor F. Hayes, Elliot Meyerson, Kajetan Schweighofer +4Evolution Strategies LLM Post-Training
1Cognizant AI Lab, San Francisco · 2The University of Texas at Austin, Austin

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

Selecting Diverse SFT Traces Improves Post-RL Generalization

Dylan Zhang, Mingyuan Wu, Jinning Li

University of Illinois Urbana-Champaign, work done at Google · Google

Reasoning Diversity RL for Language Model Reasoning Training Data Selection RL Post-Training LLM Reasoning

Open paper AskPDF arXiv

Verified solutions are not equally useful for preparing reasoning models for reinforcement learning (RL). We present a comprehensive study of route diversity, the variation in the sequences of reasoning steps in supervised fine-tuning (SFT) data, and propose a lightweight, rule-based fingerprint to select for it. From one pool at one budget, with matched training recipes and checkpoints, selecting diverse rather than similar routes improves post-RL problem coverage across puzzles and mathematics, including on problems harder than those seen in either training stage. In synthetic experiments, route-diverse SFT improves OLMo3-7B's pass@8 by 16.9 points on environments held out from SFT. In a single-model condition, where one model writes every candidate, diverse selection gains up to 6.2 points of mean pass@8 across 10 mathematics benchmarks. Pre-RL diagnostics suggest why: diverse SFT can produce both successful and failed attempts on more prompts despite slightly lower mean accuracy, giving group-relative RL more prompts with a learning signal. On 3 open-source corpora, our CPU-only selector, without model calls, outperforms more expensive alternatives in every comparison of mean post-RL performance. These results identify reasoning-route diversity as a practical criterion for selecting SFT data that better prepares models for RL.

Abstract

Explore similar work

Selecting Diverse SFT Traces Improves Post-RL Generalization

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies