Learning from the Gap Between Pass@K and Pass@1

Xuan Liu; Jingbin Qian; Haosheng Chen

cs.LGSep 17, 2026

Learning from the Gap Between Pass@K and Pass@1

Authors: Xuan Liu, Jingbin Qian, Haosheng Chen

Organizations: Shanghai Jiao Tong University · William Marsh Rice University · East China Normal University

Abstract

Sampling many responses and keeping one that passes a verifier lets large language models solve problems beyond their single-response ability, but this search must be paid again for every query, while many deployments answer with a single response. Post-training on verified responses can transfer the benefit of search into the model. With a fixed budget, selecting by correctness alone spends slots on problems the model already answers correctly, leaving fewer to correct its failures. To address this imbalance, we propose GapFT, which trains on the gap between Pass@K and Pass@1: problems that the source model fails with one response but solves within K samples. GapFT keeps the objective and training budget fixed and changes only which verified responses enter training; an exact decomposition splits the resulting Pass@1 change into corrected failures and regressions on problems the source model already solved. On LogiQA 2.0 and ReClor with three model families, GapFT is above budget-matched uniform rejection-sampling fine-tuning (RFT) in every setting, with a positive pooled effect, and on Llama-3.1-8B and Mistral-7B it recovers about two thirds to four fifths of the gain of fine-tuning on the entire verified pool with 11-34% of its problems. Further analyses reveal that the gain comes from failures that the first few search samples recover, while failures found only by deeper search displace replay and add no net gain, that filling the same budget with gold-labeled failures search cannot reach lowers accuracy, and that the gain is bounded by how many transferable failures search exposes.

Figures & tables

Figure 1: GapFT selects verified evidence for single-sample decoding. (a) The verifier checks one direct response and $K$ sampled responses from the source model, placing each problem in $G_{K}$ , $S_{K}$ , or $U_{K}$ ; (b) under a matched training budget, GapFT selects one verified response for each distinct $S_{K}$ problem, whereas uniform RFT mixes $G_{K}$ and $S_{K}$ evidence; (c) the held-out Pass@1 change decomposes into corrections and regressions.

		Problems		Pass@1 (%)					GapFT $-$ Source
Model	Task	$n$	All	Source	Replay	Uniform	GapFT	All	(pp)
Llama-3.1-8B	LogiQA 2.0	3,190	9,429	50.33	55.84	60.33	62.59	65.89	$+12.27$ [ $10.36$ , $14.24$ ]
	ReClor	913	3,050	55.40	61.27	69.13	70.67	73.80	$+15.27$ [ $12.13$ , $18.33$ ]
Qwen2.5-7B	LogiQA 2.0	275	8,797	68.54	68.50	68.20	68.81	70.06	$+0.27$ [ $-0.59$ , $1.12$ ]
	ReClor	63	2,977	72.20	72.00	71.53	72.47	73.20	$+0.27$ [ $-0.73$ , $1.33$ ]
Mistral-7B	LogiQA 2.0	887	6,554	47.26	49.74	49.57	51.87	53.21	$+4.61$ [ $3.27$ , $6.00$ ]

Table 1: Pass@1 Under Matched Evidence Budgets. Among Source, Replay, Uniform, and GapFT, the best result in each row is bold and the second-best is underlined . All is a larger-budget reference excluded from this ranking. Brackets give 95% confidence intervals.

Figure 2: GapFT above uniform RFT at a matched budget. Each point is the Pass@1 difference between GapFT and uniform RFT, with a 95% bootstrap interval over seeds and audit problems; filled markers have intervals that exclude zero.

Figure 3: One tuned decode against source search. Curves show the source model’s accuracy when a verifier picks among

K

samples; horizontal lines show the accuracy of one greedy decode from the source and from each trained model.

Figure 4: Accuracy as each control moves toward GapFT. Each point is the Pass@1 change of Llama-3.1-8B relative to the leftmost setting of its panel, which fills the varied slots with solved replay: (A)

K=0

; (B) none of the failures first recovered in samples 5–8. The rightmost point is GapFT. Bars are 95% bootstrap intervals over training seeds and audit problems; filled markers have intervals that exclude zero.

Dataset	Contrast	$\Delta$ Pass@1 (pp)	95% CI	$G$	$S$	$U$
A. Search depth, replay replaced by recovered failures, main recipe
LogiQA 2.0	$K{=}1-K{=}0$	$\mathbf{+5.46}$	[3.78, 6.95]	$-1.19$	$+3.97$	$+2.68$
LogiQA 2.0	$K{=}8-K{=}0$	$\mathbf{+6.75}$	[4.49, 8.80]	$-5.75$	$+6.39$	$+6.12$
ReClor	$K{=}1-K{=}0$	$\mathbf{+5.73}$	[3.33, 8.13]	$-2.20$	$+5.00$	$+2.93$
ReClor	$K{=}8-K{=}0$	$\mathbf{+9.40}$	[6.47, 12.53]	$-5.27$	$+9.27$	$+5.40$
B. Randomized inclusion of failures first recovered in samples 5–8, main recipe

Table 2: Pass@1 Decomposition of the Evidence Controls

Condition	Post Pass@1 (%)	$\Delta$ vs source (pp)	$\kappa_{S}$	$\kappa_{U}$	$\rho_{G}$
deployed successes	63.39	$+0.56$	41.35	3.08	11.49
search-recovered failures	66.11	$+3.28$	40.38	4.48	7.34
uniform RFT	64.11	$+1.28$	33.65	5.60	9.02
positive SFT	66.17	$+3.33$	41.35	5.04	7.69
verified correction	64.83	$+2.00$	34.62	2.52	7.16
188 distinct problems, 4 exposures each	65.78	$+2.94$	40.06	3.92	7.60

Table 3: MATH Results with Matched Training Exposures

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Models	Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-Instruct-v0.3 in the main comparison; Llama-3.1-8B-Instruct in the controls
Search	$K=8$ for the main comparison; $K\in\{0,1,8\}$ for the search-depth control; temperature $0.7$ , top- $p=0.95$ , at most 64 new tokens; shortest verified response selected, with response index as tie-breaker
Verifier / deployment	Exact option matching, on the whole response for Llama and Qwen on LogiQA 2.0 and on the first explicit answer otherwise / greedy choice decoding
Update	RFT with loss on response tokens only; AdamW; bfloat16; one epoch over the recipe repeated eight times
LoRA	Rank 8, alpha 16, dropout 0.05; seven attention and MLP projection modules listed above
Optimization	Learning rate $5\times 10^{-5}$ ; batch size 2; gradient accumulation 8; weight decay 0.01; no scheduler; gradient norm clipped to 1.0; maximum sequence length 2048

Appendix

Table 4: Implementation Details for Logic Experiments

Comparison	Task	$n$	$N=8n$	$L$	$J$
Main, Llama	LogiQA 2.0	3,190	25,520	6,049,464	1,595
	ReClor	913	7,304	1,793,896	457
Main, Qwen	LogiQA 2.0	275	2,200	512,408	138
	ReClor	63	504	120,064	32
Main, Mistral	LogiQA 2.0	887	7,096	1,901,832	444
	ReClor	252	2,016	555,896	126

Appendix

Table 5: Training Budgets of the Logic Experiments

Setting	Value
Model	Qwen2.5-7B-Instruct
Evidence	188 distinct failures recovered by search or deployed successes, each repeated four times; subject and difficulty strata differ by at most one problem
Update	752 steps using sequence-normalized cross-entropy on response tokens only; bfloat16; AdamW; constant learning rate $10^{-6}$ ; weight decay 0.01; batch size 1; maximum sequence length 2048
Budget matching	752 training examples and model positions; 270,912 versus 269,980 supervised response tokens, below the prespecified 2% tolerance
LoRA	Rank 8, alpha 16, dropout 0.05; the seven attention and MLP projection modules listed above
Audit	600 disjoint MATH problems ( Hendrycks et al., 2021 ) ; deterministic decoding with at most 1,024 new tokens

Appendix

Table 6: MATH Matched-Diversity Control

Setting	Value
Models	Qwen2.5-Coder-7B-Instruct ( Hui et al., 2024 ) and DeepSeek-Coder-6.7B-Instruct ( Guo et al., 2024 )
Search	$K=16$ independent source episodes; temperature $0.7$ , top- $p=0.95$ ; at most three attempts and 1,536 new tokens per attempt; shortest successful response from the final turn selected, with sample index and seed as tie-breakers
Verifier / deployment	All official public and hidden tests / one episode with at most three internal attempts and public test feedback
Update	RFT with loss on response tokens only; AdamW; bfloat16; one epoch; gradient checkpointing
LoRA	Rank 16, alpha 32, dropout 0.05; the same seven projection modules as the logic experiments
Optimization	Learning rate $5\times 10^{-5}$ ; one sequence per microbatch; training examples distributed across exactly 64 update groups; weight decay 0.01; no scheduler; gradient norm clipped to 1.0; maximum sequence length 8192

Appendix

Table 7: LiveCodeBench Allocation and Replication ( Jain et al., 2025 )

Dataset	Model	$K$	$G/S/U$	$p_{S}$ (%)	$\kappa_{S}$	$\kappa_{U}$	$\rho_{G}$	Net gain (pp)	$S$	$U$
					observed rates (%)				redistribution null (%)
LogiQA 2.0	Llama-3.1-8B	4	770 / 306 / 496	19.47	47.17	20.50	8.23	$+11.62$	–	–
LogiQA 2.0	Llama-3.1-8B	8	770 / 416 / 386	26.46	50.72	16.93	6.88	$+14.21$	21.90	11.11
LogiQA 2.0	Qwen2.5-7B	4	1,050 / 28 / 494	1.78	51.19	8.37	2.79	$+1.68$	–	–
LogiQA 2.0	Qwen2.5-7B	8	1,050 / 37 / 485	2.35	54.95	7.56	2.73	$+1.80$	27.83	4.92
ReClor	Llama-3.1-8B	4	277 / 96 / 127	19.20	46.88	38.58	6.38	$+15.27$	–	–

Appendix

Table 8: Logic Scope Matrix and Answer-Redistribution Control

Method	Question	Assumption on verified evidence	Mechanism	Regime	Difference from GapFT
RFT ( Shao et al., 2024 )	Improve reasoning from self-generated data	All verified responses are useful	Sample, keep verifier-accepted responses, fine-tune	Offline	Ignores state; budget not matched across evidence sets
Distinct paths ( Yuan et al., 2023 )	What drives RFT gains	Distinct correct paths matter more than samples	Augment with diverse verified paths	Offline	Diversity of responses, not state of the problem
DART-Math ( Tong et al., 2024 )	Correct the easy-query bias of rejection sampling	Difficult queries are under-covered	Give more synthesis trials to queries with a high fail rate under a separate synthesis model	Offline	Selects by query difficulty for the synthesizer, not the deployed state of the trained policy; budget not matched
DAPO dynamic sampling ( Yu et al., 2025 )	Keep gradient signal in group RL	Prompts with uniform group outcome carry no signal	Drop all-correct and all-incorrect prompts online	Online	Filters on current-policy reward variance, which Appendix B.5 shows is not the source state
BOND ( Sessa et al., 2025 )	Amortize Best-of- $N$	The Best-of- $N$ distribution is the target	Distill Best-of- $N$ via Jeffreys divergence	Online	Targets a distribution, not the failure set; no state distinction
SCoRe ( Kumar et al., 2025 )	Self-correction at test time	Off-policy correction traces mismatch the policy	Two-stage on-policy RL with reward shaping	Online	Repairs mismatch by RL; GapFT selects evidence by state offline

Appendix

Table 9: Comparison with Closely Related Methods

Training condition	Pass@1 (%)	$\Delta$ vs replay (pp)	Pass@1 (%)	$\Delta$ vs replay (pp)
	LogiQA 2.0		ReClor
no post-training (source)	50.33	–	55.40	–
solved replay, $K{=}0$	54.18	0 (ref.)	59.27	0 (ref.)
search evidence, $K{=}1$	56.91	$\mathbf{+2.73}$ [1.51, 3.93]	62.40	$\mathbf{+3.13}$ [0.47, 5.60]
search evidence, $K{=}8$	62.33	$\mathbf{+8.14}$ [6.78, 9.61]	67.93	$\mathbf{+8.67}$ [6.27, 10.93]
failures selected with gold labels	63.37	$\mathbf{+9.19}$ [7.14, 11.09]	73.20	$\mathbf{+13.93}$ [11.33, 16.67]

Appendix

Table 10: Pass@1 of the Evidence Controls with a Shared Core

Model	Task	$p_{S}$ (%)	Condition	$\kappa_{S}$	$\kappa_{U}$	$\rho_{G}$	$S$	$U$	$G$
				Rates (%)			Term difference vs uniform (pp)
Llama-3.1-8B	LogiQA 2.0	26.6	replay	30.7	4.4	7.3	$-4.22$	$-2.88$	$+2.61$
			uniform	46.5	16.9	12.5	ref.	ref.	ref.
			GapFT	54.7	31.0	18.8	$+2.17$	$+3.24$	$-3.15$
			all verified	56.1	26.7	11.0	$+2.56$	$+2.24$	$+0.76$
Llama-3.1-8B	ReClor	26.0	replay	27.4	15.8	7.6	$-5.73$	$-3.60$	$+1.47$

Appendix

Table 11: Decomposition of the Main Comparison

Model	Task	Greedy	Pass@4	Pass@8	Replay	Uniform	GapFT	All
		Source (%)			$K_{\mathrm{eq}}$
Llama-3.1-8B	LogiQA 2.0	50.33	68.40	76.81	1	2	2	3
	ReClor	55.40	73.32	81.40	1	2	3	4
Qwen2.5-7B	LogiQA 2.0	68.54	70.17	70.67	$<1$	$<1$	1	3
	ReClor	72.20	73.69	74.40	$<1$	$<1$	1	2
Mistral-7B-v0.3	LogiQA 2.0	47.26	52.53	54.94	1	1	3	4

Appendix

Table 12: Source Pass@

K

Reached by One Decode

Figure 5: Outcome flows by source state. (a) Where the audit problems of each source state end up after GapFT; (b) the same transitions as rates within each source state. Llama-3.1-8B on LogiQA 2.0, averaged over three training seeds on the deduplicated audit set.

Figure 6: Code Coverage and Transfer Effects

	Answer only		Permissive
Condition	Pass@1 (%)	$\Delta$ vs GapFT	Pass@1 (%)	$\Delta$ vs GapFT
source	48.98	–	49.36	–
GapFT	63.40	ref.	63.40	ref.
GRPO, all prompts	42.56	$-20.84$ [ $-23.1$ , $-18.5$ ]	51.55	$-11.85$ [ $-14.2$ , $-9.7$ ]
GRPO, flat groups masked	19.02	$-44.38$ [ $-46.6$ , $-42.1$ ]	47.43	$-15.97$ [ $-18.3$ , $-13.7$ ]

Appendix

Table 13: Online RL Control at Matched Updates

A. Partition and flows for each item
Quantity	LogiQA 2.0	ReClor
Unstable $S/U$ share among items the source gets wrong (%)	35.91	44.39
Mean $S/U$ agreement across partitions (%)	79.90	74.89
$S$ Jaccard across partitions (%)	67.67	63.87
Mean $S$ flow (pp)	15.07	14.87
Mean $U$ flow (pp)	6.39	7.53

Appendix

Table 14: Stability of the Source-State Partition

Explore similar work

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See
Aug 18, 2026Ayoub Kirouane, Christos PetrocheilosSupervised Fine-Tuning Low-Resource Language Processing
Sophea AI, KIEFER SA, Athens, Greece
Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage
Sep 16, 2026Eric Fithian, Kirill Skobelev, X. Y. HanSupervised Fine-Tuning Diverse Text Generation
University of Chicago · Northwestern University
TailSFT: Filtered Fine-Tuning Improves Post-Training Performance
Aug 26, 2026Sadhika Malladi, Samy Jelassi, Dylan Foster +2Supervised Fine-Tuning Instruction-Tuning Data Selection
University of California San Diego · Microsoft Research NYC

Aug 18, 2026, cs.CLJ/K move · Enter open · S save

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

Ayoub Kirouane, Christos Petrocheilos

Sophea AI, KIEFER SA, Athens, Greece

Supervised Fine-Tuning Low-Resource Language Processing RL for Language Models

Open paper AskPDF arXiv

Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.

Abstract

Explore similar work

Thinking in a Low-Resource Language: What SFT Builds, What RL Fixes, What Accuracy Cannot See

Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance