Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile

Ely Sheikh

cs.LGSep 27, 2026

Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile

Authors: Ely Sheikh

Organizations: Aix-Marseille University

Abstract

Recipes such as s1 and LIMO teach a model to reason with little data by showing it the same thousand or fewer worked solutions many times over. Judged when that training ends, the repetition looks harmless. But reasoning models are often trained again, and we find that repetition leaves their reasoning fragile to that next stage, even when the stage has nothing to do with reasoning. We fine-tuned Qwen3.5-9B-Base on its own correct solutions to competition math problems, either drilling a few hundred of them about eight times each or showing many more once; with the same amount of training, both solve about 95% of held-out problems. A single pass of ordinary instruction tuning leaves the once-trained model where it was, while the drilled one falls to 86.0%, and harsher later stages take it to 59.3% or below. A third model that visited the drilled problems just as often, with a new solution at every visit, was unharmed, so the damage comes from seeing the same texts again rather than from having few problems. The break recurs with a stronger model's traces, in further training runs and on other models and tasks. It is also cheap to undo: the reasoning is suppressed rather than erased, and five updates of reasoning training bring almost all of it back, as does brief training on the reasoning format with almost no mathematics. Fresh solutions prevented the damage, and so did replaying 6.25% of the original solutions in a gentler later stage, so our claim concerns later training without such replay. Sharpening alone does not explain the break, since a model sharpened three-quarters as much without repetition was unharmed. On a skill the base model could not perform within a token budget, repetition mainly cost learning.

Figures & tables

					Stress tests
Training texts	Model	Passes	Handoff	One epoch	Instructions	Answer-only
None	Base	–	94.7	94.5	95.6	91.7
Own solutions	Drilled	1.1	95.4	–	94.5	94.3
	Drilled	3.3	95.1	–	94.0	92.8
	Drilled	7.7	95.4	86.0	59.3	22.5
	Once-trained	1.0	94.6	94.8	94.7	94.8

Table 1: Forced accuracy (%) of the 9B models on MATH-500 (levels 3–5) at handoff and after each later stage. Passes: how many times the average training text was seen. One epoch: instruction tuning at

10^{-4}

; stress tests: twenty updates at

3\times 10^{-4}

. The sharpened model is trained like the once-trained one on solutions sampled at temperature 0.5 (Section 5 ). Dashes mark stages we did not run. Table 3 adds free accuracy,

B

, the other models and the arithmetic task.

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Arm	Trained on	Updates	Passes
Math	D	579 of the base model’s correct solutions	20–140	1.1–7.7
	O	4,480 of the base model’s correct solutions	60, 140	0.43, 1.0
	P	D’s 579 problems, a fresh correct solution at each visit	140	1.0
	D2	579 other solutions from O’s set	140	7.7
	D-r128	D’s solutions, rank-128 adapters	140	7.7
	D-T	gpt-oss-120b traces for 575 problems	20, 140	1.1, 7.8

Appendix

Table 2: Acquisition arms. Passes: how many times the average training text was seen. All arms use learning rate

10^{-4}

, Adam without clipping and a token loss normalized by the set’s mean length, in batches of 32; FA and FN add a second stream of 32 examples per batch. The 35B and Nemotron runs repeat the base, D and O on their own solutions.

Task	Model	Passes	Free	Forced	$B$ (%)	Instr. tuning	Answer-only	after AO
			At handoff			Forced after		At cap
Math, 9B	Base	–	74.7	94.7	0	95.6	91.7	7.0
	D-u20	1.1	95.0	95.4	0.1	94.5	94.3	5.8
	D-u60	3.3	97.3	95.1	3.4	94.0	92.8	4.2
	D-u140	7.7	95.2	95.4	24	59.3	22.5	2.6
	O-u140	1.0	94.1	94.6	0.1	94.7	94.8	4.5

Appendix

Table 3: Every arm at handoff and after twenty updates of one later stage. Forced accuracy (%) within 16,384 tokens; free accuracy at handoff.

B

is a percentage of the starting model’s own per-token negative log-likelihood (9B math: 0.459 nats; 35B: 0.499; Nemotron: 0.640; arithmetic, scored to 4,096 tokens: 0.390). Math: MATH-500 levels 3–5, four forced draws. Teacher traces: the same design on gpt-oss-120b’s traces, where

B

is descriptive only. Arithmetic: 192 targeted problems, two draws; its

B

is scored on samples drawn at top-p 0.95, which gives every checkpoint a shared offset, so small negative values occur (Appendix I ). Beyond Table 1 , the table adds the second subset (D2), rank 128 (D-r128), the 35B and Nemotron runs, Nemotron’s starting model, the arithmetic task with its replicate U2, and the exploratory anchor arms FA025, FA1 and FN (Appendix G ). The last column is the share of forced responses (%) at the 16,384-token cap after the answer-only stage. P-u140 saw each problem 7.7 times with a fresh solution each time.

MATH-500, levels 3–5
	35B-A3B			Nemotron
State	H	IT	AO	H	IT	AO
Base	96.0	48.1	93.7	93.6	83.6	94.1
D-u20	96.0	93.4	96.0	93.1	88.8	90.4
D-u60	96.3	91.6	79.6	95.6	69.8	92.4
D-u140	94.9	50.8	21.5	93.4	22.3	54.2

Appendix

Table 4: A second scale and a second family: Qwen3.5-35B-A3B-Base and Nemotron 3 Nano. Forced accuracy (%) within 16,384 tokens (AIME: 32,768), four draws, at handoff (H) and after twenty updates of instruction tuning (IT) or of the answer-only stage (AO). Nemotron’s starting model is already post-trained. Last row: D-u140’s excess loss over O-u140, in points.

Model and stage	Correct	integers	Unboxed	tokens	tokens	its turn	block	At cap
		Crediting		Median	Under 100	Ends	Closes
D-u140, answer-only	22.5	32.2	40.3	103	49	97	0.9	2.6
D-u140, instruction tuning	59.3	67.2	20.1	451	7.5	78	29	5.9
O-u140, answer-only	94.8	95.0	3.6	1,294	0.0	0.0	96	4.5
Base, answer-only	91.7	92.2	5.7	1,474	1.0	11	84	7.0

Appendix

Table 5: Forced math responses after twenty updates (884 per model). “Ends its turn”: the response stops by opening a new user turn.

Excess over O-u140
	Instr. tuning	Answer-only
First pass	36.2 [31.9, 40.4]	73.1 [69.3, 76.7]
Budget rule	21.3 [16.9, 25.7]	68.3 [64.3, 72.2]
Answer rule	26.5 [22.3, 30.5]	58.0 [53.7, 62.2]
D-u140, forced accuracy
First pass	59.3	22.5

Appendix

Table 6: Re-forcing the drilled 9B model. Top: D-u140’s excess loss over O-u140, in points with 95% paired intervals, with every model scored under the same rule; the first pass is the registered result. Bottom: D-u140’s forced accuracy (%). Budget rule (primary): a response that ends before 1,024 tokens or without a boxed answer is continued with “Wait”. Answer rule: only unboxed responses are continued.

State	First pass	Budget rule	Answer rule
Base	94.7	92.8	94.7
Base, instr. tuning	95.6	93.3	95.7
Base, answer-only	91.7	82.5	92.3
D-u140	95.4	93.7	95.4
D-u140, instr. tuning	59.3	71.7	69.0
D-u140, answer-only	22.5	26.1	37.6

Appendix

Table 7: Forced accuracy (%) on MATH-500 levels 3–5 under re-forcing: first pass, budget rule and answer rule.

Comparison	Later stage	Excess	Interval
Drilled vs. once-trained	instructions, one epoch	9.6	[6.9, 12.4]
Fresh solutions vs. once-trained	instructions, one epoch	$-$ 0.2	[ $-$ 1.9, 1.6]
Drilled vs. once-trained	instructions, stress test	36.2	[31.9, 40.4]
	answer-only, stress test	73.1	[69.3, 76.7]
Drilled vs. fresh solutions	instructions, stress test	36.4	[32.4, 40.4]
	answer-only, stress test	72.5	[68.8, 76.0]

Appendix

Table 8: Excess loss of forced accuracy, in points with 95% paired intervals: MATH-500 for the 9B model unless noted, and the arithmetic task’s targeted problems. A positive excess means that the first model lost more than the second between handoff and the end of the later stage. The stress tests run 20 updates at

3\times 10^{-4}

; one epoch is 267 updates at

10^{-4}

	At handoff			After twenty answer-only updates
Arm	Forced, 4,096	Forced, 16,384	Free, 4,096	4,096	16,384	At cap	Strict	$B$
F-u140	61.7	81.5	55.7	4.2	11.7	87.5	4.9	0.139
FA1	51.0	75.3	62.2	25.8	48.2	49.0	13.0	0.0085
FA025	53.6	78.6	61.7	7.6	27.9	71.1	10.2	0.022
FN	58.3	85.7	55.2	4.9	25.3	72.4	6.5	0.013
U2	45.3	70.6	22.4	0.3	6.5	92.2	2.1	0.154

Appendix

Table 9: The anchor arms, forced accuracy (%) at both budgets.

B

over the full samples. Differences from F-u140 after the answer-only stage within 16,384 tokens, with paired intervals: FA1 +36.5 [30.5, 42.2], FA025 +16.1 [10.9, 21.4], FN +13.5 [7.8, 19.3], U2

-

5.2 [

-

9.6,

-

1.0].

State	4–7	8–15	16–31	32–63	64–127	128–255	256–511	512–1,024
T-u30	0.0057	0.0003	$-$ 0.0010	$-$ 0.0017	$-$ 0.0004	$-$ 0.0006	$-$ 0.0003	$-$ 0.0002
S-u230	$-$ 0.018	$-$ 0.0040	$-$ 0.0033	$-$ 0.0044	$-$ 0.0044	$-$ 0.0039	$-$ 0.0033	$-$ 0.0032
U-u20	0.0059	$-$ 0.0026	$-$ 0.0019	$-$ 0.0030	$-$ 0.0037	$-$ 0.0029	$-$ 0.0026	$-$ 0.0027
F-u20	0.0017	$-$ 0.0021	$-$ 0.0003	$-$ 0.0025	$-$ 0.0032	$-$ 0.0030	$-$ 0.0026	$-$ 0.0024
U-u60	0.025	$-$ 0.0020	$-$ 0.0008	0.019	0.020	0.019	0.017	0.013
F-u60	0.015	0.0004	0.0072	0.0092	0.014	0.020	0.016	0.016

Appendix

Table 10:

B

(nats per token) by position on M0’s forced texts, first 1,024 tokens. Positions 1–3 are the forced prefix.

State	Passes	$B$ , full	$B$ , 1,024	$A$ (%)	Kept
T-u30	–	$-$ 0.0001	$-$ 0.0004	1.3	0.98
S-u230	1.15	$-$ 0.0034	$-$ 0.0035	2.2	0.91
U-u20	1.1	$-$ 0.0027	$-$ 0.0027	1.9	0.89
F-u20	1.1	$-$ 0.0024	$-$ 0.0025	1.9	0.90
U-u60	3.3	0.012	0.015	3.7	0.74
F-u60	3.3	0.013	0.016	3.8	0.74

Appendix

Table 11:

B

(nats per token) scored over the full samples and over the first 1,024 tokens, with

A

and the share of M0’s probability kept on the alternatives it sampled, both over the full samples.

Origin	Update 1	Update 2	Update 3
M0	10.45	8.06	9.31
T-u30	10.46	7.89	9.10
S-u230	10.43	7.74	10.10
U-u20	10.89	8.05	9.08
F-u20	10.97	7.86	9.11
U-u60	10.56	8.52	9.38

Appendix

Table 12: Loss on the next batch of the answer-only stage (summed over 32 examples); the batches are identical across origins.

State	Resumed	Strict	Share resumed	Free
M0	54.9	13.8	52	13.5
T-u30	62.2	22.1	48	44.0
S-u230	57.0	14.6	51	37.8
U-u20	59.9	10.4	61	43.8
F-u20	58.6	22.9	43	51.8
U-u60	64.8	14.1	62	46.1

Appendix

Table 13: Accuracy (%) at handoff within 4,096 tokens: forced, resumed and strict, the share of forced responses resumed, and free accuracy. Free accuracy on the arithmetic task was probed only at this budget.

State	Answer-only, $3\times 10^{-4}$	Answer-only, $10^{-4}$	Instr. tuning
M0	45.1 / 9.9	–	53.6 / 26.3
T-u30	59.4 / 22.9	–	–
S-u230	57.6 / 8.6	–	–
U-u20	58.1 / 25.3	61.5 / 10.4	58.9 / 26.8
F-u20	62.5 / 22.1	60.9 / 21.6	57.8 / 37.0
U-u60	46.4 / 7.6	–	–

Appendix

Table 14: Forced accuracy (%) after twenty updates of each later stage, within 4,096 tokens, as resumed / strict.

State	Handoff	After twenty answer-only updates
M0	77.6 / 19.3 (67)	73.4 / 18.8 (61)
T-u30	78.1 / 22.9 (63)	74.7 / 34.6 (45)
S-u230	77.9 / 26.3 (59)	75.8 / 10.9 (81)
U-u140	75.5 / 17.4 (66)	6.0 / 0.8 (35)
F-u140	81.5 / 68.5 (14)	11.7 / 4.9 (20)

Appendix

Table 15: Forced accuracy (%) within 16,384 tokens, as resumed / strict (share resumed).

	At handoff			After twenty answer-only updates
State	4,096	8,192	16,384	4,096	8,192	16,384	Over	Cap
M0	50.0	65.1	77.6	50.8	64.8	73.4	46	22.1
T-u30	58.6	71.6	78.1	55.5	65.6	74.7	41	18.0
S-u230	54.2	68.8	77.9	54.2	65.6	75.8	41	16.1
U-u140	52.9	65.4	75.5	1.8	2.9	6.0	98	93.2
F-u140	60.4	73.4	81.5	4.2	7.3	11.7	95	87.5

Appendix

Table 16: Arithmetic forced accuracy (%) within each budget, from the 16,384-token probes; “Over” and “Cap” are the shares of responses longer than 4,096 tokens and at the 16,384 cap.

Model and stage	2,048	4,096	8,192	16,384	Capped	Unboxed	Mean tokens
Base, handoff	67.2	79.6	88.3	94.7	5.1	3.8	3,340
Base, instruction tuning	71.8	83.7	91.1	95.6	2.5	1.9	2,712
Base, answer-only	67.9	78.5	86.5	91.7	7.0	5.7	3,608
O-u140, handoff	68.3	80.1	88.8	94.6	5.4	3.8	3,370
O-u140, instruction tuning	67.5	81.3	89.3	94.7	4.8	4.1	3,196
O-u140, answer-only	67.9	79.5	89.6	94.8	4.5	3.6	3,162

Appendix

Table 17: Math forced accuracy (%) within each budget, MATH-500 levels 3–5, with the shares of responses at the cap and without a boxed answer, and the mean response length.

Removed	Problems
Answer not a single integer	12,493
Solution boxes several different answers	2,614
Duplicate copies disagree	107
Figures	331
Prompt over 512 tokens	28
Contamination hits	91

Appendix

Table 18: Screens of the math pool, in order of removal.

Source	Forced	Free	Direct	Median tokens
MATH-500 levels 1–2	1.00	1.00	0.58	850
MATH-500 level 3	0.92	0.83	0.42	1,326
MATH-500 level 4	0.88	0.71	0.17	954
MATH-500 level 5	0.62	0.58	0.08	2,928
OlympiadBench (integer)	0.46	0.42	0.00	6,144 (cap)
NuminaMath-TIR (integer)	0.73	0.67	0.19	1,410

Appendix

Table 19: Calibration of the base model, one draw per problem and mode. Direct: an empty think block with a 256-token cap.

Model	Handoff	After	Loss	Interval
Base, new adapter	94.7	94.5	0.2	[ $-$ 1.2, 1.6]
Drilled	95.4	86.0	9.4	[6.8, 12.1]
Once-trained	94.6	94.8	$-$ 0.2	[ $-$ 1.5, 1.0]
Fresh solutions	94.6	95.0	$-$ 0.5	[ $-$ 1.6, 0.7]

Appendix

Table 20: One epoch of instruction tuning at

10^{-4}

: forced accuracy (%) and each model’s loss, in points with its 95% interval.

Training texts	Model	Handoff	After
Own solutions	Drilled	95.4	94.3
	Once-trained	94.6	94.7
gpt-oss-120b traces	Drilled	95.0	95.5
	Once-trained	96.3	95.8

Appendix

Table 21: Replay: forced accuracy (%) at handoff and after 60 updates of instruction tuning at

10^{-4}

with 6.25% of each batch replayed from the model’s own training texts.

Origin	$k=0$	1	2	5	20	$R_{5}$	Habit, $k=20$
D + IT	59.3	84.7	90.3	93.9	94.3	0.98 [0.94, 1.01]	94.0 (0.98)
D + AO	22.5	51.0	79.5	92.8	93.3	0.97 [0.95, 1.00]	92.3 (0.97)
D-T + IT	33.7	76.4	87.1	93.1	93.3	0.98 [0.94, 1.00]	–
D-T + AO	1.1	21.6	70.5	91.5	93.0	0.96 [0.94, 0.99]	–
O + IT	94.7	95.2	94.2	94.3	94.6	–	–

Appendix

Table 22: Relearning: forced accuracy (%) after

k

updates of reasoning training from each broken model, the share

R_{5}

recovered after five updates with its 95% interval, and forced accuracy after twenty updates of the habit data, with its share

R_{20}

. IT: the instruction stress test; AO: the answer-only stage.

Model	$B$	Handoff	Instructions	Answer-only
Sharpened	0.082	93.1	95.0	92.4
Once-trained	0.0004	94.6	94.7	94.8
Drilled	0.111	95.4	59.3	22.5

Appendix

Table 23: Sharpening without repetition:

B

at handoff (nats per token) and forced accuracy (%) at handoff and after the two stress tests.

Run	Model	Handoff	IT	Gentle
Second	Drilled	94.8	32.7	80.9
	Once-trained	96.2	95.1	95.4
	Fresh solutions	95.6	94.6	94.0
	Drilled’s excess		61.1 [57.0, 65.2]	13.1 [10.0, 16.3]
Third	Drilled	94.6	45.9	76.1
	Once-trained	94.9	95.1	94.2

Appendix

Table 24: The two new runs of the 9B design: forced accuracy (%) at handoff and after each stage, and the drilled model’s excess over the once-trained model, in points with its 95% interval. IT: the instruction stress test; gentle: 60 updates at

10^{-4}

Model	Handoff	Gentle	Stress test	Loss, gentle
Base	0.2	–	–	–
Drilled (D-S)	37.9	26.6	0.1	11.3 [7.8, 14.8]
Once-trained (O-S)	66.3	50.8	0.4	15.5 [11.7, 19.3]
Fresh solutions (P-S)	58.1	38.7	0.4	19.4 [15.4, 23.3]

Appendix

Table 25: The synthetic skill: forced accuracy (%) on 300 held-out five-digit base-7 products within 2,048 tokens, four draws each, and each model’s loss in the gentler stage (60 updates at

10^{-4}

) with its 95% interval.

ID	Prediction	Threshold	Observed	Outcome	Claim
RS1	One epoch breaks D	excess $\geq$ 5, CI $>0$	9.6	held	break
RS2	One epoch spares P	P $-$ O within $\pm$ 3	$-$ 0.2	held	cause
RP1	Replay mitigates, own solutions	reduction $\geq$ 50%	91%, prevents	held	remedy
RL1	Relearning is fast	$R_{5}$ of D + IT $\geq$ 0.5	0.98	held	competence
RL2	The ceiling control holds	O + IT within 2 at every $k$	at most 0.6	held	competence
SH1	Sharpening alone does less than half	AO excess of O-sharp $<$ 36.5	0.9	held	signature

Appendix

Table 26: Registered predictions of the revision experiments, confirmatory, one per row; columns, outcomes and claims as in Table 27 . SD’s secondary prediction, an excess of at least 5 points in each run and stage, also held; secondary predictions are listed in the records.

Math pilot
ID	Prediction	Threshold	Observed	Outcome	Claim
MP1	Drilling over-sharpens	$B$ (D) $-$ $B$ (O) $\geq$ 5% of the base’s NLL, CI $>0$	24% against 0.1%	held	signature
MP2	$B$ rises with passes	D-u20 $<$ D-u60 $<$ D-u140	0.1, 3.4, 24%	held	signature
MP3	The drilled model loses more, each stage	more than D-u20 and O, CIs $>0$	36.2 (IT), 73.1 (AO) over O	held	break
MP5	Free accuracy falls more than forced	base and O, both stages	all four	held	decision
Fresh solutions (P)

Appendix

Table 27: Registered predictions on mathematics (confirmatory), one per row. Threshold: the registered criterion; Observed: the result, in points of forced accuracy unless marked. Outcomes: held; failed; failed* (in part); rival, not supported (T2 was registered as the alternative to T1, so at most one could hold). Claim: break (the drilled model breaks where the once-trained one does not), cause (repeated text is the cause), teacher (teacher traces), competence (whether the competence is lost or suppressed), stage (dependence on the later stage), signature (

B

and over-sharpening), mechanism (what the later stage does), decision (the decision to reason), remedy (replay). “As MP1” repeats the pilot’s criterion; IT is instruction tuning and AO the answer-only stage.

Depth profile and cap test
ID	Prediction	Threshold	Observed	Outcome	Claim
DP1	Repetition, not training amount, moves the reasoning	$B$ (S) below $B$ (U) and $B$ (F), CIs $<0$	$-$ 0.0035 against 0.107, 0.161	held	signature
DP2	Every lasting $B$ below every drilled $B$	separation	highest lasting $-$ 0.0004, lowest drilled 0.061	held	signature
DP3	$B$ rises with passes in U and F	monotone	monotone	held	signature
DP4	$B$ ranks the loss	Spearman $\geq$ 0.7	0.93	held	signature
CT1	Competence lost, not slowed	U, F $<$ 15% within 16,384 tokens	6.0, 11.7	held	break

Appendix

Table 28: Registered predictions on the arithmetic task, an exploratory line run before and alongside the math pilot;

B

is over the first 1,024 tokens, as registered. Columns and outcomes as in Table 27 ; not readable: a registered check failed, or the item was never evaluated. Claims as in Table 27 , plus budget (token budgets), anchor (the anchor arms), horizon (long-horizon probes) and earlier study (claims of the earlier replay study, re-examined; its records are in the repository). Accuracy is forced, in percent; U and F are drilled to 7.7 passes unless marked.

Explore similar work

Reasoning Quality Emerges Early: Data Curation for Reasoning Models
Jun 25, 2026Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari +2Large Reasoning Models Reasoning Traces
Department of Computer Science, University of California Los Angeles (UCLA), California, USA · Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA
Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning
Sep 22, 2026Ismail Labiad, Matthieu Kowalski, Marc Schoenauer +2LLM Reasoning Strategies Reasoning Skills
Meta FAIR · Université Paris-Saclay, LISN, Inria, CNRS · NYU Courant Institute and CDS
Thinking with Reasoning Skills: Fewer Tokens, More Accuracy
Apr 23, 2026Guangxiang Zhao, Qilong Shi, Xusen Xiao +3LLM Reasoning Strategies Reasoning Skills
Qiyuan Tech · Tsinghua University · The University of Hong Kong +1

Jun 25, 2026, cs.LGJ/K move · Enter open · S save

Reasoning Quality Emerges Early: Data Curation for Reasoning Models

Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato+1

Department of Computer Science, University of California Los Angeles (UCLA), California, USA · Optum AI, UnitedHealth Group, Minneapolis, Minnesota, USA

Large Reasoning Models Reasoning Traces Data-Curation

Open paper AskPDF arXiv

Supervised fine-tuning (SFT) on a small, high-quality set of long reasoning traces is an effective approach for eliciting strong reasoning capabilities in Large Language Models (LLMs). However, existing methods for curating high-quality SFT data rely heavily on strong reasoning models to filter examples based on diversity and difficulty, making the curation process costly while often yielding suboptimal data quality. In this work, we show that diverse and challenging reasoning examples can be identified using only the initial reasoning tokens. Specifically, we demonstrate that difficult problems can be reliably detected based on the loss of the first 100 reasoning tokens evaluated at a randomly perturbed checkpoint of the pretrained model. We further show that examples exhibiting similar loss patterns over their first 1k reasoning tokens across a small number of perturbed checkpoints extrapolating along the fine-tuning trajectory provably induce similar gradients. We validate our approach through extensive experiments on fine-tuning Qwen2.5-7B and Llama3.1-8B models on the M23K medical reasoning and OpenThoughts-Math datasets. Our method outperforms existing baselines by up to 1.7% while being 91% more token efficient.

Abstract

Explore similar work

Reasoning Quality Emerges Early: Data Curation for Reasoning Models

Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy