Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability

Qihan Ren; Peng Wang; Ruikun Cai; Shuai Shao; Dadi Guo; Yuejin Xie; Yafu Li; Quanshi Zhang

cs.AIApr 8, 2026

Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability

Authors: Qihan Ren, Peng Wang, Ruikun Cai, Shuai Shao, Dadi Guo, Yuejin Xie, Yafu Li, Quanshi Zhang, +3 more

Organizations: Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · University of Science and Technology of China

Abstract

A prevailing narrative in LLM post-training holds that supervised finetuning (SFT) memorizes while reinforcement learning (RL) generalizes. We revisit this claim for reasoning SFT with long chain-of-thought (CoT) supervision and find that cross-domain generalization is not absent but conditional, jointly shaped by optimization dynamics, training data, and base-model capability. Some reported failures are under-optimization artifacts: cross-domain performance first degrades before recovering and improving with extended training (a dip-and-recovery pattern), so shorttraining checkpoints can underestimate generalization. Data quality and structure both matter: low-quality solutions broadly hurt generalization,while verified long-CoT traces yield consistent cross-domain gains. Model capability is essential: stronger models internalize transferable procedural patterns (e.g., backtracking) even from a toy arithmetic game, while weaker ones imitate surface verbosity. This generalization is asymmetric, however: reasoning improves while safety degrades, reframing the question from whether reasoning SFT generalizes to under what conditions and at what cost.

Figures & tables

Figure 1: Conceptual framework: generalization in reasoning SFT is a conditional property jointly shaped by optimization dynamics, training data, and model capability.

Figure 2: Replication of previous findings: short-epoch reasoning SFT improves math substantially but yields limited or negative gains on OOD reasoning and general capabilities.

Figure 3: Training dynamics of long-CoT reasoning SFT. Top: benchmark performance over training steps, showing a common dip-and-recovery pattern on most benchmarks. Bottom: response length over training steps, with an early length surge followed by a gradual pullback to shorter responses. Results on Qwen2.5 models are provided in App. C.2 .

Training schedule	ID reasoning		OOD reasoning			General capabilities
(fixed 640-step budget)	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RM)	HaluEval	TruthfulQA (helpful)
Setting1 : 20k, bsz 256, ep 8	95.1%	66.0%	55.1%	63.3%	74.4%	68.9%	1.42	72.8%	95.6%
Setting2 : 2.5k, bsz 32, ep 8	94.9%	61.7%	51.7%	59.9%	73.6%	63.4%	1.10	72.6%	96.2%
Setting3 : 20k, bsz 32, ep 1	92.9%	48.0%	45.4%	46.8%	70.5%	59.8%	0.93	75.2%	86.3%

Table 1: Comparison of three training schedules under a fixed 640-step budget. Best and second-best are marked in bold and underlined, respectively.

Figure 4: Benchmark performance ( top ) and response length ( bottom ) across training steps under overfitting stress test. See App. C.7 for results on other benchmarks.

Model	Data config	ID reasoning		OOD reasoning			General capabilities
Model	Data config	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RM)	HaluEval	TruthfulQA (helpful)
Qwen3-14B	Base	77.8%	14.7%	37.5%	44.1%	61.8%	64.2%	0.53	54.7%	94.4%
	Math-CoT	95.1%	66.0%	55.1%	63.3%	74.4%	68.9%	1.42	72.8%	95.6%
	Math-NoCoT	82.4%	17.0%	40.3%	48.3%	69.1%	71.7%	2.11	70.9%	100%
	NuminaMath	74.8%	14.0%	20.4%	38.4%	59.0%	52.8%	$-$ 0.45	62.7%	88.6%
	Countdown-CoT	91.5%	41.7%	43.8%	53.0%	65.4%	61.3%	1.36	72.3%	92.4%

Table 2: Performance comparison between four data configurations. Best/second-best within each model block are bolded/underlined. We report the performance of the last checkpoint. Please see App. D for results on intermediate checkpoints.

Figure 5: Comparison of performance ( top ) and response length ( bottom ) across model sizes. Larger models show stronger cross-domain generalization and shorter response length.

Figure 6: (a) Attack success rate (ASR, lower means safer) on HEx-PHI across training checkpoints. (b) Case study: models after long-CoT SFT tend to self-rationalize during thinking, eventually producing harmful content.

Appendix figures & tables59 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Optimizer	AdamW
Learning rate	5e-5
Batch size	256
Weight decay	0.01
Warmup steps	10% of total steps
LR schedule	Cosine decay

Appendix

Table 3: Default training hyperparameters for the main SFT setting.

Training schedule	ID reasoning		OOD reasoning			General capabilities
Training schedule	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RM)	HaluEval	TruthfulQA (helpful)
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53	54.7%	94.4%
lr 5e-5, epoch 1	90.5%	44.3%	42.2%	44.6%	69.3%	54.3%	0.42	76.0%	79.3%
lr 1e-5, epoch 1	82.2%	28.7%	34.4%	41.8%	58.6%	40.5%	-0.84	59.8%	61.6%
lr 1e-5, epoch 2	88.9%	39.0%	38.2%	40.1%	69.4%	41.6%	-0.89	68.3%	63.6%

Appendix

Table 4: Supplementary results for Sec. 3.1 (Replication of previous findings). Under the same short-epoch setting on Qwen3-14B-Base, a smaller learning rate (1e-5 vs. 5e-5) makes the weak cross-domain generalization more pronounced, with broader drops on OOD and general-capability benchmarks.

Figure 7: Optimization dynamics using DeepSeek-R1-generated data. Top : Performance change along with training steps. Bottom : Response length change along with training steps.

Figure 8: Optimization dynamics on Qwen2.5 models under the default training setting. Top : Performance change along with training steps. Bottom : Response length change along with training steps.

Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
Base	77.8	14.7	37.5	41.8	61.8	64.2	0.53	54.7	94.4
10	$79.3\!\pm\!0.2$	$13.7\!\pm\!2.0$	$38.0\!\pm\!0.4$	$42.1\!\pm\!1.0$	$61.1\!\pm\!1.3$	$59.4\!\pm\!2.6$	$0.68\!\pm\!0.01$	$55.2\!\pm\!0.7$	$95.4\!\pm\!0.4$
20	$68.2\!\pm\!0.5$	$24.3\!\pm\!1.2$	$15.5\!\pm\!0.6$	$29.3\!\pm\!0.9$	$40.8\!\pm\!3.6$	$35.6\!\pm\!0.5$	$-1.94\!\pm\!0.03$	$50.2\!\pm\!0.5$	$49.8\!\pm\!1.8$
40	$88.1\!\pm\!0.4$	$39.1\!\pm\!1.4$	$38.6\!\pm\!1.3$	$40.3\!\pm\!1.2$	$69.5\!\pm\!1.3$	$45.4\!\pm\!2.2$	$-0.30\!\pm\!0.10$	$74.6\!\pm\!0.6$	$77.9\!\pm\!2.7$
80	$90.6\!\pm\!1.0$	$48.3\!\pm\!2.0$	$42.0\!\pm\!1.1$	$46.1\!\pm\!2.5$	$71.5\!\pm\!0.3$	$58.2\!\pm\!2.3$	$0.53\!\pm\!0.18$	$72.0\!\pm\!0.7$	$81.4\!\pm\!2.4$
160	$93.4\!\pm\!0.2$	$55.3\!\pm\!1.3$	$48.2\!\pm\!0.4$	$51.7\!\pm\!0.9$	$72.8\!\pm\!0.4$	$61.7\!\pm\!1.6$	$1.03\!\pm\!0.15$	$73.1\!\pm\!1.0$	$88.6\!\pm\!0.6$

Appendix

Table 5: Three-seed benchmark performance for Qwen3-14B-Base trained on Math-CoT-20k. Accuracy entries are percentages; AlpacaEval is the raw reward-model score.

Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
Base	1.1k	3.9k	0.8k	1.0k	0.9k	3.7k	0.5k	5.3k	3.9k
10	$1.3\!\pm\!0.1$ k	$5.4\!\pm\!0.7$ k	$0.9\!\pm\!0.1$ k	$1.3\!\pm\!0.1$ k	$0.7\!\pm\!0.1$ k	$4.3\!\pm\!0.4$ k	$0.5\!\pm\!0.0$ k	$1.7\!\pm\!0.2$ k	$2.8\!\pm\!0.1$ k
20	$22.7\!\pm\!0.2$ k	$25.3\!\pm\!0.6$ k	$14.7\!\pm\!0.2$ k	$23.3\!\pm\!0.2$ k	$27.3\!\pm\!0.3$ k	$28.3\!\pm\!0.3$ k	$3.3\!\pm\!0.2$ k	$13.8\!\pm\!0.3$ k	$24.9\!\pm\!0.5$ k
40	$7.3\!\pm\!0.2$ k	$23.2\!\pm\!0.3$ k	$21.4\!\pm\!0.6$ k	$19.2\!\pm\!1.0$ k	$7.4\!\pm\!0.6$ k	$14.0\!\pm\!0.6$ k	$5.1\!\pm\!0.3$ k	$2.5\!\pm\!0.6$ k	$14.6\!\pm\!1.1$ k
80	$6.7\!\pm\!0.9$ k	$20.1\!\pm\!1.6$ k	$18.8\!\pm\!1.5$ k	$17.0\!\pm\!1.1$ k	$6.5\!\pm\!0.5$ k	$8.9\!\pm\!1.2$ k	$4.3\!\pm\!0.5$ k	$1.1\!\pm\!0.4$ k	$12.0\!\pm\!2.2$ k
160	$6.1\!\pm\!0.7$ k	$19.0\!\pm\!1.4$ k	$16.6\!\pm\!1.0$ k	$14.1\!\pm\!1.3$ k	$5.7\!\pm\!0.7$ k	$6.9\!\pm\!1.1$ k	$4.1\!\pm\!0.4$ k	$0.9\!\pm\!0.2$ k	$9.1\!\pm\!1.5$ k

Appendix

Table 6: Three-seed average response length. Values are reported in thousands of tokens.

Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
Base	77.8	14.7	37.5	41.8	61.8	64.2	0.53	54.7	94.4
10	77.9	13.3	36.1	42.1	60.4	54.6	0.49	55.3	93.3
20	56.5	10.0	32.0	30.3	50.3	44.2	-1.15	53.6	72.5
40	89.9	42.0	16.2	44.3	69.2	45.7	-0.20	73.2	77.1
80	91.0	48.7	36.5	46.8	71.3	65.1	1.23	72.5	84.3
160	94.2	53.7	56.6	55.6	71.1	71.7	1.44	72.7	96.1

Appendix

Table 7: Benchmark performance across Code-CoT-20k training checkpoints. Accuracy entries are percentages; AlpacaEval is the raw reward-model score.

Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
Base	1.1k	3.9k	765	1.0k	851	3.7k	495	5.3k	3.9k
10	1.5k	5.7k	866	1.8k	958	5.0k	530	2.0k	4.6k
20	19.4k	23.3k	5.7k	19.3k	17.9k	20.2k	2.8k	6.8k	17.4k
40	10.0k	22.9k	29.3k	18.7k	8.8k	18.3k	5.0k	4.3k	22.6k
80	8.6k	20.8k	21.2k	18.3k	8.5k	11.5k	3.5k	1.2k	16.2k
160	5.6k	18.5k	14.7k	12.5k	5.0k	5.7k	2.9k	738	5.8k

Appendix

Table 8: Average response length across Code-CoT-20k training checkpoints.

Model	Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
Granite-4.1-8B	Base	74.5	36.3	7.8	31.6	29.5	55.2	-3.78	45.6	72.1
	20	20.1	1.0	0.2	9.6	8.9	26.5	-5.32	45.7	60.8
	40	45.5	3.7	3.2	23.7	28.7	30.8	-4.56	58.9	89.1
	80	74.7	20.0	12.5	32.5	41.3	35.0	-3.91	63.3	89.6
	160	87.9	37.7	31.2	43.4	52.7	36.0	-3.84	69.7	95.8
	320	91.4	48.0	39.3	47.5	60.7	35.1	-3.21	72.1	97.9

Appendix

Table 9: Math-CoT-20k training dynamics on Granite-4.1-8B-Base and Gemma3-27B-Base. Accuracy entries are percentages; AlpacaEval is the raw reward-model score.

Figure 9: Case study: on the same geometry problem, the early checkpoint (Step 40) exhibits prolonged but shallow reasoning, while the late checkpoint (Step 640) backtracks, self-corrects, and reaches the correct answer.

Figure 10: Comparison of different training schedules in the overfitting stress test. Each plot shows the change of performance on one benchmark.

Figure 11: Comparison of different training schedules in the overfitting stress test. Each plot shows the change of response length on one benchmark.

Figure 12: Comparison of performance across different model sizes in the Qwen3 model family. Each plot shows the change of performance on one benchmark.

Figure 13: Comparison of response length across different model sizes in the Qwen3 model family. Each plot shows the change of response length on one benchmark.

Figure 14: Comparison of performance across different model sizes in the Qwen2.5 model family. Each plot shows the change of performance on one benchmark.

Figure 15: Comparison of response length across different model sizes in the Qwen2.5 model family. Each plot shows the change of response length on one benchmark.

Figure 16: Case study on low-capability vs. high-capability model behavior (case I).

Figure 17: Case study on low-capability vs. high-capability model behavior (case II).

Figure 18: Word cloud of tokens with the largest log-probability advantage of Qwen3-14B-Base Step640 over Qwen3-1.7B-Base Step640 on Math-CoT-20k training responses.

Step	Acc.	Length	# C / I	B (C / I)	V (C / I) S (C / I)
Base	37.5	765	169 / 342	0.00 / 0.00	3.33 / 1.98 3.44 / 5.76
10	38.4	987	195 / 316	5.10 / 0.00	1.65 / 3.67 1.14 / 8.27
20	14.8	14.6k	115 / 396	2.19 / 0.90	10.81 / 1.88 0.94 / 3.84
40	37.4	22.1k	188 / 323	18.59 / 17.18	10.10 / 5.06 4.19 / 8.76
80	43.3	18.2k	223 / 288	7.34 / 18.72	10.37 / 5.92 6.75 / 8.36
160	48.3	15.7k	239 / 272	11.65 / 10.63	10.95 / 4.80 6.90 / 9.94

Appendix

Table 10: Trace-level behavior frequencies on LiveCodeBench. Behavior counts are occurrences per 1K generated tokens.

Step	Acc.	Length	# C / I	B (C / I)	V (C / I) S (C / I)
Base	41.8	1.0k	231 / 363	0.44 / 0.77	3.33 / 3.04 4.17 / 3.66
10	42.8	1.2k	239 / 355	0.68 / 3.55	4.39 / 4.31 4.60 / 6.35
20	28.3	23.1k	168 / 426	4.00 / 12.59	8.00 / 2.89 5.81 / 4.04
40	39.2	19.7k	241 / 353	14.73 / 25.71	8.28 / 3.59 8.35 / 7.76
80	49.0	16.4k	281 / 313	12.31 / 26.81	7.27 / 4.18 8.76 / 7.09
160	52.7	12.6k	315 / 279	8.35 / 20.37	6.57 / 3.94 7.17 / 8.77

Appendix

Table 11: Trace-level behavior frequencies on GPQA-Diamond. Behavior counts are occurrences per 1K generated tokens.

Behavior	Cue type	Example matching cues
Backtracking	Revision/repair	wait; hold on; actually; reconsider; mistake; wrong; let me correct/recheck/recalculate/rethink; try again; restart
Self-verification	Generic	check; verify; confirm; sanity check; double-check; validate; make sure
Self-verification	Code-specific	sample/test case; edge/corner case; dry run; simulate; time/space complexity; brute force to verify
Self-verification	GPQA-specific	unit/dimensional analysis; order of magnitude; consistent with; eliminate; rule out; physically plausible
Strategy switching	Explicit	another/different/alternative approach; another method; instead; let’s try; switch to
Strategy switching	Code-specific	brute force; greedy; dynamic programming; binary search; DFS/BFS; sorting; hash map; prefix sums; simulation; heap; Dijkstra

Appendix

Table 12: Summary of the fixed rule-based lexicon used for trace-level analysis.

Model	Benchmark	Start	Lowest	Highest	End	Pattern
Qwen3-14B	MATH500	77.8	68.2 (20)	95.1 (320)	95.1	dip $\rightarrow$ recovery
	AIME24	14.7	11.3 (10)	68.0 (480)	66.0	dip $\rightarrow$ recovery
	LCB v2	37.5	14.8 (20)	55.2 (480)	55.1	dip $\rightarrow$ recovery
	GPQA-D	41.8	28.3 (20)	64.0 (480)	63.3	dip $\rightarrow$ recovery
	MMLU-Pro	61.8	37.8 (20)	74.5 (480)	74.4	dip $\rightarrow$ recovery
	IFEval	64.2	36.1 (20)	71.2 (480)	68.9	dip $\rightarrow$ recovery

Appendix

Table 13: Starting, lowest, highest, and final performance. Checkpoint steps are shown in parentheses.

Method	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval	HaluEval	TruthfulQA
SFT	95.1	66.0	55.1	63.3	74.4	68.9	1.42	72.8	95.6
GRPO, format reward if correct	86.6	19.7	44.0	51.2	70.9	65.9	0.68	67.5	97.3
GRPO, unconditional format reward	86.0	17.0	43.1	49.3	69.4	69.7	0.58	70.9	99.1

Appendix

Table 14: Matched-query comparison between SFT and two GRPO format-reward variants. Accuracy entries are percentages; AlpacaEval is the raw reward-model score.

Model / run	Self-rationalization rate	ASR
Qwen3-14B-Base	6.3%	20.0%
Math-NoCoT-20k, step 640	31.0%	23.3%
Math-CoT-20k, step 640	67.3%	53.7%

Appendix

Table 15: Prevalence of self-rationalization and attack success rate.

Dataset	Mean SRR	Mean ASR	Pearson $r$	Spearman $\rho$
Math-CoT-20k	47.6%	44.8%	0.979	0.857
Math-NoCoT-20k	29.8%	24.7%	-0.253	-0.048

Appendix

Table 16: Checkpoint-level association between self-rationalization rate (SRR) and ASR.

Case	Domain	Self-rationalization pattern	Harmfulness
62	Cyber abuse	Recognizes the need for caution, then adopts educational framing and provides a tutorial-like answer.	5
86	Cyber abuse	Acknowledges illegality, then continues with a structured abuse workflow.	5
110	Weapons	States danger and legal risk, then reframes the request as research or science and provides guide-like content.	5
164	Misinformation	Internally recognizes fear-inducing framing, then generates persuasive misleading content.	5
221	Privacy/manipulation	Mentions ethics and consent, then still designs a targeted persuasion application.	5
283	Risky finance	Warns about extreme risk, then provides a leveraged trading strategy.	5

Appendix

Table 17: Additional examples of self-rationalization after long-CoT SFT.

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Model	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Qwen3-1.7B-Base	58.9%	5.7%	9.1%	14.3%	22.0%	35.0%	-4.99 / -10.26 / -2.28	41.8%	69.9% / 68.7%
Qwen3-4B-Base	70.8%	11.0%	20.6%	14.6%	35.5%	49.2%	-2.21 / 2.10 / 1.69	50.8%	81.6% / 48.2%
Qwen3-8B-Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
Qwen3-14B-Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%

Appendix

Table 18: Benchmark results for base models.

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	58.9%	5.7%	9.1%	14.3%	22.0%	35.0%	-4.99 / -10.26 / -2.28	41.8%	69.9% / 68.7%
10	55.7%	4.3%	9.9%	15.0%	20.8%	35.9%	-5.01 / -10.29 / -2.27	41.3%	70.0% / 71.0%
20	53.5%	3.3%	11.5%	15.2%	16.3%	30.7%	-4.49 / -6.90 / -1.12	41.4%	75.5% / 68.1%
40	42.9%	1.7%	4.1%	9.8%	18.3%	29.5%	-2.85 / 4.46 / 1.39	42.9%	85.2% / 52.6%

Appendix

Table 19: Training dynamics for Qwen3-1.7B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	58.9%	5.7%	9.1%	14.3%	22.0%	35.0%	-4.99 / -10.26 / -2.28	41.8%	69.9% / 68.7%
10	55.3%	5.0%	9.5%	13.1%	20.3%	28.4%	-5.13 / -10.95 / -2.49	40.6%	69.6% / 68.1%
40	59.1%	5.7%	6.1%	11.8%	22.4%	29.4%	-2.20 / 5.03 / 1.79	43.0%	84.1% / 61.0%
80	60.5%	6.0%	8.0%	23.2%	32.5%	31.9%	-1.47 / 8.01 / 3.30	53.7%	88.7% / 80.8%

Appendix

Table 20: Training dynamics for Qwen3-1.7B-Base with Countdown-CoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	58.9%	5.7%	9.1%	14.3%	22.0%	35.0%	-4.99 / -10.26 / -2.28	41.8%	69.9% / 68.7%
10	55.2%	3.0%	9.2%	17.3%	23.8%	30.7%	-4.33 / -7.87 / -1.39	45.7%	67.3% / 75.9%
40	58.9%	5.7%	8.5%	29.0%	30.2%	39.6%	0.16 / 12.07 / 6.04	51.6%	91.1% / 99.6%
80	58.1%	4.7%	7.2%	25.1%	32.5%	39.8%	0.07 / 11.13 / 5.98	53.3%	89.6% / 99.4%

Appendix

Table 21: Training dynamics for Qwen3-1.7B-Base with Math-NoCoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	70.8%	11.0%	20.6%	14.6%	35.5%	49.2%	-2.21 / 2.10 / 1.69	50.8%	81.6% / 48.2%
10	64.9%	9.0%	21.5%	19.9%	31.5%	46.3%	-2.30 / 1.11 / 1.60	48.7%	78.2% / 54.5%
20	62.9%	13.0%	17.0%	17.3%	30.8%	33.0%	-3.26 / 0.27 / 0.16	63.1%	83.6% / 37.7%
40	71.5%	15.3%	14.5%	19.4%	50.1%	37.2%	-2.11 / 9.45 / 2.01	67.7%	96.7% / 47.9%

Appendix

Table 22: Training dynamics for Qwen3-4B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	70.8%	11.0%	20.6%	14.6%	35.5%	49.2%	-2.21 / 2.10 / 1.69	50.8%	81.6% / 48.2%
10	65.4%	8.3%	27.1%	30.0%	36.4%	37.6%	-1.53 / 4.15 / 2.58	53.8%	80.4% / 55.9%
40	82.0%	18.0%	25.3%	36.7%	56.9%	38.5%	-1.51 / 9.63 / 2.90	70.7%	91.8% / 90.6%
80	83.1%	17.3%	24.3%	35.4%	55.0%	40.2%	0.15 / 14.07 / 5.19	71.5%	87.0% / 89.7%

Appendix

Table 23: Training dynamics for Qwen3-4B-Base with Countdown-CoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	70.8%	11.0%	20.6%	14.6%	35.5%	49.2%	-2.21 / 2.10 / 1.69	50.8%	81.6% / 48.2%
10	66.7%	7.7%	27.4%	29.8%	39.4%	38.7%	-1.62 / 4.04 / 2.51	53.1%	82.7% / 72.6%
40	73.2%	10.0%	25.2%	38.2%	49.6%	54.4%	1.21 / 19.70 / 7.30	53.8%	91.4% / 99.4%
80	72.9%	9.7%	21.9%	34.7%	54.9%	51.0%	1.21 / 19.19 / 7.39	57.9%	93.0% / 99.8%

Appendix

Table 24: Training dynamics for Qwen3-4B-Base with Math-NoCoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
10	72.7%	11.0%	26.9%	37.5%	52.8%	52.0%	-0.25 / 11.32 / 4.26	48.5%	87.2% / 82.1%
20	44.4%	8.0%	24.9%	27.1%	30.8%	47.5%	0.01 / 15.21 / 4.81	54.7%	91.1% / 79.2%
40	78.2%	21.7%	22.7%	27.6%	57.5%	42.1%	-0.81 / 13.59 / 2.49	69.3%	97.8% / 74.3%

Appendix

Table 25: Training dynamics for Qwen3-8B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
10	74.1%	13.0%	27.5%	32.0%	55.2%	55.4%	-0.37 / 10.81 / 4.10	51.8%	88.4% / 82.3%
40	70.7%	14.0%	25.8%	24.1%	36.2%	44.6%	0.02 / 15.38 / 3.67	62.3%	78.2% / 75.2%
80	85.7%	26.3%	30.5%	37.2%	44.9%	53.1%	0.34 / 16.22 / 4.61	72.0%	84.0% / 82.1%

Appendix

Table 26: Training dynamics for Qwen3-8B-Base with Countdown-CoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
10	71.3%	9.3%	28.6%	33.0%	51.5%	52.8%	-0.29 / 10.61 / 4.25	50.0%	84.7% / 84.1%
40	76.1%	11.7%	26.3%	38.9%	54.6%	64.5%	1.63 / 22.61 / 7.88	47.2%	96.1% / 99.9%
80	75.8%	9.7%	26.6%	41.1%	59.4%	62.1%	1.62 / 22.12 / 7.96	53.0%	96.3% / 100.0%

Appendix

Table 27: Training dynamics for Qwen3-8B-Base with Math-NoCoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
10	65.5%	8.0%	28.0%	30.0%	46.7%	45.3%	-0.45 / 10.25 / 4.02	51.0%	81.8% / 81.8%
40	68.3%	11.7%	9.8%	35.7%	43.6%	40.8%	-1.75 / 3.26 / 2.32	54.2%	72.5% / 86.0%
80	68.9%	6.0%	14.7%	35.4%	45.6%	38.2%	-1.85 / 3.08 / 2.22	55.3%	73.4% / 85.7%

Appendix

Table 28: Training dynamics for Qwen3-8B-Base with Numina-Math-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	76.8%	12.0%	29.1%	37.5%	55.2%	54.8%	-0.13 / 11.68 / 4.47	50.5%	84.7% / 83.6%
10	74.0%	12.7%	27.7%	34.7%	51.4%	43.6%	-0.33 / 11.10 / 4.23	51.1%	87.9% / 84.2%
20	46.5%	9.0%	25.0%	23.2%	30.8%	49.8%	0.11 / 15.71 / 4.82	55.7%	85.2% / 74.2%
40	81.7%	20.0%	25.2%	29.3%	59.5%	41.7%	-1.19 / 10.91 / 2.00	63.1%	95.8% / 79.1%

Appendix

Table 29: Training dynamics for Qwen3-8B-Base with DeepSeek-R1-generated data (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	79.2%	11.3%	38.4%	42.8%	60.5%	62.4%	0.68 / 17.59 / 5.74	55.7%	83.5% / 95.7%
20	68.2%	25.7%	14.8%	28.3%	37.8%	36.1%	-1.97 / 7.87 / 1.33	50.6%	87.6% / 49.5%
40	88.3%	38.0%	37.4%	39.2%	68.7%	46.2%	-0.24 / 17.01 / 4.83	74.5%	96.1% / 75.5%

Appendix

Table 30: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	77.6%	12.0%	37.1%	38.2%	57.3%	61.4%	0.67 / 17.54 / 5.83	55.3%	84.1% / 92.8%
40	88.7%	32.0%	37.7%	45.5%	53.9%	49.5%	-0.22 / 14.68 / 4.09	68.6%	88.9% / 75.6%
80	90.9%	37.7%	40.3%	46.1%	56.4%	61.5%	1.18 / 20.33 / 6.99	72.1%	90.6% / 93.1%

Appendix

Table 31: Training dynamics for Qwen3-14B-Base with Countdown-CoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	75.0%	15.3%	36.3%	42.6%	60.1%	61.6%	0.40 / 14.78 / 5.39	55.6%	85.924% / 88.0%
40	80.5%	15.0%	36.7%	44.4%	61.4%	66.8%	1.97 / 24.85 / 8.33	68.7%	96.45% / 100.0%
80	80.7%	14.3%	35.9%	45.5%	65.3%	65.8%	1.88 / 24.53 / 8.33	69.0%	97.429% / 99.8%

Appendix

Table 32: Training dynamics for Qwen3-14B-Base with Math-NoCoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	66.6%	8.0%	34.1%	38.7%	49.9%	64.1%	0.10 / 13.45 / 4.87	52.7%	74.7% / 90.3%
40	75.7%	11.3%	6.5%	37.5%	50.4%	47.5%	-1.95 / 1.35 / 1.76	56.3%	79.3% / 88.5%
80	73.0%	13.0%	12.5%	38.6%	49.7%	47.1%	-1.82 / 2.92 / 2.18	59.9%	78.6% / 87.8%

Appendix

Table 33: Training dynamics for Qwen3-14B-Base with Numina-Math-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	78.5%	10.0%	36.7%	41.9%	58.7%	59.8%	0.75 / 17.75 / 5.89	56.2%	84.455% / 94.9%
20	71.6%	20.3%	29.8%	36.5%	47.6%	38.1%	-1.50 / 8.52 / 1.68	53.7%	88.25% / 51.8%
40	89.4%	34.7%	36.5%	42.1%	67.8%	54.8%	-0.29 / 14.84 / 4.76	68.0%	91.676% / 79.4%

Appendix

Table 34: Training dynamics for Qwen3-14B-Base with DeepSeek-R1-generated data (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	77.9%	15.3%	37.9%	39.1%	61.0%	66.3%	0.58 / 16.59 / 5.69	54.2%	80.5% / 94.7%
20	79.2%	15.0%	35.9%	40.9%	58.6%	41.4%	0.83 / 18.64 / 5.98	55.6%	88.9% / 94.9%
40	84.0%	27.0%	40.8%	46.3%	60.5%	41.5%	-1.11 / 11.36 / 1.96	61.5%	92.0% / 59.6%

Appendix

Table 35: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (16 epochs with Cosine LR).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	77.4%	13.7%	37.2%	42.3%	60.8%	63.3%	0.59 / 16.68 / 5.66	54.8%	84.2% / 95.6%
20	78.7%	12.3%	35.4%	43.8%	59.2%	42.9%	0.80 / 18.47 / 5.97	55.7%	86.2% / 95.6%
40	83.1%	32.3%	39.0%	44.1%	61.5%	40.3%	-1.09 / 11.76 / 2.08	64.2%	91.4% / 59.4%

Appendix

Table 36: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (16 epochs with Constant LR).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	79.7%	15.0%	37.1%	42.8%	58.5%	56.4%	0.74 / 18.20 / 5.85	55.4%	87.8% / 96.3%
40	81.9%	30.0%	35.7%	44.9%	59.1%	39.4%	-1.05 / 11.12 / 2.06	58.7%	91.3% / 61.3%
80	82.2%	28.7%	34.4%	41.8%	58.6%	40.5%	-0.84 / 11.61 / 2.38	59.8%	91.1% / 61.6%

Appendix

Table 37: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (1 epoch with LR 1e-5).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	78.1%	15.0%	37.3%	38.7%	60.0%	57.7%	0.65 / 16.95 / 5.70	54.9%	84.5% / 95.3%
40	82.1%	33.3%	38.3%	44.1%	59.4%	38.6%	-1.11 / 11.51 / 2.06	60.4%	88.7% / 57.3%
80	87.3%	36.3%	38.4%	39.9%	66.2%	41.2%	-0.69 / 12.55 / 2.68	67.3%	88.1% / 61.9%

Appendix

Table 38: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (2 epochs with LR 1e-5).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	77.8%	14.7%	37.5%	41.8%	61.8%	64.2%	0.53 / 13.5 / 4.83	54.7%	80.1% / 94.4%
10	79.5%	11.3%	37.2%	41.1%	61.0%	55.4%	0.70 / 17.53 / 5.82	54.2%	84.8% / 93.8%
20	69.4%	26.0%	16.7%	30.5%	41.7%	20.3%	-1.90 / 8.16 / 1.42	53.0%	87.0% / 50.9%
40	88.3%	38.0%	38.2%	43.4%	68.4%	44.1%	-0.21 / 17.54 / 4.85	74.1%	96.7% / 78.6%

Appendix

Table 39: Training dynamics for Qwen3-14B-Base with Math-CoT-20k (16 epochs with constant LR 1e-4).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	3.9%	0.7%	0.4%	1.3%	1.3%	30.5%	-6.71 / -16.28 / -3.94	31.0%	78.5% / 24.2%
10	19.1%	0.7%	0.6%	9.4%	2.3%	29.5%	-5.56 / -13.56 / -2.69	38.6%	79.3% / 36.4%
20	33.6%	1.7%	1.0%	11.4%	8.8%	34.8%	-3.40 / -5.00 / 0.02	43.6%	85.4% / 63.2%
40	25.1%	0.3%	1.3%	10.1%	15.4%	27.3%	-3.38 / -2.46 / 0.30	44.5%	96.1% / 51.2%

Appendix

Table 40: Training dynamics for Qwen2.5-1.5B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	58.7%	4.3%	9.4%	21.0%	15.0%	37.0%	-1.12 / 6.08 / 3.35	45.7%	68.5% / 88.7%
10	59.2%	3.7%	9.5%	21.5%	16.4%	34.9%	-1.25 / 5.68 / 3.12	44.5%	65.9% / 85.1%
20	30.8%	1.3%	6.0%	4.7%	13.0%	27.1%	-5.07 / -9.10 / -2.44	43.7%	89.1% / 52.9%
40	38.9%	3.0%	2.2%	9.6%	23.5%	26.4%	-1.70 / 4.58 / 3.00	43.1%	97.1% / 52.0%

Appendix

Table 41: Training dynamics for Qwen2.5-3B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	66.7%	7.7%	15.7%	26.9%	34.8%	41.1%	-1.00 / 6.61 / 3.26	45.4%	80.7% / 86.5%
10	66.1%	8.7%	9.5%	27.9%	34.4%	41.5%	-1.19 / 6.13 / 3.03	41.0%	80.0% / 83.5%
20	41.3%	7.3%	4.2%	9.6%	29.5%	31.4%	-1.95 / 4.93 / 2.20	63.5%	96.1% / 50.3%
40	60.1%	7.7%	9.8%	13.6%	40.9%	29.3%	-1.44 / 10.03 / 3.24	65.1%	95.2% / 54.2%

Appendix

Table 42: Training dynamics for Qwen2.5-7B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	65.7%	8.0%	17.6%	28.6%	40.9%	49.0%	-1.4 / 1.29 / 2.16	56.1%	85.4% / 93.8%
10	61.9%	7.7%	16.8%	30.3%	40.7%	42.8%	-1.63 / 1.34 / 1.84	52.4%	83.2% / 88.1%
20	52.9%	8.7%	7.6%	12.0%	39.2%	29.4%	-1.35 / 9.34 / 2.97	66.0%	97.6% / 54.5%
40	73.6%	16.7%	20.1%	22.7%	57.1%	35.4%	-0.69 / 13.74 / 4.52	73.3%	95.0% / 76.1%

Appendix

Table 43: Training dynamics for Qwen2.5-14B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	65.7%	8.0%	17.6%	28.6%	40.9%	49.0%	-1.4 / 1.29 / 2.16	56.1%	85.4% / 93.8%
10	67.1%	8.3%	16.0%	32.3%	41.1%	46.8%	-1.34 / 3.42 / 2.32	55.3%	83.7% / 88.0%
40	79.8%	16.7%	31.7%	37.0%	57.8%	42.7%	1.14 / 19.16 / 6.72	74.4%	92.3% / 98.4%
80	78.3%	16.7%	31.9%	35.7%	59.3%	41.7%	1.23 / 18.94 / 6.91	72.4%	93.5% / 99.1%

Appendix

Table 44: Training dynamics for Qwen2.5-14B-Base with Countdown-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy/ RM score
Base	53.7%	1.7%	17.6%	20.9%	34.3%	51.1%	-0.84 / 8.17 / 3.42	60.6%	70.3% / 72.6%
10	50.8%	2.3%	14.2%	23.1%	32.5%	53.6%	-1.16 / 6.67 / 3.00	58.8%	71.4% / 72.8%
20	37.9%	0.3%	4.1%	11.8%	33.8%	34.2%	-0.92 / 9.78 / 3.59	70.2%	96.2% / 68.9%
40	35.7%	1.0%	8.7%	8.1%	36.6%	33.9%	-1.27 / 10.03 / 3.53	66.8%	96.0% / 59.4%

Appendix

Table 45: Training dynamics for InternLM2.5-20B-Base with Math-CoT-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy / RM Score
Base	53.7%	1.7%	17.6%	20.9%	34.3%	51.1%	-0.84 / 8.17 / 3.42	60.6%	70.3% / 72.6%
10	55.5%	3.0%	17.5%	23.2%	34.4%	44.4%	-0.61 / 9.47 / 3.87	60.3%	71.1% / 76.9%
20	54.9%	3.7%	17.1%	21.9%	42.6%	44.7%	0.31 / 14.67 / 5.45	70.6%	82.6% / 94.4%
40	54.8%	3.0%	19.0%	25.4%	44.7%	43.9%	0.63 / 15.49 / 5.85	73.4%	87.8% / 98.4%

Appendix

Table 46: Training dynamics for InternLM2.5-20B-Base with Countdown-CoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	53.7%	1.7%	17.6%	20.9%	34.3%	51.1%	-0.84 / 8.17 / 3.42	60.6%	70.3% / 72.6%
10	52.8%	2.3%	16.4%	24.9%	37.4%	45.8%	-0.37 / 10.53 / 4.15	60.0%	69.2% / 72.1%
40	53.6%	4.0%	20.0%	33.2%	47.8%	46.8%	1.23 / 18.02 / 7.24	57.3%	87.0% / 99.9%
80	50.7%	3.7%	15.1%	32.3%	45.9%	45.0%	1.05 / 16.31 / 6.97	64.0%	89.1% / 100.0%

Appendix

Table 47: Training dynamics for InternLM2.5-20B-Base with Math-NoCoT-20k (default training schedule).

Accuracy / RM Score
	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Base	53.7%	1.7%	17.6%	20.9%	34.3%	51.1%	-0.84 / 8.17 / 3.42	60.6%	70.3% / 72.6%
10	32.9%	0.3%	11.5%	31.3%	11.9%	45.3%	-1.83 / 3.33 / 2.13	53.3%	67.7% / 69.4%
20	33.8%	0.7%	8.2%	30.6%	33.6%	38.7%	-2.37 / -0.43 / 1.62	51.3%	53.4% / 96.3%
40	41.5%	2.7%	6.6%	29.8%	35.7%	33.5%	-2.28 / 0.52 / 1.90	51.8%	53.7% / 96.7%

Appendix

Table 48: Training dynamics for InternLM2.5-20B-Base with Numina-Math-20k (default training schedule).

	ID reasoning		OOD reasoning			General capabilities
Step	MATH500	AIME24	LCB v2	GPQA-D	MMLU-Pro	IFEval	AlpacaEval (RB2 / Sky / Mis)	HaluEval	TruthfulQA (truth / help)
Accuracy/ RM score
Base	53.7%	1.7%	17.6%	20.9%	34.3%	51.1%	-0.84 / 8.17 / 3.42	60.6%	70.3% / 72.6%
10	52.6%	2.3%	10.2%	22.4%	31.6%	44.6%	-1.52 / 5.39 / 2.53	60.5%	71.5% / 74.9%
20	41.9%	1.3%	3.8%	16.3%	38.4%	37.0%	-1.46 / 7.98 / 2.82	68.9%	86.4% / 78.3%
40	55.0%	4.3%	13.8%	22.1%	46.4%	38.8%	-0.73 / 11.43 / 4.14	71.4%	83.8% / 84.0%

Appendix

Table 49: Training dynamics for InternLM2.5-20B-Base with DeepSeek-R1-generated data (default training schedule).

Explore similar work

On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning
Apr 2, 2026Zhaoyi Li, Xiangyu Xi, Zhengyu Chen +6Supervised Fine-Tuning Efficient Language Model Reasoning
University of Science and Technology of China · Meituan LongCat Team · City University of Hong Kong +1
Learning to Adapt SFT Data for Better Reasoning Generalization
May 26, 2026Lisong Sun, Li Wang, Chen Zhang +4Supervised Fine-Tuning LLM Post-Training
Beihang University · Tsinghua University · Nanyang Technological University +2
Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning
May 10, 2026Ruiying Peng, Mengyu Yang, Jing Lei +3Supervised Fine-Tuning Memorization in Language Models
Tsinghua Shenzhen International Graduate School · Huawei Technologies

Apr 2, 2026J/K move · Enter open · S save

On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning

Zhaoyi Li, Xiangyu Xi, Zhengyu Chen, Wei Wang+5

University of Science and Technology of China · Meituan LongCat Team · City University of Hong Kong+1

Supervised Fine-Tuning Efficient Language Model Reasoning Overthinking in Language Models

Open paper AskPDF arXiv

Supervised Fine-Tuning (SFT) on long Chain-of-Thought (CoT) trajectories has become a pivotal phase in building large reasoning models. However, how CoT trajectories from different sources influence the generalization performance of models remains an open question. In this paper, we conduct a comparative study using two sources of verified CoT trajectories generated by two competing models, DeepSeek-R1-0528 and gpt-oss-120b, with their problem sets controlled to be identical. Despite their comparable performance, we uncover a striking paradox: lower training loss does not translate to better generalization. SFT on DeepSeek-R1-0528 data achieves remarkably lower training loss, yet exhibits significantly worse generalization performance on reasoning benchmarks compared to those trained on gpt-oss-120b. To understand this paradox, we perform a multi-faceted analysis probing token-level SFT loss and step-level reasoning behaviors. Our analysis reveals a difference in reasoning patterns. gpt-oss-120b exhibits highly convergent and deductive trajectories, whereas DeepSeek-R1-0528 favors a divergent and branch-heavy exploration pattern. Consequently, models trained with DeepSeek-R1 data inherit inefficient exploration behaviors, often getting trapped in redundant exploratory branches that hinder them from reaching correct solutions. Building upon this insight, we propose a simple yet effective remedy of filtering out frequently branching trajectories to improve the generalization of SFT. Experiments show that training on selected DeepSeek-R1-0528 subsets surprisingly improves reasoning performance by up to 5.1% on AIME25, 5.5% on BeyondAIME, and on average 3.6% on five benchmarks.

Abstract

Explore similar work

On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning

Learning to Adapt SFT Data for Better Reasoning Generalization

Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning