Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit

Jimson Huang; Maxwell Nickell; Ruowang Zhang; Abulhair Saparov

cs.LGOct 19, 2026

Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit

Authors: Jimson Huang, Maxwell Nickell, Ruowang Zhang, Abulhair Saparov

Organizations: Purdue University West Lafayette, IN 47907, USA

Abstract

LLMs "reason" by externalizing intermediate steps as tokens through chain-of-thought (CoT), yet even frontier reasoning models hit abrupt performance cliffs on long-horizon tasks. Implicit reasoning (reasoning within a single forward pass) is useful during CoT generation, as the model can "search ahead" to decide the correct next move without explicit search. Models with weak implicit reasoning are more likely to commit to wrong paths and must attempt to recover via backtracking, while models with strong implicit reasoning identify the correct proof strategy directly, shortening the CoT. However, prior work shows LLMs struggle to reason implicitly beyond shallow depths, with no relief from scale. We investigate whether curriculum learning can extend this depth by fine-tuning on examples with progressively increasing vocabulary size and lookahead (the number of steps a model must search ahead to identify the correct next move). At matched compute, curriculum-trained models achieve lookaheads where no-curriculum training fails. Qwen3-0.6B reaches 98% accuracy on problems with lookahead up to 256 with curriculum, compared to 45% without. However, progress slows as fine-tuning continues, following a saturating exponential that suggests a maximum intrinsic lookahead to which models can learn to reason implicitly. Larger models reach higher fitted ceilings, fine-tuning pretrained models far outpaces training from scratch, and curriculum step sizes that are too large stall progression. Finally, the capability acquired on synthetic data transfers to natural language reasoning, with curriculum-trained models outperforming models trained without curriculum at matched compute on 11 natural language reasoning benchmarks.

Figures & tables

Figure 1: Curriculum versus non-curriculum fine-tuning of the same pretrained LLM. The curriculum orders training problems by increasing difficulty, whereas the baseline draws from the full range of difficulties from the start. On the example proof, the curriculum-trained model takes the correct next step each time, while the baseline commits to a wrong branch and must backtrack. Section 4.1 reports the measured comparison.

Figure 2: Eval accuracy of Qwen3-0.6B at maximum

L\in\{96,128,256\}

vs. cumulative training compute (PFLOPs). The dashed line marks the

98\%

threshold; dotted lines mark where each setting crosses it (Table 1 ).

$L$	Curriculum	No curriculum	Speedup
96	66.5K	82.8K	$1.25\times$
104	66.5K	106K	$1.59\times$
112	74.3K	140K	$1.89\times$
120	74.3K	106K	$1.42\times$
128	81.8K	186K	$2.27\times$
256	259K	–	–

Table 1: Cumulative training compute (PFLOPs) to reach

98\%

accuracy at each target lookahead

L

. A dash marks a setting that never reaches

98\%

within budget.

Figure 3: Achieved lookahead

L

vs. cumulative training compute (PFLOPs) for Qwen3-0.6B at

s{=}8

, with the attribute vocabulary grown as

|\mathcal{V}_{k}|=bL_{k}+2

or held fixed at its maximum size. The bold entry (

b{=}2

, growing

|\mathcal{V}_{k}|

) is the configuration used in all other experiments.

Figure 4: Achieved lookahead

L

vs. cumulative training compute (PFLOPs, log–log scale) for Qwen3-0.6B ( left ) and Pythia-1.4B ( right ) trained under the curriculum with

s{=}1

, with the power-law and saturating exponential fits overlaid on the observed trajectory.

Trajectory	Form	Fitted equation	RMSE
Qwen3-0.6B	Power law	$L\approx 1.03\,C^{0.390}$	6.10
Qwen3-0.6B	Saturating exponential	$L\approx 324\bigl(1-\exp\bigl(-(C/7.7{\times}10^{5})^{0.547}\bigr)\bigr)$	2.22
Pythia-1.4B	Power law	$L\approx 2.45\,C^{0.228}$	2.98
Pythia-1.4B	Saturating exponential	$L\approx 82\bigl(1-\exp\bigl(-(C/5.0{\times}10^{5})^{0.371}\bigr)\bigr)$	0.53

Table 2: Fitted forms of achieved lookahead

L

against cumulative training compute

C

(PFLOPs), with in-sample RMSE.

	Held-out RMSE
Trajectory	Power law	Sat. exp.	$p$ -value
Qwen3-0.6B	23.88	17.08	$7{\times}10^{-10}$
Pythia-1.4B	16.14	0.68	$1.5{\times}10^{-4}$

Table 3: Held-out RMSE for achieved lookahead

L

p

-value: significance of the saturating exponential’s advantage under the per-stage iteration test, two-sided. Additional fits on less-trained trajectories in Table 17 .

Figure 5: Achieved lookahead

L

vs. cumulative training compute for Qwen3-0.6B with step sizes

s\in\{1,4,8,16,32,64\}

Figure 6: Achieved lookahead

L

vs. cumulative training compute for the four Pythia sizes at

s{=}1

. At 1M PFLOPs, the curves reach

L{=}9

44

58

, and

94

, from smallest to largest.

Figure 7: Achieved lookahead

L

vs. cumulative training compute (PFLOPs), starting from the pretrained checkpoint vs. random initialization. Left: Pythia-1.4B , Right: Qwen3-0.6B .

	Curriculum $L$				No curr.	$\Delta$
Benchmark	8	16	32	48	No curr.	$\Delta$
ProofWriter OWA	41.1	33.6	39.3	54.9 ∗	40.6	$+14.3$
ProofWriter CWA	66.8	57.2	67.6	67.9 ∗	55.2	$+12.8$
ParaRules	46.1	36.2	39.1	55.1 ∗	50.4	$+4.7$
ZebraLogic-MC	47.4	53.4	51.8	53.5 ∗	48.3	$+5.2$
PrOntoQA †	52.5	55.3	56.8	57.1 ∗	52.0	$+5.1$

Table 4: Downstream reasoning accuracy (%) for Qwen3-1.7B at curriculum stages

L

, vs. trained directly at

L{=}48

without curriculum at matched compute.

\Delta

: accuracy of curriculum at

L{=}48

minus that of no curriculum. Best per row in bold. ∗ : curriculum wins are statistically significant (two-sided exact McNemar test on paired items,

p<0.05

); confidence intervals in Appendix F . † Extended by us to test for greater depths (Appendix G ).

Appendix figures & tables27 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Training loop of our curriculum over lookahead and attribute vocabulary, as described in Section 3.2 .

Setting	Curriculum	No-curriculum
Model	Qwen3-0.6B (full FT)	Qwen3-0.6B (full FT)
Batch size	768	768
Learning rate	$5\times 10^{-5}$	$5\times 10^{-5}$
Curriculum step $s$	8	N/A
$L$	$8\rightarrow 256$	$\{96,104,112,120,128,256\}$

Appendix

Table 5: Hyperparameters for Section 4.1 ( Qwen3-0.6B ).

Model	Batch size	LR
Pythia-160M	192	$1\times 10^{-4}$
Pythia-410M	192	$5\times 10^{-5}$
Pythia-1.4B	192	$2.7\times 10^{-5}$
Pythia-2.8B	1,536	$2.7\times 10^{-5}$

Appendix

Table 6: Per-model hyperparameters for Section 4.4 ( Pythia model-size sweep). All sizes:

s{=}1

, AdamW, constant LR.

Condition	Batch size	LR	Init
Pythia-1.4B , pretrained	192	$2.7\times 10^{-5}$	Pythia-1.4B base
Pythia-1.4B , random init	768	$1\times 10^{-4}$	default initializer
Qwen3-0.6B , pretrained	768	$5\times 10^{-5}$	Qwen3-0.6B post-trained release
Qwen3-0.6B , random init	1,536	$3\times 10^{-4}$	default initializer

Appendix

Table 7: Hyperparameters for Section 4.5 (

s{=}1

; curriculum to

L{=}96

for Pythia-1.4B and to

L{=}256

for Qwen3-0.6B ). The Qwen3-0.6B pretrained row reuses the training from Section 4.2 .

Configuration	Mix
Qwen3-1.7B , curriculum	6%
Qwen3-1.7B , no curriculum (direct at $L{=}48$ )	6%
Qwen3-1.7B , instruction only	100%

Appendix

Table 8: Hyperparameters for Section 4.6 . All: AdamW, constant LR

5\times 10^{-5}

, batch 768, chat template with enable_thinking=False . The curriculum uses

s{=}1

up to

L{=}48

Figure 9: Same layout as Figure 2 , for the full sweep

L\in\{96,104,112,120,128,256\}

Stage $L$	Joint curriculum	Lookahead only
8	144	401
16	983	2.54K
24	3.14K	4.30K
32	6.19K	7.66K
48	13.6K	20.6K
64	23.5K	43.7K

Appendix

Table 9: Cumulative training compute (PFLOPs) at which each configuration completes stage

L

for Qwen3-0.6B with

s{=}8

, and the stage completed within a shared budget of 100K PFLOPs. The joint curriculum grows the attribute vocabulary as

|\mathcal{V}_{k}|=2(L_{k}+1)

, and the lookahead-only curriculum holds it fixed at its maximum size. A dash (–) means the model never reached that lookahead during training. The fixed-vocabulary configuration ends with

L{=}96

completed after 107K PFLOPs. PFLOPs are computed as

6N

per token with actual parameter counts.

Model	Achieved lookahead $L$	Completed at (PFLOPs)
Pythia-160M	9	921K
Pythia-410M	46	1.72M
Pythia-1.4B	75	4.97M
Pythia-2.8B	133	2.55M

Appendix

Table 10: Achieved lookahead

L

at the end of training for each Pythia size under the curriculum with

s{=}1

, and the cumulative training compute (PFLOPs, computed as

6N

per token) at which that stage is completed.

	$L$ completed at cumulative PFLOPs						End of training
Model	100K	200K	300K	500K	750K	1M	$L$	PFLOPs	Tokens
Pythia-160M	7	7	8	8	8	9	9	995,662	1,022B
Pythia-410M	30	34	37	40	42	44	46	2,017,056	828B
Pythia-1.4B	34	41	46	52	56	58	75	5,000,829	589B
Pythia-2.8B	31	46	60	73	84	94	133	2,561,267	154B

Appendix

Table 11: Curriculum stage completed by each Pythia size (

s{=}1

) at fixed cumulative training compute and at the end of training. Pythia-410M leads Pythia-1.4B only below

{\sim}30

K PFLOPs, and Pythia-2.8B overtakes Pythia-1.4B between 100K and 200K PFLOPs. Pythia-2.8B completes

L{=}130

at 2.40M PFLOPs and

L{=}133

at 2.55M PFLOPs; Pythia-1.4B completes

L{=}75

at 4.97M PFLOPs.

Model	8	16	24	32	48	64	96	128
Pythia-160M	253K	–	–	–	–	–	–	–
Pythia-410M	748	7.64K	34.3K	134K	–	–	–	–
Pythia-1.4B	1.07K	7.75K	28.2K	73.6K	385K	1.65M	–	–
Pythia-2.8B	2.62K	15.2K	33.6K	102K	212K	347K	1.04M	2.30M

Appendix

Table 12: Cumulative training compute (PFLOPs) at which each Pythia model completes stage

L

under the curriculum with

s{=}1

. A dash (–) means the model never reached that lookahead during training. Training ends with

L{=}9

(160M, 996K PFLOPs),

46

(410M, 2.02M),

75

(1.4B, 5.00M) and

133

(2.8B, 2.56M) completed. PFLOPs are computed as

6N

per token with actual parameter counts.

Model	8	16	24	32	48	64	96	128
Pythia-160M	260	–	–	–	–	–	–	–
Pythia-410M	0.31	3.1	14.1	55.1	–	–	–	–
Pythia-1.4B	0.13	0.91	3.3	8.7	45.4	194.8	–	–
Pythia-2.8B	0.16	0.91	2.0	6.2	12.7	20.9	62.5	138.4

Appendix

Table 13: Training tokens (billions) at which each Pythia model completes stage

L

under the curriculum with

s{=}1

; the same stage completions as Table 12 , in tokens rather than PFLOPs. A dash (–) means the model never reached that lookahead during training.

	Pythia-1.4B		Qwen3-0.6B
Stage $L$	pretrained	random	pretrained	random
8	1.07K	79.7K	618	74.8K
16	7.75K	636K	2.35K	285K
24	28.2K	1.64M	5.78K	618K
32	73.6K	4.41M	10.8K	1.36M
48	385K	–	25.4K	2.22M

Appendix

Table 14: Cumulative training compute (PFLOPs) at which each condition completes stage

L

under the curriculum with

s{=}1

, and the stage completed at 300K and 1.5M PFLOPs. A dash (–) means the model never reached that lookahead during training. Training from random initialization ends with

L{=}35

completed ( Pythia-1.4B , 6.02M PFLOPs) and

L{=}61

completed ( Qwen3-0.6B , 4.82M PFLOPs). PFLOPs are computed as

6N

per token with actual parameter counts.

Figure 10: Achieved lookahead

L

vs. cumulative training compute (PFLOPs) for Pythia-1.4B fine-tuned from intermediate pretraining checkpoints, each stopped at 500K PFLOPs. The dashed curve is the random-initialization trajectory from Section 4.5 .

Stage $L$	Step 1,000	Step 10,000	Step 100,000
	( ${\sim}$ 2.1B tokens)	( ${\sim}$ 21B)	( ${\sim}$ 210B)
8	9.95K	3.79K	1.04K
16	83.0K	43.1K	7.25K
24	259K	203K	27.7K
32	–	–	66.7K
48	–	–	235K

Appendix

Table 15: Cumulative training compute (PFLOPs) at which Pythia-1.4B , fine-tuned from intermediate pretraining checkpoints, completes stage

L

under the curriculum with

s{=}1

, and the stage completed at three budgets. Column headers give the pretraining step and the approximate number of pretraining tokens. A dash (–) means the model never reached that lookahead during training. Each trajectory ends at 500K PFLOPs.

	Compute (PFLOPs)			Optimizer steps
Stage $L$	Full FT	$r{=}8$	$r{=}64$	Full FT	$r{=}8$	$r{=}64$
32	4.25K	7.97K	4.49K	3,250	8,575	4,375
64	25.6K	34.3K	22.1K	10,700	22,175	12,600
96	66.5K	60.4K	45.2K	19,550	31,000	19,475
128	116K	110K	98.1K	27,575	43,125	31,200

Appendix

Table 16: Cumulative training compute and optimizer steps at which Qwen3-0.6B completes stage

L

under the curriculum with

s{=}8

, with full fine-tuning (Full FT) and with LoRA at rank

r

. LoRA compute is counted as

4N+6N_{\text{LoRA}}

per token.

	Held-out RMSE
Trajectory	Power law	Sat. exp.	Ratio	$p$
Qwen3-0.6B	23.88	17.08	1.40	$7{\times}10^{-10}$
Pythia-410M	8.23	0.84	9.80	$0.018$
Pythia-1.4B	16.14	0.68	23.77	$1.5{\times}10^{-4}$
Pythia-2.8B	9.25	3.72	2.49	$0.65$

Appendix

Table 17: Held-out prediction error for achieved lookahead

L

on all four

s{=}1

trajectories, fitting on the first

70\%

of stage completions and predicting the remainder. Ratio is power-law over saturating-exponential RMSE.

p

: significance of the saturating exponential’s advantage under the per-stage iteration test of Appendix D.1 , two-sided.

Trajectory	$n$	$L_{\text{end}}$	Power-law RMSE [95% CI]	Saturating exponential RMSE [95% CI]	$\Delta$ AICc
Qwen3-0.6B	244	244	$6.10$ $[3.27,6.75]$	$2.22$ $[1.57,2.35]$	493
Pythia-410M	46	46	$1.93$ $[0.71,2.05]$	$0.47$ $[0.32,0.53]$	130
Pythia-1.4B	75	75	$2.98$ $[1.69,3.24]$	$0.53$ $[0.40,0.57]$	259
Pythia-2.8B	133	133	$2.67$ $[1.92,3.16]$	$2.15$ $[1.46,2.54]$	57

Appendix

Table 18: Fit comparison for the four long

s{=}1

trajectories.

\Delta

AICc is (power-law

-

saturating exponential); positive favors the saturating exponential, and since both forms have three parameters it reflects fit quality alone. RMSE confidence intervals are 95% moving-block bootstrap. Both in-sample criteria favor the saturating exponential on every trajectory. Held-out prediction is compared in Table 17 and Appendix D.1 .

Trajectory	$L_{\max}$ [95% CI]	$C_{0}$ (PFLOPs)	$k$
Qwen3-0.6B	$323.6$ $[304,404]$	$7.7{\times}10^{5}$	0.547
Pythia-410M	$51.2$ $[49.2,56.9]$	$1.4{\times}10^{5}$	0.341
Pythia-1.4B	$81.9$ $[77.8,83.0]$	$5.0{\times}10^{5}$	0.371
Pythia-2.8B	$235.1$ $[175,315]$	$3.8{\times}10^{6}$	0.497

Appendix

Table 19: Fitted saturating exponential parameters,

L=L_{\max}\bigl(1-\exp(-(C/C_{0})^{k})\bigr)

, for the four long

s{=}1

trajectories.

L_{\max}

is the fitted ceiling plotted in Figure 11 . Brackets give 95% moving-block bootstrap percentile intervals for

L_{\max}

(block length

{\approx}\,n^{1/3}

, 300 resamples). Their lower ends are stable under longer blocks, while the upper ends for Qwen3-0.6B and Pythia-2.8B widen substantially, so those upper bounds are indicative only.

Figure 11: Saturating exponential fits (black dotted, over the observed range) to the achieved-lookahead trajectories of Pythia-410M , -1.4B , and -2.8B trained with

s{=}1

(log–log). Legends give each fitted ceiling

L_{\max}

(Table 19 ).

	Cumulative lookahead $L(C)$		Per-stage iterations $\Delta I(L)$
Trajectory	lag-1 autocorr.	Ljung–Box $p$	lag-1 autocorr.	Ljung–Box $p$
Qwen3-0.6B	$0.95$	$2{\times}10^{-199}$	$-0.14$	$0.15$
Pythia-410M	$0.25$	$0.36$	$-0.28$	$0.36$
Pythia-1.4B	$0.33$	$0.08$	$-0.33$	$0.046$
Pythia-2.8B	$0.92$	$4{\times}10^{-94}$	$-0.10$	$0.13$

Appendix

Table 20: Residual dependence of the saturating exponential fit on cumulative lookahead and on per-stage training iterations (log units). Ljung–Box tests use five lags. Per-stage iterations remove the serial dependence that invalidates a Diebold–Mariano test on cumulative lookahead.

		Held-out RMSE ( $\log\Delta I$ )			Mean advantage $\alpha$		Trend $\beta$ (per stage)
Trajectory	$T$	Sat. exp.	Power law	Ratio	Estimate [95% CI]	$p$	Estimate [95% CI]	$p$
Qwen3-0.6B	73	$0.93$	$0.96$	$1.04$	$0.068$ $[0.049,0.087]$	$7{\times}10^{-10}$	$0.0025$ $[0.0016,0.0034]$	$1{\times}10^{-6}$
Pythia-410M	14	$1.14$	$1.90$	$1.67$	$2.33$ $[0.47,4.18]$	$0.018$	$0.51$ $[0.08,0.95]$	$0.025$
Pythia-1.4B	23	$1.06$	$1.76$	$1.66$	$1.97$ $[1.08,2.86]$	$1.5{\times}10^{-4}$	$0.18$ $[0.05,0.32]$	$0.010$
Pythia-2.8B	40	$1.13$	$1.13$	$1.00$	$6{\times}10^{-6}$ $[-2{\times}10^{-5},3{\times}10^{-5}]$	$0.65$	$-9{\times}10^{-7}$ $[-3{\times}10^{-6},1{\times}10^{-6}]$	$0.37$

Appendix

Table 21: Held-out prediction of per-stage training iterations.

T

is the number of held-out stages.

\alpha

and

\beta

are the intercept and slope of

d_{L}

regressed on the centered stage index, with Newey–West standard errors;

\alpha>0

favors the saturating exponential and

\beta>0

indicates an advantage that grows with depth.

Figure 12: Mean training compute per completed curriculum stage for Pythia-1.4B with

s{=}1

. Each bar averages five consecutive completed stages; labels show the mean in PFLOPs. Compute for a stage is measured between consecutive stage completions, with the first stage measured from the start of training. The unfinished stage

L{=}76

is excluded.

Benchmark	Canonical LLM protocol	Our protocol
Reasoning benchmarks
ProofWriter OWA	none	0-shot Direct
ProofWriter CWA	none	0-shot Direct
ParaRules	none	0-shot Direct
ZebraLogic-MC	none	3-shot Direct
PrOntoQA	8-shot CoT ( Saparov et al., 2023 )	8-shot Direct

Appendix

Table 22: Per-benchmark prompting protocol. Where a benchmark has a canonical zero-shot protocol, we keep it; where it has a canonical few-shot protocol, we use its shot count; where the canonical protocol uses chain-of-thought or no LLM protocol exists, we use Direct prompting. All demonstrations show the question and label only and are removed from the scored items. The seven general-capability benchmarks without a canonical protocol use the lm-evaluation-harness default of zero-shot.

	Base	Curriculum $L$				No curr.	Instr. only
Benchmark	Base	8	16	32	48	No curr.	Instr. only
ProofWriter OWA	40.3 $\pm$ 1.8	41.1 $\pm$ 1.8	33.6 $\pm$ 1.7	39.3 $\pm$ 1.7	54.9 $\pm$ 1.8 ∗	40.6 $\pm$ 1.8	47.8 $\pm$ 1.8
ProofWriter CWA	52.7 $\pm$ 1.8	66.8 $\pm$ 1.7 ∗	57.2 $\pm$ 1.8 ∗	67.6 $\pm$ 1.7 ∗	67.9 $\pm$ 1.7 ∗	55.2 $\pm$ 1.8	58.0 $\pm$ 1.8
ParaRules	43.8 $\pm$ 1.4	46.1 $\pm$ 1.4	36.2 $\pm$ 1.4	39.1 $\pm$ 1.4	55.1 $\pm$ 1.4 ∗	50.4 $\pm$ 1.5	57.1 $\pm$ 1.4
ZebraLogic-MC	25.5 $\pm$ 2.7	47.4 $\pm$ 3.1	53.4 $\pm$ 3.1 ∗	51.8 $\pm$ 3.1 ∗	53.5 $\pm$ 3.1 ∗	48.3 $\pm$ 3.1	53.2 $\pm$ 3.1
PrOntoQA †	53.1 $\pm$ 1.5	52.5 $\pm$ 1.5	55.3 $\pm$ 1.5 ∗	56.8 $\pm$ 1.5 ∗	57.1 $\pm$ 1.5 ∗	52.0 $\pm$ 1.5	52.2 $\pm$ 1.5

Appendix

Table 23: Downstream reasoning accuracy (%) of Qwen3-1.7B from Table 4 , with 95% confidence intervals, given by a simple normal approximation. Base is the non-finetuned Qwen3-1.7B under the same prompts, with the answer cue prefilled in its own markdown answer format. Instruction-only is trained on the instruction data alone at compute matched to curriculum

L{=}48

. ∗ : curriculum checkpoint significantly above no curriculum (two-sided exact McNemar test on paired items,

p<0.05

). † Extended by us to greater depths (Appendix G ).

Model	MMLU	ARC-E	ARC-C	HellaSwag	WinoG.	PIQA	BoolQ	LAMBADA	Mean	IFEval
Base Qwen3-1.7B	55.5	72.6	42.7	60.5	61.7	72.5	77.6	50.3	61.7	33.1
Curriculum, $L{=}8$	56.9	76.2	48.0	62.4	63.2	72.8	76.4	57.4	64.2	35.1
Curriculum, $L{=}16$	57.1	75.7	45.9	62.5	62.4	73.3	77.1	58.0	64.0	39.0
Curriculum, $L{=}32$	57.3	74.3	45.8	62.5	61.6	73.0	77.8	59.2	63.9	34.6
Curriculum, $L{=}48$	56.1	74.4	45.6	64.0	59.8	73.3	77.2	56.5	63.4	41.2
No curriculum	57.5	75.5	45.3	62.7	62.9	73.7	73.2	55.1	63.2	40.1

Appendix

Table 24: General model capability of Qwen3-1.7B across curriculum stages. Accuracy on eight standard benchmarks (zero-shot, lm-evaluation-harness v0.4.11, identical settings for every model) for the non-finetuned model, four curriculum stage checkpoints, the model trained without curriculum, and an instruction-only control, the last two compute-matched to the

L{=}48

checkpoint. “Mean” is the unweighted mean of the eight language understanding benchmarks. IFEval (prompt-level strict accuracy) is reported separately and is excluded from the mean.

Benchmark	Reasoning type	Answer format	Split / subset	$n$
Reasoning benchmarks
ProofWriter OWA † ( Tafjord et al., 2021 )	Synthetic deductive	True/False/ Unknown	OWA, depths 0–5, label-balanced	3000
ProofWriter CWA ( Tafjord et al., 2021 )	Synthetic deductive	True/False	CWA, depths 0–5	3000
ParaRules † ( Clark et al., 2020 )	Synthetic deductive, paraphrased	True/False/ Unknown	ProofWriter OWA NatLang test, depths 0–5, label-balanced	4566
ZebraLogic-MC ( Lin et al., 2025a )	Constraint satisfaction	Multiple choice	mc_mode	1000
PrOntoQA † ( Saparov et al., 2023 )	Synthetic deductive	True/False	generated, 8 depths up to 64	4000

Appendix

Table 25: The fourteen downstream reasoning benchmarks and the nine general-capability benchmarks, with the number of evaluation items

n

. General-capability benchmarks use the default lm-evaluation-harness splits. ProofWriter appears twice because its open- and closed-world splits differ in label set. NLGraph is scored with the authors’ per-task scorers, and we report the mean over its eight sub-tasks (Appendix E ). † Modified by us, as described in this appendix.

Figure 13: Curriculum over lookahead and attribute vocabulary size. The three stages use maximum lookahead

L{=}1,2,3

and attribute vocabulary size

|\mathcal{V}_{k}|=4,6,8

, respectively. The arrows indicate stage advancement. Each DAG is converted into natural-language premises; for example, the edge vimba

\to

vorga reads “If Alice is vimba, then Alice is vorga.” The abbreviated examples below the graphs show the goal and proof prefix; the model predicts the next attribute at the blank. Green marks goal-reaching paths and red marks distractor paths. Neither the colors nor the graph diagrams are part of the model input.

Long-Horizon LLM Reasoning

See all 21 papers

Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum
Mar 18, 2026Nived Rajaraman, Audrey Huang, Miro Dudik +3Efficient Language Model Training Curriculum RL
Microsoft Research · University of Illinois Urbana Champaign
Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Traces
Sep 22, 2026Minghui Liu, Thomas Magelinski, Dehao Yuan +2Numerical Reasoning in Language Models Curriculum Learning
University of Maryland, College Park · Capital One
Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
Jun 21, 2026Pengxiang Cai, Tianchen Fang, Xiaohan Li +3RL for Language Model Reasoning Curriculum RL
The Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Zhejiang University

Mar 18, 2026J/K move · Enter open · S save

Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum

Nived Rajaraman, Audrey Huang, Miro Dudik, Robert Schapire+2

Microsoft Research · University of Illinois Urbana Champaign

Efficient Language Model Training Curriculum RL RL for Language Model Reasoning RL Fine-Tuning

Open paper AskPDF arXiv

Chain-of-thought reasoning, where language models expend additional computation by producing thinking tokens prior to final responses, has driven significant advances in model capabilities. However, training these reasoning models is extremely costly in terms of both data and compute, as it involves collecting long traces of reasoning behavior from humans or synthetic generators and further post-training the model via reinforcement learning. Are these costs fundamental, or can they be reduced through better algorithmic design? We show that autocurriculum, where the model uses its own performance to decide which problems to focus training on, provably improves upon standard training recipes for both supervised fine-tuning (SFT) and reinforcement learning (RL). For SFT, we show that autocurriculum requires exponentially fewer reasoning demonstrations than non-adaptive fine-tuning, by focusing teacher supervision on prompts where the current model struggles. For RL fine-tuning, autocurriculum decouples the computational cost from the quality of the reference model, reducing the latter to a burn-in cost that is nearly independent of the target accuracy. These improvements arise purely from adaptive data selection, drawing on classical techniques from boosting and learning from counterexamples, and requiring no assumption on the distribution or difficulty of prompts.

Abstract

Long-Horizon LLM Reasoning

Explore similar work

Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum

Ladders of Thought: A Self-Evolving Curriculum of Progressively Simplified Reasoning Traces

Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model