FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Models

Kehan Jiang; Haonan Dong; Zhaolu Kang; Zhengzhou Zhu; Guojie Song

cs.AIApr 3, 2026

FoE: Forest of Errors Makes the First Solution the Best in Large Reasoning Models

Authors: Kehan Jiang, Haonan Dong, Zhaolu Kang, Zhengzhou Zhu, Guojie Song

Organizations: School of Software and Microelectronics, Peking University · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University

Abstract

Recent Large Reasoning Models (LRMs) like DeepSeek-R1 have demonstrated remarkable success in complex reasoning tasks, exhibiting human-like patterns in exploring multiple alternative solutions. Upon closer inspection, however, we uncover a surprising phenomenon: The First is The Best, where alternative solutions are not merely suboptimal but potentially detrimental. This observation challenges widely accepted test-time scaling laws, leading us to hypothesize that errors within the reasoning path scale concurrently with test time. Through comprehensive empirical analysis, we characterize errors as a forest-structured Forest of Errors (FoE) and conclude that FoE makes the First the Best, which is underpinned by rigorous theoretical analysis. Leveraging these insights, we propose RED, a self-guided efficient reasoning framework comprising two components: I) Refining First, which suppresses FoE growth in the first solution; and II) Discarding Subs, which prunes subsequent FoE via dual-consistency. Extensive experiments across five benchmarks and six backbone models demonstrate that RED outperforms eight competitive baselines, achieving performance gains of up to 19.0% while reducing token consumption by 37.7% ~ 70.4%. Moreover, comparative experiments on FoE metrics shed light on how RED achieves effectiveness.

Figures & tables

Figure 1: ( Upper ) The First is The Best . ( Lower ) Forest of Errors.

Figure 2: ( Left ) Key observations within the FoE . ( Right ) Our proposed RED framework.

Dataset	Model	✓ $\rightarrow$ ✗	✗ $\rightarrow$ ✗	✗ $\rightarrow$ ✓
AIME25	Qwen-distilled-7b	18.3	77.4	4.3
	Qwen-distilled-32b	16.7	76.9	6.4
	Qwen3-8b	15.5	79.1	5.4
	Qwen3-32b	13.0	79.9	7.1
	Llama-distilled-70b	18.8	75.4	5.8
GPQA- Diamond	Qwen-distilled-7b	16.0	81.1	2.9

Table 1: Distribution of the Influence of Subs on First , excluding (✓

\rightarrow

✓ ) (%). Notably, 93.7% of cases favor First , defined as instances where it is either uniquely correct or more robust than Subs (

\blacktriangleright

Appendix E ).

Figure 3: Manual correction on distinct error node types. ( Upper ) Impact of rectifying individual G randchild, C hild, and R oot nodes. ( Lower ) Consequences of delayed root correction in a formed tree, demonstrating substantial mitigation of subsequent node proliferation.

Dataset	Static Metrics			Dynamic Metrics
	FS	N/T	D/T	Repro
AIME25	6.9/ 8.1 $\uparrow$ $\uparrow$ 1.2	7.1/ 8.4 $\uparrow$ $\uparrow$ 1.3	4.9/ 5.7 $\uparrow$ $\uparrow$ 0.8	0.084/ 0.126 $\uparrow$ $\uparrow$ 50.0%
MATH500	3.9/ 5.6 $\uparrow$ $\uparrow$ 1.7	5.6/ 7.7 $\uparrow$ $\uparrow$ 2.1	3.2/ 4.4 $\uparrow$ $\uparrow$ 1.2	0.047/ 0.102 $\uparrow$ $\uparrow$ 117.0%
GSM8K	1.1/ 2.1 $\uparrow$ $\uparrow$ 1.0	2.1/ 3.7 $\uparrow$ $\uparrow$ 1.6	1.5/ 2.0 $\uparrow$ $\uparrow$ 0.5	0.009/ 0.027 $\uparrow$ $\uparrow$ 200.0%
GPQA	4.2/ 6.1 $\uparrow$ $\uparrow$ 1.9	5.1/ 7.6 $\uparrow$ $\uparrow$ 2.5	3.4/ 4.9 $\uparrow$ $\uparrow$ 1.5	0.052/ 0.107 $\uparrow$ $\uparrow$ 105.8%

Table 2: Average statistics of error trees across datasets (First Solution / Average Subsequent Solutions).

Figure 4: Distribution of various node types with respect to entropy and entropy variance. Experiments were conducted using the Qwen3-8B model on BS-17K-subset.

Figure 5: Average distribution of correction types (True, Refuse, Fake) on BS-17k-subset + Qwen3-8B. We manually inject error-signaling prompts to probe early-stage errors (

<20\%

). The green and red vertical lines mark the completion of First and Subs , respectively.

Figure 6: Evaluation of intra-solution reflection metrics.

Model	Method	AIME24		AIME25		MATH500		GSM8K		GPQA-Diamond
		Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$
Qwen3-8B-thinking	Vanilla	70.0	11125	61.1	12490	95.9	4486	94.1	1573	58.8	6638
	DEER	71.1	7903	58.9	9271	95.6	2889	94.3	711	56.9	2926
	DAST	67.8	5964	60.0	6055	96.0	2158	94.1	483	58.4	3007
	Think or Not	68.9	5387	62.2	6674	96.3	2408	94.2	599	59.1	3678
	AlphaOne	73.3	8343	63.3	8607	96.7	4311	94.5	1095	59.8	6002

Table 3: Main results. Best results are highlighted in bold , with runners-up underlined .

Model	Method	AIME25
		Pass@1 $\uparrow$	FS $\downarrow$	N/T $\downarrow$	D/T $\downarrow$	Repro $\downarrow$
Qwen3-32B-thinking	Vanilla	68.9	6.8	7.4	5.3	0.081
	DEER	66.7	6.3	7.2	5.4	0.083
	RL + LP	66.7	5.9	6.9	4.7	0.071
	S-GRRO	71.1	6.1	7.0	5.1	0.074
	RED	72.2	3.1	4.2	3.1	0.026

Table 4: Results on AIME25 with FoE-related metrics. Additional results are provided in Appendix B.1 .

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Method	AIME25
		Pass@1 $\uparrow$	FS $\downarrow$	N/T $\downarrow$	D/T $\downarrow$	Repro $\downarrow$
Qwen3-32B-Thinking	Vanilla	68.9	6.8	7.4	5.3	0.081
	DEER	66.7	6.3	7.2	5.4	0.083
	RL + LP	66.7	5.9	6.9	4.7	0.071
	S-GRRO	71.1	6.1	7.0	5.1	0.074
	RED	72.2	3.1	4.2	3.1	0.026

Appendix

Table 5: Results on AIME25 with FoE-related metrics.

Model	Method	MATH500
		Pass@1	FS $\downarrow$	N/T $\downarrow$	D/T $\downarrow$	Repro $\downarrow$
Qwen3-32B-Thinking	Vanilla	96.8	4.1	4.4	3.2	0.049
	DEER	96.4	3.8	4.3	3.2	0.050
	RL + LP	97.2	3.5	4.1	2.8	0.043
	S-GRRO	97.3	3.7	4.2	3.1	0.044
	RED	97.9	1.9	2.5	1.9	0.016

Appendix

Table 6: Results on MATH500 with FoE-related metrics.

Model	Method	GPQA-Diamond
		Pass@1 $\uparrow$	FS $\downarrow$	N/T $\downarrow$	D/T $\downarrow$	Repro $\downarrow$
Qwen3-32B-Thinking	Vanilla	65.3	5.1	5.6	4.0	0.061
	DEER	64.8	4.7	5.4	4.1	0.062
	RL + LP	66.0	4.4	5.2	3.5	0.053
	S-GRRO	66.5	4.6	5.3	3.8	0.056
	RED	66.8	2.3	3.2	2.3	0.020

Appendix

Table 7: Results on GPQA-Diamond with FoE-related metrics.

Panel of Figure 7 — Figure 7: Test-time scalability under self-consistency.

Model	Method	AIME24		AIME25		MATH500		GSM8K		GPQA-Diamond
		Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$	Pass@1 $\uparrow$	Token $\downarrow$
DeepSeek-R1	w/o All (Vanilla)	76.7	11638	67.8	12625	97.2	4716	95.1	1325	70.9	7544
	w/o Refining First	76.9	6739	67.9	7067	97.2	2769	95.0	519	71.0	4179
	w/o Discarding Subs	77.9	9831	69.3	10469	97.6	3887	94.8	947	71.8	5947
	RED (ours)	78.9	6148	70.0	6404	98.1	2408	95.1	447	72.4	3704
Qwen3-8B Thinking	w/o All (Vanilla)	70.0	11125	61.1	12490	95.9	4486	94.1	1573	58.8	6638

Appendix

Table 8: Ablation study.

Model	Setting	MATH500
		Pass@1 $\uparrow$	Token $\downarrow$
Qwen3-32B- thinking	$T{=}2.0$	97.6	2058
	$T{=}2.1$	97.7	2042
	$T{=}2.2$	97.6	2027
	$T{=}2.3$	97.8	2011
	$T{=}2.4$	97.9	1995

Appendix

Table 9: Sensitivity of the entropy-variance threshold

T

for Refine First on MATH500 (Qwen3-32B-thinking). Best results are highlighted in bold .

Model	Setting	MATH500
		Pass@1 $\uparrow$	Token $\downarrow$
Qwen3-32B- thinking	$P{=}0.4$	87.1	1741
	$P{=}0.5$	87.4	1877
	$P{=}0.6$	97.9	1995
	$P{=}0.7$	97.4	2434

Appendix

Table 10: Sensitivity of the internal consistency rate

P

for Discarding Subs on MATH500 (Qwen3-32B-thinking). Best results are highlighted in bold .

Figure 8: Average sampling error rate (%, lower is better) under rollback sampling. For each solution variant, we truncate an originally correct trajectory at rollback ratio

K\in\{10,20,\ldots,80\}\%

, restore the KV cache at the truncation point, and re-sample the remaining continuation

N=100

times; Curves report error rates averaged over problems. First remains highly robust (error

<1\%

across all

K

), whereas Subs exhibit substantially higher and largely

K

invariant error rates, revealing persistent latent instability even when the original final answers are correct.

Dataset	#Pairs	$\boldsymbol{\rho}$	QWK	MAE	Acc@ $\tau$	F1@ $\tau$
AIME25	150	0.90	0.84	0.24	0.94	0.93
MATH500	150	0.87	0.82	0.27	0.93	0.92
GSM8K	100	0.85	0.79	0.30	0.92	0.91
GPQA-Diamond	100	0.84	0.80	0.32	0.92	0.90
Overall	500	0.88	0.82	0.28	0.94	0.92

Appendix

Table 11: Human-LLM agreement for PCS few-shot judging. Spearman’s

\rho

and MAE are computed on raw one-decimal PCS scores. QWK is computed after rounding PCS to the nearest integer in

\{1,2,3,4,5\}

. Acc@

\tau

and F1@

\tau

treat PCS

\geq 4.0

as a positive edge.

Figure 9: Distribution of absolute PCS disagreement between the LLM judge and the median human score.

Order	Candidate	Short description	Human PCS	LLM PCS
Nearest	$e_{j-1}$	Wrong binding reused by child ( $T{=}13$ )	4.9	5.0
2nd	$e_{j-2}$	Ancestor value error (total $=13$ )	3.6	3.7
Far	$e_{j-6}$	Same segment, no artifact reuse	2.1	2.0

Appendix

Table 12: Single-case example of near-to-far candidate scoring for one child error node. The LLM and humans agree on the first candidate reaching

\tau{=}4.0

, which becomes the parent.

Metric	Fix-Root	No-Fix
$\|V\|$	0.42 $\times$	1.00 $\times$
$\bar{R}$	0.35 $\times$	1.00 $\times$
$D$	0.51 $\times$	1.00 $\times$

Appendix

Table 13: Quantitative impact of root-node intervention on the FoE structure. All metrics are reported as relative ratios to the No-Fix baseline (

1.00\times

), averaged over the BS-17k-subset. Metric definitions:

|V|

denotes the total number of error nodes per tree;

\bar{R}

is the error reproduction rate; and

D

represents the average tree depth.

Region	First			Subs
( $h$ / $v$ )	NTR	RTR	AND	NTR	RTR	AND
LL (low/low)	0.082	0.014	1.12	0.115	0.028	1.25
HL (high- $h$ only)	0.245	0.063	1.48	0.312	0.094	1.62
LH (high- $v$ only)	0.198	0.052	2.85	0.224	0.071	3.14
HH (high/high)	0.312	0.187	2.14	0.386	0.264	2.41

Appendix

Table 14: Node generation statistics across entropy quadrants (default

L{=}15

). NTR, RTR, and AND metrics are reported for both First and Subs settings.

Feature Set	AUC ( First ) $\uparrow$	AUC ( Subs ) $\uparrow$
$h$ only	0.642	0.658
$v$ only	0.615	0.631
$h+v$	0.724	0.742
$h+v+(h\times v)$	0.816	0.835

Appendix

Table 15: Predictive ablation for root-node triggers using logistic regression. AUC scores quantify the predictive power of entropy (

h

) and variance (

v

) features. The interaction term (

h{\times}v

) provides a substantial gain.

Window $L$	$\text{RTR}_{\text{HH}}$ ( First ) $\uparrow$	$\text{RTR}_{\text{HH}}$ ( Subs ) $\uparrow$
10	0.174	0.251
12	0.182	0.258
15 (Default)	0.187	0.264
18	0.185	0.261
20	0.179	0.255

Appendix

Table 16: Robustness to window length

L

. Across all

L\in[10,20]

, the HH region consistently yields the highest RTR, with First maintaining a lower trigger rate than Subs under identical conditions.

Comparison	First		Subs
(RTR)	OR $\uparrow$	Holm- $p$ $\downarrow$	OR $\uparrow$	Holm- $p$ $\downarrow$
HH vs. LL	16.20	$<10^{-12}$	12.45	$<10^{-12}$
HH vs. HL	3.42	$2.4\times 10^{-7}$	3.46	$1.1\times 10^{-8}$
HH vs. LH	4.19	$8.6\times 10^{-9}$	4.69	$3.2\times 10^{-10}$
First vs. Subs	OR = 0.64 (Holm- $p=0.0042$ )

Appendix

Table 17: Effect sizes and statistical significance for root-node generation. Odds ratios (OR) quantify the relative increase in root-trigger probability, with

p

-values Holm-Bonferroni corrected. Note : The cross-setting OR (

0.64

) is calculated for First relative to Subs as the reference.

Figure 10: Convergence (A, success). For a fixed probe prompt, the modal answer ratio (mode count /

N

) increases over checkpoints and exceeds the threshold

P

before the First answer becomes explicit; bars are green since the mode is correct.

Figure 11: Convergence (A, failure). A single prompt probe can wrongly converge: the mode switches from the correct answer (green) to a stable wrong answer (red) while still surpassing

P

Variant	ESC (%)	WESR (%)	Crisk@Agree (%)
A	73.20	0.88	1.20

Appendix

Table 18: Baseline A.

Figure 12: Robustness (B, success). With

M

diverse probe prompts and one sample per prompt (

N{=}1

), the number of unique induced answers decreases over time and eventually reaches 1 (green), indicating cross-prompt agreement.

Figure 13: Robustness (B, failure). Cross prompt agreement with

N{=}1

can be spurious: unique answers briefly drop to 1 on a wrong answer (red) due to high variance single draws.

Variant	ESC (%)	WESR (%)	Crisk@Agree (%)
B	70.50	0.92	1.30

Appendix

Table 19: Baseline B.

Figure 14: Robustness+Parallelism (B+C, success). Adding per-prompt parallel samples stabilizes prompt-wise modes; once modes align, the trigger is correct (green), and the minimum internal mode ratio rises.

Figure 15: Robustness+Parallelism (B+C, failure). Without enforcing an internal threshold, prompt-wise modes may align too early when internal mode ratios are still low, producing a wrong trigger (red).

Variant	ESC (%)	WESR (%)	Crisk@Agree (%)
B+C	76.80	0.46	0.60

Appendix

Table 20: Baseline B+C.

Variant	ESC (%)	WESR (%)	crisk@agr (%)
A/ IC	73.20	0.88	1.20
B/ CP1	70.50	0.92	1.30
B+C/ CPN	76.80	0.46	0.60
A+B+C/ DC	62.10	0.12	0.19
RED (ours)	66.70	0.04	0.06

Appendix

Table 21: Reverse ablation results under a unified evaluation setting.

Model	Method	AIME24	AIME25	MATH500	GSM8K	GPQA-Diamond
Qwen3-8B-Thinking	Vanilla	02:26	02:38	06:24	05:47	04:12
Qwen3-8B-Thinking	RED (raw)	02:27	02:38	06:34	06:03	04:20
Qwen3-32B-Thinking	Vanilla	04:25	04:34	10:04	08:09	06:21
Qwen3-32B-Thinking	RED (raw)	04:35	04:49	10:27	08:26	06:38
DeepSeek-R1-Qwen-7B	Vanilla	02:13	02:12	04:11	02:26	04:52
DeepSeek-R1-Qwen-7B	RED (raw)	02:24	02:21	04:23	02:38	05:15

Appendix

Table 22: Total inference runtime (mm:ss) comparison across benchmarks. Vanilla represents the baseline runtime, while RED (raw) denotes our method under stress test, where early-exit is disabled to isolate the operational overhead of probing and interventions

Model	Method	AIME24	AIME25	MATH500	GSM8K	GPQA-Diamond
Qwen3-8B-Thinking	Vanilla	02:26	02:38	06:24	05:47	04:12
	Think or Not	02:14	02:40	06:32	04:11	04:25
	DAST	01:35	01:28	03:50	02:08	02:19
	RL +length penalty	01:48	01:56	04:57	03:38	02:21
	S-GRPO	01:44	01:46	04:23	03:22	02:18
	RED (ours)	01:09	01:16	03:02	01:49	02:13

Appendix

Table 23: Total inference runtime (mm:ss) comparison across baselines. The experiments enable the early-exit mechanism. Vanilla represents the baseline runtime, while RED (ours) denotes our method.

Explore similar work

BFS-PO: Best-First Search for Large Reasoning Models
Feb 16, 2026Fiorenzo Parascandolo, Wenhui Tan, Enver Sangineto +2Efficient Language Model Reasoning RL for Language Model Reasoning
Department of Engineering, University of Modena and Reggio Emilia, Italy. · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models
Jun 1, 2026Simone Caldarella, Davide Talon, Rahaf Aljundi +2LLM Evaluation Overthinking in Language Models
University of Trento · 3Fondazione Bruno Kessler · 2Toyota Motor Europe
LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models
May 10, 2026Songtao Wei, Yi Li, Zhikai Li +7RL for Language Model Reasoning Efficient Language Model Reasoning
University of Texas at Dallas · Emory University · University of Texas at Arlington +2

Feb 16, 2026J/K move · Enter open · S save

BFS-PO: Best-First Search for Large Reasoning Models

Fiorenzo Parascandolo, Wenhui Tan, Enver Sangineto, Ruihua Song+1

Department of Engineering, University of Modena and Reggio Emilia, Italy. · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.

Efficient Language Model Reasoning RL for Language Model Reasoning Overthinking in Language Models Large Reasoning Models Tree Search

Open paper AskPDF arXiv

Large Reasoning Models (LRMs) such as OpenAI o1 and DeepSeek-R1 have shown excellent performance in reasoning tasks using long reasoning chains. However, this has also led to a significant increase of computational costs and the generation of verbose output, a phenomenon known as overthinking. The tendency to overthinking is often exacerbated by Reinforcement Learning (RL) algorithms such as GRPO/DAPO. In this paper, we propose BFS-PO, an RL algorithm which alleviates this problem using a Best-First Search exploration strategy. Specifically, BFS-PO looks for the shortest correct answer using a backtracking mechanism based on maximum entropy nodes. By generating progressively shorter responses during training, BFS-PO learns to produce concise reasoning chains. Using different benchmarks and base LRMs, we show that BFS-PO can simultaneously increase the LRM accuracy and shorten its answers.

Abstract

Explore similar work

BFS-PO: Best-First Search for Large Reasoning Models

Thinking Past the Answer: Evaluating Harmful Overthinking in Large Reasoning Models

LEAD: Length-Efficient Adaptive and Dynamic Reasoning for Large Language Models