Parallel Thinking for Large Language Model with Path Diversity
Authors: Jiaru Qian, Yihong Dong, Xue Jiang, Chun-Wah Hsu, Zhi Jin, Ge Li
Organizations: School of Computer Science, Peking University · Key Laboratory of High Confidence Software Technologies, Peking University, Ministry of Education · Shanghai Jiao Tong University · School of Computer Science, Wuhan University
Parallel thinking has emerged as a promising paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs) by exploring multiple reasoning paths simultaneously. However, we identify that parallel thinking via Reinforcement Learning (RL) typically suffers from the mode collapse of reasoning paths, where parallel thinking trajectories degenerate into similar ones. This homogeneity undermines the fundamental motivation of parallel thinking, that is, maintaining multiple hypotheses concurrently to prevent premature loss into a suboptimal solution. In this paper, we propose Diverse Parallel Thinking (DPT), a simple yet effective RL approach for parallel thinking designed to encourage path diversity. DPT rewards the model to develop distinct reasoning strategies by penalizing the prefix similarity of parallel thinking paths, thereby enhancing a more comprehensive exploration of the reasoning space. We conduct various experiments and analyses across multiple reasoning benchmarks to demonstrate the effectiveness of DPT. Results show that DPT successfully outperforms previous parallel thinking methods, achieving state-of-the-art performance over baselines. We validate path diversity as a key element to improve parallel thinking. Our code is available at https://github.com/JiaruQian/DPT.
Figures & tables
Figure 1: Comparison of Traditional Sequential Thinking, Naive Parallel Thinking (RL with outcome reward), and Ideal Parallel Thinking.
Figure 2: Path Mode Collapse in Naive Parallel Thinking.
Figure 3: Overview of DPT. (I) Format learning teaches the parallel thinking format via SFT on easy problems. (II) Diverse parallel thinking develops autonomous parallel thinking on hard problems with combined rewards.
Method
AMC23
AIME25
AIME24
MATH
Average
pass@1
pass@16
pass@1
pass@16
pass@1
pass@16
pass@1
Base Model
8.1
51.2
1.3
10.2
2.9
16.5
13.9
14.9
SFT
39.1
77.3
4.4
17.2
6.5
24.6
65.8
33.6
GRPO
61.4
88.8
16.0
33.3
17.9
39.9
82.9
48.6
LTE ( Tang et al., 2026 )
56.7
85.0
14.2
46.7
18.3
30.0
74.6
46.5
TTRL ( Zuo et al., 2025 )
42.5
75.2
6.8
28.6
12.1
23.2
75.4
37.7
Table 1: Performance comparison on mathematical reasoning benchmarks, where the best results shown in bold and second-best results underlined.
Method
HumanEval
ARC-c
MMLU-Pro
GRPO
60.98
64.68
43.40
Parallel-R1
67.12
64.88
47.76
DPT
69.51
65.27
49.40
Table 2: Reported scores (%) on three additional on coding and knowledge reasoning tasks.
Method
AIME24
AIME25
AMC23
MATH
Parallel-R1
0.581
0.542
0.529
0.567
DPT
0.923
0.927
0.911
0.886
Δ
↑ 0.342
↑ 0.385
↑ 0.382
↑ 0.319
Table 3: Lexical path diversity measured by Jaccard distance. Higher values indicate lower lexical overlap; Δ denotes the increase over Parallel-R1.
Figure 4: Multi-dimensional diversity evaluation. LLM-as-judge comparing DPT and Parallel-R1 across five dimensions. Lower score denotes more diversity.
AIME25
AIME24
AMC23
Setting
pass@1
pass@16
pass@1
pass@16
pass@1
pass@16
DPT ( K=2 )
18.5
40.2
18.1
40.2
64.4
89.9
DPT ( K=4 )
17.5
41.2
17.5
38.2
61.9
90.4
Table 4: Results (%) with two or four internal reasoning paths. K counts paths within a trajectory. We report pass@1 and pass@16 accuracy on three mathematical reasoning benchmarks.
Method
pass@ k (%)
1
16
64
128
256
GRPO
16.0
33.3
48.3
52.4
56.0
Parallel-R1
18.3
38.5
47.9
49.3
50.9
DPT
18.5
40.2
49.2
53.5
57.0
Table 5: AIME25 pass@ k accuracy (%) with increasing numbers of sampled complete solutions.
Figure 5: Training dynamics of accuracy and parallel ratio, where the parallel ratio (purple) decreases from near 100% to ∼ 65% as the model learns selective parallel thinking, while accuracy (blue) continues improving.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Prompt Construction.
Metric
AIME25
AIME24
AMC23
MATH
Average
pass@1
pass@16
pass@1
pass@16
pass@1
pass@16
pass@1
No diversity reward
18.33
38.51
16.45
37.11
63.59
88.20
83.22
49.34
Prefix-1 exact
18.54
40.18
18.12
43.34
64.37
89.88
84.49
51.27
Prefix-2 exact
17.91
42.56
16.66
35.88
62.81
88.50
85.12
49.92
Prefix-3 exact
17.08
40.92
17.70
33.72
63.75
89.10
84.49
49.54
Prefix-4 exact
20.83
42.88
17.91
36.77
64.53
91.26
82.59
50.97
Appendix
Table 6: Ablation study on diversity metric design across four mathematical reasoning benchmarks. Scores are percentages; Average is the arithmetic mean of the seven reported accuracy metrics. The best results in each column are bold.
Metric
AIME25
AIME24
AMC23
MATH
No diversity reward
54.58
52.08
77.81
94.62
Prefix-1 exact
60.42
61.67
82.50
95.89
Prefix-2 exact
75.00
77.71
81.80
95.89
Prefix-3 exact
48.75
49.17
73.75
92.72
Prefix-4 exact
51.88
51.04
77.34
89.87
Prefix-2 similarity
42.08
44.79
71.88
92.72
Appendix
Table 7: Parallel ratio (%) for each diversity metric setting, measuring how often the model triggers parallel thinking during the reasoning process.
Method
Tokens/problem
Latency (s)
TFLOPs/problem
Tokens/solved problem
GRPO (sequential)
1,783
0.778
17.25
3,544
Parallel-R1
1,666
0.710
15.64
4,004
DPT
1,584
0.678
14.99
3,785
Appendix
Table 8: Inference cost in the checkpoint-175 validation analysis. FLOPs are reported in trillions per problem. The last column normalizes token consumption by solved problems.
Method
Median tokens
At token cap (%)
GRPO
1,715
24.0
Parallel-R1
1,491
13.0
DPT
1,401
10.2
Appendix
Table 9: Response lengths and frequency of reaching the 3,000-token cap in the validation cost analysis.