Scaling test-time reasoning has substantially improved the problem-solving ability of large language models (LLMs), but standard autoregressive decoding still executes long reasoning traces sequentially, creating severe latency for difficult tasks (up to days and weeks). Parallel reasoning offers a natural remedy. However, prior systems primarily focus on Subtask Parallelism, where the model learns to decompose a high-level task into smaller chunks that can be solved independently. This approach overlooks another pervasive form of parallelism: Trial Parallelism, where multiple speculative attempts explore, verify, and aggregate competing hypotheses in parallel. In this paper, we introduce Parason, which reveals and learns both forms of parallelism in LLM reasoning. Our analysis identifies Trial Parallelism as the majority of parallelizable reasoning computation (65.5% in DeepSeek-V4's reasoning steps in HLE), and it becomes increasingly dominant on hard problems. Guided by this taxonomy, Parason converts sequential reasoning traces into structured parallel trajectories with a context-free grammar, then trains models with Parallelism-Aware Group Relative Policy Optimization (PA-GRPO), whose reward jointly balances accuracy, latency, and the two parallelism ratios. At inference time, Parason executes the learned parallel structure through tool calls, translating theoretical savings to real-world wall-clock acceleration. Experiments on mathematical reasoning benchmarks including AIME24 and AIME25 show that Parason achieves an average acceleration about 1.7× while maintaining competitive accuracy.
Figures & tables
Figure 1: Overview of two types of parallelism. The figure contrasts Subtask Parallelism, where all decomposed branches are needed for the final answer, with Trial Parallelism, where multiple uncertain paths are explored as alternative (OR) branches. Despite this OR relation, the outputs of all Trial branches are concatenated into the subsequent context, making every branch’s content available for final synthesis. While previous work mainly focuses on Subtask Parallelism, our analysis shows that Trial Parallelism accounts for the majority of parallelizable reasoning on HLE for every model we study, and exceeds 58% for DeepSeek-R1 and DeepSeek-V4 across both datasets.
Figure 2: Visualization of Trial and Subtask Parallelism. The 24 Puzzle trace generated by DeepSeek-R1 ( DeepSeek-AI et al., 2025 ) highlights Subtask Parallelism in blue and Trial Parallelism in green . Subtask branches implement divide-and-conquer execution, while Trial branches explore and verify competing hypotheses; Trial steps occupy the majority of reasoning tokens.
Table 1: Trial Parallelism dominates on harder reasoning datasets. We report Subtask and Trial ratios in reasoning traces from OpenMathReasoning ( Moshkov et al., 2025 ) and Humanity’s Last Exam (HLE) ( Phan et al., 2025 ) . Ratios are computed on the first 100 problems of each dataset, and annotation details are provided in Appendix A. For closed-source models, the original thinking trace is hidden, thus we counted summary statistics instead.
Figure 3: Solving the 24 Puzzle in Parason format. This example is adapted from a real 24 Puzzle reasoning trace. The blue region gives the Subtask plan, splitting the problem into three independent subtasks. The green regions show Trial Parallelism, where each subtask explores candidate routes in parallel. The correct solution appears in the first Trial branch of the second subtask.
Figure 4: Context-Free Grammar for Parason. Each parallel region contains an Outlines block with one or more correctly matched Subtask or Trial entries, followed by one or more Thread bodies. As in Figure 3 , each Thread may contain free-form reasoning or a nested parallel region.
Method
AIME24
AIME25
Math500
AMC
Avg
Avg. Latency #Tokens
Parallel-R1 (4B)
19.4
19.2
-
-
-
-
ShorterBetter (7B)
53.3
-
-
-
-
-
ThinkPrune (32B)
72.5
-
93.8
95.9
-
-
DYNASOR-COT (32B)
78.0
-
93.0
94.0
-
-
Dynamic Early Exit (32B)
70.0
-
94.8
95.0
-
-
AdaptThink (7B)
55.6
-
92.0
-
-
-
Table 2: Trial rewards improve accuracy, while Subtask rewards reduce latency. We compare Parason with prior systems and ablate βsubtask and βtrial . Avg and Token Latency average AIME24, AIME25, Math500, and AMC for each PA-GRPO row. Bold and underlined values mark the highest and second-highest PA-GRPO results, except Token Latency, where lower is better.
Table 3: Accuracy under token-latency budgets. Each column reports AIME24 accuracy for a fixed thinking budget B on the longest token path. Subscripts show absolute accuracy-point gains relative to SFT only under the same budget; red indicates a decrease. Subtask-aware PA-GRPO provides the largest gains in low-budget regimes: at B=2,048 , βsubtask=0.100 reaches 34.7% accuracy, compared with 16.8% for SFT only.
Method
Trigger Ratio
Token Latency
Acceleration Ratio
Avg. Math Acc.
ThreadWeaver (8B)
83.7
13.3k
1.18x
77.6
Multiverse (32B)
65.6
10.3k
1.16x
63.8
Parason-8B (w/ SFT only)
69.3
14.1k
1.27x
78.3
Original runs with α=0.000
+ PA-GRPO ( βtrial=0.025 , α=0.000 )
77.3
15.2k
1.46x
79.9
+ PA-GRPO ( βtrial=0.050 , α=0.000 )
84.0
15.8k
1.61x
80.4
Table 4: PA-GRPO produces usable parallelism across latency-penalty settings. For PA-GRPO, Trigger Ratio and Avg. Math Acc. are averaged over AIME24, AIME25, and Math500. Acceleration ratio is total tokens / longest-path tokens, and trigger ratio is the fraction of samples containing at least one <Parallel> block. Bold and underlined values mark the best and second-best PA-GRPO results among available measurements.
Token-Level Metrics
Wall-Clock Metrics
Difficulty
#Problems
Generated #Tokens
Latency #Tokens
Acceleration Ratio
Saved #Tokens
Parallel (s)
Sequential (s)
Easy
13
21.3k
12.5k
1.70×
8.8k
188.3
304.2
Medium
11
36.8k
21.2k
1.74×
15.6k
336.2
464.7
Hard
6
50.3k
29.0k
1.73×
21.3k
487.3
716.3
Table 5: Harder questions save more tokens, while parallel execution reduces wall-clock latency across all difficulty levels. We break down AIME24 thinking tokens and measured wall-clock time by difficulty. Latency #Tokens denotes the length of the longest token path, Acceleration Ratio is generated tokens divided by latency tokens, and Saved #Tokens is generated tokens minus latency tokens. The experiment is executed on A800 GPUs.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Difficulty
Problem Indices
Hard
2, 3, 13, 21, 28, 29
Mid
1, 4, 5, 10, 16, 17, 18, 20, 25, 26, 27
Easy
0, 6, 7, 8, 9, 11, 12, 14, 15, 19, 22, 23, 24
Appendix
Table 6: AIME 2024 Problem Classification by Difficulty.
Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but their inherently sequential decoding incurs substantial latency, motivating parallelization of the generation process. However, existing parallel reasoning approaches suffer from performance degradation compared to their sequential counterparts, and often rely on specialized inference engines. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that matches the accuracy of comparably sized sequential reasoning models while significantly reducing inference latency via three key innovations: 1) a two-stage parallel trajectory generator that produces high-quality parallel chain-of-thought data for supervised fine-tuning; 2) a trie-based rollout design that enables parallel reasoning on any off-the-shelf autoregressive inference engine; and 3) a parallelization-aware reinforcement learning framework that trains the model to balance reasoning accuracy with effective parallelization. Across six challenging math reasoning benchmarks, ThreadWeaver trained on top of Qwen3-8B achieves performance on par with cutting-edge sequential reasoning models (79.9% on AIME24 and 71.9% on average) while delivering up to 1.53x speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.
Long Lian, Sida Wang, Felix Juefei-Xu +7
Meta Superintelligence Labs (MSL), Menlo Park, CA, United States · UC Berkeley, Berkeley, CA, United States · UCSF, San Francisco, CA, United States
Scaling inference-time compute for Large Language Models (LLMs) has unlocked unprecedented reasoning capabilities. However, existing inference-time scaling methods typically rely on inefficient and suboptimal discrete search algorithms or trial-and-error prompting to improve the online policy. In this paper, we propose ∇-Reasoner, an iterative generation framework that integrates differentiable optimization over token logits into the decoding loop to refine the policy on the fly. Our core component, Differentiable Textual Optimization (DTO), leverages gradient signals from both the LLM's likelihood and a reward model to refine textual representations. ∇-Reasoner further incorporates rejection sampling and acceleration design to robustify and speed up decoding. Theoretically, we show that performing inference-time gradient descent in the sample space to maximize reward is dual to aligning an LLM policy via KL-regularized reinforcement learning. Empirically, ∇-Reasoner achieves over 20% accuracy improvement on a challenging mathematical reasoning benchmark, while reducing number of model calls by approximately 10-40% compared to strong baselines. Overall, our work introduces a paradigm shift from zeroth-order search to first-order optimization at test time, offering a cost-effective path to amplify LLM reasoning.
Peihao Wang, Ruisi Cai, Zhen Wang +4
The University of Texas at Austin · UC San Diego · TTIC +1
Long chain-of-thought reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical reasoning. However, LLM generation is highly sequential, and long CoTs lead to a high latency. We propose to train Divide-and-Conquer CoT (DC-CoT) to reduce the latency. With DC-CoT, the model can act as a director that identifies distinct subtasks that can be performed in parallel in its reasoning process, and then spawns workers to execute the subtasks. Our goal is to achieve high accuracy, with a low longest path length, which is a theoretical measure of the latency needed for the response. We start with a long CoT base model (DeepScaleR-1.5B-Preview), and first use SFT with a small curated demonstration set to initialize its ability to spawn workers in a certain format. Because SFT degrades the accuracy significantly, we design a multi-stage RL algorithm, with various data filtering strategies, to recover the accuracy while decreasing the longest path length. Across several benchmarks including AIME 2024 and HMMT 2025, DC-CoT achieves similar accuracy as DeepScaleR-1.5B-Preview while decreasing longest path length by 35-40%. Our code, SFT dataset and models are publicly available at https://github.com/amahankali10/DC_CoT_RL_for_Low_Latency_CoT_with_Parallel_Reasoning.