Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning

Arvind Mahankali; Kaiyue Wen; Tengyu Ma

cs.LGJan 30, 2026

Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning

Authors: Arvind Mahankali, Kaiyue Wen, Tengyu Ma

Organizations: Stanford University

Abstract

Long chain-of-thought reasoning (Long CoT) is now fundamental to state-of-the-art LLMs, especially in mathematical reasoning. However, LLM generation is highly sequential, and long CoTs lead to a high latency. We propose to train Divide-and-Conquer CoT (DC-CoT) to reduce the latency. With DC-CoT, the model can act as a director that identifies distinct subtasks that can be performed in parallel in its reasoning process, and then spawns workers to execute the subtasks. Our goal is to achieve high accuracy, with a low longest path length, which is a theoretical measure of the latency needed for the response. We start with a long CoT base model (DeepScaleR-1.5B-Preview), and first use SFT with a small curated demonstration set to initialize its ability to spawn workers in a certain format. Because SFT degrades the accuracy significantly, we design a multi-stage RL algorithm, with various data filtering strategies, to recover the accuracy while decreasing the longest path length. Across several benchmarks including AIME 2024 and HMMT 2025, DC-CoT achieves similar accuracy as DeepScaleR-1.5B-Preview while decreasing longest path length by 35-40%. Our code, SFT dataset and models are publicly available at https://github.com/amahankali10/DC_CoT_RL_for_Low_Latency_CoT_with_Parallel_Reasoning.

Figures & tables

Figure 1 : Accuracy (pass@1) and longest path length for DC-CoT and baselines on AIME 2024. DSR-32K and DSR-12K denote DeepScaleR-1.5B-Preview evaluated with response lengths $32,768$ and $12,000$ respectively. DC-CoT-HLP denotes DC-CoT trained with additional RL with a high length penalty, and DSR-HLP-24K and DSR-HLP-12K denotes baselines trained similarly, starting from DeepScaleR-1.5B-Preview. Maj-DSR-3 denotes DSR-12K evaluated using majority voting (maj@3), and DC-CoT-Maj denotes DC-CoT evaluated using maj@3. DC-CoT achieves higher accuracy than DSR-32K while requiring less longest path length.

Figure 4 : Accuracy (pass@1) and longest path length for all methods on all benchmarks.

Method	AIME 24		AMC		HMMT		MATH		MM		OB
	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL
DSR-32K	$40.20$	$9540$	$81.35$	$5151$	$18.28$	$10685$	$89.04$	$3146$	$36.65$	$4879$	$60.94$	$5875$
DSR-12K	$39.43$	$8047$	$80.66$	$4792$	$18.28$	$8666$	$88.81$	$2963$	$36.63$	$4496$	$60.52$	$5341$
Maj-DSR-3	$44.19$	$9690$	$84.55$	$5803$	$19.39$	$9970$	$91.02$	$3638$	$38.69$	$5716$	$63.34$	$6479$
DSR-HLP-24K	$40.08$	$7903$	$81.90$	$4535$	$18.32$	$8375$	$89.29$	$2790$	$37.50$	$4124$	$61.72$	$4909$
DSR-HLP-12K	$39.03$	$6023$	$80.83$	$3574$	$16.78$	$6391$	$88.08$	$2215$	$37.36$	$3122$	$60.34$	$3880$

Table 1 : Accuracy and longest path length (LPL) for all methods and benchmarks discussed in Section 5.1 . Error bars for all accuracies and longest path lengths shown in Appendix J , Table 3 and Table 4 .

Method	Baseline	AIME 24		AMC		HMMT		MATH		MM		OB
		Acc	LPL	Acc	LPL	Acc	LPL	Acc	LPL	Acc	LPL	Acc	LPL
DC-CoT	DSR-32K	+1.67	-37.4%	-0.97	-34.6%	-0.21	-36.3%	-0.3	-37.1%	-1.49	-41.3%	-0.75	-36.3%
DC-CoT-HLP	DSR-HLP-12K	+1.82	-21.3%	-0.28	-22.0%	-0.3	-19.0%	+0.36	-21.7%	-2.34	-21.4%	-0.27	-21.6%
DC-CoT-Maj	Maj-DSR-3	+1.83	-23.5%	-0.85	-27.0%	+0.15	-18.7%	-0.36	-32.8%	-1.64	-35.8%	-0.89	-29.0%

Table 2 : Change in accuracy and longest path length of our methods compared to appropriate baselines. For DC-CoT we use DSR-32K as the baseline, for DC-CoT-HLP we use DSR-HLP-12K as the baseline, and for DC-CoT-Maj we use Maj-DSR-3 as the baseline.

Figure 5 : Accuracy and longest path length of DC-CoT on AIME 2024 during all stages of RL.

Figure 6 : Accuracy (pass@1) and longest path length on AIME 2024 for DSR-HLP-12K , DSR-HLP-24K , and DC-CoT-HLP .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Algorithm 1 Algorithm for inference with a language model $M$ that spawns worker threads.

Figure 8 : DAPO plateaus in accuracy during Stage 1, while accuracy continues improving monotonically once we switch to CISPO.

Figure 9 : We encounter a plateau in accuracy in Stage 2, where we use filtering strategy Include-Easy . Accuracy continues improving monotonically in Stage 3 when we switch to filtering strategy Remove-Easy .

Panel of Figure 10 — Figure 10 : Entropy statistics during stage 1, when using DAPO with $\varepsilon_{\textup{high}}=0.28$ .

Method	AIME 24		AMC		HMMT
	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL
DSR-32K	$40.20_{\pm 0.63}$	$9540_{\pm 83}$	$81.35_{\pm 0.44}$	$5151_{\pm 51}$	$18.28_{\pm 0.50}$	$10685_{\pm 89}$
DSR-12K	$39.43_{\pm 0.63}$	$8047_{\pm 43}$	$80.66_{\pm 0.44}$	$4792_{\pm 38}$	$18.28_{\pm 0.50}$	$8666_{\pm 41}$
Maj-DSR	$44.19_{\pm 0.01}$	$9690_{\pm 0.0}$	$84.55_{\pm 0.0}$	$5803_{\pm 1}$	$19.39_{\pm 0.01}$	$9970_{\pm 0.0}$
DSR-HLP-24K	$40.08_{\pm 0.63}$	$7903_{\pm 54}$	$81.90_{\pm 0.43}$	$4535_{\pm 40}$	$18.32_{\pm 0.50}$	$8375_{\pm 55}$
DSR-HLP-12K	$39.03_{\pm 0.63}$	$6023_{\pm 38}$	$80.83_{\pm 0.44}$	$3574_{\pm 29}$	$16.78_{\pm 0.48}$	$6391_{\pm 38}$

Appendix

Table 3 : Results in Table 1 with sample mean and sample standard deviation (AIME 2024, AMC 23, HMMT 2025).

Method	MATH		MM		OB
	Acc (%)	LPL	Acc (%)	LPL	Acc (%)	LPL
DSR-32K	$89.04_{\pm 0.35}$	$3146_{\pm 39}$	$36.65_{\pm 0.73}$	$4879_{\pm 71}$	$60.94_{\pm 0.47}$	$5875_{\pm 49}$
DSR-12K	$88.81_{\pm 0.35}$	$2963_{\pm 29}$	$36.63_{\pm 0.73}$	$4496_{\pm 47}$	$60.52_{\pm 0.47}$	$5341_{\pm 33}$
Maj-DSR	$91.02_{\pm 0.05}$	$3638_{\pm 6}$	$38.69_{\pm 0.12}$	$5716_{\pm 9}$	$63.34_{\pm 0.08}$	$6479_{\pm 6}$
DSR-HLP-24K	$89.29_{\pm 0.35}$	$2790_{\pm 30}$	$37.50_{\pm 0.73}$	$4124_{\pm 49}$	$61.72_{\pm 0.47}$	$4909_{\pm 34}$
DSR-HLP-12K	$88.08_{\pm 0.36}$	$2215_{\pm 22}$	$37.36_{\pm 0.73}$	$3122_{\pm 33}$	$60.34_{\pm 0.47}$	$3880_{\pm 25}$

Appendix

Table 4 : Sample mean and sample standard deviation for Table 1 (MATH 500, Minerva Math, Olympiad Bench).

Method	AIME 24	AMC	HMMT	MATH	MM	OB
DC-CoT	$6831$	$4185$	$7603$	$2554$	$3526$	$4489$
DC-CoT-HLP	$6185$	$4022$	$6411$	$2537$	$3401$	$4154$
DC-CoT-Maj	$20493$	$12554$	$22809$	$7662$	$10579$	$13468$

Appendix

Table 5 : Total tokens used by each response, averaged across responses. For DC-CoT-Maj , we average across all subsets of

3

responses to the same problem. For each subset of

3

responses, we compute the sum of the tokens used by each of the

3

responses.

Figure 11 : Degree of parallelism of DC-CoT on AIME 2024 during all stages of RL.

Figure 12 : Degree of parallelism on AIME 2024 for DC-CoT-HLP

Figure 13 : Validation accuracy and longest path length over the course of the three-stage algorithm. We write "stage 2" to indicate that the hyperparameters from stage 2 of our original run are being used, and similarly for the other stages.

Figure 14 : Training entropy over the course of the three-stage algorithm.

Figure 15 : Validation accuracy and longest path length over the course of the two-stage algorithm. We write "stage 3" to indicate that the hyperparameters from stage 3 of our original run are being used, and similarly for "stage 4".

Method	Avg time per Problem (s)	Std. Dev across seeds (s)
DSR-32K	84.33	31.19
DSR-12K	63.74	2.29
DC-CoT	47.51	5.69
DSR-HLP-24K	61.53	5.68
DSR-HLP-12K	48.94	1.56
DC-CoT-HLP	35.49	1.68

Appendix

Table 6 : Latency for generation with batch size 1 on AIME 2024.

Method	$N=1$	$N=4$	$N=8$	$N=16$
DC-CoT	124.4 $\pm$ 2.6	289.7 $\pm$ 21.6	579.5 $\pm$ 16.6	1039.7 $\pm$ 12.4
DC-CoT-HLP	90.8 $\pm$ 9.9	224.0 $\pm$ 9.3	435.2 $\pm$ 43.7	911.1 $\pm$ 16.6
DSR-12K	152.2 $\pm$ 2.6	421.4 $\pm$ 23.9	862.4 $\pm$ 14.3	1706.5 $\pm$ 55.7
DSR-32K	332.9 $\pm$ 93.9	725.4 $\pm$ 42.1	1499.4 $\pm$ 66.8	2756.0 $\pm$ 79.5
DSR-HLP-12K	140.6 $\pm$ 3.2	307.5 $\pm$ 11.8	525.4 $\pm$ 38.7	1132.2 $\pm$ 5.3
DSR-HLP-24K	208.4 $\pm$ 6.5	461.2 $\pm$ 44.8	892.7 $\pm$ 80.6	1815.0 $\pm$ 50.4

Appendix

Table 7 : Wall-clock time (seconds) for generation with batch size

30\cdot N

, where

N

is the number of rollouts per problem, on AIME 2024. Each entry is the mean

\pm

standard deviation across 3 random seeds.

Explore similar work

May 25, 2026cs.CL

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains

Explicit chain-of-thought (CoT) reasoning substantially improves the reasoning ability of large language models (LLMs), but incurs high inference cost due to lengthy autoregressive traces. Existing latent reasoning methods offer a promising alternative, yet they often treat reasoning as uniformly compressible, causing precision-critical intermediate steps to be overly compressed and thereby degrading reasoning accuracy. In this work, we propose Selective Latent Thinking (SLT), a framework that selectively compresses redundant reasoning spans into latent representations while preserving precision-critical spans as explicit CoT within the same reasoning trajectory. Specifically, SLT first uses a lightweight decoder to anticipate a short upcoming reasoning span, and then applies confidence-based gating to determine the longest span that can be reliably compressed. The accepted span is encoded into a compact latent representation to improve reasoning efficiency, while uncertain or precision-critical reasoning remains in explicit CoT form to preserve accuracy. To learn this selective compression policy, SLT adopts a three-stage training strategy that combines span-level latent compression, reliability-aware future reasoning prediction, and trajectory-level reinforcement learning to optimize the trade-off between answer correctness and reasoning cost. Extensive experiments across four mathematical reasoning benchmarks demonstrate that SLT achieves 22.7% higher accuracy than latent reasoning baselines at comparable compression ratios, while reducing reasoning chain length by 58.4% with only 2.8% accuracy degradation compared to explicit CoT,Our code can be found in https://github.com/hunshi34/SLT.

Hui Xie, Jie Liu, Ziyue Qiao +1

Eindhoven University of Technology, Netherlands · School of Computing and Information Technology, Great Bay University, China

Latent Thoughts Chain-of-Thought Reasoning Redundancy

Ask PDF

Nov 24, 2025cs.LG

ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models

Scaling inference-time computation has enabled Large Language Models (LLMs) to achieve strong reasoning performance, but their inherently sequential decoding incurs substantial latency, motivating parallelization of the generation process. However, existing parallel reasoning approaches suffer from performance degradation compared to their sequential counterparts, and often rely on specialized inference engines. We introduce ThreadWeaver, a framework for adaptive parallel reasoning that matches the accuracy of comparably sized sequential reasoning models while significantly reducing inference latency via three key innovations: 1) a two-stage parallel trajectory generator that produces high-quality parallel chain-of-thought data for supervised fine-tuning; 2) a trie-based rollout design that enables parallel reasoning on any off-the-shelf autoregressive inference engine; and 3) a parallelization-aware reinforcement learning framework that trains the model to balance reasoning accuracy with effective parallelization. Across six challenging math reasoning benchmarks, ThreadWeaver trained on top of Qwen3-8B achieves performance on par with cutting-edge sequential reasoning models (79.9% on AIME24 and 71.9% on average) while delivering up to 1.53x speedup in token latency, establishing a new Pareto frontier between accuracy and efficiency.

Long Lian, Sida Wang, Felix Juefei-Xu +7

Meta Superintelligence Labs (MSL), Menlo Park, CA, United States · UC Berkeley, Berkeley, CA, United States · UCSF, San Francisco, CA, United States

LLM Reasoning Strategies Inference Latency Weave

Ask PDF

May 4, 2026cs.AI

Distilling Long-CoT Reasoning through Collaborative Step-wise Multi-Teacher Decoding

Distilling large reasoning models is essential for making Long-CoT reasoning practical, as full-scale inference remains computationally prohibitive. Existing curation-based approaches select complete reasoning traces post-hoc, overlooking collaboration among heterogeneous teachers and lacking dynamic exploration, which leads to redundant sampling and missed complementary reasoning. We introduce CoRD, a collaborative multi-teacher decoding framework that performs step-wise reasoning synthesis guided by predictive perplexity-based scoring and beam search. This enables heterogeneous LRMs to jointly construct coherent reasoning trajectories while efficiently preserving diverse, high-potential hypotheses. Experiments show that CoRD produces higher-quality reasoning data and achieves near teacher-level student performance with fewer, structured supervision signals, without substantial efficiency overhead. CoRD further generalizes well to out-of-domain and open-ended settings. The dataset and model are available at https://github.com/DISL-Lab/CoRD.

Taewon Yun, Jisu Shin, Jeonghwan Choi +2

KAIST · UNIST

Reasoning Trajectory Teacher-Student Distillation Teacher Low-Quality Data

Ask PDF