Contextual Correction of Parallel Draftsfor Speculative Decoding

Roy Eisenstadt; Harel Lidar; Tal Gonen; Omer Luxembourg; Itamar Zimerman; Lior Wolf; Eli David; Ilan Naiman

cs.CLOct 8, 2026

Contextual Correction of Parallel Draftsfor Speculative Decoding

Authors: Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman

Organizations: Stealth Startup, Tel Aviv · Tel Aviv University · Ben-Gurion University

Abstract

Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predictions, while drafting multiple future positions in parallel avoids the sequential cost of autoregressive generation. This parallelism, however, comes at a cost. Tokens within a draft block are predicted without access to the realized identities of preceding draft tokens, removing intra-block dependencies that would naturally be available under autoregressive generation. We introduce DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores these dependencies while preserving parallel drafting. Rather than committing to each position's top prediction, DCOCO retains multiple candidate tokens and evaluates their contextual compatibility across adjacent draft positions in parallel, enabling contextually informed token selection at low additional cost. Across four verifier backbones and ten benchmarks, DCOCO achieves the highest average throughput in every evaluated configuration, spanning greedy and stochastic decoding with and without extended reasoning, with speedups of up to 6.6x over vanilla decoding. Its advantage persists under concurrent serving: DCOCO outperforms the strongest competing method by 6-23% across 4-32 concurrent requests, while other sequential correctors fall below uncorrected DFlash at these loads. These results demonstrate that recovering intra-block dependencies through parallel contextual scoring delivers consistent acceleration across decoding regimes and serving loads.

Figures & tables

Figure 1: Candidate coverage gains grow deeper in the block. Fraction of slots where DFlash’s top-

k

candidates contain the verifier’s committed token. (a) Qwen3-8B; shading marks the top-16–top-1 coverage gap. (b) The same gap across verifiers. Averages over the 15 slots appear in Tab. 18 .

Panel of Figure 2 — Figure 2: DCOCO : parallel scoring, prefix-first selection. (a) One DFlash pass supplies slot states, candidate IDs, and logits. Candidate and pair representations feed a shared MLP, computing all residual scores in parallel: $S_{p}(a,b)=\ell_{p+1}(b)+\Delta_{\theta}$ . (b) Starting from the first-slot draft argmax, selection follows the best transition from each chosen predecessor. Rows retain draft rank; scores in slots 2–4 condition on the selected predecessor. Tokens and scores are illustrative.

Figure 3: At high concurrency, DCOCO achieves the highest throughput, while Domino and DSpark fall below DFlash. Qwen3-8B throughput at (a)

T=0

and (b)

T=1

. (c) DCOCO ’s relative throughput gain over the strongest baseline at each concurrency.

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	265.2	1293.8 (4.88 $\times$ )	1412.3 (5.33 $\times$ )	1257.9 (4.74 $\times$ )	1625.2 (6.13 $\times$ )
	AIME-25	263.7	1133.7 (4.30 $\times$ )	1279.9 (4.85 $\times$ )	1188.4 (4.51 $\times$ )	1436.8 (5.45 $\times$ )
	GSM8K	267.0	936.9 (3.51 $\times$ )	1018.1 (3.81 $\times$ )	921.6 (3.45 $\times$ )	1175.8 (4.40 $\times$ )
Code	HumanEval	266.6	873.9 (3.28 $\times$ )	950.5 (3.57 $\times$ )	857.5 (3.22 $\times$ )	1075.6 (4.04 $\times$ )
	MBPP	267.0	800.9 (3.00 $\times$ )	891.7 (3.34 $\times$ )	778.2 (2.92 $\times$ )	979.1 (3.67 $\times$ )
	LiveCodeBench	264.2	1080.1 (4.09 $\times$ )	1161.6 (4.40 $\times$ )	1084.7 (4.11 $\times$ )	1328.5 (5.03 $\times$ )

Table 1: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Qwen3-8B, thinking off , concurrency 1, max 2048 generated tokens. Each speculative cell reports TPS with its speedup over the no-spec column in parentheses. Bold: best method per row.

		DFlash		Domino		DSpark		DCOCO
Verifier	Think.	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Qwen3-4B	off	2.98 $\times$	2.52 $\times$	3.27 $\times$	2.71 $\times$	2.66 $\times$	2.28 $\times$	3.62 $\times$	2.95 $\times$
Qwen3-4B	on	2.08 $\times$	1.87 $\times$	2.17 $\times$	1.93 $\times$	1.85 $\times$	1.69 $\times$	2.38 $\times$	2.12 $\times$
Qwen3-8B	off	2.95 $\times$	2.45 $\times$	3.16 $\times$	2.53 $\times$	2.88 $\times$	2.37 $\times$	3.62 $\times$	2.87 $\times$
Qwen3-8B	on	1.95 $\times$	1.78 $\times$	1.97 $\times$	1.79 $\times$	1.83 $\times$	1.68 $\times$	2.26 $\times$	2.03 $\times$
Qwen3-Coder-30B-A3B	off	1.89 $\times$	1.67 $\times$	1.94 $\times$	1.67 $\times$	1.84 $\times$	1.62 $\times$	2.31 $\times$	1.98 $\times$

Table 2: All configurations at a glance: average speedup over vanilla decoding , unweighted mean over the ten evaluation suites. Bold: best per row and temperature. Qwen3-Coder-30B-A3B has no thinking mode. Per-dataset throughput and accepted length: App. D .

Figure 4: At high concurrency, overhead outweighs acceptance gains for Domino and DSpark, while DCOCO remains faster than DFlash. Accepted length

\tau

(solid) and throughput (dashed), relative to DFlash; Shading highlights the separation between the accepted-length and throughput ratios. Qwen3-8B, thinking off,

T=0

(dark) and

T=1

(light), averaged over all ten benchmark suites. Absolute values: Tab. 19 .

		Accepted length $\tau$			Throughput (tok/s)
Domain	Dataset	Greedy	Geometric	Global-Sum	Greedy	Geometric	Global-Sum
Math	MATH-500	8.039	8.053	7.999	1627.2	1543.6	1565.9
	AIME-25	7.263	7.292	7.226	1437.7	1404.1	1392.3
	GSM8K	6.118	6.115	6.075	1176.9	1160.5	1169.4
Code	HumanEval	5.443	5.445	5.401	1077.0	1051.8	1059.0
	MBPP	5.215	5.231	5.181	979.3	971.1	969.7

Table 3: Path selection over identical trained scores (Qwen3-8B from-pretrained corrector,

T{=}0

, no thinking, 2048 max tokens). Bold: best rule per row and metric. Between greedy and geometric the

\tau

differences carry both signs and are an order of magnitude smaller than either rule’s gap to Global-Sum; the throughput ordering is uniform.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Arm	$\tau$ @ 6 epochs	$\tau$ @ 12 epochs	$\Delta\tau$	$\Delta$ TPS
DFlash	3.913	3.948	+0.035 (+0.9%)	+11.0 (+1.4%)
Domino	4.613	4.701	+0.088 (+1.9%)	+10.9 (+1.3%)
DSpark	4.254	4.321	+0.067 (+1.6%)	+3.1 (+0.4%)
DCOCO (6 + 6 joint)	3.913 (drafter-only @ 6)	4.985	+1.072 (+27.4%)	+176.6 (+22.6%)

Appendix

Table 4: Extending baseline training to match our total budget , Q8B no-thinking, max generated tokens 2048, T=0 averaged across the 10 benchmarks. Baselines show no more than 2% improvement from a second six epochs; the joint phase accounts for our arm’s improvement.

Domain	Dataset	DCOCO	$-$ pair state	$-$ cand state	Frozen backbone
Math	MATH-500	1599.3	1497.7 ( $-$ 6.4%)	1521.8 ( $-$ 4.8%)	1333.6 ( $-$ 16.6%)
	AIME-25	1475.6	1382.0 ( $-$ 6.3%)	1387.9 ( $-$ 5.9%)	1236.1 ( $-$ 16.2%)
	GSM8K	1130.3	1042.3 ( $-$ 7.8%)	1073.2 ( $-$ 5.0%)	920.2 ( $-$ 18.6%)
Code	HumanEval	1113.9	1035.6 ( $-$ 7.0%)	1055.9 ( $-$ 5.2%)	929.5 ( $-$ 16.6%)
	MBPP	1121.6	1048.7 ( $-$ 6.5%)	1067.8 ( $-$ 4.8%)	929.5 ( $-$ 17.1%)
	LiveCodeBench	1202.9	1121.2 ( $-$ 6.8%)	1139.7 ( $-$ 5.3%)	1019.2 ( $-$ 15.3%)

Appendix

Table 5: Ablations: throughput (output tokens/s, per-stream), Qwen3-8B, thinking off , concurrency 1, max 2048 generated tokens, all ten suites. Each ablation cell reports its value with the change relative to DCOCO in parentheses.

-

pair state removes the corrector’s dedicated previous-state channel;

-

cand state zeroes the candidate representations entering the transition scorer; Frozen backbone trains only the corrector on the fixed six-epoch DFlash backbone. Bold: best measured system per row.

Domain	Dataset	DCOCO	$-$ pair state	$-$ cand state	Frozen backbone
Math	MATH-500	8.256	7.588 ( $-$ 8.1%)	7.802 ( $-$ 5.5%)	6.801 ( $-$ 17.6%)
	AIME-25	7.506	6.935 ( $-$ 7.6%)	7.043 ( $-$ 6.2%)	6.250 ( $-$ 16.7%)
	GSM8K	6.010	5.482 ( $-$ 8.8%)	5.697 ( $-$ 5.2%)	4.845 ( $-$ 19.4%)
Code	HumanEval	5.746	5.340 ( $-$ 7.1%)	5.439 ( $-$ 5.3%)	4.764 ( $-$ 17.1%)
	MBPP	5.858	5.379 ( $-$ 8.2%)	5.575 ( $-$ 4.8%)	4.800 ( $-$ 18.1%)
	LiveCodeBench	6.153	5.712 ( $-$ 7.2%)	5.790 ( $-$ 5.9%)	5.196 ( $-$ 15.5%)

Appendix

Table 6: Ablations: accepted length

\tau

, Qwen3-8B, thinking off , concurrency 1, max 2048 generated tokens, all ten suites. Each ablation cell reports its value with the change relative to DCOCO in parentheses.

-

pair state removes the corrector’s dedicated previous-state channel;

-

cand state zeroes the candidate representations entering the transition scorer; Frozen backbone trains only the corrector on the fixed six-epoch DFlash backbone. Bold: best measured system per row.

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	6.173	4.751 $\pm$ 0.024	7.500	5.518 $\pm$ 0.022	6.815	5.245 $\pm$ 0.053	8.039	5.755 $\pm$ 0.073
	AIME-25	5.554	3.751 $\pm$ 0.064	6.803	4.246 $\pm$ 0.076	6.289	3.975 $\pm$ 0.059	7.263	4.486 $\pm$ 0.037
	GSM8K	4.632	4.217 $\pm$ 0.024	5.563	4.960 $\pm$ 0.122	4.992	4.526 $\pm$ 0.019	6.118	5.440 $\pm$ 0.141
Code	HumanEval	4.245	3.771 $\pm$ 0.009	4.971	4.319 $\pm$ 0.075	4.576	4.046 $\pm$ 0.033	5.443	4.646 $\pm$ 0.043
	MBPP	3.992	3.507 $\pm$ 0.014	4.857	4.121 $\pm$ 0.117	4.345	3.767 $\pm$ 0.009	5.215	4.428 $\pm$ 0.008

Appendix

Table 7: Mechanism: mean accepted length

\tau

, Qwen3-8B, thinking off , companion to Table 1 ; no-spec commits one token per round by construction and is omitted.

\tau

explains the TPS movement; the corrector’s own overhead is visible only in throughput.

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	262.0	608.8 (2.32 $\times$ )	617.1 (2.36 $\times$ )	567.4 (2.17 $\times$ )	724.9 (2.77 $\times$ )
	AIME-25	260.3	586.9 (2.26 $\times$ )	612.5 (2.35 $\times$ )	574.9 (2.21 $\times$ )	703.1 (2.70 $\times$ )
	GSM8K	264.1	555.6 (2.10 $\times$ )	559.7 (2.12 $\times$ )	518.6 (1.96 $\times$ )	646.5 (2.45 $\times$ )
Code	HumanEval	260.6	562.9 (2.16 $\times$ )	572.8 (2.20 $\times$ )	524.7 (2.01 $\times$ )	654.4 (2.51 $\times$ )
	MBPP	262.8	535.6 (2.04 $\times$ )	548.2 (2.09 $\times$ )	498.2 (1.90 $\times$ )	618.1 (2.35 $\times$ )
	LiveCodeBench	259.2	541.9 (2.09 $\times$ )	554.5 (2.14 $\times$ )	506.6 (1.95 $\times$ )	624.2 (2.41 $\times$ )

Appendix

Table 8: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Qwen3-8B, thinking on , concurrency 1, max 4096 generated tokens. Same format as Table 1 .

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	2.947	2.759 $\pm$ 0.017	3.298	3.057 $\pm$ 0.018	3.102	2.884 $\pm$ 0.016	3.590	3.304 $\pm$ 0.021
	AIME-25	2.887	2.625 $\pm$ 0.035	3.250	2.981 $\pm$ 0.015	3.039	2.788 $\pm$ 0.021	3.526	3.170 $\pm$ 0.013
	GSM8K	2.717	2.560 $\pm$ 0.004	3.025	2.820 $\pm$ 0.005	2.788	2.649 $\pm$ 0.012	3.287	3.040 $\pm$ 0.015
Code	HumanEval	2.786	2.443 $\pm$ 0.004	3.099	2.702 $\pm$ 0.016	2.876	2.561 $\pm$ 0.004	3.331	2.879 $\pm$ 0.018
	MBPP	2.588	2.326 $\pm$ 0.003	2.892	2.555 $\pm$ 0.020	2.681	2.420 $\pm$ 0.009	3.093	2.703 $\pm$ 0.011

Appendix

Table 9: Mean accepted length

\tau

, Qwen3-8B, thinking on , companion to Table 8 ; DCOCO attains the best

\tau

in every cell in this configuration.

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	278.4	1356.9 (4.88 $\times$ )	1477.5 (5.31 $\times$ )	1367.6 (4.91 $\times$ )	1636.0 (5.88 $\times$ )
	AIME-25	274.8	1465.6 (5.33 $\times$ )	1601.0 (5.83 $\times$ )	1491.2 (5.43 $\times$ )	1811.7 (6.59 $\times$ )
	GSM8K	278.4	891.0 (3.20 $\times$ )	915.5 (3.29 $\times$ )	865.8 (3.11 $\times$ )	1160.8 (4.17 $\times$ )
Code	HumanEval	276.4	1049.9 (3.80 $\times$ )	1144.7 (4.14 $\times$ )	1038.0 (3.76 $\times$ )	1260.3 (4.56 $\times$ )
	MBPP	282.3	1036.3 (3.67 $\times$ )	1128.5 (4.00 $\times$ )	1039.0 (3.68 $\times$ )	989.7 (3.51 $\times$ )
	LiveCodeBench	278.7	990.0 (3.55 $\times$ )	1050.5 (3.77 $\times$ )	972.1 (3.49 $\times$ )	1448.8 (5.20 $\times$ )

Appendix

Table 10: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Llama-3.1-8B, thinking off , concurrency 1, max 2048 generated tokens. Same format as Table 1 .

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	7.194	2.514 $\pm$ 0.075	7.837	2.393 $\pm$ 0.278	7.540	2.314 $\pm$ 0.067	8.141	2.635 $\pm$ 0.18
	AIME-25	7.937	1.758 $\pm$ 0.149	8.470	1.662 $\pm$ 0.145	8.306	1.573 $\pm$ 0.137	8.916	1.714 $\pm$ 0.15
	GSM8K	4.529	3.071 $\pm$ 0.374	4.692	3.496 $\pm$ 0.339	4.543	2.897 $\pm$ 0.242	5.984	3.404 $\pm$ 0.25
Code	HumanEval	5.126	4.186 $\pm$ 0.092	5.545	4.523 $\pm$ 0.059	5.254	4.258 $\pm$ 0.179	6.454	4.490 $\pm$ 0.39
	MBPP	5.293	3.929 $\pm$ 0.129	5.806	4.276 $\pm$ 0.091	5.503	4.000 $\pm$ 0.201	5.560	4.274 $\pm$ 0.43

Appendix

Table 11: Mean accepted length

\tau

, Llama-3.1-8B, thinking off , companion to Table 10 . Domino attains best

\tau

on some datasets here, yet the lower average throughput (Table 10 ), the

\tau

-vs-TPS trade at the aggregate level.

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	343.2	1043.3 (3.04 $\times$ )	1138.2 (3.32 $\times$ )	1050.8 (3.06 $\times$ )	1231.8 (3.59 $\times$ )
	AIME-25	342.9	928.1 (2.71 $\times$ )	941.5 (2.75 $\times$ )	906.7 (2.64 $\times$ )	1074.5 (3.13 $\times$ )
	GSM8K	348.0	733.5 (2.11 $\times$ )	728.3 (2.09 $\times$ )	698.2 (2.01 $\times$ )	854.8 (2.46 $\times$ )
Code	HumanEval	335.2	874.8 (2.61 $\times$ )	880.9 (2.63 $\times$ )	847.7 (2.53 $\times$ )	1003.8 (3.00 $\times$ )
	MBPP	333.5	759.5 (2.28 $\times$ )	779.0 (2.34 $\times$ )	734.6 (2.20 $\times$ )	974.0 (2.92 $\times$ )
	LiveCodeBench	338.9	684.5 (2.02 $\times$ )	743.9 (2.20 $\times$ )	698.9 (2.06 $\times$ )	890.8 (2.63 $\times$ )

Appendix

Table 12: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Qwen3-Coder-30B-A3B, thinking off , concurrency 1, max 2048 generated tokens. Same format as Table 1 .

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	6.372	5.639 $\pm$ 0.062	6.947	6.017 $\pm$ 0.104	6.536	5.626 $\pm$ 0.055	7.224	6.221 $\pm$ 0.016
	AIME-25	5.566	4.276 $\pm$ 0.122	6.057	4.439 $\pm$ 0.091	5.695	4.309 $\pm$ 0.091	6.312	4.693 $\pm$ 0.114
	GSM8K	4.717	4.457 $\pm$ 0.138	4.998	4.616 $\pm$ 0.109	4.652	4.372 $\pm$ 0.029	5.322	5.036 $\pm$ 0.064
Code	HumanEval	5.805	5.279 $\pm$ 0.155	6.189	5.598 $\pm$ 0.031	5.789	5.285 $\pm$ 0.044	6.435	5.803 $\pm$ 0.103
	MBPP	4.981	4.787 $\pm$ 0.023	5.391	5.208 $\pm$ 0.048	4.989	4.828 $\pm$ 0.029	6.141	4.987 $\pm$ 0.063

Appendix

Table 13: Mean accepted length

\tau

, Qwen3-Coder-30B-A3B, thinking off , companion to Table 12 .

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	366.2	1764.7 (4.82 $\times$ )	1974.6 (5.39 $\times$ )	1599.5 (4.37 $\times$ )	2165.5 (5.91 $\times$ )
	AIME-25	364.4	1640.5 (4.50 $\times$ )	1922.1 (5.27 $\times$ )	1566.0 (4.30 $\times$ )	2085.9 (5.72 $\times$ )
	GSM8K	369.6	1330.9 (3.60 $\times$ )	1450.5 (3.93 $\times$ )	1127.0 (3.05 $\times$ )	1638.4 (4.43 $\times$ )
Code	HumanEval	370.6	1226.4 (3.31 $\times$ )	1348.7 (3.64 $\times$ )	1075.4 (2.90 $\times$ )	1475.8 (3.98 $\times$ )
	MBPP	370.2	1137.6 (3.07 $\times$ )	1267.5 (3.42 $\times$ )	978.6 (2.64 $\times$ )	1435.3 (3.88 $\times$ )
	LiveCodeBench	369.2	1482.9 (4.02 $\times$ )	1620.5 (4.39 $\times$ )	1309.8 (3.55 $\times$ )	1754.1 (4.75 $\times$ )

Appendix

Table 14: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Qwen3-4B, thinking off , concurrency 1, max 2048 generated tokens. Same format as Table 1 .

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	6.443	5.104 $\pm$ 0.039	7.719	5.799 $\pm$ 0.119	6.125	4.910 $\pm$ 0.068	7.967	6.041 $\pm$ 0.098
	AIME-25	5.996	3.894 $\pm$ 0.084	7.176	4.520 $\pm$ 0.067	5.729	3.872 $\pm$ 0.068	7.368	4.623 $\pm$ 0.178
	GSM8K	4.872	4.604 $\pm$ 0.036	5.825	5.422 $\pm$ 0.084	4.474	4.230 $\pm$ 0.021	6.082	5.610 $\pm$ 0.065
Code	HumanEval	4.416	4.163 $\pm$ 0.012	5.263	4.904 $\pm$ 0.034	4.091	3.848 $\pm$ 0.021	5.351	4.949 $\pm$ 0.046
	MBPP	4.270	3.812 $\pm$ 0.084	5.085	4.508 $\pm$ 0.062	3.933	3.541 $\pm$ 0.004	5.297	4.603 $\pm$ 0.007

Appendix

Table 15: Mean accepted length

\tau

, Qwen3-4B, thinking off , companion to Table 14 . DCOCO leads

\tau

in 19 of 20 cells and both averages.

Domain	Dataset	No-spec	DFlash	Domino	DSpark	DCOCO
Math	MATH-500	361.0	868.8 (2.41 $\times$ )	921.7 (2.55 $\times$ )	785.8 (2.18 $\times$ )	1000.4 (2.77 $\times$ )
	AIME-25	359.5	861.5 (2.40 $\times$ )	909.9 (2.53 $\times$ )	755.0 (2.10 $\times$ )	989.8 (2.75 $\times$ )
	GSM8K	362.0	846.8 (2.34 $\times$ )	869.7 (2.40 $\times$ )	735.9 (2.03 $\times$ )	974.0 (2.69 $\times$ )
Code	HumanEval	357.9	824.4 (2.30 $\times$ )	875.9 (2.45 $\times$ )	739.7 (2.07 $\times$ )	934.4 (2.61 $\times$ )
	MBPP	359.5	797.0 (2.22 $\times$ )	836.3 (2.33 $\times$ )	731.1 (2.03 $\times$ )	923.9 (2.57 $\times$ )
	LiveCodeBench	361.2	780.1 (2.16 $\times$ )	842.8 (2.33 $\times$ )	706.5 (1.96 $\times$ )	896.5 (2.48 $\times$ )

Appendix

Table 16: Throughput (output tokens/s, per-stream) and speedup over no-spec decoding, Qwen3-4B, thinking on , concurrency 1, max 4096 generated tokens. Same format as Table 1 .

		DFlash		Domino		DSpark		DCOCO
Domain	Dataset	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$	$T{=}0$	$T{=}1$
Math	MATH-500	3.216	2.957 $\pm$ 0.008	3.607	3.289 $\pm$ 0.005	2.969	2.771 $\pm$ 0.001	3.644	3.317 $\pm$ 0.012
	AIME-25	3.140	2.845 $\pm$ 0.032	3.520	3.134 $\pm$ 0.026	2.900	2.664 $\pm$ 0.017	3.568	3.234 $\pm$ 0.020
	GSM8K	3.099	2.753 $\pm$ 0.008	3.437	3.030 $\pm$ 0.010	2.848	2.563 $\pm$ 0.018	3.538	3.088 $\pm$ 0.010
Code	HumanEval	3.100	2.689 $\pm$ 0.010	3.517	3.020 $\pm$ 0.020	2.858	2.524 $\pm$ 0.012	3.451	2.968 $\pm$ 0.002
	MBPP	2.961	2.564 $\pm$ 0.004	3.337	2.850 $\pm$ 0.005	2.749	2.425 $\pm$ 0.012	3.327	2.847 $\pm$ 0.000

Appendix

Table 17: Mean accepted length

\tau

, Qwen3-4B, thinking on , companion to Table 16 .

	top-1	top-4	top-8	top-16
Qwen3-8B, mean over drafted slots	0.400	0.587	0.676	0.755
first drafted slot	0.814	0.915	0.936	0.951
last drafted slot	0.154	0.355	0.483	0.608
Qwen3-4B, mean over drafted slots	0.413	0.574	0.651	0.722
first drafted slot	0.815	0.907	0.928	0.943
last drafted slot	0.169	0.348	0.461	0.573

Appendix

Table 18: Coverage by verifier : means over the 15 drafted slots of Figure 1 , with the first and last slot shown per verifier. Top-1 is the drafter’s own per-slot accuracy; DCOCO uses

K{=}16

Figure 5: At high concurrency, DCOCO achieves the highest throughput, while Domino and DSpark fall below DFlash. Qwen3-4B throughput at (a)

T=0

and (b)

T=1

, averaged over the ten suites. (c) DCOCO ’s relative throughput gain over the strongest baseline at each concurrency. DCOCO ’s margin is

10

–

14\%

T{=}0

and

7

–

11\%

T{=}1

Figure 6: DCOCO achieves the highest throughput at every concurrency, while Domino and DSpark stay at or below DFlash. Qwen3-Coder-30B-A3B throughput at (a)

T=0

and (b)

T=1

, averaged over the ten suites. (c) DCOCO ’s relative throughput gain over the strongest baseline at each concurrency. DCOCO ’s margin is

10

–

14\%

T{=}0

and

8

–

10\%

T{=}1

Figure 7: DCOCO achieves the highest throughput at every concurrency, while Domino falls increasingly behind DFlash as load grows. Llama-3.1-8B throughput at (a)

T=0

and (b)

T=1

, averaged over the ten suites. (c) DCOCO ’s relative throughput gain over the strongest baseline at each concurrency. DCOCO ’s margin is

9

–

16\%

T{=}0

and

6

–

12\%

T{=}1

Figure 8: Domino’s acceptance gain is consumed by per-cycle overhead as concurrency grows, and DSpark never matches DFlash’s acceptance, while DCOCO remains faster than DFlash. Accepted length

\tau

(solid) and throughput (dashed), relative to DFlash; Shading highlights the separation between the accepted-length and throughput ratios.. Qwen3-4B, thinking off,

T=0

(dark) and

T=1

(light), averaged over all ten benchmark suites. Absolute values: Tab. 20 .

Figure 9: At high concurrency, overhead outweighs Domino’s acceptance gain and DSpark gains no acceptance over DFlash, while DCOCO remains faster than DFlash. Accepted length

\tau

(solid) and throughput (dashed), relative to DFlash; Shading highlights the separation between the accepted-length and throughput ratios. Qwen3-Coder-30B-A3B, thinking off,

T=0

(dark) and

T=1

(light), averaged over all ten benchmark suites. Absolute values: Tab. 21 .

Figure 10: As concurrency grows, Domino’s acceptance gain is consumed by per-cycle overhead, while DCOCO remains faster than DFlash. Accepted length

\tau

(solid) and throughput (dashed), relative to DFlash; Shading highlights the separation between the accepted-length and throughput ratios. Llama-3.1-8B, thinking off,

T=0

(dark) and

T=1

(light), averaged over all ten benchmark suites. Absolute values: Tab. 22 .

Concurrency	DFlash	Domino	DSpark	DCOCO	DCOCO / DFlash	DCOCO vs. best
Greedy decoding, $T{=}0$
1	783.1	839.3	765.6	959.7	$1.23\times$	$+14.3\%$
4	2806.4	3008.6	2719.0	3507.3	$1.25\times$	$+16.6\%$
8	4889.1	5051.0	4614.4	6082.6	$1.24\times$	$+20.4\%$
16	7229.1	7098.4	7092.9	8894.3	$1.23\times$	$+23.0\%$
32	9655.0	9013.9	9490.3	11512.4	$1.19\times$	$+19.2\%$

Appendix

Table 19: Aggregate throughput (output tokens/s) as concurrency grows , Qwen3-8B, thinking off, averaged over the ten suites per concurrency workload.

Concurrency	DFlash	Domino	DSpark	DCOCO	DCOCO / DFlash	DCOCO vs. best
Greedy decoding, $T{=}0$
1	1096.9	1202.7	980.1	1332.6	$1.21\times$	$+10.8\%$
4	4097.1	4032.1	3517.5	4660.3	$1.14\times$	$+13.7\%$
8	6942.6	6560.9	5953.9	7781.7	$1.12\times$	$+12.1\%$
16	10940.3	9633.9	9448.6	12215.5	$1.12\times$	$+11.7\%$
32	14181.1	11966.5	12746.7	15602.7	$1.10\times$	$+10.0\%$

Appendix

Table 20: Aggregate throughput (output tokens/s) as concurrency grows , Qwen3-4B, thinking off, averaged over the ten suites. Bold: best method per row.

Concurrency	DFlash	Domino	DSpark	DCOCO	DCOCO / DFlash	DCOCO vs. best
Greedy decoding, $T{=}0$
1	642.4	660.1	627.2	785.1	$1.22\times$	$+18.9\%$
4	1862.6	1852.5	1812.2	2077.2	$1.12\times$	$+11.5\%$
8	3110.6	3055.5	2972.4	3550.5	$1.14\times$	$+14.1\%$
16	4886.7	4610.1	4784.7	5477.8	$1.12\times$	$+12.1\%$
32	7155.2	6606.8	7057.2	7867.9	$1.10\times$	$+10.0\%$

Appendix

Table 21: Aggregate throughput (output tokens/s) as concurrency grows , Qwen3-Coder-30B-A3B, thinking off, averaged over the ten suites. Bold: best method per row.

Concurrency	DFlash	Domino	DSpark	DCOCO	DCOCO / DFlash	DCOCO vs. best
Greedy decoding, $T{=}0$
1	844.2	900.8	838.0	1042.4	$1.23\times$	$+15.7\%$
4	3128.0	3139.2	3145.0	3659.2	$1.17\times$	$+16.3\%$
8	5388.7	5217.4	5367.7	6137.4	$1.14\times$	$+13.9\%$
16	8007.2	7379.9	7940.5	8916.8	$1.11\times$	$+11.4\%$
32	10730.3	9435.8	10660.3	11731.7	$1.09\times$	$+9.3\%$

Appendix

Table 22: Aggregate throughput (output tokens/s) as concurrency grows , Llama-3.1-8B, thinking off, averaged over the ten suites. Bold: best method per row.

Speculative Decoding

Speculative decoding accelerates generation by drafting multiple tokens and verifying them in parallel with a larger target model.

See all 291 papers

Sep 23, 2026

When Parallel Drafter Meets Parallel Speculative Decoding

DSpark-style parallel drafters have made speculative decoding highly effective, yet their draft phase remains serialized on the critical path of every round. Parallel speculative decoding (PSD) overlaps drafting with verification, yet existing methods must guess the accepted prefix and bonus token in advance: a wrong guess reverts the whole batch to serial drafting. We present DPara, a PSD framework that reuses effective parallel drafters yet guarantees backbone--verification overlap in every round, thereby eliminating this probabilistic fallback altogether. While the target verifies, DPara's diffusion backbone precomputes draft representations for every acceptance boundary with the bonus left unspecified; a lightweight autoregressive head then combines the revealed verification outcome with the matching precomputed representation to emit the next round's draft tokens almost instantly---fully parallelizing the dominant backbone forward with verification and leaving only the negligible head cost serial. Experiments on Qwen3-8B and Qwen3-14B across seven math, coding, and chat benchmarks show that DPara achieves average speedups of

3.21\times

and

3.52\times

over autoregressive decoding, surpassing the strongest serial and parallel speculative decoding baselines alike.

Fuliang Liu, Xue Li, Kun Qian +4

State Key Laboratory of Novel Software Technology, Nanjing University · Alibaba Group

Parallel Speculative Decoding Speculative Decoding LLM Inference Acceleration Parallel Decoding Diffusion Language Model Decoding

Ask PDF

Jul 8, 2026

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.

Hong-Kai Zheng, Piji Li

College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, China

Speculative Decoding LLM Inference Acceleration Parallel Speculative Decoding

Ask PDF

Aug 20, 2026

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

Speculative decoding accelerates language-model inference by drafting future tokens the target model verifies in parallel. A diffusion-style drafter such as DFlash drafts an entire block in one forward pass. It is trained on the per-position marginals rather than on the joint distribution over the block, so the tokens it emits are individually plausible yet jointly incoherent. We introduce LiLiCorr, a Lightweight Likelihood-based model that Correlates the per-position marginals such a drafter produces. It keeps the top-K tokens at each position and processes them jointly, emitting an in and an out vector for each. Two candidates at consecutive positions match when the earlier out vector aligns, in cosine similarity, with the later in vector. Training scores the correct pairings highest and pushes competing ones down, so coherent blocks outscore incoherent ones. The joint distribution over the block, exponential in its length, is never materialized. One lightweight network pass produces all the vectors, the pairwise scores follow as batched matrix operations, leaving only a cheap greedy walk sequential. We co-train the DFlash drafter with LiLiCorr, so it proposes candidates that correlate into longer accepted sequences. Over the vanilla DFlash drafter it builds on, LiLiCorr accepts more and serves faster at all 72 settings we test: nine benchmarks at two target sizes under greedy and temperature-one decoding, plus a throughput sweep over six concurrencies, two input lengths and three output-entropy tiers. It raises acceptance length by 7 to 19%, while its single-pass scoring head costs only about 3% of the per-block latency. Against three concurrently developed methods that also restore coherence at draft time, all equally optimized on a common stack, LiLiCorr holds the highest throughput in 63 of those settings, ties within a measured noise floor in 6, and trails in only 3.

Matan Rusanovsky, Yoav Miron, Roy Uziel +5

NVIDIA

Speculative Decoding LLM Inference Acceleration Diffusion Language Model Decoding

Ask PDF

Abstract

Speculative Decoding

Explore similar work

When Parallel Drafter Meets Parallel Speculative Decoding

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding