Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

Yanxiao Liu; Sicheng Wan; Zhan Gao; Deniz Gündüz

cs.CRSep 18, 2026

Watermarkable Multi-Draft Speculative Sampling via Poisson Processes

Authors: Yanxiao Liu, Sicheng Wan, Zhan Gao, Deniz Gündüz

Organizations: Imperial College London · University of Washington

Abstract

Large language models (LLMs) have achieved state-of-the-art performance across a wide range of tasks, motivating two important aspects of deployment: inference efficiency and output provenance, which can be tackled by speculative sampling and watermarking, respectively. However, recent works have shown that combining these two goals is highly nontrivial and can be potentially impossible. In this work, we develop a novel multi-draft speculative sampling algorithm based on Poisson processes that improves the frontier of this fundamental trade-off. The proposed algorithm has strong sampling efficiency on its own and, more interestingly, is naturally watermarkable: we can embed an unbiased watermark without degrading speculative acceptance. Moreover, our algorithm is based on an exact list-coupling-without-communication scheme, which yields a drafter invariance property that benefits both sampling and watermarking. It is the first multi-draft, drafter-invariant speculative sampling scheme that maintains both watermark strength and sampling efficiency, and we experimentally verify its strong performance in both aspects.

Figures & tables

Table 1

Figure 1: The trade-off between watermark and efficiency on existing methods ( Hu and Huang, 2024 ) : Vanilla Speculative Sampling (VSpS), Maintain Watermark Strength (MWS), and Maintain Sampling Efficiency (MSE), and our algorithms (PFR, and multi-draft MPFR that is presented in Section 5 ). For both PFR (lower-left) and MPFR (upper), we embed watermark (measured by Average Negative Log P-value Per Token (ANLPPT)) without degrading the Average Accepted Tokens Per Step (AATPS). This figure uses Qwen2.5-7B-Instruct and Qwen2.5-0.5B-Instruct as the target and draft, respectively, on dataset cnn_dailymail . Experiments on more models and datasets are shown in Appendix D .

		Qwen-CNN/ DM		Qwen- ELI5		Llama-CNN/ DM		Llama- ELI5
Decoder	$B$	AATPS	TR	AATPS	TR	AATPS	TR	AATPS	TR
MPFR	2	$2.797$	$\mathbf{25.11}$	$\mathbf{2.563}$	$\mathbf{23.57}$	$\mathbf{3.459}$	$\mathbf{37.44}$	$\mathbf{3.103}$	$\mathbf{34.29}$
MPFR	4	$\mathbf{3.123}$	$\mathbf{27.50}$	$\mathbf{2.918}$	$\mathbf{26.46}$	$\mathbf{3.784}$	$\mathbf{40.20}$	$\mathbf{3.455}$	$\mathbf{37.79}$
MPFR	6	$\mathbf{3.294}$	$\mathbf{28.50}$	$\mathbf{3.109}$	$\mathbf{27.81}$	$\mathbf{3.935}$	$\mathbf{41.71}$	$\mathbf{3.635}$	$\mathbf{39.59}$
MPFR	8	$\mathbf{3.409}$	$\mathbf{29.13}$	$\mathbf{3.235}$	$\mathbf{28.73}$	$\mathbf{4.034}$	$\mathbf{42.04}$	$\mathbf{3.749}$	$\mathbf{39.76}$
GLS	2	$\mathbf{2.803}$	$24.78$	$2.523$	$22.73$	$3.431$	$36.79$	$3.080$	$33.72$

Table 1: Multi-draft results on Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct on CNN/ DailyMail and ELI5 ,

L=4

. We report the average accepted tokens per step (AATPS) and token rate (TR). The better result between MPFR and GLS for each setting with the same

B

is in bold. Each cell is computed over the

1000

prompts and

3

different seeds. See more detailed data in Appendix D.4 .

Figure 2: Detection and robustness to drafter substitution. (a) TPR@1%FPR versus detection token budget $T_{\mathrm{eval}}$ at the default drafter. PFR and MPFR match the strong-watermark references Basic-UWM and MWS , and outperform MSE / MSE-Pseudo . (b) TPR@1%FPR under drafter substitution: the target, watermark key, and prompts are fixed and only the drafter is swapped. PFR drifts by $<\!1$ point, whereas MSE drifts by $12$ – $22$ points and MSE-Pseudo by $6$ – $11$ points.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

(S_{i_{1}},i_{1},Z_{i_{1}}),\ldots,(S_{i_{B}},i_{B},Z_{i_{B}})

Appendix

Algorithm 5 Finite implementation of the

B

-sample mapped PFR

(S_{(1)},U_{(1)},J_{(1)}),\ldots,(S_{(B)},U_{(B)},J_{(B)})

Appendix

Algorithm 6 Direct finite-support implementation of

B

-sample MPFR

(X_{1}(c),\ldots,X_{\nu(c)}(c))\leftarrow\textsc{MPFR}(Q_{k}(\cdot|c),\Pi_{\mathsf{k}}(c),\nu(c)).

Appendix

Algorithm 7 Batched-target direct-top-

k

MPFR speculative sampling

Figure 3: Single-draft (AATPS, ANLPPT-U) frontier on Qwen2.5-7B-Instruct / Qwen2.5-0.5B-Instruct $\times$ ELI5 . PFR dominates MWS / MSE simultaneously on both axes; attaching the watermark does not decrease AATPS.

Figure 4: Single-draft (AATPS, ANLPPT-U) frontier on Vicuna-7B-v1.5 / Vicuna-68m $\times$ CNN/DailyMail . PFR dominates MWS / MSE simultaneously on both axes; attaching the watermark does not decrease AATPS.

Figure 5: Single-draft (AATPS, ANLPPT-U) frontier on Vicuna-7B-v1.5 / Vicuna-68m $\times$ ELI5 . PFR dominates MWS / MSE simultaneously on both axes; attaching the watermark does not decrease AATPS.

$L$	Method	AATPS	Token rate	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
1	Basic-UWM	$1.000\pm 0.000$	$33.353\pm 2.815$	$0.048\pm 0.035$	$0.035\pm 0.029$	$0.051\pm 0.036$	$0.473\pm 0.134$
	VSpS	$1.641\pm 0.070$	$33.229\pm 1.944$	$0.003\pm 0.005$	$0.003\pm 0.006$	$0.003\pm 0.006$	$0.477\pm 0.136$
	PFR-NoWM	$1.623\pm 0.075$	$32.147\pm 2.161$	$0.002\pm 0.006$	$0.003\pm 0.006$	$0.002\pm 0.006$	$0.474\pm 0.140$
	PFR	$1.625\pm 0.072$	$33.321\pm 1.706$	$0.051\pm 0.036$	$0.036\pm 0.030$	$0.053\pm 0.038$	$0.470\pm 0.143$
	MSE	$1.642\pm 0.070$	$20.578\pm 2.645$	$0.023\pm 0.021$	$0.016\pm 0.017$	$0.025\pm 0.023$	$0.472\pm 0.136$
	MWS	$1.621\pm 0.072$	$20.136\pm 2.952$	$0.049\pm 0.035$	$0.034\pm 0.029$	$0.052\pm 0.036$	$0.475\pm 0.133$

Appendix

Table 2: Single-draft results on Qwen2.5-7B-Instruct / Qwen2.5-0.5B-Instruct

\times

CNN/DailyMail . Each cell:

\mathrm{mean}\pm\mathrm{std}

across

1000

prompts.

$L$	Method	AATPS	Token rate	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
1	Basic-UWM	$1.000\pm 0.000$	$26.132\pm 1.924$	$0.100\pm 0.045$	$0.072\pm 0.039$	$0.104\pm 0.046$	$0.696\pm 0.141$
	VSpS	$1.593\pm 0.060$	$32.090\pm 1.889$	$0.002\pm 0.005$	$0.002\pm 0.005$	$0.002\pm 0.005$	$0.701\pm 0.133$
	PFR-NoWM	$1.567\pm 0.061$	$33.046\pm 1.405$	$0.002\pm 0.004$	$0.002\pm 0.004$	$0.002\pm 0.004$	$0.703\pm 0.139$
	PFR	$1.570\pm 0.062$	$33.213\pm 1.487$	$0.104\pm 0.048$	$0.075\pm 0.039$	$0.107\pm 0.049$	$0.699\pm 0.134$
	MSE	$1.595\pm 0.060$	$18.218\pm 2.022$	$0.048\pm 0.030$	$0.032\pm 0.024$	$0.053\pm 0.032$	$0.704\pm 0.135$
	MWS	$1.565\pm 0.063$	$20.177\pm 2.688$	$0.102\pm 0.047$	$0.075\pm 0.040$	$0.106\pm 0.048$	$0.700\pm 0.137$

Appendix

Table 3: Single-draft results on Qwen2.5-7B-Instruct / Qwen2.5-0.5B-Instruct

\times

ELI5 . Each cell:

\mathrm{mean}\pm\mathrm{std}

across

1000

prompts.

$L$	Method	AATPS	Token rate	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
1	Basic-UWM	$1.000\pm 0.000$	$37.058\pm 0.848$	$0.021\pm 0.024$	$0.016\pm 0.019$	$0.022\pm 0.025$	$0.274\pm 0.190$
	VSpS	$1.557\pm 0.086$	$48.890\pm 3.439$	$0.002\pm 0.006$	$0.003\pm 0.006$	$0.002\pm 0.006$	$0.275\pm 0.167$
	PFR-NoWM	$1.544\pm 0.088$	$37.027\pm 5.064$	$0.002\pm 0.006$	$0.003\pm 0.006$	$0.002\pm 0.005$	$0.269\pm 0.121$
	PFR	$1.550\pm 0.090$	$38.256\pm 6.092$	$0.022\pm 0.024$	$0.017\pm 0.019$	$0.023\pm 0.025$	$0.272\pm 0.128$
	MSE	$1.557\pm 0.088$	$45.513\pm 4.490$	$0.009\pm 0.013$	$0.007\pm 0.011$	$0.010\pm 0.015$	$0.269\pm 0.123$
	MWS	$1.547\pm 0.091$	$30.814\pm 4.078$	$0.022\pm 0.024$	$0.017\pm 0.020$	$0.023\pm 0.026$	$0.276\pm 0.192$

Appendix

Table 4: Single-draft results on Vicuna-7B-v1.5 / Vicuna-68m

\times

CNN/DailyMail . Each cell:

\mathrm{mean}\pm\mathrm{std}

across

1000

prompts.

$L$	Method	AATPS	Token rate	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
1	Basic-UWM	$1.000\pm 0.000$	$38.536\pm 1.896$	$0.060\pm 0.040$	$0.044\pm 0.033$	$0.062\pm 0.042$	$0.507\pm 0.152$
	VSpS	$1.410\pm 0.063$	$46.306\pm 3.178$	$0.002\pm 0.005$	$0.003\pm 0.005$	$0.002\pm 0.005$	$0.511\pm 0.150$
	PFR-NoWM	$1.399\pm 0.063$	$33.636\pm 4.757$	$0.003\pm 0.006$	$0.003\pm 0.007$	$0.002\pm 0.005$	$0.509\pm 0.152$
	PFR	$1.397\pm 0.065$	$35.231\pm 5.397$	$0.061\pm 0.043$	$0.045\pm 0.038$	$0.063\pm 0.042$	$0.513\pm 0.162$
	MSE	$1.415\pm 0.062$	$44.719\pm 3.192$	$0.020\pm 0.020$	$0.014\pm 0.017$	$0.022\pm 0.022$	$0.509\pm 0.147$
	MWS	$1.399\pm 0.064$	$26.309\pm 3.816$	$0.060\pm 0.041$	$0.044\pm 0.034$	$0.062\pm 0.042$	$0.507\pm 0.151$

Appendix

Table 5: Single-draft results on Vicuna-7B-v1.5 / Vicuna-68m

\times

ELI5 . Each cell:

\mathrm{mean}\pm\mathrm{std}

across

1000

prompts.

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	$2.797\pm 0.001$	$25.11\pm 0.22$	$0.051\pm 0.001$	$0.036\pm 0.001$	$0.053\pm 0.001$	$0.474\pm 0.002$
MPFR	4	$3.123\pm 0.002$	$27.50\pm 0.31$	$0.051\pm 0.001$	$0.036\pm 0.001$	$0.053\pm 0.001$	$0.474\pm 0.002$
MPFR	6	$3.294\pm 0.003$	$28.50\pm 0.49$	$0.050\pm 0.001$	$0.036\pm 0.001$	$0.053\pm 0.001$	$0.474\pm 0.001$
MPFR	8	$3.409\pm 0.002$	$29.13\pm 0.59$	$0.051\pm 0.002$	$0.036\pm 0.001$	$0.053\pm 0.002$	$0.474\pm 0.002$
Invariant	2	$2.803\pm 0.007$	$24.78\pm 0.14$	$0.002\pm 0.000$	$0.003\pm 0.000$	$0.002\pm 0.000$	$0.483\pm 0.005$
Invariant	4	$3.109\pm 0.008$	$27.12\pm 0.20$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.484\pm 0.003$

Appendix

Table 6: Multi-draft results on Qwen2.5-7B-Instruct

\times

CNN/DailyMail ,

L{=}4

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	$2.563\pm 0.007$	$23.57\pm 0.15$	$0.104\pm 0.001$	$0.075\pm 0.001$	$0.108\pm 0.001$	$0.705\pm 0.001$
MPFR	4	$2.918\pm 0.008$	$26.46\pm 0.25$	$0.104\pm 0.001$	$0.076\pm 0.001$	$0.108\pm 0.001$	$0.706\pm 0.000$
MPFR	6	$3.109\pm 0.005$	$27.81\pm 0.35$	$0.103\pm 0.001$	$0.075\pm 0.001$	$0.107\pm 0.001$	$0.705\pm 0.000$
MPFR	8	$3.235\pm 0.010$	$28.73\pm 0.49$	$0.104\pm 0.002$	$0.075\pm 0.002$	$0.108\pm 0.001$	$0.706\pm 0.001$
Invariant	2	$2.523\pm 0.004$	$22.73\pm 0.22$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.747\pm 0.005$
Invariant	4	$2.850\pm 0.010$	$25.60\pm 0.26$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.736\pm 0.002$

Appendix

Table 7: Multi-draft results on Qwen2.5-7B-Instruct

\times

ELI5 ,

L{=}4

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	$3.459\pm 0.015$	$37.44\pm 0.57$	$0.058\pm 0.001$	$0.041\pm 0.000$	$0.061\pm 0.001$	$0.514\pm 0.002$
MPFR	4	$3.784\pm 0.013$	$40.20\pm 0.28$	$0.058\pm 0.000$	$0.041\pm 0.000$	$0.062\pm 0.001$	$0.514\pm 0.003$
MPFR	6	$3.935\pm 0.005$	$41.71\pm 0.30$	$0.058\pm 0.000$	$0.041\pm 0.000$	$0.061\pm 0.001$	$0.514\pm 0.003$
MPFR	8	$4.034\pm 0.005$	$42.04\pm 0.81$	$0.058\pm 0.001$	$0.041\pm 0.000$	$0.061\pm 0.001$	$0.514\pm 0.003$
Invariant	2	$3.431\pm 0.005$	$36.79\pm 0.37$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.514\pm 0.003$
Invariant	4	$3.741\pm 0.007$	$39.31\pm 0.29$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.508\pm 0.001$

Appendix

Table 8: Multi-draft results on Llama-3.1-8B-Instruct

\times

CNN/ DailyMail ,

L=4

. Mean

\pm

std over 3 random seeds,

1000

prompts each. The drafter is Llama-3.2-1B-Instruct, the official distillation of the target sharing the same tokenizer.

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	$3.103\pm 0.006$	$34.29\pm 0.35$	$0.135\pm 0.001$	$0.099\pm 0.001$	$0.139\pm 0.001$	$0.811\pm 0.004$
MPFR	4	$3.455\pm 0.010$	$37.79\pm 0.38$	$0.135\pm 0.002$	$0.099\pm 0.001$	$0.139\pm 0.002$	$0.812\pm 0.003$
MPFR	6	$3.635\pm 0.001$	$39.59\pm 0.43$	$0.135\pm 0.002$	$0.099\pm 0.001$	$0.139\pm 0.002$	$0.812\pm 0.005$
MPFR	8	$3.749\pm 0.002$	$39.76\pm 0.80$	$0.135\pm 0.001$	$0.098\pm 0.000$	$0.139\pm 0.001$	$0.811\pm 0.004$
Invariant	2	$3.080\pm 0.011$	$33.72\pm 0.60$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.803\pm 0.003$
Invariant	4	$3.408\pm 0.006$	$37.11\pm 0.12$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.002\pm 0.000$	$0.794\pm 0.004$

Appendix

Table 9: Multi-draft results on Llama-3.1-8B-Instruct

\times

ELI5,

L=4

. Mean

\pm

std over 3 random seeds,

1000

prompts each.

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	2.799	23.18	0.052	0.037	0.055	0.475
MPFR	4	3.124	24.89	0.052	0.037	0.055	0.474
MPFR	6	3.298	25.39	0.052	0.037	0.055	0.476
MPFR	8	3.410	25.17	0.053	0.037	0.055	0.476
Invariant	2	2.804	25.03	0.003	0.003	0.002	0.487
Invariant	4	3.114	26.63	0.002	0.002	0.002	0.481

Appendix

Table 10: Multi-draft results on Qwen2.5-7B-Instruct

\times

CNN/DailyMail ,

L{=}4

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	2.563	22.38	0.104	0.076	0.108	0.704
MPFR	4	2.919	24.48	0.104	0.076	0.107	0.703
MPFR	6	3.111	25.54	0.105	0.076	0.108	0.706
MPFR	8	3.237	25.83	0.105	0.076	0.108	0.704
Invariant	2	2.519	23.18	0.002	0.002	0.002	0.750
Invariant	4	2.852	25.75	0.002	0.002	0.002	0.733

Appendix

Table 11: Multi-draft results on Qwen2.5-7B-Instruct

\times

ELI5 ,

L{=}4

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	2.422	52.10	0.022	0.017	0.022	0.270
MPFR	4	2.664	48.28	0.022	0.017	0.023	0.271
MPFR	6	2.803	44.44	0.022	0.017	0.023	0.270
MPFR	8	2.899	40.76	0.022	0.017	0.023	0.270
Invariant	2	2.478	58.15	0.002	0.002	0.002	0.287
Invariant	4	2.690	53.12	0.002	0.002	0.002	0.286

Appendix

Table 12: Multi-draft results on Vicuna-7B-v1.5

\times

CNN/DailyMail ,

L{=}4

Decoder	$B$	AATPS	TR	ANLPPT-U	ANLPPT-Li	ANLPPT-PL	LPPL
MPFR	2	1.889	45.89	0.058	0.043	0.061	0.504
MPFR	4	2.133	47.43	0.058	0.042	0.061	0.504
MPFR	6	2.278	47.42	0.058	0.042	0.062	0.503
MPFR	8	2.377	46.42	0.059	0.043	0.062	0.504
Invariant	2	1.884	50.29	0.002	0.002	0.002	0.555
Invariant	4	2.108	53.42	0.002	0.002	0.002	0.552

Appendix

Table 13: Multi-draft results on Vicuna-7B-v1.5

\times

ELI5 ,

L{=}4

Figure 6: Full detection across $(\textsc{target},\textsc{dataset})$ cells. TPR@1%FPR (Aaronson Gamma-tail) versus detection token budget $T_{\mathrm{eval}}$ at the default drafter $D_{0}$ , for the seven schemes plotted in Figure 2 (a). Each panel is one cell of the main $2{\times}2$ result: target $\in$ {Qwen2.5-7B-Instruct, Vicuna-7B-v1.5} $\times$ dataset $\in$ { CNN/DailyMail , ELI5 }. PFR / MPFR (ours) match the strong-watermark ceiling ( Basic-UWM , MWS ) in every cell, while MSE and MSE-Pseudo are uniformly weaker. The empirical $H_{0}$ (no-watermark) floor sits near the nominal $1\%$ FPR, calibrating the test.

	Single-draft baselines							Multi-draft ( $B{=}4$ )
Metric	VSpS	PFR-NoWM	Basic-UWM	MSE	MWS	MSE-Pseudo	PFR	Invariant	MPFR
LPPL $\downarrow$	0.474	0.469	0.472	0.470	0.475	0.477	0.465	0.484	0.474
ROUGE-L $\uparrow$	0.219	0.222	0.223	0.221	0.223	0.219	0.223	0.222	0.222
AATPS $\uparrow$	2.51	2.46	1.00	2.53	2.43	2.52	2.46	3.12	3.13

Appendix

Table 14: Output quality. LPPL

\downarrow

under target. ROUGE-L

\uparrow

vs. gold highlights . AATPS

\uparrow

accepted target tokens per step.

Decoder	min-pair mean ROUGE-L
Basic-UWM (no drafter, vacuous bound)	1.000
PFR (ours)	0.990
MWS	0.922
PFR-NoWM (no wm)	0.436
MSE-Pseudo	0.426
MSE	0.406

Appendix

Table 15: Output stability under drafter substitution. Pairwise ROUGE-L F1 between the realised token sequences across each

(D_{i},D_{j})

pair, per-prompt minimum averaged over prompts. An unbiased verify rule produces identical sequences across all drafters, so ROUGE-L

\to 1

Speculative Decoding

Speculative decoding accelerates generation by drafting multiple tokens and verifying them in parallel with a larger target model.

See all 308 papers

Speculative Diffusion is a Tale of Two Tradeoffs
Oct 22, 2026Aaron Zorel, Guy Van den Broeck, Aditya Grover +1LLM Inference Speculative Decoding
Independent Researcher · University of California, Los Angeles
WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency
Jul 14, 2026Z Sun, Q Jiang, S Sheng +1LLM Inference Efficiency LLM Watermarking
Shanghai Jiao Tong University · Shanghai Innovation Institute · Tianjin University
ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping
Aug 25, 2026Zhi-Kai Chen, Jun-Jie Tao, Wei-Xiang Mao +2Speculative Decoding LLM Inference Acceleration
School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China · Nanjing University, China

Oct 22, 2026J/K move · Enter open · S save

Speculative Diffusion is a Tale of Two Tradeoffs

Aaron Zorel, Guy Van den Broeck, Aditya Grover, Daniel Israel

Independent Researcher · University of California, Los Angeles

LLM Inference Speculative Decoding Speculative Decoding for Diffusion Language Models Diffusion Language Model Decoding

Open paper AskPDF arXiv

Speculative decoding is a technique that accelerates the inference speed of autoregressive large language models (LLMs) by sampling from a faster, lower quality draft model. Traditionally, a fundamental characteristic of speculative decoding is that it is lossless with respect to the verifier distribution, guaranteeing speed without sacrificing quality. This differs from recently developed diffusion LLMs (dLLMs), which are notably able to achieve substantial speedups by degrading the quality of their outputs with less diffusion effort. Because diffusion models are able to sample faster than autoregressive models by generating tokens in parallel, they are considered strong candidates for draft models in speculative diffusion. While it is widely understood that diffusion models can sacrifice quality for speed, speculative decoding itself can also be generalized to generate lossy samples faster using more lenient acceptance criteria with respect to the verifier distribution. Presented with the two parameters of diffusion effort and speculative decoding leniency, this paper explores the full achievable Pareto frontier via joint variations of these parameters. Our experiments show that jointly varying diffusion effort and acceptance leniency can extend the speed-quality Pareto frontier over single step diffusion drafting.

Abstract

Speculative Decoding

Explore similar work

Speculative Diffusion is a Tale of Two Tradeoffs

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping