SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains

Siva Rajesh Kasa; Yasong Dai; Sumit Negi; Hongdong Li

cs.CLNov 5, 2026

SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains

Authors: Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li

Organizations: School of Computing, Australian National University

Abstract

Masked diffusion language models offer parallel text generation, but independently sampling multiple tokens can disrupt their dependencies and change the output distribution. Exact continuous-time sampling avoids this approximation, yet typically requires one sequential model evaluation per token. We introduce SpecCTMC, a training-free speculative sampler that reduces this sequential bottleneck without a separate draft model. SpecCTMC proposes successive token updates using previously computed model predictions, evaluates their intermediate states in parallel, and applies an exact acceptance-and-correction procedure. We prove that SpecCTMC preserves the joint distribution of event times, generation order, and token values of the underlying continuous-time Markov chain. It therefore introduces neither time-discretization error nor additional factorization error from parallel decoding. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across five benchmarks demonstrate 1.73-4.38x fewer sequential model evaluations than direct continuous-time sampling, while preserving the reference sampling distribution. Our code is available at: https://github.com/kasakh/specctmc

Figures & tables

Figure 1: Sampling a masked diffusion LLM as a CTMC. (a) Direct event simulation is exact but requires one sequential target model evaluation per unmasking event. (b) Speculative dLLM decoders verify drafts against a finite-step greedy decoder, with restricted lossless guarantees. (c) SpecCTMC preserves the exact random event clock and the distribution of joint marks. The diagram’s

N/K

label is the full-acceptance ideal when all drafted states fit in one target-model call; generally, one speculative round commits up to

K

events.

Panel of Figure 2 — Figure 2: Exact law preserving of SpecCTMC on NFE acceleration. On GSM8K with LLaDA-8B-Instruct (256 generated tokens), SpecCTMC reduces target NFEs by $2.88\times$ relative to DirectCTMC while preserving expected accuracy. Finite-step samplers trade accuracy for fewer evaluations.

Continuous-time Markov chain simulation
Method	Training free	Exact event times	Natural-event CTMC law	No separate drafter	Within-trajectory parallelism	No added factorization error
Fixed-step ancestral / tau-leaping ( Campbell et al., 2022 ; Ren et al., 2025 )	Yes	No	No	Yes	Yes	No
Sequential FHS (DirectCTMC) ( Zheng et al., 2025 )	Yes	Yes	Yes	Yes	No	Yes
Parallel FHS ( Zheng et al., 2025 )	Yes	Yes	No	Yes	Yes	No
Parallel masked-diffusion decoding
Fast-dLLM ( Wu et al., 2026 )	Yes	No	No	Yes	Yes	No

Table 1: Comparison of masked-diffusion inference methods. “Natural-event CTMC law” is the stochastic process induced by the unchanged denoiser. “Within-trajectory parallelism” means that one target round can advance multiple events or positions.

	GSM8K		MATH		MBPP		HumanEval		IFEval		Avg.
$L$	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE
32	1.00 $\times$	1.75 (2)	1.15 $\times$	1.75 (2)	1.10 $\times$	2.39 (3)	2.19 $\times$	3.02 (5)	2.65 $\times$	3.42 (7)	1.62 $\times$	2.46
64	1.01 $\times$	1.83 (2)	1.18 $\times$	1.85 (2)	1.05 $\times$	1.85 (2)	2.48 $\times$	3.88 (6)	2.77 $\times$	3.56 (5)	1.70 $\times$	2.59
128	1.06 $\times$	2.70 (3)	1.18 $\times$	2.71 (3)	1.12 $\times$	1.90 (2)	2.37 $\times$	3.94 (5)	2.88 $\times$	4.02 (5)	1.72 $\times$	3.05
256	1.08 $\times$	2.81 (3)	1.10 $\times$	2.81 (3)	1.10 $\times$	1.93 (2)	1.92 $\times$	2.80 (3)	2.22 $\times$	4.38 (5)	1.48 $\times$	2.95
512	1.09 $\times$	3.81 (4)	1.07 $\times$	2.89 (3)	1.07 $\times$	1.96 (2)	1.38 $\times$	2.87 (3)	1.62 $\times$	2.91 (3)	1.25 $\times$	2.89

Table 2: Exact speculative CTMC sampling on LLaDA. For each generated length

L

we report wall-clock speedup of SpecCTMC over DirectCTMC (Wall-clock) and the reduction in sequential target-model calls (NFE); the draft depth used is in parentheses. Gray speedups are at parity (the paired 95% interval contains one); all others are faster, and no condition is slower. The last row reports accuracy (%) of DirectCTMC

\rightarrow

SpecCTMC averaged over lengths; every paired 95% confidence interval for the SpecCTMC-minus-DirectCTMC accuracy difference contains zero. Bold marks the best value.

	GSM8K		MATH		MBPP		HumanEval		IFEval		Avg.
$L$	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE	Wall-clock	NFE
32	0.91 $\times$	1.73 $\times$ (2)	0.99 $\times$	1.74 $\times$ (2)	0.96 $\times$	1.75 $\times$ (2)	1.94 $\times$	2.67 $\times$ (4)	1.80 $\times$	2.87 $\times$ (5)	1.32 $\times$	2.15 $\times$
64	0.96 $\times$	1.83 $\times$ (2)	1.02 $\times$	1.84 $\times$ (2)	1.03 $\times$	1.83 $\times$ (2)	1.97 $\times$	3.46 $\times$ (5)	2.62 $\times$	3.72 $\times$ (6)	1.52 $\times$	2.54 $\times$
128	0.99 $\times$	1.89 $\times$ (2)	1.05 $\times$	2.71 $\times$ (3)	1.02 $\times$	2.74 $\times$ (3)	2.00 $\times$	2.69 $\times$ (3)	2.46 $\times$	3.97 $\times$ (5)	1.50 $\times$	2.80 $\times$
256	0.99 $\times$	1.95 $\times$ (2)	1.00 $\times$	2.82 $\times$ (3)	1.03 $\times$	3.72 $\times$ (4)	1.61 $\times$	4.32 $\times$ (5)	1.96 $\times$	3.63 $\times$ (4)	1.32 $\times$	3.29 $\times$
512	1.00 $\times$	2.94 $\times$ (3)	1.00 $\times$	3.82 $\times$ (4)	1.02 $\times$	3.88 $\times$ (4)	1.23 $\times$	3.82 $\times$ (4)	1.33 $\times$	2.91 $\times$ (3)	1.12 $\times$	3.48 $\times$

Table 3: Exact speculative CTMC sampling on Dream. For each generated length

L

we report the wall-clock speedup of SpecCTMC over DirectCTMC (Wall-clock) and the reduction in sequential target model function evaluations (NFE); depth used shown in () . The last row reports accuracy (%) of DirectCTMC

\rightarrow

SpecCTMC, averaged over lengths.

Panel of Figure 3 — Figure 3: Batching cost and complete-generation speedup depend on the hardware. (a, b) Forward-only relative cost of one three-candidate target call against one single-candidate call, versus complete sequence length for LLaDA-8B-Instruct and Dream-7B-Instruct on a B200 and an L40S GPU. The dashed line marks the cost of three separate calls and the dotted line one single-candidate call. At short lengths on the B200, one three-state call costs less than three separate calls. As sequence length grows, the three-state call approaches the cost of three separate calls on both GPUs, with the L40S reaching that regime at shorter lengths. (c) Complete-generation speedup of SpecCTMC over DirectCTMC versus speculation depth for Dream-7B-Instruct on HumanEval and GSM8K with 256 generated tokens, on a B200 (solid) and an L40S (dashed); shaded bands are paired 95% intervals. The same depth, acceptance, and evaluation reduction give different wall-clock outcomes on the two devices, and depth 8 on GSM8K exceeds L40S memory.

Dataset	$L$	Depth	Direct.	Spec.	NFE reduction	Accept.
MathVerse	256	8	256.00	43.17	5.93 $\times$	0.698
	512	8	512.00	78.04	6.56 $\times$	0.784
MathVista	256	8	256.00	42.41	6.04 $\times$	0.718
	512	8	512.00	76.48	6.69 $\times$	0.806

Table 4: DirectCTMC vs. SpecCTMC on multimodal LLaDA-V. NFE is averaged across the fixed evaluation set and sampling seeds.

Panel of Figure 4 — Figure 4: Output quality versus sequential evaluation reduction. LLaDA-8B-Instruct, GSM8K, 256 tokens, $T=1.0$ . Columns: $K$ answers per question. Rows: mean accuracy and Pass@ $K$ , normalized to DirectCTMC at the same $K$ (absolute DirectCTMC score in the lower-left corner). The horizontal axis is the ratio of median cumulative sequential target-model evaluations (DirectCTMC / method). From left to right, $\tau$ -leaping uses 256, 192, 64 and 16 steps and trapezoidal uses 64 and 8 steps; at $S+1$ and $2S+1$ evaluations per answer, 16-step $\tau$ -leaping and 8-step trapezoidal are cost matched.

Panel of Figure 5 — Figure 5: Stochastic sampling expands solution coverage, and exact speculation reduces its sequential cost. All results use LLaDA-8B-Instruct on five-shot GSM8K with 256 generated tokens. The dashed line is the strongest matched deterministic configuration, Fast-dLLM factor $f=1.0$ (81.25%; Table 7 ). (a): empirical Pass@ $K$ for four selected fixed token temperatures; $T=0.5$ is highlighted. (b): measured DirectCTMC and SpecCTMC results at $T=1.0$ , with points corresponding to $K\in\{1,2,4,8,16,32,64\}$ . At $K=32$ , both attain 85.94% observed coverage, while median cumulative sequential evaluations decrease from 8,140 to 2,901 ( $2.81\times$ ).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Term or symbol	Meaning
Process
DirectCTMC	Event-by-event exact simulation of the absorbing-mask CTMC: one target-model evaluation per unmasking event.
SpecCTMC	Exact speculative simulation of the same CTMC.
Natural event	One masked position unmasked at a random event time.
Mark $z=(d,v)$	Position $d$ and replacement token $v$ of one event.
State update $\Phi(x,(d,v))$	Sequence equal to $x$ except that position $d$ is replaced by token $v$ .

Appendix

Table 5: Terminology and notation used throughout the main paper and appendix.

Position choice	Token choice	Accuracy
Uniform random	Sampled	26.56%
Highest confidence	Sampled	62.50%
Uniform random	Highest probability	26.56%
Highest confidence	Highest probability	79.69%

Appendix

Table 6: Position choice and token choice for LLaDA-8B-Instruct on GSM8K with 256 generated tokens. Accuracy is reported in percent.

Method	Selection rule	Accuracy (%)	Mean NFEs	Median time (s)
Greedy denoising	Highest-confidence position; highest-probability token	79.69	256.00	34.78
Fast-dLLM, threshold 0.9	Confidence threshold $>0.9$	78.13	80.11	10.52
Fast-dLLM, factor 1.0	Confidence-profile factor rule	81.25	63.06	8.13

Appendix

Table 7: Matched deterministic configurations on LLaDA-8B-Instruct and five-shot GSM8K with 256 generated tokens. All rows use the same checkpoint revision, prompts, scorer, and timing procedure. Scores and costs are our measured results. These configurations are deterministic under the tested settings, so repeated identical runs have the same Pass@

K

as their single-answer accuracy.

Method/configuration	Own Greedy accuracy (%)	Method accuracy (%)	Mean NFEs	Greedy time / method time
Parallel Speculative Decoding, confidence 0.9	79.69	76.56	60.39	1.684
Trajectory-Level, single-block exact	73.44	73.44	160.17	1.544
Trajectory-Level, practical cross-block	73.44	76.56	203.44	1.016
Self Cascaded Speculative Decoding, no cache	82.81	79.69	81.59	0.599
Self Cascaded Speculative Decoding, approximate cache	73.44	78.13	100.98	1.268
Spiffy, independent-state implementation	79.69	78.13	127.19	0.261

Appendix

Table 8: Additional greedy-derived speculative implementations. Each row is reported with its own measured Greedy reference because implementation, input shape, or cache definition differs. The final column reports the within-implementation ratio of Greedy time to method time; a value above one indicates acceleration relative to that reference. Absolute latency is not pooled across these implementations. Each method is named according to its evaluated configuration, and all numerical results are from our runs.

Refresh	Cached seq.	Cached Spec	Cache change	SpecCTMC change
interval	accuracy	accuracy	from uncached	from cached seq.
8	25.0%	35.9%	$-7.8$ [ $-18.8$ , $1.6$ ]	$10.9$ [ $-1.6$ , $23.4$ ]
16	26.6%	28.1%	$-6.3$ [ $-15.6$ , $3.1$ ]	$1.6$ [ $-10.9$ , $14.1$ ]
32	17.2%	20.3%	$-15.6$ [ $-26.6$ , $-4.7$ ]	$3.1$ [ $-6.3$ , $12.5$ ]

Appendix

Table 9: Task quality for prompt-cached sequential CTMC and prompt-cached SpecCTMC. Changes are percentage points with 95% CI. Cache change compares cached sequential CTMC with uncached DirectCTMC. SpecCTMC change compares the two cached methods.

Refresh	Seq. target	Spec target	Refresh	Spec total	Call	Time ratio
interval	calls	calls	calls	calls	reduction	with 95% interval
8	253.9	100.6	32.9	133.5	2.530	1.697 [1.692, 1.700]
16	253.9	98.4	17.0	115.3	2.586	1.940 [1.936, 1.949]
32	253.9	92.7	9.0	101.6	2.751	2.211 [2.201, 2.217]

Appendix

Table 10: Target-model evaluation accounting for the 256-token prompt-cache experiment. Refresh calls apply to both cached methods. The final two columns compare cached sequential CTMC with cached SpecCTMC at the same refresh interval.

Quantity	Event by event	SpecCTMC
Sequential target calls	32.00	10.25
Sequential-call reduction	1.00	3.122
Target states evaluated	32.00	38.50
Additional proposal calls	0.00	1.50
Total checkpoint calls	32.00	11.75
Accepted events per round	Not applicable	2.631

Appendix

Table 11: Fast-dLLM-v2 checkpoint transfer. The same checkpoint is used to compute the event-by-event reference, proposals, and current verification probabilities.

Speculative Decoding for Diffusion Language Models

Speculative decoding for diffusion language models speeds generation by drafting tokens in parallel and verifying them efficiently.

See all 39 papers

Token Time Continuous Diffusion for Language Modeling
May 7, 2026Parikshit Bansal, Sujay SanghaviContinuous Diffusion Language Models Diffusion Language Model Decoding
UT Austin
Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models
Oct 1, 2026Yair Schiff, Omer Belhasin, Roy Uziel +7Diffusion Language Model Inference Language Modeling
NVIDIA · Cornell University
SimSD: Simple Speculative Decoding in Diffusion Language Models
Jun 1, 2026Junxia Cui, Haotian Ye, Runchu Tian +9Masked Diffusion Models Speculative Decoding for Diffusion Language Models
University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1

May 7, 2026J/K move · Enter open · S save

Token Time Continuous Diffusion for Language Modeling

Parikshit Bansal, Sujay Sanghavi

UT Austin

Continuous Diffusion Language Models Diffusion Language Model Decoding Adaptive Computation for Diffusion Models Conditional Diffusion Models

Open paper AskPDF arXiv

In this paper we introduce token time continuous diffusion (TTCD), a new diffusion language model which (a) operates in continuous space, deterministically mapping Gaussian noise to a final token canvas with no further sampling, and crucially (b) incorporates a new notion of per-token times, with some tokens proceeding from noise to token at a faster rate than others. Continuous space modeling helps TTCD avoid the parallel sampling of multiple tokens, which is a key source of inaccuracy at high speedups for models that iterate purely in discrete space. The notion of per-token times helps TTCD to better model conditional generation, allows for more sure tokens to proceed at a faster rate, and allows for differentiated inter-token influences during refinement. TTCD outperforms discrete models at high speedups. We train a 160M parameter TTCD model on OpenWebText, and then self-distill it; we find that at high speedups we are comparable in unconditional generation quality, and outperform in conditional generation, several existing models of similar size trained, on the same data, and self-distilled. We achieve similar gains in Sudoku solving as well.

Abstract

Speculative Decoding for Diffusion Language Models

Explore similar work

Token Time Continuous Diffusion for Language Modeling

Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models

SimSD: Simple Speculative Decoding in Diffusion Language Models