DWISE: Denoising Workload-Informed Scheduling Engine

Lie Li; Wen Li; Junxiao Shen; Jindong Gu; Guosheng Hu

cs.LGOct 29, 2026

DWISE: Denoising Workload-Informed Scheduling Engine

Authors: Lie Li, Wen Li, Junxiao Shen, Jindong Gu, Guosheng Hu

Organizations: University of Bristol, Bristol, United Kingdom · University of Oxford, Oxford, United Kingdom

Abstract

Unlike autoregressive large language models (AR-LLMs), diffusion LLMs (dLLMs) generate text by iteratively denoising many tokens in parallel, offering a promising path to efficient inference. Existing serving frameworks, however, schedule dLLM requests with policies designed for AR models, namely first-come-first-served (FCFS) admission and prefill priority. We find that both policies are ill-suited to dLLMs, wasting much of their speed. FCFS ignores that a dLLM request's decoding cost is highly heterogeneous and unknown at arrival, causing severe head-of-line blocking. Prefill priority, widely used in AR-LLM serving to keep time-to-first-token low, sharply cuts dLLM efficiency, as rounds that run only prompts leave most of the batch idle. We present DWISE, a Denoising Workload-Informed Scheduling Engine for efficient dLLM serving. DWISE schedules requests by their remaining work, measured in a block-level unit shared by prefill and denoising, so both phases share a round. To predict the unknown denoising workload, DWISE combines a lightweight prompt-based predictor at admission with a single refinement from the first denoising round's hidden states, which capture model-specific information absent from the prompt and require no additional model execution. Implemented in SGLang and evaluated on five models from three dLLM families under bursty workloads, DWISE reduces mean end-to-end latency over SGLang by 5.4 $\times$ on average and up to 14.9 $\times$ while preserving task accuracy. The code will be released publicly.

Figures & tables

Figure 1: The first denoising round reveals remaining work. Spearman

\rho

( Spearman, 1904 ) between true denoising workload and estimates from prompt features, the prompt hidden state, and the first-round hidden state on LLaDA2.1-mini (five-fold CV).

Figure 2: Denoising workload varies substantially across requests (LLaDA2.1-mini). Boxes p25–p75, line median, whiskers p5–p95.

Figure 3: Prefill-first execution can stall denoising, while both phases share the same block-level scheduling granularity.

Figure 4: Overview of DWISE . Toy example illustrating block-level scheduling under SGLang, MLFQ, and DWISE .

Figure 5: DWISE per-round scheduling loop.

Workload	Serving	Train
Open-ended	1,973	986
ShareGPT ( ShareGPT Team, 2023 )	1,085	542
MBPP ( Austin et al., 2021b )	505	250
HumanEval ( Chen et al., 2021 )	87	46
IFEval ( Zhou et al., 2023 )	296	148
Single-answer	1,973	987

Table 1: Serving and predictor-training requests.

	LLaDA2.0-mini		LLaDA2.1-mini		LLaDA2.2-mini		SDAR-8B		DiffusionGemma
	B	P	B	P	B	P	B	P	B	P
Mean latency (s) $\downarrow$
SGLang	603.1	597.4	545.5	535.9	576.6	571.7	9053	8973	325.9	262.7
MLFQ	716.0	721.8	625.9	616.9	651.0	654.4	11382	12141	346.2	282.1
SSJF	739.3	728.4	602.7	584.6	647.4	611.7	11912	10936	323.5	264.2
LTR	702.7	679.6	645.3	613.9	664.2	597.3	12217	11577	326.9	266.1

Table 2: End-to-end serving performance and task accuracy across models and arrival patterns. B: burst; P: burst+Poisson.

Figure 6: Normalized p50 latency relative to SGLang; lower is better, and annotations show speedup. DGemma: DiffusionGemma.

Figure 7: Normalized p99 latency relative to SGLang; lower is better, and annotations show speedup.

Figure 8: Cumulative speedup over SGLang as DWISE’s components are added (LLaDA2.1-mini, burst+Poisson). Rank by

P_{i}

: remaining prefill blocks only (

\hat{r}_{i}{=}0

). Oracle: ranking by true denoising rounds.

		Prefill-only (%)		Idle	Filled
Policy	Rounds	rounds	time	slots (%)	slots/round
SGLang	63,926	94.6	78.2	96.1	5.0
DWISE	2,819	3.0	11.2	11.6	113.2

Table 3: Serving-round composition on LLaDA2.1-mini under burst+Poisson arrivals (

B=128

). Prefill-only: rounds with only prefill blocks, as a share of all rounds and of wall-clock time.

	FCFS	SSJF	S 3	LTR	TRAIL	DWISE
B	251.7	282.6	188.4	271.1	198.5	76.9
P	228.0	256.9	165.1	248.3	160.7	48.5

Table 4: Mean latency (s) on LLaDA2.1-mini with all ranking policies on DWISE’s mixed-phase batch former. FCFS is SGLang’s default order. B: burst; P: burst+Poisson.

Figure 9: Latency on LLaDA2.1-mini under pure Poisson arrivals without an initial backlog. Dotted line: SGLang’s throughput; grey band: the five other baselines.

Arrival	Predictor	Ratio	$\rho$	Mean (s)	p50 (s)
B	Admission only	1.03	0.714	80.8	58.4
	Raw refinement	1.16	0.890	80.2	55.2
	Calibrated	1.00	0.890	76.9	51.9
P	Admission only	1.03	0.716	51.9	17.2
	Raw refinement	1.16	0.891	50.9	15.4
	Calibrated	1.00	0.891	48.5	13.7

Table 5: Effect of refinement calibration on LLaDA2.1-mini, measured on the evaluation set. Ratio denotes the median predicted-to-actual remaining rounds. B: burst; P: burst+Poisson.

Refine	Once	Per block	Per 4 rounds	Per round
Mean (s)	48.5	49.5	51.0	51.0
p50 (s)	13.7	13.9	15.0	14.7
Preemptions / request	0.34	0.75	0.90	1.14
Reach $\hat{r}_{i}=0$ (%)	37.0	5.5	4.9	0.0
Predictor ( $\mu$ s)	795	809	828	777

Table 6: Effect of refinement frequency on LLaDA2.1-mini under burst+Poisson arrivals. Predictor: GPU time per serving round.

Figure 10: Effect of resident-set multiplier

k

on LLaDA2.1-mini.

Figure 11: DWISE overhead on LLaDA2.1-mini. (a) Cost per serving round, as a share of the 95 ms median round. (b) Admission prediction cost per request from raw prompt text (log scale).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	p50	p90	max	Prompt	1st round
	Rounds per request			$\rho$
LLaDA2.0-mini	110	236	292	0.657	0.809
LLaDA2.1-mini	46	149	279	0.740	0.868
LLaDA2.2-mini	53	189	296	0.723	0.863
SDAR-8B	97.5	252	317	0.697	0.853
DiffusionGemma	10	18	48	0.697	0.809

Appendix

Table 8: Denoising-round spread and predictor rank correlation across models. Rounds: denoising rounds per request over all 2,048 training requests; the minimum is 2 on every model.

\rho

: Spearman correlation with true denoising workload under five-fold cross-validation on the requests used to fit the predictors, with the estimators of Figure 1 : LightGBM on prompt features and a ridge probe on the mean-pooled first-round hidden state.

Predictor	$\rho$	Time (ms, single request)
Ridge	0.535	0.42
MLP (256–256)	0.701	0.44
XGBoost (100, depth 6)	0.706	0.65
LightGBM (500, 31 leaves)	0.709	0.46
LightGBM (1,000, 63 leaves)	0.689	0.54
ExtraTrees (100)	0.722	3.57

Appendix

Table 9: Model selection for the admission-time workload predictor. Time is the per-request median from raw prompt text, including tokenization and feature extraction.

Arrival	Policy	Mean latency (s)	Speedup
B	SGLang	528.0 $\pm$ 7.8	1.00 $\times$
B	DWISE	74.3 $\pm$ 0.4	7.10 $\pm$ 0.14 $\times$
P	SGLang	521.8 $\pm$ 6.8	1.00 $\times$
P	DWISE	48.7 $\pm$ 1.1	10.72 $\pm$ 0.14 $\times$

Appendix

Table 10: Seed robustness on LLaDA2.1-mini: mean

\pm

standard deviation over three traces generated with different seeds, one run each.

Policy	Runs compared	Identical (%)	$\rho$
DWISE	same trace	98	0.998
DWISE	different traces	92	0.994
SGLang	different traces	37	0.972

Appendix

Table 11: Stability of per-request denoising round counts across runs on LLaDA2.1-mini. Identical: share of requests with the same round count in both runs.

	(Hard) MCQ	Other	All
Requests	23%	77%	100%
Work	3.1%	96.9%	100%
$\rho$ , SSJF	first	$-0.02$	$0.51$
$\rho$ , LTR	first	$-0.07$	$0.48$
$\rho$ , DWISE adm.	–	$0.54$	$0.76$
$\rho$ , DWISE ref.	–	$0.84$	$0.92$

Appendix

Table 12: Output-length rankers on LLaDA2.1-mini under burst arrivals (

\Delta

averages three runs;

\rho

uses the first). Work: prefill blocks plus denoising rounds.

\rho

: Spearman correlation between a scheduler’s ranking score and work within the group; both include the exactly known prefill blocks, so

\rho

is higher than for denoising rounds alone. “first”: ranked ahead of nearly all other requests. DWISE adm./ref.: its score at admission and after refinement.

\Delta

: the group’s contribution to the change in mean latency relative to SGLang; omitted for DWISE, whose change also includes the gain from mixed-phase batching.

	B			P
Config	SGLang	DWISE	Sp.	SGLang	DWISE	Sp.
1 GPU	543.0	73.0	7.4	540.2	45.9	11.8
TP=2	498.1	74.5	6.7	483.5	46.1	10.5
TP=4	457.4	64.7	7.1	439.4	36.0	12.2
TP=4+EP=4	454.2	65.4	6.9	444.7	38.3	11.6

Appendix

Table 13: Mean latency (s) of LLaDA2.1-mini under multi-GPU deployment, averaged over three runs. B: burst; P: burst+Poisson; Sp.: speedup (

\times

) of DWISE over SGLang.

	Budget	Tok.	Rnd.	SGLang	DWISE	Sp.	$\rho$
B	256	176	62	526.2	73.0	7.21 (2.12)	0.894
	512	295	100	607.3	104.9	5.79 (1.99)	0.889
	1,024	425	144	688.7	145.3	4.74 (1.78)	0.892
	2,048	536	180	782.1	175.8	4.45 (1.74)	0.886
P	256	177	62	568.7	45.4	12.54 (2.45)	0.894
	512	294	100	610.0	76.6	7.97 (2.01)	0.890

Appendix

Table 14: Output-budget sweep on LLaDA2.1-mini. Tok. and Rnd.: mean output tokens and denoising rounds per request in the SGLang run. SGLang and DWISE: mean latency (s). Sp.: speedup (

\times

) of DWISE over SGLang in mean latency, with the p99 ratio in parentheses, from unrounded latencies.

\rho

: Spearman correlation between DWISE’s refined estimate and the denoising rounds left after the first round. The 256-token row is rerun in the same batch. B: burst; P: burst+Poisson.

Arrival	Policy	Mean	p50	p99
B	SGLang	525.2	541.1	1035.4
B	DWISE	67.1	45.2	472.3
P	SGLang	521.5	570.5	983.8
P	DWISE	40.0	6.8	403.3

Appendix

Table 15: Time to first token (s) on LLaDA2.1-mini, from arrival until the first output block is committed, averaged over three runs. B: burst; P: burst+Poisson.

	B		P
Category	p99	Max	p99	Max
Chat	1048 ${\to}$ 215	1050 ${\to}$ 244	1009 ${\to}$ 196	1016 ${\to}$ 224
Instruction	1045 ${\to}$ 246	1049 ${\to}$ 260	960 ${\to}$ 219	962 ${\to}$ 233
Code	1038 ${\to}$ 81	1041 ${\to}$ 130	950 ${\to}$ 44	952 ${\to}$ 116
Math	1040 ${\to}$ 83	1043 ${\to}$ 133	950 ${\to}$ 52	954 ${\to}$ 114
Multiling. math	1038 ${\to}$ 121	1045 ${\to}$ 125	953 ${\to}$ 86	958 ${\to}$ 105

Appendix

Table 16: Per-category p99 and maximum latency (s) on LLaDA2.1-mini, SGLang

\rightarrow

DWISE (per-run values averaged over three runs). B: burst; P: burst+Poisson.

Held-out category	Admission $\rho$	Refined $\rho$
Hard MCQ	0.200 $\rightarrow$ $-$ 0.154	0.481 $\rightarrow$ 0.463
MCQ	0.237 $\rightarrow$ $-$ 0.185	0.686 $\rightarrow$ 0.602
Code	0.251 $\rightarrow$ $-$ 0.202	0.657 $\rightarrow$ 0.539
Math	0.259 $\rightarrow$ $-$ 0.069	0.540 $\rightarrow$ 0.399
Multilingual math	0.087 $\rightarrow$ $-$ 0.163	0.755 $\rightarrow$ 0.623
Long context	0.338 $\rightarrow$ $-$ n/a	0.352 $\rightarrow$ 0.178

Appendix

Table 17: Leave-one-category-out

\rho

on LLaDA2.1-mini.

Diffusion Model Serving

Diffusion model serving optimizes the scheduling, execution, and resource use of diffusion models to meet latency and throughput goals.

See all 19 papers

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving
Sep 30, 2026Haoyu Zheng, Fangcheng Fu, Binhang Yuan +6Cost-Aware Inference Diffusion Model Serving
Wuhan University · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology +2
Sangam: Efficiently Serving Diffusion LLMs with the AR Stack
Jul 5, 2026Nitin Kedia, Saurabh Agarwal, Myungjin Lee +1LLM Inference Scheduling LLM Inference Serving
The University of Texas at Austin Austin, United States · Cisco Research Bellevue, United States
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Jul 9, 2026Yuanjie Zhu, Liangwei Yang, Ke Xu +4LLM Inference Scheduling Diffusion Model Serving
University of Illinois Chicago, USA

Sep 30, 2026J/K move · Enter open · S save

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

Haoyu Zheng, Fangcheng Fu, Binhang Yuan, Yongqiang Zhang+5

Wuhan University · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology+2

Cost-Aware Inference Diffusion Model Serving Diffusion Language Model Inference LLM Request Scheduling

Open paper AskPDF arXiv

As diffusion large language models (dLLMs) become more capable, they are moving from research settings to real-world serving, where request management (such as scheduling and resource allocation) relies on accurate estimation of per-request inference cost. However, common cost proxies fall short for dLLMs: output length ignores that one forward pass can unmask multiple tokens, and denoising-step count ignores the heterogeneous per-step costs. We observe that the block-autoregressive generation mechanism induces a two-dimensional execution structure over output blocks and within-block denoising steps, whereas these proxies collapse it into a scalar, discarding information essential for characterizing the cost. Motivated by this insight, we propose the Denoising Workload Surface (DWS), which preserves this two-dimensional block-step structure as a probability surface to weight the heterogeneous per-step costs. We then design a coarse-to-fine training scheme that enables a lightweight prompt-only predictor to accurately predict the complex DWS. This predictor runs efficiently even on a single CPU core, avoiding GPU contention with the serving model. Since DWS decouples request-dependent execution behavior from deployment-specific cost factors, the predictor transfers across hardware configurations without retraining. In real-world serving experiments, DWS reduces cost-prediction error by up to

2.50\times

over scalar-based predictors, while the DWS-guided shortest-job-first scheduler reduces end-to-end latency by up to

1.92\times

for online chatbots.

Abstract

Diffusion Model Serving

Explore similar work

Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving