Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference

Yushi Ye; Feng Hong; Huangjie Zheng; Xu Chen; Zhiyong Chen; Yanfeng Wang; Jiangchao Yao

cs.CLFeb 26, 2026

Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference

Authors: Yushi Ye, Feng Hong, Huangjie Zheng, Xu Chen, Zhiyong Chen, Yanfeng Wang, Jiangchao Yao

Organizations: Cooperative Medianet Innovation Center, Shanghai Jiao Tong University · School of Artificial Intelligence, Shanghai Jiao Tong University

Abstract

Diffusion Large Language Models (DLLMs) promise fast non-autoregressive inference but suffer a severe quality-speed trade-off in parallel decoding. This stems from the ''combinatorial contradiction'' phenomenon, where parallel tokens form semantically inconsistent combinations. We address this by integrating continuous representations into the discrete decoding process, as they preserve rich inter-position dependency. We propose ReMix (Rejection Mixing), a framework that introduces a novel Continuous Mixing State as an intermediate between the initial masked state and the final decoded token state. This intermediate state allows a token's representation to be iteratively refined in a continuous space, resolving mutual conflicts with other tokens before collapsing into a final discrete sample. Furthermore, a rejection rule reverts uncertain representations from the continuous state back to the masked state for reprocessing, ensuring stability and preventing error propagation. ReMix thus mitigates combinatorial contradictions by enabling continuous-space refinement during discrete diffusion decoding. Extensive experiments demonstrate that ReMix, as a training-free method, achieves a $2-8 \times$ inference speedup without any quality degradation.

Figures & tables

Figure 1: The ReMix decoding process and its impact on the quality-speed trade-off. (Left) By introducing an intermediate Continuous State ( $\mathbf{C}$ ), ReMix extends the standard $\mathbf{M}\rightarrow T$ transition by introducing a flexible loop involving Mixing $\mathbf{M}\rightarrow\mathbf{C}$ , self-transition $\mathbf{C}\circlearrowright$ and Rejection $\mathbf{C}\rightarrow\mathbf{M}$ , enabling in-process refinement. (Right) On GSM8K, ReMix demonstrates superior performance, turning the baseline’s 7.05% accuracy loss (when accelerating 3.39x) into a 2.80% accuracy gain while achieving similar acceleration.

Figure 2: ReMix resolves combinatorial contradictions by allowing continuous-space refinement. Using the poker hand example, this figure compares the decoding processes. (Top) Vanilla Parallel Decoding: Succumbs to the contradiction. It independently decodes “Full” and “Pair” at positions 11 and 12, resulting in the semantically incorrect “Full Pair” in 8 steps. (Middle) ReMix: Correctly generates “Full House” in only 4 steps. (Bottom) ReMix Output Distribution: This visualizes the internal mechanism. In Steps 1 & 2, both positions are in the Continuous Mixing State (State $\mathbf{C}$ , gradient). Their output distributions are iteratively refined, allowing the model to recognize the “Full” $\leftrightarrow$ “House” dependency. This mutual coordination before discretization prevents the error and leads to the correct response. The Ġ symbol ( e.g . in ĠFlush indicates a token that begins with a whitespace.

Benchmark	Method	Accuracy	Steps	Step	Latency	Latency
				Reduction		Speedup
GSM8K Math Reasoning	LLaDA	73.01	256	1.00 $\times$	13.89	1.00 $\times$
	ReMix	75.66 ( +2.65 )	51.55 ( -204.45 )	4.97 $\times$	3.00 ( -10.89 )	4.63 $\times$
MATH-500 Math Reasoning	LLaDA	32.20	256	1.00 $\times$	13.96	1.00 $\times$
	ReMix	35.00 ( +2.80 )	66.33 ( -189.67 )	3.86 $\times$	3.93 ( -10.03 )	3.55 $\times$
HumanEval Code Generation	LLaDA	43.90	256	1.00 $\times$	19.22	1.00 $\times$

Table 1: Performance and inference speedup comparison on diverse language benchmarks.

Benchmark	Method	Performance	Steps	Step	Latency	Latency
				Reduction		Speedup
Flickr30k-lite Captioning	MMaDA	57.52	256	1.00 $\times$	108.02	1.00 $\times$
	ReMix	59.59 ( +2.07 )	30.13 ( -225.87 )	8.50 $\times$	14.37 ( -93.65 )	7.52 $\times$
AI2D-lite Chart Understanding	MMaDA	62.40	256	1.00 $\times$	109.27	1.00 $\times$
	ReMix	62.60 ( +0.20 )	37.82 ( -218.18 )	6.77 $\times$	19.24 ( -90.03 )	5.68 $\times$
MathVision-mini Math Reasoning	MMaDA	12.83	256	1.00 $\times$	111.12	1.00 $\times$

Table 2: Performance and inference speedup comparison across diverse multimodal understanding and reasoning benchmarks. We use CIDEr for Flickr30k and accuracy for other benchmarks.

Benchmark	Gen	Block	Method	Accuracy	Steps	Step	Latency	Latency
	Length	Length				Reduction		Speedup
Different Generation Lengths
GSM8K	128	128	LLaDA	59.21	128	1.00 $\times$	15.15	1.00 $\times$
			ReMix	59.74 ( +0.53 )	36.63	3.49 $\times$	4.63	3.27 $\times$
	256		LLaDA	72.48	256	1.00 $\times$	38.56	1.00 $\times$
			ReMix	75.28 ( +2.80 )	58.54	4.37 $\times$	8.63	4.47 $\times$

Table 3: Experiment results on different generation lengths and block lengths

Figure 3: Comparison of GSM8K responses (LLaDA-based) score distributions addressing combinatorial contradictions on naive parallel decoding (left) and ReMix (right)

Panel of Figure 6 — Figure 6: Ablation study on the mixing ratio $\beta$ and rejection threshold $\tau_{rej}$

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Block Length = 128		Block Length = 32
Method	Acc.	TPS	Acc.	TPS
LLaDA	72.55	22.15	81.88	22.20
WINO	75.28 (+2.73)	74.82 ( $\times$ 3.38)	–	–
Fast-dLLM	–	–	80.29 (-1.59)	75.45 ( $\times$ 3.40)
Learn2PD	–	–	78.17 (-3.71)	102.72 ( $\times$ 4.63)
ReMix	75.36 (+2.81)	102.47 ( $\times$ 4.63)	81.20 (-0.68)	80.29 ( $\times$ 3.62)

Appendix

Table 4: Accuracy and throughput (TPS) comparison on GSM8K.

Component	Time (s)	Share (%)
End-to-end generation	2.52	100.00
Mixing update	0.14	5.56
Rejection	0.09	3.56
ReMix-specific total	0.23	9.12

Appendix

Table 5: Runtime breakdown of ReMix overhead on GSM8K.

Diffusion Language Model Inference

Diffusion language model inference studies how to generate, guide, and serve text efficiently through iterative denoising and parallel token updates.

See all 177 papers

DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention
Mar 9, 2026Younjoo Lee, Seungkyun Dan, Junghoo Lee +2Diffusion Language Model Inference Diffusion Language Model Decoding
Seoul National University, Seoul, Republic of Korea
Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference
Aug 27, 2026Wooje Park, Insu Lee, Minyoung Noh +4Diffusion Language Model Decoding Diffusion Language Model Inference
Seoul National University · Sungkyunkwan University
Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers
May 16, 2026Fanqin Zeng, Feng Hong, Geng Yu +6Diffusion Model Distillation Diffusion Language Model Decoding
Shanghai Jiao Tong University, Shanghai, China · Apple MLR, Cupertino, CA, USA · Tongji University, Shanghai, China +2

Mar 9, 2026J/K move · Enter open · S save

DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention

Younjoo Lee, Seungkyun Dan, Junghoo Lee, Jaiyoung Park+1

Seoul National University, Seoul, Republic of Korea

Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Model Inference Acceleration Token Pruning

Open paper AskPDF arXiv

Masked diffusion language models enable parallel token decoding, providing a promising alternative to the sequential nature of autoregressive generation. However, their iterative denoising process remains computationally expensive because it repeatedly processes the entire sequence at every step. We observe that across these diffusion steps, most token representations remain stable; only a small subset, which we term salient tokens, contributes meaningfully to the next update. Leveraging this temporal sparsity, we present DyLLM, a training-free inference framework that accelerates decoding by selectively computing only these salient tokens. DyLLM identifies saliency by measuring the cosine similarity of attention contexts between adjacent denoising steps. It recomputes feed-forward and attention operations only for salient tokens while reusing cached activations for the remainder. Across diverse reasoning and code-generation benchmarks, DyLLM achieves up to 9.6x higher throughput while largely preserving the baseline accuracy of representative open-source diffusion LLMs, LLaDA, and Dream.

Abstract

Diffusion Language Model Inference

Explore similar work

DyLLM: Efficient Diffusion LLM Inference via Saliency-based Token Selection and Partial Attention

Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

Roll Out and Roll Back: Diffusion LLMs are Their Own Efficiency Teachers