Masked Diffusion Models

Masked diffusion models learn to generate discrete sequences by corrupting tokens—often replacing them with masks—and iteratively predicting missing content using context on both sides. Work spans language, images, speech, molecules, DNA, and structured data, with methods including learned reveal-order policies, confidence- or attention-guided samplers, blockwise and speculative decoding, self-correction, and reinforcement-learning or distillation post-training. Evaluations on reasoning and coding benchmarks such as GSM8K, MATH-500, and HumanEval test whether parallel generation can match autoregressive quality, while multimodal and scientific-design tasks test the framework beyond text. The central tension is that committing more tokens per model pass improves speed but can ignore dependencies and reduce accuracy or diversity; sampler choices, likelihood evaluation, and real serving costs also complicate comparisons.