cs.LGNov 8, 2026

Discrete Diffusion Playground: A 2D Benchmark for Discrete Generative Models

Authors: Justin Shao, Sihyun Park, David Ryan Koes, Maria Chikina

Organizations: Carnegie Mellon University · University of Pittsburgh

Abstract

Discrete diffusion is a promising framework for categorical generative modeling, but current evaluation protocols often fail to reveal whether models have actually learned the target distribution. We introduce the Discrete Diffusion Playground, a diagnostic benchmark that converts known 2D continuous distributions into binary token sequences through an exact discretized bijection. This creates a rare setting in which models are trained on discrete sequences, but generated samples can be decoded back into 2D and compared directly against a known ground-truth distribution. The benchmark spans smooth Gaussian mixtures and tilted checkerboards with sharp boundaries, disconnected support, and coordinate-dependent structure, allowing it to expose failures that are invisible in standard proxy metrics. It supports exact grid divergences, Wasserstein distances, density visualizations, and nonparametric two-sample tests, providing both quantitative evaluation and interpretable failure diagnosis. Using this testbed, we show that masked diffusion models can learn meaningful adaptive reveal orders but remain less stable than autoregressive baselines on smooth targets and largely fail on tilted checkerboards, where autoregressive models preserve the target geometry. Control experiments with reversed and interleaved token orderings show that autoregressive performance is not explained by favorable bit order alone. The Playground provides a simple but stringent benchmark for stress-testing discrete generative models and for developing evaluation metrics that detect real distributional mismatch rather than proxy performance. Our code is available at https://github.com/shaojus/DiscreteDiffusionPlayground .

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Generalized Discrete Diffusion from Snapshots

    Mar 22, 2026Oussama Zekri, Théo Uscidda, Nicolas Boullé +1Diffusion ModelsDiscrete Diffusion Sampling

  2. Optimizers for Diffusion Models: A Controlled Benchmark

    Sep 19, 2026Arman Bolatov, Egor Shulgin, David Li +6Preference Optimization for Diffusion ModelsDiffusion Models

  3. Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD

    Mar 20, 2026Emiel Hoogeboom, David Ruhe, Jonathan Heek +2Diffusion Model DistillationDiffusion Models