Discrete Diffusion Model
Discrete diffusion models are a class of generative models designed to sample from complex discrete data distributions, such as text, graphs, and symbolic music, by reversing a learned noise process. Current research focuses on improving sampling efficiency (e.g., through predictor-corrector methods and non-Markov approaches), enhancing model expressiveness (e.g., by incorporating copulas to capture dependencies and using structured preferential generation), and developing training-free guidance methods for controlled generation. These advancements are significant because they offer alternatives to autoregressive models, potentially enabling faster and more flexible generation of high-quality discrete data across various scientific and engineering domains.
Papers
Simplified and Generalized Masked Diffusion for Discrete Data
Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, Michalis K. Titsias
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, Chongxuan Li