Discrete Diffusion Model
Discrete diffusion models are a class of generative models designed to sample from complex discrete data distributions, such as text, graphs, and symbolic music, by reversing a learned noise process. Current research focuses on improving sampling efficiency (e.g., through predictor-corrector methods and non-Markov approaches), enhancing model expressiveness (e.g., by incorporating copulas to capture dependencies and using structured preferential generation), and developing training-free guidance methods for controlled generation. These advancements are significant because they offer alternatives to autoregressive models, potentially enabling faster and more flexible generation of high-quality discrete data across various scientific and engineering domains.
Papers
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Xiaoxiao He, Ligong Han, Quan Dao, Song Wen, Minhao Bai, Di Liu, Han Zhang, Martin Renqiang Min, Felix Juefei-Xu, Chaowei Tan, Bo Liu, Kang Li, Hongdong Li, Junzhou Huang, Faez Ahmed, Akash Srivastava, Dimitris Metaxas
Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction
Jarrid Rector-Brooks, Mohsin Hasan, Zhangzhi Peng, Zachary Quinn, Chenghao Liu, Sarthak Mittal, Nouha Dziri, Michael Bronstein, Yoshua Bengio, Pranam Chatterjee, Alexander Tong, Avishek Joey Bose
$\textit{Jump Your Steps}$: Optimizing Sampling Schedule of Discrete Diffusion Models
Yong-Hyun Park, Chieh-Hsin Lai, Satoshi Hayakawa, Yuhta Takida, Yuki Mitsufuji