AutoRegressive Diffusion

Autoregressive diffusion models combine the strengths of autoregressive modeling (capturing sequential dependencies) with diffusion processes (generating diverse and high-quality samples) for various sequence generation tasks. Current research focuses on developing efficient architectures, such as those incorporating random walks or permutation-based corruption, to handle large-scale data and complex dependencies in diverse domains including image, speech, motion, and molecular structure generation. This approach shows promise for advancing fields like drug discovery, human-computer interaction, and multimedia generation by enabling the creation of realistic and controllable synthetic data.