The role of injected noise in diffusion model generalization
Organizations: CITEC, Bielefeld University Bielefeld, Germany
Abstract
Diffusion models trained on distinct datasets can produce highly similar samples when prompted with the same noise input, suggesting that they learn a generalized data distribution. We investigate the role of noise injected during the sampling process, and show that it plays an important role in making or breaking this strong generalization effect. Based on this finding we investigate the robustness of sample similarity to changes in the injected noise, and find that its impact is largest in the sampling regime where broad structure already exists but the precise identity of the sample is not yet determined.
Figures & tables
Diffusion Models
Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.
See all 1,251 papers