Audio Diffusion Models

Audio diffusion models learn to create or modify sound by gradually removing noise, with flow-matching methods providing a related way to generate audio. Common approaches use U-Nets or diffusion Transformers on waveform, spectrogram, or compressed codec and VAE latents; discrete masked diffusion is also used for speech and music tokens. Text, reference audio, video, spatial cues, and learned guidance can control tasks ranging from music and speech synthesis to editing, enhancement, separation, and bandwidth extension. A central challenge is balancing fidelity, controllability, and resistance to hallucinations against sampling cost and latency, while evaluating outputs with perceptual listening as well as task-specific measures such as intelligibility, alignment, and reconstruction quality.