Audio Generation

Audio generation studies how to create speech, music, sound effects, and spatial sound from inputs such as text, video, reference audio, or musical scores. Common approaches include diffusion and flow-matching models, autoregressive models that predict audio tokens or continuous latents, and neural or physical synthesis systems with explicit controls. Models increasingly combine tasks such as speech, music, and effects, while benchmarks assess sound quality, instruction following, timing, and alignment with visual scenes. A central challenge is balancing fidelity and controllability with diversity, reliable long-form timing, and fast streaming; objective metrics also do not always match listener judgments.