Sound Effects Generation

Sound effects generation creates sounds such as footsteps, impacts, and environmental ambience from text, video, reference audio, or combinations of these inputs. Common approaches include latent diffusion and flow-matching models, autoregressive audio models, retrieval-based synthesis, and procedural sound generation. To improve control, systems use cues such as object masks, event timelines, reference timbres, physical parameters, and 3D scene geometry; benchmarks assess semantic match, timing, sound quality, and audiovisual consistency. A central challenge is balancing precise control and synchronization with diversity, while objective audio metrics do not always capture human judgments of realism or usefulness.