Audio-Video Generation
Overview
The field builds models that generate or edit sound and moving images together, or use one modality to create the other. Common approaches include diffusion transformers and flow-matching models, with text, reference media, event timelines, or motion cues guiding generation; autoregressive versions support streaming and longer clips. Applications range from video soundtracks and music videos to talking avatars, with controls for identity, timing, sound sources, and scene structure. A central challenge is that perceptual quality and basic synchronization do not guarantee semantic or physical correctness, especially across long scenes, so evaluation increasingly tests human judgments, fine-grained control, and physical grounding.
Latest papers
157 papers24 papers in the last four weeks, up 71% on the four weeks before. 0.1% of all new papers.