Audio-Video Generation

The field builds models that generate or edit sound and moving images together, or use one modality to create the other. Common approaches include diffusion transformers and flow-matching models, with text, reference media, event timelines, or motion cues guiding generation; autoregressive versions support streaming and longer clips. Applications range from video soundtracks and music videos to talking avatars, with controls for identity, timing, sound sources, and scene structure. A central challenge is that perceptual quality and basic synchronization do not guarantee semantic or physical correctness, especially across long scenes, so evaluation increasingly tests human judgments, fine-grained control, and physical grounding.