cs.CVNov 17, 2026

Beyond Co-Generation: Rethinking Jointness in Joint Audio-Video Generation

Authors: Jihoo Jung, Ji-Hoon Kim, Suyeon Lee, Gia Thinh Pham, Joon Son Chung

Organizations: KAIST · Chung-Ang University

Abstract

Our aim is to advance the understanding of genuine audio-visual coordination in joint text-to-audio-video (T2AV) generation, with particular emphasis on semantic consistency. Evaluating audio-visual consistency in T2AV generation is nontrivial, as detailed text prompts obscure whether such consistency stems from genuine cross-modal coordination or merely from independent text following. To isolate the former, we introduce intrinsic jointness: the ability of audio and video to produce the same realization for details left underspecified by the prompt. Using semantically underspecified prompts (e.g., "musical instrument playing" rather than "electric guitar playing"), we investigate four recent models and reveal their limited intrinsic jointness. To address this limitation, we propose JointREPA, a simple representation alignment-based post-training method to facilitate cross-modal coordination. Since existing benchmarks use semantically detailed prompts, we introduce UNSAIDBench, a benchmark designed to evaluate intrinsic jointness under semantic underspecification. Experiments across two distinct models and three benchmarks, including UNSAIDBench, demonstrate that JointREPA consistently improves semantic consistency.

Figures & tables

Explore similar work

CardsList
  1. Do Joint Audio-Video Generation Models Understand Physics?

    May 8, 2026Zijun Cui, Xiulong Liu, Hao Fang +8Multimodal Agent EvaluationPhysical Consistency in Video Generation

  2. Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling

    Apr 26, 2026Zhen Ye, Xu Tan, Aoxiong Yin +8Talking Head GenerationAudio-Video Generation

  3. Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation

    May 24, 2026Shuyuan Tu, Qi Tian, Zihan Yang +9Cross-Modal LearningAudio-Video Generation