Audio-Video Generation

Ask about this topic

Momentum

14 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 101

All topics
CardsList

  1. JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

    Jun 1, 2026Jiashuo Yu, Yao Yao, Boyu Chen +1Audio-Video GenerationAudio-Visual Alignment

    Jen Music AI

  2. Real-Time Generation of Streamable Talking Portrait Video with Reference-Guided Deep Compression VAEs

    Jun 1, 2026Sicheng Xu, Yu Deng, Shoukang Hu +5Variational AutoencodersAudio-Video Generation

    1Microsoft Research · 2Microsoft AI

  3. Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer

    May 29, 2026Ke Lei, Yu Zhang, Changhao Pan +4Text-to-Audio GenerationAudio-Video Generation

    1Zhejiang University, China · 2ByteDance

  4. Benchmarking Single-Factor Physical Video-to-Audio Generation

    May 28, 2026Tingle Li, Siddharth Gururani, Kevin J. Shih +6Audio-Video GenerationAudio-Visual Alignment

    UC Berkeley · NVIDIA · University of Washington

  5. Native Audio-Visual Alignment for Generation

    May 28, 2026Longbin Ji, Guan Wang, Xuan Wei +6Audio-Video GenerationAudio-Visual Synchronization

    ERNIE Team, Baidu Inc.

  6. CapTalk: Text-Guided Stylization and Speech-Driven 3D Head Animation

    May 28, 2026Xuangeng Chu, Yuan Gan, Ziteng Cui +4Audio-Video GenerationTalking Head Generation

    The University of Tokyo · Snap Research, Snap Inc. · RIKEN AIP

  7. MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation

    May 27, 2026Haitian Li, Yanghao Zhou, Heyan Huang +15Audio-Video GenerationMultimodal Model Evaluation

    Shanghai University · Tsinghua University · Beijing Institute of Technology +7

  8. BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation

    May 26, 2026Yutong Wang, Yunke Wang, Xinyuan Chen +1Audio-Video GenerationVideo Generation

    The University of Sydney · Shanghai AI Laboratory

  9. LongCat-Video-Avatar 1.5 Technical Report

    May 26, 2026Meituan LongCat Team, Xunliang Cai, Meng Cheng +10Audio-Video GenerationLong Video Generation

    Meituan LongCat Team

  10. LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV

    May 25, 2026Tengfei Liu, Yang Shi, Xuanyu Zhu +17Audio-Video GenerationLong-Horizon Video Generation

    Peking University · Kling Team · Nanjing University +6

  11. StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration

    May 25, 2026Linrui Tian, Qi Wang, Bang ZhangAudio-Video GenerationLong-Horizon Video Generation

    Tongyi Lab, Alibaba Group

  12. Baton: Explicit Semantic Blueprints for Joint Video-Audio Generation

    May 24, 2026Shuyuan Tu, Qi Tian, Zihan Yang +9Cross-Modal LearningAudio-Video Generation

    Fudan University · Tencent Hunyuan

  13. AVBench: Human-Aligned and Automated Evaluation Benchmark for Audio-Video Generative Models

    May 23, 2026Jialiang Yang, Bin Xia, Ruihang Chu +6Audio-Video GenerationAudio-Visual Alignment

    Tsinghua University · The Chinese University of Hong Kong

  14. MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation

    May 19, 2026Yujie Wei, Yujin Han, Zhekai Chen +20Audio-Video GenerationAudio-Visual Alignment

    Fudan University · The University of Hong Kong · Tongyi Lab, Alibaba Group +2

  15. WavFlow: Audio Generation in Waveform Space

    May 18, 2026Feiyan Zhou, Luyuan Wang, Shoufa Chen +6Text-to-Audio GenerationAudio-Video Generation

    Meta AI · Northeastern University

  16. InstructAV2AV: Instruction-Guided Audio-Video Joint Editing

    May 18, 2026Haojie Zheng, Yixin Yang, Siqi Yang +2Audio-Video GenerationVideo Diffusion Models

    Beijing Academy of Artificial Intelligence, Beijing, China · Peking University, Beijing, China

  17. CounterFlow: A Two-Phase Inference-Time Sampling for Counterfactual Video Foley Generation

    May 18, 2026Gyubin Lee, Junwon Lee, Juhan NamText-to-Audio GenerationAudio-Video Generation

    Kim Jaechul Graduate School of AI, KAIST · 2Graduate School of Cultural Technology, KAIST

  18. Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generation

    May 17, 2026Yuheng Chen, Qingdong He, Teng Hu +4Audio-Video GenerationAudio-Visual Synchronization

    Shanghai Jiao Tong University · University of Electronic Science and Technology of China · Zhejiang University

  19. Sound Sparks Motion: Audio and Text Tuning for Video Editing

    May 14, 2026AmirHossein Naghi Razlighi, Aryan Mikaeili, Ali Mahdavi-Amiri +2Audio-Video GenerationInference-Time Optimization

    University of Cyprus, Cyprus · Simon Fraser University, Canada · Tel Aviv University, Israel +1

  20. OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation

    May 12, 2026Guohui Zhang, XiaoXiao Ma, Jie Huang +9Audio-Video GenerationAudio-Visual Synchronization

    University of Science and Technology of China · JD Explore Academy · Peking University

  21. SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning

    May 12, 2026Xin Cheng, Xihua Wang, Ying Ba +5Audio-Video GenerationDirect Preference Optimization

    Renmin University of China · Westlake University

  22. AllocMV: Optimal Resource Allocation for Music Video Generation via Structured Persistent State

    May 11, 2026Huimin Wang, Leilei Ouyang, Chang Xia +3Audio-Video GenerationLong-Horizon Video Generation

    College of Computer Science, Sichuan University.

  23. Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation

    May 9, 2026Shihao Cheng, Jiaxu Zhang, Quanyue Song +6Audio-Video GenerationAudio-Visual Synchronization

    State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan, China · ByteDance, China · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China +2

  24. Do Joint Audio-Video Generation Models Understand Physics?

    May 8, 2026Zijun Cui, Xiulong Liu, Hao Fang +8Physical Consistency in Video GenerationAudio-Video Generation

    University of Texas at Dallas · University of Washington · University of California, Los Angeles

  25. Audio-Visual Intelligence in Large Foundation Models

    May 5, 2026You Qin, Kai Liu, Shengqiong Wu +12Audio-Visual UnderstandingAudio-Video Generation

    National University of Singapore · University of Oxford · University of Toronto +5

  26. MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Video

    May 1, 2026Kazuya Tateishi, Akira Takahashi, Atsuo Hiroe +3Audio-Video GenerationSound Event Detection

    Sony Group Corporation, Japan · Sony AI, USA

  27. Generate Your Talking Avatar from Video Reference

    Apr 30, 2026Zujin Guo, Zhenhui Ye, Yi Ren +4Video-Reference Video GenerationAudio-Video Generation

    Nanyang Technological University · HeyGen Research · University of Melbourne

  28. Mutual Forcing: Dual-Mode Self-Evolution for Fast Autoregressive Audio-Video Character Generation

    Apr 28, 2026Yupeng Zhou, Lianghua Huang, Zhifan Wu +7Audio-Video GenerationLong-Horizon Video Generation

    1VCIP, School of Computer Science, Nankai University · 2Tongyi Lab · 2Tongyi Lab,3Peking University +1

  29. Talking Slide Avatars: Open-Source Multimodal Communication Approach for Teaching

    Apr 26, 2026Xinxing WuAudio-Video GenerationEducational Technology

    School of Mathematics and Computer Science, Kentucky State University, Frankfort, Kentucky, United States

  30. Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation

    Apr 26, 2026Chunyu Li, Jiaye Li, Ruiqiao Mei +4Audio-Video GenerationVideo Diffusion Models

    Shanghai Innovation Institute · Fudan University · University of Science and Technology of China +2

  31. Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling

    Apr 26, 2026Zhen Ye, Xu Tan, Aoxiong Yin +8Cross-Modal LearningAudio-Video Generation

    Hong Kong University of Science and Technology · Independent Researcher · Zhejiang University +3

  32. MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation

    Apr 21, 2026Liyang Li, Wen Wang, Canyu Zhao +4Diffusion Model GuidanceAudio-Video Generation

    Zhejiang University, Hangzhou, China

  33. OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation

    Apr 20, 2026Lei Zhu, Xing Cai, Yingjie Chen +6Audio-Video GenerationVideo Generation

    Peking University, Beijing, China · WeChat Lab, Tencent, China · Chinese Academy of Sciences, Beijing, China

  34. Hierarchical Codec Diffusion for Video-to-Speech Generation

    Apr 17, 2026Jiaxin Ye, Gaoxiang Cong, Chenhui Wang +4Audio-Video GenerationDiffusion Transformer

    Fudan University · Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +1

  35. ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling

    Apr 16, 2026Jianxuan Yang, Xinyue Guo, Zhi Cheng +10Audio-Video GenerationMultimodal Generation

    MiLM Plus, Xiaomi Inc. · Wuhan University

  36. TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation

    Apr 16, 2026Xiangyu Liu, Feng Gao, Xiaomei Zhang +4Audio-Video GenerationVideo Diffusion Models

    1MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Meituan +2

  37. EchoFoley: Event-Centric Hierarchical Control for Video Grounded Creative Sound Generation

    Dec 31, 2025Bingxuan Li, Yiming Cui, Yicheng He +4Audio-Video GenerationSound Effects Generation

    University of Illinois Urbana-Champaign · 1ByteDance Intelligent Creation · University of California, Los Angeles +1

  38. Taming Text-to-Sounding Video Generation via Advanced Modality Condition and Interaction

    Oct 3, 2025Kaisi Guan, Xihua Wang, Zhengfeng Lai +5Audio-Video GenerationVideo Diffusion Models

    1Renmin University of China · 2Apple

  39. Step-by-Step Video-to-Audio Synthesis via Negative Audio Guidance

    Jun 26, 2025Akio Hayakawa, Masato Ishii, Takashi Shibuya +1Audio-Video GenerationSound Effects Generation

    Sony AI, Tokyo, Japan · Sony Group Corporation, Tokyo, Japan

  40. From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications

    Aug 30, 2023Shreyank N Gowda, Dheeraj Pandey, Shashank Narayana GowdaAudio-Video GenerationTalking Head Generation

    School of Computer Science, University of Nottingham, Nottingham, United Kingdom · Independent Researcher

  41. PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation

    Date pendingYuchen Sun, Qian Yang, Jun Wang +4Audio-Video GenerationAudio-Visual Synchronization

    Meituan Beijing, China