Audio-Video Generation

Ask about this topic

Momentum

14 papers in the last four weeks, up 17% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 101

All topics
CardsList

  1. Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification

    Aug 6, 2026Lisai Zhang, Yidi Wu, Qi Liu +7Audio-Video GenerationLong-Horizon Video Generation

    1Vorch Team · 2Zhejiang University

  2. Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation

    Aug 5, 2026Zehua Chen, Junyou Wang, Yuxuan Jiang +5Audio-Video GenerationAudio Diffusion Models

    Tsinghua University, Beijing, China · Hefei University of Technology, Hefei, China · Monash University, Australia +1

  3. EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

    Aug 3, 2026Jiayu Chen, Xiaoyu Wu, Rongshan Gao +6Audio-Video GenerationAudio-Visual Alignment

    Peking University Beijing, China · Taiyuan University of Technology Taiyuan, China

  4. AcoustiTrace: When Plausible Sound Violates Physics

    Aug 3, 2026Shiyang Li, Yuewen Cao, Yihao Liu +4Physical Consistency in Video GenerationAudio-Video Generation

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University +3

  5. InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos

    Aug 2, 2026Chi Zhang, Haoyang Shi, Yueyi Liu +4Audio-Video GenerationMultimodal Generation

    College of AI, Tsinghua University

  6. LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

    Jul 29, 2026Rongxiang Zhang, Songhua LiuAudio-Video GenerationDiffusion Model Distillation

    School of Artificial Intelligence, Shanghai Jiao Tong University · 2Harbin Institute of Technology

  7. Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory

    Jul 29, 2026Yanbo Ding, Zhizhi Guo, Quanyue Song +4Audio-Video GenerationLong Video Generation

    Shenzhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · School of Artificial Intelligence, University of Chinese Academy of Sciences +2

  8. TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

    Jul 27, 2026Qijun Gan, Chenwei Zhang, Meiguang Jin +2Audio-Video GenerationMemory-Augmented Video Generation

    Taobao & Tmall Group of Alibaba · Nanjing University

  9. OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation

    Jul 26, 2026Jun Zhan, Chen Yang, Yitian Gong +23Variational AutoencodersAudio-Video Generation

    1Fudan University · 2Shanghai Innovation Institute · 3MOSI Intelligence +1

  10. OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars

    Jul 25, 2026Quanyue Song, Yishan He, Yanbo Ding +4Audio-Video GenerationReal-Time Interactive Video Generation

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · Shenzhen Key Laboratory of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

  11. MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation

    Jul 15, 2026Xiaohan Zhang, Yuqing Wen, Junlin Chen +9Audio-Video GenerationAudio-Visual Alignment

    Nanjing University · Kling Team · National University of Singapore +3

  12. Bring Music The Horizon: Music-Driven 360∘^\circ Video Generation

    Jul 15, 2026Kai Hsu Tsai, Yong Wei Fu, Hung I Yang +1Audio-Video Generation

    Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan

  13. Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models

    Jul 10, 2026Yijie Qian, Juncheng Wang, Chao Xu +6Audio-Video GenerationGroup Relative Policy Optimization

    Zhejiang University, Hangzhou, China · IROOTECH TECHNOLOGY, China · The Hong Kong Polytechnic University, Hong Kong, China +1

  14. Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space

    Jul 7, 2026Thanh V. T. Tran, Ngoc-Son Nguyen, Luong Tran +4Audio-Video GenerationAudio-Visual Synchronization

    FPT Software AI Center, Vietnam · NVIDIA Corporation, USA

  15. Wan-Streamer v0.2: Higher Resolution, Same Latency

    Jul 5, 2026Lianghua Huang, Zhi-Fan Wu, Yupeng Shi +23Audio-Video GenerationReal-Time Interactive Video Generation

    Wan Team, Alibaba Group

  16. Conversational Human Audio-visual Talking Dialogue Generation

    Jul 2, 2026Junhao Song, Lluis Guasch, Xilin He +8Audio-Video GenerationSpoken Dialogue Systems

    Department of Computing, Imperial College London, UK · Department of Earth Science & Engineering, Imperial College London, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE +8

  17. Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation

    Jun 30, 2026Baiqin Wang, Sen Chen, Jiankuo Zhao +3Audio-Video GenerationTalking Head Generation

    MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · CAIR, HKISI, Chinese Academy of Sciences +1

  18. AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

    Jun 29, 2026Kien T. Pham, I Chieh Chen, Qifeng Chen +1Audio-Video GenerationAudio-Visual Alignment

    The Hong Kong University of Science and Technology

  19. FacePlex: Toward Natural Full-Duplex Conversational Avatars

    Jun 29, 2026Habin Lim, Hah Min Lew, Jae-Ho Lee +4Audio-Video GenerationTalking Head Generation

    Korea University · Klleon

  20. OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data

    Jun 29, 2026Kaixing Yang, Jiashu Zhu, Xulong Tang +8Audio-Video GenerationMultimodal Generation

    Renmin University of China, Beijing, China · AMAP, Alibaba Group, Beijing, China · Malou Tech Inc, Plano, Texas, USA +2

  21. MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control

    Jun 28, 2026Kaiqi Liu, Yunyao Mao, Ziqi Cai +8Audio-Video GenerationAudio-Visual Alignment

    Peking University · Kling Team, Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences +1

  22. Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models

    Jun 23, 2026Lianghua Huang, Zhi-Fan Wu, Wei Wang +22Audio-Video GenerationUnified Multimodal Models

    Wan Team, Alibaba Group

  23. UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating

    Jun 19, 2026Jiehui Huang, Yuechen Zhang, Bin Xia +7Audio-Video GenerationMemory-Augmented Video Generation

    The Hong Kong University of Science and Technology · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +2

  24. MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model

    Jun 16, 2026Lichen Bai, Tianhao Zhang, Shitong Shao +14Audio-Video GenerationReal-Time Interactive Video Generation

    Catnip AI Team

  25. FoleyGenEx: Unified Video-to-Audio Generation with Multi-Modal Control, Temporal Alignment, and Semantic Precision

    Jun 12, 2026Shiyao Wang, Xijuan Zeng, Hui Wang +4Audio-Video GenerationAudio-Visual Synchronization

    Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin, China · Kling Team, Kuaishou Technology, Beijing, China

  26. ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance

    Jun 11, 2026Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral +1Audio-Video GenerationRL for Video Generation

    Télécom Paris, Institut Polytechnique de Paris · Max Planck Institute for Informatics

  27. CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation

    Jun 8, 2026Yuheng Chen, Teng Hu, Yuji Wang +7Audio-Video GenerationText-to-Video Generation

    Shanghai Jiao Tong University, Shanghai, China. · University of Electronic Science and Technology of China, Chengdu, China. · Zhejiang University, Hangzhou, China. +2

  28. Resonant Minds: Closed-Loop Social Avatars with Theory of Mind

    Jun 4, 2026Jianxu Shangguan, Jing Xu, Hang Ye +4Audio-Video GenerationTheory of Mind

    University of Washington · 2Peking University · 3Carnegie Mellon University +1

  29. Foley-Omni: A Unified Multimodal Generation Model from Task-Level Audio Synthesis to Complete Video Soundtrack Generation

    Jun 2, 2026Ye Tao, Lupeng Liu, Xuenan Xu +6Audio-Video GenerationUnified Multimodal Models

    School of Intelligence Science and Technology, Nanjing University · Video Rebirth · Shanghai Jiao Tong University +2

  30. Inference-Time Scaling for Joint Audio-Video Generation

    Jun 2, 2026Jaemin Jung, Kyeongha Rho, Inkyu Shin +1Audio-Video GenerationInference-Time Scaling

    Korea Advanced Institute of Science and Technology · Luma AI