Audio-Video Generation

Ask about this topic

Momentum

11 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 87

All topics
CardsList

  1. Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation

    Oct 4, 2026Team Kandinsky, Julia Agafonova, Bulat Akhmatov +85Audio-Video Generation

  2. Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training

    Oct 4, 2026Shuyuan Tu, Qi Tian, Yinming Huang +8Audio-Video GenerationSpatio-Temporal Attention

    Fudan University · Tencent Hunyuan Foundation Model Team · Zhejiang University

  3. Soundwich: Video Generation with Layered and Controllable Audio

    Sep 30, 2026Zhuo Ning, AmirHossein Naghi Razlighi, Sagi Polaczek +2Audio-Video GenerationVideo Generation

    Simon Fraser University · University of Cyprus · CYENS Centre of Excellence +1

  4. Here the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation

    Sep 30, 2026Hanmo Chen, Chengcheng Liu, Tianxiao Chen +7Audio-Visual ConsistencyAudio-Video Generation

    Xidian University · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.

  5. Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation

    Sep 29, 2026Xingtong Ge, Yutong Wang, Lunjie Zhu +7Audio-Video GenerationFew-Step Generation

    The Hong Kong University of Science and Technology · Vivix Group Limited · The University of Sydney +1

  6. EvolvingAvatar: Interactive 3D Head Generation That Adapts as Conversations Unfold

    Sep 28, 2026Junjie Chen, Fei Wang, Kun Li +5AvatarsAudio-Video Generation

    Hefei University of Technology · EPIC Lab, Shanghai Jiao Tong University · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center +4

  7. ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

    Sep 28, 2026Jiacheng Hua, Xiaokun Feng, Jiaqi Hua +3Audio-Video GenerationMultimodal Generation

    College of AI, Tsinghua University · Tencent Hy

  8. Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

    Sep 28, 2026Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21Audio-Video GenerationAudio-Visual Consistency

    University of Massachusetts Amherst · University of Texas at Dallas · Virginia Tech +10

  9. AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

    Sep 24, 2026Zhiyu Xu, Weilong Yan, Yufei Shi +4Audio-Video GenerationDiffusion-Based Reinforcement Learning Methods

    Shanghai AI Laboratory · The Hong Kong Polytechnic University · National University of Singapore +2

  10. TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculum

    Sep 22, 2026Xinyue Guo, Jianxuan Yang, Daiguo Zhou +4Audio-Visual ConsistencyVideo Object Removal

    MiLM Plus, Xiaomi Inc.

  11. Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation

    Sep 8, 2026Tianyi Zeng, Junchao Liao, Yujie Wei +9Video EditingAudio-Video Generation

    2Shanghai Jiao Tong University · *Work done during internship at Alibaba · 1Alibaba Group +4

  12. The Attention Triangle in Audio-Video Models

    Sep 3, 2026Sagi Polaczek, Noa Kraicer, Gal Metzer +4Cross-Modal AttentionAudio-Video Generation

    1Tel Aviv University · 2Simon Fraser University

  13. TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models

    Sep 1, 2026Chao Zhou, Yiling Chen, Qi Chu +3Audio-Video GenerationDiffusion Models

  14. DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution

    Aug 31, 2026Jiashu Zhu, Yanhao Zheng, Ruitian Tian +7Audio-Video GenerationMultimodal Generation

    DreamX Team, Alibaba Group

  15. TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

    Aug 25, 2026Xiaoda Yang, Yuxiang Liu, Kaiwen Zheng +12Audio-Video GenerationText-To-Video Generation Model

    Zhejiang University, Hangzhou, China · Tianjin University, Tianjin, China · Tsinghua University, Beijing, China +4

  16. VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

    Aug 19, 2026Yinming Huang, Shuyuan Tu, Xi Yan +6Audio-Video GenerationPreference Datasets

    Fudan University · Shanghai Innovation Institution · Yinwang Intelligent Technology Co., Ltd +1

  17. Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation

    Aug 14, 2026Jian Xu, Delu Zeng, John PaisleyAudio-Video GenerationShortcut Learning

    RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1

  18. StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos

    Aug 12, 2026Julian Spravil, Sebastian Houben, Sven BehnkeAudio-Video GenerationAudio Understanding

    Fraunhofer IAIS, Germany · A2S, University of Applied Sciences Bonn-Rhein-Sieg, Germany · AIS, Computer Science Institute VI, University of Bonn, Germany +2

  19. HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion

    Aug 12, 2026Wenshuo Peng, Kaipeng ZhangAudio-Video GenerationHarmonization

    Electornic Engineering Department, Tsinghua University, Haidian District, Beijing, 100084 China. · OpenGVlab, Shanghai AI Laboratory, Xuhui District, Shanghai, 200030, China.

  20. UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos

    Aug 12, 2026Yuxuan Zhang, Haozhong Xiong, Jiayi Song +5Audio-Video GenerationStreaming

    The Chinese University of Hong Kong · Qwen Applications Business Group of Alibaba

  21. Dialogue-Aware Video-to-Music Generation Using Public Domain Film Collections

    Aug 12, 2026Haven Kim, Zachary Novack, Julian McAuley +1Audio-Video GenerationFilms

    University of California San Diego · University of Michigan

  22. Towards Expressive and Faithful Audio-to-Image Generation: A Unified Multimodal Dataset and Synthesis Framework

    Aug 10, 2026Dongxu Ge, Shansong Liu, Cheng Gong +3Audio-Video GenerationMultimodal Generation

    University of Science and Technology of China Hefei, China · Institute of Artificial Intelligence, China Telecom (TeleAI) Beijing, China · Northwest Polytechnical University Xi’an, China

  23. MMAG: A Multi-Control Mixed Audio Generation Benchmark

    Aug 7, 2026Zihao Zheng, Xuenan Xu, Jiahao Mei +5Modern Generative Audio ModelsAudio-Video Generation

    MoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University, China · Shanghai AI Lab, China

  24. Vorch-Omni: Multi-Task Orchestration of Sight and Sound

    Aug 6, 2026Vorch Team, Xiaoyu Chen, Yang Ding +25Audio-Video GenerationGenerative Video Models

  25. Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification

    Aug 6, 2026Lisai Zhang, Yidi Wu, Qi Liu +7Audio-Video GenerationDramadirector

    1Vorch Team · 2Zhejiang University

  26. Vorch-Streamer: Extending Human Audio-Visual Generation to Real-Time Long-Form Streaming

    Aug 6, 2026Menglin Han, Yang Ding, Yulei Lu +6Audio-Video GenerationStreaming

    Vorch Team · Tongji University · Harbin Institute of Technology, Shenzhen +1

  27. Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation

    Aug 5, 2026Zehua Chen, Junyou Wang, Yuxuan Jiang +5Audio-Video GenerationText-To-Video Diffusion Models

    Tsinghua University, Beijing, China · Hefei University of Technology, Hefei, China · Monash University, Australia +1

  28. OmniVR: Audio-Video Conditional Generation for Archival Footage Restoration

    Aug 4, 2026Xin Lu, Zihao Fan, Jie Huang +4Video RestorationAudio-Video Generation

    University of Science and Technology of China · JD Explore Academy

  29. EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

    Aug 3, 2026Jiayu Chen, Xiaoyu Wu, Rongshan Gao +6Audio-Video GenerationAudio-Visual Consistency

    Peking University Beijing, China · Taiyuan University of Technology Taiyuan, China

  30. AcoustiTrace: When Plausible Sound Violates Physics

    Aug 3, 2026Shiyang Li, Yuewen Cao, Yihao Liu +4Audio-Video GenerationAcoustic

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University +3

  31. StreamTalk: Streaming Co-Speech Gesture Generation with Key-Pose Anchoring

    Aug 3, 2026Xiangyue Zhang, Jianfang Li, Jiaxu Zhang +2Co-Speech Gesture GenerationAudio-Video Generation

    The University of Tokyo · Alibaba Group · Nanyang Technological University +1

  32. InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos

    Aug 2, 2026Chi Zhang, Haoyang Shi, Yueyi Liu +4Video Multimodal Large Language ModelsInteractivity

    College of AI, Tsinghua University

  33. Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory

    Jul 29, 2026Yanbo Ding, Zhizhi Guo, Quanyue Song +4Audio-Video GenerationReal-Time Systems

    Shenzhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · School of Artificial Intelligence, University of Chinese Academy of Sciences +2

  34. TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

    Jul 27, 2026Qijun Gan, Chenwei Zhang, Meiguang Jin +2Audio-Video GenerationReal-Time Systems

    Taobao & Tmall Group of Alibaba · Nanjing University

  35. AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

    Jul 27, 2026Hengyuan Zhang, Jingna Sun, Meiguang Jin +1Audio-Video GenerationVideo Generation

    Alibaba Group, Taobao and Tmall Group

  36. OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation

    Jul 26, 2026Jun Zhan, Chen Yang, Yitian Gong +23Audio-Video GenerationCross-Modal Alignment

    1Fudan University · 2Shanghai Innovation Institute · 3MOSI Intelligence +1

  37. OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars

    Jul 25, 2026Quanyue Song, Yishan He, Yanbo Ding +4Audio-Video GenerationAudio-Visual Consistency

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · Shenzhen Key Laboratory of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

  38. MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation

    Jul 15, 2026Xiaohan Zhang, Yuqing Wen, Junlin Chen +9Audio-Video GenerationAudio-Visual Consistency

    Nanjing University · Kling Team · National University of Singapore +3

  39. Bring Music The Horizon: Music-Driven 360∘^\circ Video Generation

    Jul 15, 2026Kai Hsu Tsai, Yong Wei Fu, Hung I Yang +1Dance-To-Music GenerationAudio-Video Generation

    Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan

  40. Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space

    Jul 7, 2026Thanh V. T. Tran, Ngoc-Son Nguyen, Luong Tran +4Audio-Video GenerationCross-Modal Alignment

    FPT Software AI Center, Vietnam · NVIDIA Corporation, USA

  41. Lights, Camera, Carbon: Architectural Scaling Laws for Video Generation Energy Consumption

    Jul 5, 2026Nidhal Jegham, Boris Gamazaychikov, Sasha LuccioniVideo Diffusion ModelsVideo Generation

    University of Rhode Island, Sustainable AI Group Rhode Island, United States · Sustainable AI Group Paris, France · Sustainable AI Group Montreal, Canada

  42. Conversational Human Audio-visual Talking Dialogue Generation

    Jul 2, 2026Junhao Song, Lluis Guasch, Xilin He +8Conversational DatasetsAudio-Video Generation

    Department of Computing, Imperial College London, UK · Department of Earth Science & Engineering, Imperial College London, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE +8

  43. AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

    Jun 29, 2026Kien T. Pham, I Chieh Chen, Qifeng Chen +1Audio-Video GenerationAudio Tokenizers

    The Hong Kong University of Science and Technology

  44. FacePlex: Toward Natural Full-Duplex Conversational Avatars

    Jun 29, 2026Habin Lim, Hah Min Lew, Jae-Ho Lee +4Precise Lip SynchronizationFull-Duplex Speech Models

    Korea University · Klleon

  45. MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control

    Jun 28, 2026Kaiqi Liu, Yunyao Mao, Ziqi Cai +8Audio-Video GenerationNarratives

    Peking University · Kling Team, Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences +1

  46. Integrating Facial Generation into Full-Duplex Spoken Dialogue Systems

    Jun 20, 2026Jingjing Jiang, Atsumoto Ohashi, Ryuichiro HigashinakaFull-Duplex Speech ModelsAudio-Video Generation

    Graduate School of Informatics, Nagoya University, Japan

  47. UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating

    Jun 19, 2026Jiehui Huang, Yuechen Zhang, Bin Xia +7Audio-Video GenerationMotion Coherence

    The Hong Kong University of Science and Technology · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +2

  48. FoleyGenEx: Unified Video-to-Audio Generation with Multi-Modal Control, Temporal Alignment, and Semantic Precision

    Jun 12, 2026Shiyao Wang, Xijuan Zeng, Hui Wang +4Audio-Video Generation

    Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin, China · Kling Team, Kuaishou Technology, Beijing, China

  49. CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation

    Jun 8, 2026Yuheng Chen, Teng Hu, Yuji Wang +7Audio-Video GenerationInteractive Video Generation

    Shanghai Jiao Tong University, Shanghai, China. · University of Electronic Science and Technology of China, Chengdu, China. · Zhejiang University, Hangzhou, China. +2

  50. dots.tts Technical Report

    Jun 5, 2026Shi Lian, Changtao Li, Bohan Li +6Autoregressive Text-To-SpeechAudio-Video Generation

  51. Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation

    Jun 2, 2026Xuan Wei, Jiahui Chen, Kaiheng Li +2Human Motion GenerationAudio-Video Generation

    Department of Digital Media Technology, School of Film, Xiamen University, Xiamen, China · 2National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China

  52. Inference-Time Scaling for Joint Audio-Video Generation

    Jun 2, 2026Jaemin Jung, Kyeongha Rho, Inkyu Shin +1Audio-Video GenerationPerceptual Quality

    Korea Advanced Institute of Science and Technology · Luma AI

  53. JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

    Jun 1, 2026Jiashuo Yu, Yao Yao, Boyu Chen +1Audio-Video GenerationGenerative Music

    Jen Music AI

  54. Benchmarking Single-Factor Physical Video-to-Audio Generation

    May 28, 2026Tingle Li, Siddharth Gururani, Kevin J. Shih +6Audio-Video Generation

    UC Berkeley · NVIDIA · University of Washington

  55. Native Audio-Visual Alignment for Generation

    May 28, 2026Longbin Ji, Guan Wang, Xuan Wei +6Audio-Video GenerationAudio-Visual Consistency

    ERNIE Team, Baidu Inc.

  56. MTAVG-Bench 2.0: Diagnosing Failure Modes of Cinematic Expressiveness in Multi-Talker Audio-Video Generation

    May 27, 2026Haitian Li, Yanghao Zhou, Heyan Huang +15Audio-Video GenerationAudio-Visual Consistency

    Shanghai University · Tsinghua University · Beijing Institute of Technology +7

  57. LongCat-Video-Avatar 1.5 Technical Report

    May 26, 2026Meituan LongCat Team, Xunliang Cai, Meng Cheng +10Audio-Video GenerationVisual Fidelity

    Meituan LongCat Team

  58. LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV

    May 25, 2026Tengfei Liu, Yang Shi, Xuanyu Zhu +17Audio-Video GenerationAudio-Visual Consistency

    Peking University · Kling Team · Nanjing University +6

  59. StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestration

    May 25, 2026Linrui Tian, Qi Wang, Bang ZhangAudio-Video GenerationStreaming

    Tongyi Lab, Alibaba Group

  60. Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation

    May 25, 2026Zhicheng Zhang, Lei Wang, Yu Zhang +1Audio-Video GenerationGenerative Video Models

    School of Business, University of New South Wales (UNSW), Australia · School of Engineering and Built Environment, Griffith University, Australia