Audio-Video Generation

Ask about this topic

Momentum

14 papers in the last four weeks, up 27% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 99

All topics
CardsList

  1. WorldSonus: Bringing Sound to Worlds

    Oct 6, 2026Pengjun Fang, Jingyi Fa, Kam Man Wu +9Audio-Video GenerationAudio Diffusion Models

    The Hong Kong University of Science and Technology · Noiz AI · MetaX +1

  2. UltraDub: Towards Authentic Dubbing by Unifying Visually-Steered Flow Learning and Trajectory Guidance

    Oct 5, 2026Gaoxiang Cong, Liang Li, Jianwei Wen +3Audio-Video GenerationAudio-Visual Alignment

    Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Hangzhou Dianzi University +2

  3. Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation

    Oct 4, 2026Team Kandinsky, Julia Agafonova, Bulat Akhmatov +85Audio-Video GenerationVideo Diffusion Models

  4. Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training

    Oct 4, 2026Shuyuan Tu, Qi Tian, Yinming Huang +8Audio-Video GenerationVideo Generation

    Fudan University · Tencent Hunyuan Foundation Model Team · Zhejiang University

  5. Soundwich: Video Generation with Layered and Controllable Audio

    Sep 30, 2026Zhuo Ning, AmirHossein Naghi Razlighi, Sagi Polaczek +2Audio-Video GenerationAudio-Visual Synchronization

    Simon Fraser University · University of Cyprus · CYENS Centre of Excellence +1

  6. PLACE: Positional Latent Adaptation via Conditioned Embeddings for Binaural Audio Generation

    Sep 30, 2026Tiernon Riesenmy, You Zhang, Gautam Bhattacharya +1Text-to-Audio GenerationAudio-Video Generation

    The Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA, USA · Advanced Technology Group, Dolby Laboratories, San Francisco, CA, USA

  7. Here the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation

    Sep 30, 2026Hanmo Chen, Chengcheng Liu, Tianxiao Chen +7Audio-Video GenerationAudio-Visual Alignment

    Xidian University · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd.

  8. Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation

    Sep 29, 2026Xingtong Ge, Yutong Wang, Lunjie Zhu +7Audio-Video GenerationVideo Diffusion Models

    The Hong Kong University of Science and Technology · Vivix Group Limited · The University of Sydney +1

  9. AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenes

    Sep 29, 2026Weihan Xu, Kan Jen Cheng, Koichi Saito +8Audio-Video GenerationAudio Editing

    University of Washington · Massachusetts Institute of Technology · University of Maryland, College Park +3

  10. Enabling Immersive Audio-Visual Experience from Any Video

    Sep 28, 2026Zitong Lan, Mutian Tong, Jiatao Gu +1Audio-Video GenerationAudio-Visual Alignment

    University of Pennsylvania

  11. ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

    Sep 28, 2026Jiacheng Hua, Xiaokun Feng, Jiaqi Hua +3Audio-Video GenerationControllable Video Generation

    College of AI, Tsinghua University · Tencent Hy

  12. Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

    Sep 28, 2026Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21Audio-Video GenerationMultimodal Generation

    University of Massachusetts Amherst · University of Texas at Dallas · Virginia Tech +10

  13. Video-to-Music Generation for Gameplay Videos

    Sep 25, 2026Felipe Marra, Lucas N. FerreiraAudio-Video GenerationMusic Generation

    Universidade Federal de Viçosa Departamento de Informática Viçosa, Minas Gerais, Brazil · Universidade Federal de Minas Gerais Departamento de Ciência da Computação Belo Horizonte, Minas Gerais, Brazil

  14. AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

    Sep 24, 2026Zhiyu Xu, Weilong Yan, Yufei Shi +4Audio-Video GenerationAudio-Visual Alignment

    Shanghai AI Laboratory · The Hong Kong Polytechnic University · National University of Singapore +2

  15. Decoupled Self-Forcing Distillation for Streaming Talking Head Generation

    Sep 9, 2026Yanru An, Ruiyan Wang, Wenwu Wei +7Audio-Video GenerationDiffusion Model Distillation

    Shanghai Jiao Tong University · Ant Group

  16. Beyond Coherence: Benchmarking Professional Editing-Technique Execution in Multi-Shot Audio-Video Generation

    Sep 8, 2026Tianyi Zeng, Junchao Liao, Yujie Wei +9Video Editing BenchmarksAudio-Video Generation

    2Shanghai Jiao Tong University · *Work done during internship at Alibaba · 1Alibaba Group +4

  17. The Attention Triangle in Audio-Video Models

    Sep 3, 2026Sagi Polaczek, Noa Kraicer, Gal Metzer +4Audio-Video GenerationAttention Control in Diffusion Models

    1Tel Aviv University · 2Simon Fraser University

  18. TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models

    Sep 1, 2026Chao Zhou, Yiling Chen, Qi Chu +3Audio-Video GenerationLatent Diffusion Models

  19. DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution

    Aug 31, 2026Jiashu Zhu, Yanhao Zheng, Ruitian Tian +7Audio-Video GenerationVideo Diffusion Models

    DreamX Team, Alibaba Group

  20. TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

    Aug 25, 2026Xiaoda Yang, Yuxiang Liu, Kaiwen Zheng +12Audio-Video GenerationDiffusion Model Distillation

    Zhejiang University, Hangzhou, China · Tianjin University, Tianjin, China · Tsinghua University, Beijing, China +4

  21. VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

    Aug 19, 2026Yinming Huang, Shuyuan Tu, Xi Yan +6Reward ModelingAudio-Video Generation

    Fudan University · Shanghai Innovation Institution · Yinwang Intelligent Technology Co., Ltd +1

  22. Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation

    Aug 14, 2026Jian Xu, Delu Zeng, John PaisleyAudio-Video GenerationCausal Inference

    RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1

  23. Avatar-Forever: Decoupled Parallel Training for High-Quality Real-Time Infinite Avatars

    Aug 12, 2026Ruibin Li, Tao Yang, Zhiyuan Ma +3Audio-Video GenerationReal-Time Interactive Video Generation

    The Hong Kong Polytechnic University · ByteDance · AMD

  24. HarmoniDPO: Video-guided Audio Generation via Preference-Optimized Diffusion

    Aug 12, 2026Wenshuo Peng, Kaipeng ZhangAudio-Video GenerationAudio Diffusion Models

    Electornic Engineering Department, Tsinghua University, Haidian District, Beijing, 100084 China. · OpenGVlab, Shanghai AI Laboratory, Xuhui District, Shanghai, 200030, China.

  25. UniSwap: Streaming Audio-Visual Identity Swapping for Talking Videos

    Aug 12, 2026Yuxuan Zhang, Haozhong Xiong, Jiayi Song +5Audio-Video GenerationVideo Diffusion Models

    The Chinese University of Hong Kong · Qwen Applications Business Group of Alibaba

  26. Dialogue-Aware Video-to-Music Generation Using Public Domain Film Collections

    Aug 12, 2026Haven Kim, Zachary Novack, Julian McAuley +1Audio-Video GenerationMusic Generation

    University of California San Diego · University of Michigan

  27. Ex-Omni-2D: Expressive Omni-Modal Dialogue Models with Native Visual Presence

    Aug 11, 2026Haoyu Zhang, Zhipeng Li, Xiaoying Tang +2Audio-Video GenerationStreaming Video Generation

    The Chinese University of Hong Kong, Shenzhen · LIGHTSPEED · Independent Researcher

  28. Vorch-Omni: Multi-Task Orchestration of Sight and Sound

    Aug 6, 2026Vorch Team, Xiaoyu Chen, Yang Ding +25Audio-Video GenerationUnified Multimodal Models

  29. Vorch-Director: Interactive World Story Model via Noise-Aware Error Rectification

    Aug 6, 2026Lisai Zhang, Yidi Wu, Qi Liu +7Audio-Video GenerationLong-Horizon Video Generation

    1Vorch Team · 2Zhejiang University

  30. Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation

    Aug 5, 2026Zehua Chen, Junyou Wang, Yuxuan Jiang +5Audio-Video GenerationAudio Diffusion Models

    Tsinghua University, Beijing, China · Hefei University of Technology, Hefei, China · Monash University, Australia +1

  31. EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

    Aug 3, 2026Jiayu Chen, Xiaoyu Wu, Rongshan Gao +6Audio-Video GenerationAudio-Visual Alignment

    Peking University Beijing, China · Taiyuan University of Technology Taiyuan, China

  32. AcoustiTrace: When Plausible Sound Violates Physics

    Aug 3, 2026Shiyang Li, Yuewen Cao, Yihao Liu +4Physical Consistency in Video GenerationAudio-Video Generation

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University +3

  33. InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videos

    Aug 2, 2026Chi Zhang, Haoyang Shi, Yueyi Liu +4Audio-Video GenerationMultimodal Generation

    College of AI, Tsinghua University

  34. LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

    Jul 29, 2026Rongxiang Zhang, Songhua LiuAudio-Video GenerationDiffusion Model Distillation

    School of Artificial Intelligence, Shanghai Jiao Tong University · 2Harbin Institute of Technology

  35. Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory

    Jul 29, 2026Yanbo Ding, Zhizhi Guo, Quanyue Song +4Audio-Video GenerationLong Video Generation

    Shenzhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · School of Artificial Intelligence, University of Chinese Academy of Sciences +2

  36. TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

    Jul 27, 2026Qijun Gan, Chenwei Zhang, Meiguang Jin +2Audio-Video GenerationMemory-Augmented Video Generation

    Taobao & Tmall Group of Alibaba · Nanjing University

  37. OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation

    Jul 26, 2026Jun Zhan, Chen Yang, Yitian Gong +23Variational AutoencodersAudio-Video Generation

    1Fudan University · 2Shanghai Innovation Institute · 3MOSI Intelligence +1

  38. OmniMate: Open-Ended Real-Time Streaming Audio-Visual Generation for Interactive Avatars

    Jul 25, 2026Quanyue Song, Yishan He, Yanbo Ding +4Audio-Video GenerationReal-Time Interactive Video Generation

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, China · China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd., China · Shenzhen Key Laboratory of Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

  39. MultiRef-Compass: Towards Comprehensive Evaluation of Multi-Reference-to-Audio-Video Generation

    Jul 15, 2026Xiaohan Zhang, Yuqing Wen, Junlin Chen +9Audio-Video GenerationAudio-Visual Alignment

    Nanjing University · Kling Team · National University of Singapore +3

  40. Bring Music The Horizon: Music-Driven 360∘^\circ Video Generation

    Jul 15, 2026Kai Hsu Tsai, Yong Wei Fu, Hung I Yang +1Audio-Video Generation

    Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan

  41. Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models

    Jul 10, 2026Yijie Qian, Juncheng Wang, Chao Xu +6Audio-Video GenerationGroup Relative Policy Optimization

    Zhejiang University, Hangzhou, China · IROOTECH TECHNOLOGY, China · The Hong Kong Polytechnic University, Hong Kong, China +1

  42. Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space

    Jul 7, 2026Thanh V. T. Tran, Ngoc-Son Nguyen, Luong Tran +4Audio-Video GenerationAudio-Visual Synchronization

    FPT Software AI Center, Vietnam · NVIDIA Corporation, USA

  43. Wan-Streamer v0.2: Higher Resolution, Same Latency

    Jul 5, 2026Lianghua Huang, Zhi-Fan Wu, Yupeng Shi +23Audio-Video GenerationReal-Time Interactive Video Generation

    Wan Team, Alibaba Group

  44. Conversational Human Audio-visual Talking Dialogue Generation

    Jul 2, 2026Junhao Song, Lluis Guasch, Xilin He +8Audio-Video GenerationSpoken Dialogue Systems

    Department of Computing, Imperial College London, UK · Department of Earth Science & Engineering, Imperial College London, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE +8

  45. Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation

    Jun 30, 2026Baiqin Wang, Sen Chen, Jiankuo Zhao +3Audio-Video GenerationTalking Head Generation

    MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · CAIR, HKISI, Chinese Academy of Sciences +1

  46. AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

    Jun 29, 2026Kien T. Pham, I Chieh Chen, Qifeng Chen +1Audio-Video GenerationAudio-Visual Alignment

    The Hong Kong University of Science and Technology

  47. FacePlex: Toward Natural Full-Duplex Conversational Avatars

    Jun 29, 2026Habin Lim, Hah Min Lew, Jae-Ho Lee +4Audio-Video GenerationTalking Head Generation

    Korea University · Klleon

  48. OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data

    Jun 29, 2026Kaixing Yang, Jiashu Zhu, Xulong Tang +8Audio-Video GenerationMultimodal Generation

    Renmin University of China, Beijing, China · AMAP, Alibaba Group, Beijing, China · Malou Tech Inc, Plano, Texas, USA +2

  49. MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control

    Jun 28, 2026Kaiqi Liu, Yunyao Mao, Ziqi Cai +8Audio-Video GenerationAudio-Visual Alignment

    Peking University · Kling Team, Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences +1

  50. Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models

    Jun 23, 2026Lianghua Huang, Zhi-Fan Wu, Wei Wang +22Audio-Video GenerationUnified Multimodal Models

    Wan Team, Alibaba Group

  51. UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating

    Jun 19, 2026Jiehui Huang, Yuechen Zhang, Bin Xia +7Audio-Video GenerationMemory-Augmented Video Generation

    The Hong Kong University of Science and Technology · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +2

  52. MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model

    Jun 16, 2026Lichen Bai, Tianhao Zhang, Shitong Shao +14Audio-Video GenerationReal-Time Interactive Video Generation

    Catnip AI Team

  53. FoleyGenEx: Unified Video-to-Audio Generation with Multi-Modal Control, Temporal Alignment, and Semantic Precision

    Jun 12, 2026Shiyao Wang, Xijuan Zeng, Hui Wang +4Audio-Video GenerationAudio-Visual Synchronization

    Academy for Advanced Interdisciplinary Studies, Nankai University, Tianjin, China · Kling Team, Kuaishou Technology, Beijing, China

  54. ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance

    Jun 11, 2026Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral +1Audio-Video GenerationRL for Video Generation

    Télécom Paris, Institut Polytechnique de Paris · Max Planck Institute for Informatics

  55. CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation

    Jun 8, 2026Yuheng Chen, Teng Hu, Yuji Wang +7Audio-Video GenerationText-to-Video Generation

    Shanghai Jiao Tong University, Shanghai, China. · University of Electronic Science and Technology of China, Chengdu, China. · Zhejiang University, Hangzhou, China. +2

  56. Resonant Minds: Closed-Loop Social Avatars with Theory of Mind

    Jun 4, 2026Jianxu Shangguan, Jing Xu, Hang Ye +4Audio-Video GenerationTheory of Mind

    University of Washington · 2Peking University · 3Carnegie Mellon University +1

  57. Foley-Omni: A Unified Multimodal Generation Model from Task-Level Audio Synthesis to Complete Video Soundtrack Generation

    Jun 2, 2026Ye Tao, Lupeng Liu, Xuenan Xu +6Audio-Video GenerationUnified Multimodal Models

    School of Intelligence Science and Technology, Nanjing University · Video Rebirth · Shanghai Jiao Tong University +2

  58. Inference-Time Scaling for Joint Audio-Video Generation

    Jun 2, 2026Jaemin Jung, Kyeongha Rho, Inkyu Shin +1Audio-Video GenerationInference-Time Scaling

    Korea Advanced Institute of Science and Technology · Luma AI

  59. JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

    Jun 1, 2026Jiashuo Yu, Yao Yao, Boyu Chen +1Audio-Video GenerationAudio-Visual Alignment

    Jen Music AI

  60. Real-Time Generation of Streamable Talking Portrait Video with Reference-Guided Deep Compression VAEs

    Jun 1, 2026Sicheng Xu, Yu Deng, Shoukang Hu +5Variational AutoencodersAudio-Video Generation

    1Microsoft Research · 2Microsoft AI