Audio-Driven Facial Animation

Overview

Audio-driven facial animation studies how speech can produce synchronized mouth movements and broader facial behavior in talking-head videos, 3D meshes, blendshapes, and Gaussian avatars. Systems use acoustic and phonetic features, often alongside text, emotion, identity, or conversational context, with Transformer, diffusion, and flow-matching models to generate motion or video. Current work aims to balance precise lip articulation with natural expression, identity preservation, personalization, and real-time or long-form generation. Evaluation remains an open challenge because frame-based metrics can miss timing differences, coarticulation, and whether motion looks natural to people.

Latest papers

CardsList

89 papers5 papers in the last four weeks, down 64% on the four weeks before. 0.0% of all new papers.

  1. LongCat-Video-Avatar 1.6 Technical Report

    Nov 13, 2026Meituan LongCat Team, Xunliang Cai, Yiqi Chen +8Audio-Video GenerationAudio-Driven Facial Animation

  2. Emo-LiveSync: Real-Time Universal Lip Synchronization with Explicit Emotion Control

    Nov 6, 2026Sangjoon Yu, Seungeun Lee, JoonHeyok Shin +1Lip SynchronizationAudio-Driven Facial Animation

  3. Fréchet Oral Distance: A Coordinate-Free Metric and Benchmark for Speech-Driven Oral Animation

    Nov 1, 2026Jaehong Myung, Seungho Eum, Hyunjung Chung +1Audio-Driven Facial Animation

  4. TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation

    Oct 29, 2026Xinmu Wang, Fumi Honda, Xiang Gao +3Human Motion GenerationCo-Speech Gesture Generation

  5. AudioLift: Audio-Driven Single Image to Full-Head Gaussian Avatar via Multi-view Video Generation

    Oct 22, 2026Zhuoyuan Jiang, Weijie Lyu, Xiaoyuan Wang +23D Avatar GenerationAudio-Driven Facial Animation

  6. CASE: Complementary Articulation-Aware Speech Encoder to Enhance Speech-Driven 3D Facial Animation

    Oct 18, 2026Jaehong Myung, Seungho Eum, Taegon Kim +2Audio-Driven Facial AnimationAudio-Visual Representation Learning

  7. Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation

    Oct 13, 2026Nikolay Semenov, Artem Riabinin, Sergey Bassenko +8Audio-Video GenerationAudio-Driven Facial Animation

  8. CatPortrait: Taming Video Generative Models for Portrait Animation via In-context Conditioning

    Oct 10, 2026Ziqi Ni, Zhou Fang, Shijian Wang +2Controllable Video GenerationVideo Generation

  9. Talk Like You: Imitating How You Speak in Real-Time Talking Head Generation

    Oct 5, 2026Baiqin Wang, Zhixing Ding, Jijie Li +3Talking Head GenerationAudio-Driven Facial Animation

  10. The Shape of Speech: A Geometric Measure of Coarticulation for Speech-Driven 3D Facial Animation

    Oct 2, 2026Danzel Serrano, Przemyslaw MusialskiAudio-Driven Facial Animation

  11. MegaAvatar: Controllable Talking Avatar Generation

    Sep 30, 2026Junyao Gao, Sibo Liu, Weidong Zhang +2Talking Head Generation3D Avatar Generation

  12. EvolvingAvatar: Interactive 3D Head Generation That Adapts as Conversations Unfold

    Sep 28, 2026Junjie Chen, Fei Wang, Kun Li +5Talking Head Generation3D Avatar Generation

  13. REALM: A Coarse-to-Fine Generative Framework for Embodied Reactive Listening

    Sep 27, 2026Peizhen Li, Longbing Cao, Yang ZhangAudio-Driven Facial AnimationFacial Expression Synthesis

  14. Where and When to Force: Routed Forcing for Streaming Avatars

    Sep 25, 2026Zihan Su, Siwen Lu, Junhao Zhuang +6Streaming Video GenerationDiffusion Model Distillation

  15. Seeing Speech: Learning Visible Articulatory Dynamics for Speech-Driven 3D Facial Animation

    Sep 24, 2026Hyung Kyu Kim, Byungchan Hwang, Hak Gu KimAudio-Driven Facial Animation

  16. ComplexSync: High-Fidelity and Real-Time Lip Sync in Complex Scenarios

    Sep 24, 2026Jiaran Cai, Xingpei Ma, Shenneng HuangVideo Diffusion ModelsLip Synchronization

  17. VibeAvatar: Aligning Phonetic Kinematics and Human Aesthetics for High-Fidelity Talking Avatar Synthesis

    Sep 16, 2026Qilin Wang, Mingyu Li, Hao TangHuman Avatar AnimationAudio-Video Generation

  18. Hi-Singers: A Comprehensive High-Quality Dataset for Expressive Audio-Driven Singing Head Synthesis

    Sep 7, 2026Yichi Zhang, Hui Zhang, Guanjun Liu +3Audio-Driven Facial Animation

  19. TBDub: Production-Oriented Visual Dubbing

    Sep 5, 2026Bihan Li, Xinyang Li, Zeran Xu +2Video Diffusion ModelsAudio-Driven Facial Animation

  20. InterSing: Explicit Interaction Dynamics for 3D Duet Singing Animation and Beyond

    Sep 4, 2026Yihan Zhou, Zikai Huang, Yuyang Yu +3Human Avatar AnimationAudio-Driven Facial Animation

  21. Audio-Driven Adversarial Defense for 3D Talking Face Generation with totally Visual Fidelity Preservation

    Aug 31, 2026Rui-Qing Sun, Chen-Hao Cui, Hui-Yang Zhao +3Audio-Driven Facial AnimationAdversarial Examples

  22. ECHO: Dyadic 3D Facial Motion Generation with Asymmetric Deterministic Articulation and Stochastic Reaction

    Aug 29, 2026Zhuoqiang Cai, Yujie Sun, Chaoyue Niu +4Human Avatar AnimationAudio-Driven Facial Animation

  23. EfficientSync: Real-Time Lip Synchronization via Deformation-Based Reference Texture Mixing

    Aug 19, 2026Fa-Ting Hong, Runzhen Liu, Luchuan Song +2Lip SynchronizationAudio-Visual Synchronization

  24. EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation

    Aug 19, 2026Harsh Kumar Agarwal, Xavier Alameda-Pineda, Olivier PerrotinHuman Motion GenerationCo-Speech Gesture Generation

  25. DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation

    Aug 18, 2026Yubo Huang, Sirui Zhao, Xinchen Yao +5Audio-Driven Facial AnimationSelf-Distillation

  26. SingDance: Compositional Zero-Shot Singing-and-Dancing Video Generation with Role-Aware Audio Conditioning

    Aug 17, 2026Tao Feng, Xu Li, Xiangyang Luo +4Video GenerationControllable Video Generation