Talking Head Generation

Follow this topic Ask about this topic

Talking head generation studies how to animate a person’s face or avatar so speech, lip movements, identity, and expressions remain coordinated. Approaches range from audio-conditioned diffusion and flow-matching models to autoregressive streaming systems and 3D methods using facial meshes or Gaussian splatting. Recent work adds controls for emotion, speaking style, and conversational reactions, while aiming to make generation interactive and efficient. A central challenge is balancing natural, temporally consistent motion and identity preservation with low latency, flexible control, and evaluation that reflects perceived quality.

Latest papers

41 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

No Distillation Needed: Single-Pass Real-Time Talking Heads via Acausal Noise Shaping
Oct 8, 2026Yu Han, Dejan Markovic, Alexander Richard +4Talking Head Generation Audio-Driven Facial Animation
Meta Reality Labs
Talk Like You: Imitating How You Speak in Real-Time Talking Head Generation
Oct 5, 2026Baiqin Wang, Zhixing Ding, Jijie Li +3Talking Head Generation Audio-Driven Facial Animation
MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · CAIR, HKISI, Chinese Academy of Sciences +1
GLARE: Generating Listening Heads with Appropriate Reactions
Sep 30, 2026Zikai Liao, Yumin Suh, Yi Ouyang +3Talking Head Generation
Department of Computer Science, Stony Brook University · Atmanity Inc.
MegaAvatar: Controllable Talking Avatar Generation
Sep 30, 2026Junyao Gao, Sibo Liu, Weidong Zhang +2Talking Head Generation 3D Avatar Generation
Tencent · Tongji University
EvolvingAvatar: Interactive 3D Head Generation That Adapts as Conversations Unfold
Sep 28, 2026Junjie Chen, Fei Wang, Kun Li +5Talking Head Generation 3D Avatar Generation
Hefei University of Technology · EPIC Lab, Shanghai Jiao Tong University · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center +4
Decoupled Self-Forcing Distillation for Streaming Talking Head Generation
Sep 9, 2026Yanru An, Ruiyan Wang, Wenwu Wei +7Audio-Video Generation Diffusion Model Distillation
Shanghai Jiao Tong University · Ant Group
CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation
Aug 15, 2026Peng Jia, Li Dai, Zhen Xiao +2Talking Head Generation Audio-Driven Facial Animation
Hefei University of Technology, Hefei 230009, Anhui, China
LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation
Jul 29, 2026Rongxiang Zhang, Songhua LiuAudio-Video Generation Diffusion Model Distillation
School of Artificial Intelligence, Shanghai Jiao Tong University · 2Harbin Institute of Technology
Conversational Human Audio-visual Talking Dialogue Generation
Jul 2, 2026Junhao Song, Lluis Guasch, Xilin He +8Audio-Video Generation Spoken Dialogue Systems
Department of Computing, Imperial College London, UK · Department of Earth Science & Engineering, Imperial College London, UK · Mohamed bin Zayed University of Artificial Intelligence, UAE +8
GaussianEmoTalker: Real-Time Emotional Talking Head Synthesis with Audio-Driven and Blendshape-Based 3D Gaussian Splatting
Jul 1, 2026Haijie Yang, Zhenyu Zhang, Yixuan Dong +23D Gaussian Splatting Talking Head Generation
PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, 215000, China · Tsientang Institute for Advanced Study, Hangzhou, 310000, China
Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation
Jun 30, 2026Baiqin Wang, Sen Chen, Jiankuo Zhao +3Audio-Video Generation Talking Head Generation
MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · CAIR, HKISI, Chinese Academy of Sciences +1
FacePlex: Toward Natural Full-Duplex Conversational Avatars
Jun 29, 2026Habin Lim, Hah Min Lew, Jae-Ho Lee +4Audio-Video Generation Talking Head Generation
Korea University · Klleon
Avatar V: Scaling Video-Reference Avatar Video Generation
Jun 11, 2026Benjamin Liang, Ce Chen, Desmond Lin +20Video-Reference Video Generation Identity-Preserving Video Generation
HeyGen Research
ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance
Jun 11, 2026Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral +1Audio-Video Generation RL for Video Generation
Télécom Paris, Institut Polytechnique de Paris · Max Planck Institute for Informatics
Temporally-Aligned Evaluation for Audio-Driven Talking Head Generation
May 31, 2026Zhicheng Zhang, Lei Wang, Yu Zhang +1Talking Head Generation Audio-Driven Facial Animation
School of Business, University of New South Wales (UNSW), Australia. · School of Engineering and Built Environment, Griffith University, Australia. · Data61/CSIRO, Australia.
TokTalk: Expressive Real-time Facial Animation from Audio-LLM Tokens
May 29, 2026Qingcheng Zhao, Yifang Pan, Karan SinghTalking Head Generation Human Avatar Animation
University of Toronto · Toronto, Canada
IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation
May 28, 2026Hao Wu, Xiangyang Luo, Hao Wang +3Talking Head Generation Audio-Driven Facial Animation
Information Engineering University · Huai’an University · Chongqing University of Post and Telecommunications +1
CapTalk: Text-Guided Stylization and Speech-Driven 3D Head Animation
May 28, 2026Xuangeng Chu, Yuan Gan, Ziteng Cui +4Audio-Video Generation Talking Head Generation
The University of Tokyo · Snap Research, Snap Inc. · RIKEN AIP
Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation
May 25, 2026Zhicheng Zhang, Lei Wang, Yu Zhang +1Test-Time Adaptation Talking Head Generation
School of Business, University of New South Wales (UNSW), Australia · School of Engineering and Built Environment, Griffith University, Australia
SDTalk: Structured Facial Priors and Dual-Branch Motion Fields for Generalizable Gaussian Talking Head Synthesis
May 11, 2026Peng Jia, Zhen Xiao, Jia Li +33D Gaussian Splatting Talking Head Generation
Hefei University of Technology, Hefei, China · University of Science and Technology of China, Hefei, China
MoCoTalk: Multi-Conditional Diffusion with Adaptive Router for Controllable Talking Head Generation
May 8, 2026Xinyan Ye, Jiankang Deng, Abbas EdalatVideo Diffusion Models Talking Head Generation
Imperial College London London, UK
AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation
May 1, 2026Yuxin Lu, Jiayang Sun, Guibo Zhu +1Video Diffusion Models Long Video Generation
Soochow University · Institute of Automation, Chinese Academy of Sciences
Generate Your Talking Avatar from Video Reference
Apr 30, 2026Zujin Guo, Zhenhui Ye, Yi Ren +4Video-Reference Video Generation Audio-Video Generation
Nanyang Technological University · HeyGen Research · University of Melbourne
Talking Slide Avatars: Open-Source Multimodal Communication Approach for Teaching
Apr 26, 2026Xinxing WuAudio-Video Generation Educational Technology
School of Mathematics and Computer Science, Kentucky State University, Frankfort, Kentucky, United States
Talker-T2AV: Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling
Apr 26, 2026Zhen Ye, Xu Tan, Aoxiong Yin +8Talking Head Generation Audio-Video Generation
Hong Kong University of Science and Technology · Independent Researcher · Zhejiang University +3
EmbodiedHead: Real-Time Listening and Speaking Avatar for Conversational Agents
Apr 19, 2026Yu Zhang, Kaiyuan Shen, Yang LiStreaming Video Generation Talking Head Generation
School of Computer Science and Technology, East China Normal University, Shanghai, China · Garabido Shanghai Technology Co., Ltd., Shanghai, China
TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation
Apr 16, 2026Xiangyu Liu, Feng Gao, Xiaomei Zhang +4Audio-Video Generation Video Diffusion Models
1MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Meituan +2
SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization
Apr 14, 2026Farzaneh Jafari, Stefano Berretti, Anup BasuAudio-Driven Facial Animation Talking Head Generation
University of Alberta, Edmonton, AB, Canada · University of Florence, Florence, Italy
Beyond Monologue: Interactive Talking-Listening Avatar Generation with Conversational Audio Context-Aware Kernels
Apr 11, 2026Yuzhe Weng, Haotian Wang, Xinyi Yu +4Audio-Video Generation Talking Head Generation
University of Science and Technology of China · iFLYTEK
PrivFedTalk: Privacy-Aware Federated Diffusion with Identity-Stable Adapters for Personalized Talking-Head Generation
Apr 9, 2026Soumya Mazumdar, Vineet Kumar Rakesh, Tapas SamantaTalking Head Generation Federated Learning
Department of Computer Science and Business Systems Gargi Memorial Institute of Technology Affiliated to Maulana Abul Kalam Azad University of Technology Balarampur, Mouza Beralia, Baruipur, Kolkata 700144, West Bengal, India · Engineering Sciences, Homi Bhabha National Institute Anushaktinagar, Mumbai, Maharashtra 400094, India · Computer and Informatics Group / Radioactive Ion Beam Facilities Group Variable Energy Cyclotron Centre 1/AF, Bidhannagar, Kolkata 700064, West Bengal, India

Oct 8, 2026J/K move · Enter open · S save

No Distillation Needed: Single-Pass Real-Time Talking Heads via Acausal Noise Shaping

Yu Han, Dejan Markovic, Alexander Richard, Wojciech Zielonka+3

Meta Reality Labs

Talking Head Generation Audio-Driven Facial Animation Facial Expression Synthesis Generative Adversarial Networks One-Step Generative Modeling

Open paper AskPDF arXiv

Audio-driven facial animation underpins real-time avatars, telepresence, and embodied virtual agents. And it must run online: each frame emitted from audio observed up to the current time, at interactive rates. Recent progress is dominated by diffusion models, which need many network evaluations per sample and are therefore a poor fit for streaming. We argue the cost is unnecessary in this domain. Audio-conditioned facial motion occupies a comparatively low-dimensional manifold, a regime where a single-pass GAN suffices. The obstacle is not capacity but stochastic structure. We show that a causal, time-invariant generator driven by i.i.d. noise cannot suppress its output spectrum over a band without collapsing its per-step innovation. We proposed FaceGAN, which dissolved the limitation by shaping the noise pathway acausally. Because the driving noise is synthetic, its future can be sampled now, so the audio-to-expression path stays causal, and the model supports fully causal operation. FaceGAN emits expression and head pose in a single forward pass per frame and matches or outperforms state-of-art approaches in generation quality. Being feed-forward with bounded attention windows, it generates indefinitely without drift.