cs.CVDec 16, 2025

FacEDiT: Talking Head Video Editing via Facial Motion Infilling

Authors: Kim Sung-Bin, Joohyun Chang, David Harwath, Tae-Hyun Oh

Organizations: Dept. of Electrical Engineering, POSTECH · School of Computing, KAIST · The University of Texas at Austin

Abstract

Editing a local segment of a talking head video without reshooting the scene remains challenging and underexplored. Local edits, such as insertion, deletion, or substitution, require synthesizing a replacement segment while leaving the rest of the video unchanged. This involves variable-duration local rewriting while preserving identity, unedited regions, and boundary continuity, which standard speech-driven generation and lip synchronization methods do not directly address. Moreover, direct supervision is impractical, as it would require paired videos of the same person and scene that differ only in a local spoken segment. We instead formulate talking head video editing as facial motion infilling, a self-supervised pretext task that recovers masked facial motion from speech and surrounding motion context using ordinary video--speech pairs. Our key insight is that local video editing can be simulated during training by masking a motion span and reconstructing it from speech and visible motion context. Building on this formulation, we introduce FacEDiT, a mask-controlled talking head model with a local temporal attention bias and temporal smoothness regularization to improve lip--speech alignment and transition continuity. We also introduce FacEDiTBench, the first benchmark for talking head video editing, covering diverse edit types and lengths with dedicated evaluation metrics. Extensive experiments show that FacEDiT produces accurate, speech-aligned edits with strong identity preservation and seamless boundary transitions. Beyond editing, the same facial motion infilling model extends to portrait animation and lip synchronization by simply changing the mask pattern, establishing FacEDiT as a unified framework for talking head video editing and generation.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SyncEdit: Rethinking Lip Synchronization as Editing with Audio-Driven Diffusion Models

    Mar 10, 2026Lixiang Lin, Siyuan Jin, Jinshan ZhangLip SynchronizationVideo Editing

  2. LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

    Jul 29, 2026Rongxiang Zhang, Songhua LiuAudio-Video GenerationDiffusion Model Distillation

  3. Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation

    May 25, 2026Zhicheng Zhang, Lei Wang, Yu Zhang +1Test-Time AdaptationTalking Head Generation