FacEDiT: Talking Head Video Editing via Facial Motion Infilling
Organizations: Dept. of Electrical Engineering, POSTECH · School of Computing, KAIST · The University of Texas at Austin
Abstract
Editing a local segment of a talking head video without reshooting the scene remains challenging and underexplored. Local edits, such as insertion, deletion, or substitution, require synthesizing a replacement segment while leaving the rest of the video unchanged. This involves variable-duration local rewriting while preserving identity, unedited regions, and boundary continuity, which standard speech-driven generation and lip synchronization methods do not directly address. Moreover, direct supervision is impractical, as it would require paired videos of the same person and scene that differ only in a local spoken segment. We instead formulate talking head video editing as facial motion infilling, a self-supervised pretext task that recovers masked facial motion from speech and surrounding motion context using ordinary video--speech pairs. Our key insight is that local video editing can be simulated during training by masking a motion span and reconstructing it from speech and visible motion context. Building on this formulation, we introduce FacEDiT, a mask-controlled talking head model with a local temporal attention bias and temporal smoothness regularization to improve lip--speech alignment and transition continuity. We also introduce FacEDiTBench, the first benchmark for talking head video editing, covering diverse edit types and lengths with dedicated evaluation metrics. Extensive experiments show that FacEDiT produces accurate, speech-aligned edits with strong identity preservation and seamless boundary transitions. Beyond editing, the same facial motion infilling model extends to portrait animation and lip synchronization by simply changing the mask pattern, establishing FacEDiT as a unified framework for talking head video editing and generation.
Figures & tables
| Method / Setting | Duration-changing local edit | Unedited preservation | Boundary continuity | Portrait animation | Lip sync |
| Portrait animation Zhang et al. (2023) ; Wei et al. (2024) ; Xu et al. (2024) ; Cui et al. (2025a) ; Bigata et al. (2025) ; Wang et al. (2024) ; Chen et al. (2025c) ; Meng et al. (2025) ; Cui et al. (2025b) | No | No | No | Yes | Partial |
| Lip synchronization Prajwal et al. (2019) ; Prajwal et al. (2020) ; Zhou et al. (2021) ; Mukhopadhyay et al. (2024) ; Li et al. (2024) ; Peng et al. (2025) ; Ma et al. (2025) | Partial | Partial | Partial | No | Yes |
| FacEDiT | Yes | Yes | Yes | Yes | Yes |
| \RowStyle Method | LSE-D | LSE-C | IDSIM | FVD | P | M |
| Speech-driven portrait animation baselines Editing | ||||||
| V-Express Wang et al. (2024) | 7.719 | 6.381 | 0.845 | 201.392 | 40.668 | 11.470 |
| AniPortrait Wei et al. (2024) | 10.601 | 0.904 | 0.905 | 141.574 | 8.445 | 8.281 |
| EchoMimic Chen et al. (2025c) | 9.313 | 4.289 | 0.885 | 158.631 | 12.924 | 17.892 |
| EchoMimicV2 Meng et al. (2025) | 9.698 | 3.559 | 0.866 | 142.929 | 12.357 | 16.513 |
| SadTalker Zhang et al. (2023) | 7.644 | 6.202 | 0.851 | 112.951 | 8.243 | 7.442 |
| \RowStyle Edit duration | Avg. dur. | LSE-D | LSE-C | P | M | IDSIM | |
| 10–20 s | 18 | 15.89 s | 7.419 | 6.061 | 1.259 | 0.360 | 0.9474 |
| 20–25 s | 13 | 21.64 s | 7.740 | 5.975 | 1.204 | 0.312 | 0.9469 |
| 25–30 s | 9 | 26.93 s | 7.582 | 5.583 | 1.772 | 0.388 | 0.9290 |
| 30 s | 12 | 33.13 s | 7.673 | 5.178 | 1.501 | 0.549 | 0.9200 |
| \RowStyle | Fusion | Speech Enc. | Biased Attn. | LSE-D | LSE-C | IDSIM | FVD | P | M | |
| (a) | Early | WavLM | ✓ | ✗ | 7.292 | 6.581 | 0.965 | 63.229 | 2.447 | 0.825 |
| (b) | CA | WavLM | ✓ | ✗ | 7.179 | 6.673 | 0.966 | 61.955 | 2.466 | 0.829 |
| (c) | CA | Wav2Vec | ✓ | ✗ | 7.223 | 6.639 | 0.965 | 62.171 | 2.448 | 0.821 |
| (d) | CA | WavLM | ✗ | ✗ | 7.301 | 6.655 | 0.955 | 65.559 | 2.816 | 1.097 |
| (e) | CA | WavLM | ✓ | ✓ | 7.135 | 6.670 | 0.966 | 61.930 | 2.420 | 0.800 |
| \RowStyle Method | LSE-D | LSE-C | IDSIM | FVD | LPIPS |
| V-Express Wang et al. (2024) | 7.470 | 7.842 | 0.906 | 65.158 | 0.341 |
| AniPortrait Wei et al. (2024) | 11.168 | 2.866 | 0.914 | 102.414 | 0.310 |
| EchoMimic Chen et al. (2025c) | 9.085 | 5.555 | 0.929 | 89.129 | 0.320 |
| EchoMimicV2 Meng et al. (2025) | 9.966 | 4.051 | 0.890 | 131.603 | 0.322 |
| SadTalker Zhang et al. (2023) | 7.557 | 7.421 | 0.848 | 81.572 | 0.309 |
| Hallo Xu et al. (2024) | 7.824 | 7.080 | 0.925 | 42.825 | 0.283 |
| 3 Task | Ours | SadTalker Zhang et al. (2023) | Hallo Xu et al. (2024) | Hallo2 Cui et al. (2025a) | Hallo3 Cui et al. (2025b) | KeyFace Bigata et al. (2025) | |
| Editing | Overall | 1.57 1.40 | 3.93 1.38 | 3.58 1.29 | 3.90 1.40 | 3.32 1.51 | 4.70 1.51 |
| Boundary | 1.39 1.25 | 4.19 1.29 | 3.35 1.07 | 3.90 1.34 | 3.23 1.49 | 4.93 1.39 | |
| Portrait animation | Overall | 2.32 1.60 | 4.83 1.50 | 3.04 1.35 | 2.91 1.41 | 3.68 1.59 | 4.23 1.47 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Edit type | Original transcript | Edited transcript |
| Substitution (5 words) | However, if that naturally suits you , then do | However, if your heart tells you to , then do |
| Insertion (6 words) | And I forget the story behind it, but it is really cool to see | And I forget the story behind it, about how the festival first began , but it is really cool to see |
| Deletion (6 words) | So we are not looking at a murder, we are looking at negligence | So we are not looking at negligence |
| Method | LSE-D ( ) | LSE-C ( ) | IDSIM ( ) | FVD ( ) | LPIPS ( ) |
| V-Express [ 20 ] | 7.718 | 7.264 | 0.756 | 196.974 | 0.426 |
| AniPortrait [ 19 ] | 11.421 | 2.651 | 0.847 | 151.998 | 0.330 |
| EchoMimic [ 6 ] | 9.511 | 4.890 | 0.832 | 143.894 | 0.350 |
| EchoMimicV2 [ 10 ] | 10.136 | 3.819 | 0.817 | 124.219 | 0.347 |
| SadTalker [ 4 ] | 7.578 | 7.119 | 0.779 | 132.646 | 0.324 |
| Hallo [ 5 ] | 8.210 | 6.591 | 0.842 | 116.429 | 0.315 |
| Method | Inference (seconds) |
| Ours | 73.787 |
| V-Express [ 20 ] | 639.694 |
| AniPortrait [ 19 ] | 280.817 |
| EchoMimic [ 6 ] | 808.065 |
| EchoMimicV2 [ 10 ] | 512.349 |
| Variant | Velocity | Acceleration |
| w/o | 0.001983 | 0.001924 |
| w/ | 0.001872 | 0.001671 |
| Relative reduction | 5.6% | 13.2% |
| Method | LSE-D ( ) | LSE-C ( ) | IDSIM ( ) | P ( ) | M ( ) |
| Sparse selection | 7.769 | 5.137 | 0.968 | 3.609 | 1.587 |
| Interpolation | 7.716 | 5.215 | 0.954 | 3.139 | 0.829 |
| Our resampling | 7.336 | 5.588 | 0.960 | 3.118 | 0.825 |
Video Editing
Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.
See all 188 papers