ReXMo: End-to-End Video Character Replacement with Expression and Motion Preservation
Organizations: College of Computer Science and Technology, Jilin University, Changchun, Jilin, China
Abstract
Video character replacement aims to replace actors while preserving facial expressions, motions, and scene dynamics. Existing methods typically rely on masked generation with pose guidance, limiting fine-grained expression and motion preservation in complex scenes. We propose ReXMo, an end-to-end framework that supports single-, multi-, and selective character replacement scenarios. ReXMo introduces paired triplet supervision, where synthesized driving videos and original videos share identical performances but different identities, enabling scalable training through synthetic data generation. An In-Context Diffusion Transformer learns identity transformation by jointly conditioning on driving videos, reference identities, and text prompts, while an Expression Enhancement Adapter improves facial expression preservation in multi-person scenes. Extensive experiments demonstrate that ReXMo outperforms existing methods in identity fidelity, motion consistency, expression preservation, and visual quality. Our dataset and code will be publicly released.
Figures & tables
| Setting | Method | Motion Preservation | Reconstruction Quality | ||||||
| Pose-DTW | [email protected] | AJVE | Accel.Err. | PSNR | LPIPS | SSIM | MSE | ||
| Single- | Wan-Animate | 0.0345 | 0.9824 | 0.0185 | 0.0278 | 18.85 | 0.2371 | 0.7947 | 847.1 |
| MultiAnimate | 0.0387 | 0.9791 | 0.0184 | 0.0281 | 18.55 | 0.2380 | 0.7864 | 907.7 | |
| SCAIL | 0.0298 | 0.9830 | 0.0179 | 0.0283 | 19.86 | 0.2323 | 0.8021 | 676.1 | |
| SCAIL-2 | 0.0263 | 0.9879 | 0.0177 | 0.0267 | 19.89 | 0.2349 | 0.8062 | 666.9 | |
| ReXMo (Ours) | 0.0258 | 0.9890 | 0.0174 | 0.0265 | 19.91 | 0.2315 | 0.8075 | 663.6 | |
| Method | Image Quality | Motion Smooth. | Temporal Consist. | Appearance Consist. |
| Wan-Animate | 3.78 | 3.88 | 3.93 | 4.18 |
| MultiAnimate | 3.71 | 3.74 | 3.84 | 3.90 |
| SCAIL | 4.18 | 3.92 | 3.98 | 4.20 |
| SCAIL-2 | 4.27 | 3.87 | 4.08 | 4.32 |
| ReXMo (Ours) | 4.32 | 3.90 | 4.12 | 4.39 |
| Method | F-LMD | M-LMD | Expr-LMD | LVD |
| Wan-Animate | 0.3345 | 0.3248 | 0.3189 | 0.0374 |
| MultiAnimate | 0.5013 | 0.4977 | 0.5086 | 0.0458 |
| SCAIL | 0.3309 | 0.3381 | 0.3485 | 0.0388 |
| SCAIL-2 | 0.3350 | 0.3411 | 0.3414 | 0.0394 |
| w/o EEA | 0.3341 | 0.3251 | 0.3415 | 0.0389 |
| w/ EEA | 0.2858 | 0.2929 | 0.2905 | 0.0288 |
Video Diffusion Models
Video diffusion models generate or edit temporally coherent video by iteratively denoising latent sequences conditioned on text, images, motion, or other signals.
See all 783 papers