MultiAnimate: Pose-Guided Image Animation Made Extensible
Organizations: State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · ShanghaiTech University · Shanghai Jiao Tong University
Abstract
Pose-guided human image animation aims to synthesize realistic videos of a reference character driven by a sequence of poses. While diffusion-based methods have achieved remarkable success, most existing approaches are limited to single-character animation. We observe that naively extending these methods to multi-character scenarios often leads to identity confusion and implausible occlusions between characters. To address these challenges, in this paper, we propose an extensible multi-character image animation framework built upon modern Diffusion Transformers (DiTs) for video generation. At its core, our framework introduces two novel components-Identifier Assigner and Identifier Adapter - which collaboratively capture per-person positional cues and inter-person spatial relationships. This mask-driven scheme, along with a scalable training strategy, not only enhances flexibility but also enables generalization to scenarios with more characters than those seen during training. Remarkably, trained on only a two-character dataset, our model generalizes to multi-character animation while maintaining compatibility with single-character cases. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in multi-character image animation, surpassing existing diffusion-based baselines.
Figures & tables
| Dataset | Method | PSNR | SSIM | L1(E-3) | LPIPS | FVD | FID-VID |
| Swing Dance | Mimicmotion [ 48 ] | 13.47 | 0.390 | 145.10 | 0.501 | 780.31 | 49.69 |
| DisPose [ 19 ] | 14.08 | 0.444 | 137.69 | 0.466 | 746.29 | 46.24 | |
| UniAnimate-DiT [ 36 ] | 16.15 | 0.619 | 98.72 | 0.427 | 891.89 | 27.71 | |
| VACE [ 17 ] | 11.15 | 0.311 | 216.98 | 0.563 | 763.75 | 29.88 | |
| Ours ( Stage 1 model ) | 19.40 | 0.687 | 64.94 | 0.335 | 648.84 | 22.50 | |
| Gen-dataset | Mimicmotion [ 48 ] | 11.96 | 0.286 | 173.14 | 0.444 | 865.36 | 86.99 |
| Method | PSNR | SSIM | L1(E-3) | LPIPS | FVD | FID-VID |
| DisPose [ 19 ] | 17.17 | 0.691 | 74.97 | 0.261 | 615.27 | 64.74 |
| UniAnimate-DiT [ 36 ] | 17.76 | 0.781 | 68.16 | 0.337 | 649.30 | 50.16 |
| Ours | 23.68 | 0.867 | 33.65 | 0.250 | 342.48 | 41.85 |
Pose-Conditioned Video Generation
Pose-conditioned video generation uses body motion cues to steer video synthesis while preserving identity, appearance, and temporal coherence.
See all 27 papers