AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation
Organizations: Mohamed bin Zayed University of Artificial Intelligence, UAE · University of Tromsø (UiT) – The Arctic University of Norway, Norway · Shenzhen campus of Sun Yet-sen University, China
Abstract
Controllable character animation has advanced rapidly in recent years, yet multi-character animation remains underexplored. As the number of characters grows, multi-character reference encoding becomes more susceptible to latent identity entanglement, resulting in identity bleeding and reduced controllability. Moreover, learning precise and spatio-temporally consistent correspondences between reference identities and driving pose sequences becomes increasingly challenging, often leading to identity-pose mis-binding and inconsistency in generated videos. To address these challenges, we propose AnyCrowd, a Diffusion Transformer (DiT)-based video generation framework capable of scaling to an arbitrary number of characters. Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement. Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global foreground-background coordination. Furthermore, to mitigate token ambiguity in overlapping regions, an Adaptive Gated Fusion (AGF) module is integrated within TSDA to predict identity-aware weights, effectively fusing competing token groups into identity-consistent representations...
Figures & tables
| Methods | Self-Driven | Cross-Driven | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Frame-wise | Distribution-wise | ID Consistency | Distribution-wise | ||||||||
| PSNR | SSIM | LPIPS | FID | FID-VID | FVD | CLIP-I | DINO-I | FID | FID-VID | FVD | |
| StableAnimator [ 39 ] | 16.84 | 0.659 | 0.302 | 26.29 | 29.87 | 321.8 | 0.756 | 0.581 | 42.13 | 45.89 | 596.3 |
| UniAnimate-DiT [ 42 ] | 16.12 | 0.641 | 0.340 | 16.86 | 21.90 | 153.6 | 0.797 | 0.637 | 32.46 | 40.04 | 354.0 |
| Wan-Animate [ 5 ] | 18.16 | 0.688 | 0.272 | 17.03 | 18.76 | 160.5 | 0.770 | 0.568 | 33.21 | 39.32 | 431.4 |
| VACE [ 21 ] | 17.67 | 0.683 | 0.276 | 14.03 | 17.36 | 115.6 | 0.793 | 0.630 | 29.94 | 37.69 | 316.9 |
| Methods | Self-Driven | Cross-Driven | ||||||||||
| Frame-wise | Distribution-wise | ID Consistency | Distribution-wise | |||||||||
| PSNR | SSIM | LPIPS | FID | FID-VID | FVD | CLIP-I | DINO-I | FID | FID-VID | FVD | ||
| (a) | Baseline [ 21 ] | 17.67 | 0.683 | 0.276 | 14.03 | 17.36 | 115.6 | 0.793 | 0.630 | 29.94 | 37.69 | 316.9 |
| AnyCrowd | 18.76 | 0.692 | 0.257 | 12.73 | 14.85 | 99.22 | 0.811 | 0.678 | 29.42 | 36.80 | 307.4 | |
| AnyCrowd | 18.88 | 0.693 | 0.251 | 12.57 | 14.66 | 98.26 | 0.811 | 0.681 | 29.11 | 37.15 | 309.1 | |
| (b) | AnyCrowd | 18.47 | 0.686 | 0.258 | 12.62 | 14.58 | 97.48 | 0.813 | 0.682 | 29.13 | 36.98 | 310.9 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Benchmark | Persons / Clip | Total Clips |
|---|---|---|---|
| Single-character | DisPose [ 23 ] | 1 | 30 |
| HumanVid [ 43 ] | 1 | 80 | |
| UBC-Fashion [ 54 ] | 1 | 100 | |
| TikTokDataset [ 20 ] | 1 | 340 | |
| Multi-character | Multi-Character [ 49 ] | 2 | 20 |
| DanceTogEval-100 [ 4 ] | 2 | 100 |
Identity-Preserving Video Generation
Identity-preserving video generation keeps a person or character recognizable as appearance, pose, motion, and scenes change.
See all 65 papers