AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation

Zhenyu Xie; Ji Xia; Michael Kampffmeyer; Panwen Hu; Zehua Ma; Yujian Zheng; Jing Wang; Zheng Chong

cs.CVMar 16, 2026

AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation

Authors: Zhenyu Xie, Ji Xia, Michael Kampffmeyer, Panwen Hu, Zehua Ma, Yujian Zheng, Jing Wang, Zheng Chong,

Organizations: Mohamed bin Zayed University of Artificial Intelligence, UAE · University of Tromsø (UiT) – The Arctic University of Norway, Norway · Shenzhen campus of Sun Yet-sen University, China

Abstract

Controllable character animation has advanced rapidly in recent years, yet multi-character animation remains underexplored. As the number of characters grows, multi-character reference encoding becomes more susceptible to latent identity entanglement, resulting in identity bleeding and reduced controllability. Moreover, learning precise and spatio-temporally consistent correspondences between reference identities and driving pose sequences becomes increasingly challenging, often leading to identity-pose mis-binding and inconsistency in generated videos. To address these challenges, we propose AnyCrowd, a Diffusion Transformer (DiT)-based video generation framework capable of scaling to an arbitrary number of characters. Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement. Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global foreground-background coordination. Furthermore, to mitigate token ambiguity in overlapping regions, an Adaptive Gated Fusion (AGF) module is integrated within TSDA to predict identity-aware weights, effectively fusing competing token groups into identity-consistent representations...

Figures & tables

Figure 2: Failure cases of baseline method [ 21 ] in multi-character scenarios. (a) Identity-Pose Mis-Binding ( red ): identity swap. (b) Identity Entanglement ( green ): appearance blending. Boxes track specific identities across Reference, GT, and generated frames.

Figure 3: Overview of AnyCrowd. (a) Instance-Isolated Latent Representation (IILR): The reference image with $C$ identities is decoupled into $C+1$ isolated images and encoded into identity-decoupled reference tokens. (b) Architecture: AnyCrowd is built upon a dual-stream DiT architecture, where the Context and DiT branches process conditioning signals and perform iterative denoising. (c) Tri-Stage Decoupled Attention (TSDA): This mechanism facilitates explicit identity-pose binding during the self-attention process, incorporating an Adaptive Gated Fusion (AGF) module to adaptively fuse overlapping tokens from different categories.

Figure 4: (a, b) Character number distribution of MCD-7K (train) and MCD-300 (test). (c) Preference results on MCD-300 under cross-driven setting.

Figure 5: Qualitative comparison on MCD-300 under the self-driven setting. Red dashed boxes highlight typical artifacts in baseline results. Please zoom in for details.

Figure 6: Qualitative comparison on MCD-300 under the cross-driven setting. Red dashed boxes highlight typical artifacts in baseline results. Please zoom in for details.

Figure 7: Ablation study. (a) Effectiveness of TSDA and AGF. (b) Necessity of IILR. (c) Influence of masking strategies (under cross-driven setting). Please zoom in for details.

Methods	Self-Driven								Cross-Driven
	Frame-wise			Distribution-wise			ID Consistency		Distribution-wise
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	FID-VID $\downarrow$	FVD $\downarrow$	CLIP-I $\uparrow$	DINO-I $\uparrow$	FID $\downarrow$	FID-VID $\downarrow$	FVD $\downarrow$
StableAnimator [ 39 ]	16.84	0.659	0.302	26.29	29.87	321.8	0.756	0.581	42.13	45.89	596.3
UniAnimate-DiT [ 42 ]	16.12	0.641	0.340	16.86	21.90	153.6	0.797	0.637	32.46	40.04	354.0
Wan-Animate [ 5 ]	18.16	0.688	0.272	17.03	18.76	160.5	0.770	0.568	33.21	39.32	431.4
VACE [ 21 ]	17.67	0.683	0.276	14.03	17.36	115.6	0.793	0.630	29.94	37.69	316.9

Table 1: Quantitative comparison of AnyCrowd with baselines on MCD-300.

	Methods	Self-Driven								Cross-Driven
		Frame-wise			Distribution-wise			ID Consistency		Distribution-wise
		PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	FID-VID $\downarrow$	FVD $\downarrow$	CLIP-I $\uparrow$	DINO-I $\uparrow$	FID $\downarrow$	FID-VID $\downarrow$	FVD $\downarrow$
(a)	Baseline [ 21 ]	17.67	0.683	0.276	14.03	17.36	115.6	0.793	0.630	29.94	37.69	316.9
	AnyCrowd $\star$	18.76	0.692	0.257	12.73	14.85	99.22	0.811	0.678	29.42	36.80	307.4
	AnyCrowd $\ast$	18.88	0.693	0.251	12.57	14.66	98.26	0.811	0.681	29.11	37.15	309.1
(b)	AnyCrowd $\dagger$	18.47	0.686	0.258	12.62	14.58	97.48	0.813	0.682	29.13	36.98	310.9

Table 2: Ablation study of core compoments on MCD-300.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Benchmark	Persons / Clip	Total Clips
Single-character	DisPose [ 23 ]	1	30
	HumanVid [ 43 ]	1	80
	UBC-Fashion [ 54 ]	1	100
	TikTokDataset [ 20 ]	1	340
Multi-character	Multi-Character [ 49 ]	2	20
	DanceTogEval-100 [ 4 ]	2	100

Appendix

Table 3: Quantitative comparison of benchmarks in the character animation field. While existing datasets primarily focus on single or low-density scenarios, our MCD-300 introduces a large-scale, high-density testbed specifically designed to advance the state-of-the-art in multi-character animation and motion fidelity in crowded environments.

Figure 8: The Interface of User Study Questionnaire.

Figure 9: (a) showcases the diversity of the MCD-7K with 2–6 performers across various dance styles, while (b) highlights high-density scenarios with 6–9 performers from MCD-300 to evaluate model performance under out-of-distribution conditions. Tracking masks and 2D pose annotations are superimposed for visualization.

Figure 10: More qualitative comparisons on MCD-300 under the self-driven setting. Please zoom in for details.

Figure 11: More qualitative comparisons on MCD-300 under the cross-driven setting. Please zoom in for details.

Figure 12: More application results of AnyCrowd. (a) Flexible Identity Input: AnyCrowd supports an arbitrary number of reference images from different sources. (b) One-to-Many Animation: A single reference character can be driven by diverse pose sequences. (c) Many-to-One Animation: Multiple distinct identities are animated simultaneously by a single driving pose sequence. (d) Arbitrary Pose-ID Mapping: AnyCrowd enables customized animation by reassigning specific identities to arbitrary target poses.

Figure 13: Limitation Analysis. (a) The model produces artifacts when the input pose sequence is extremely noisy or incomplete. (b) Fine-grained anatomical structures, such as hands, are prone to artifacts due to the 480p resolution training bottleneck.

Identity-Preserving Video Generation

Identity-preserving video generation keeps a person or character recognizable as appearance, pose, motion, and scenes change.

See all 65 papers

MultiAnimate: Pose-Guided Image Animation Made Extensible
Feb 25, 2026Yingcheng Hu, Haowen Gong, Chuanguang Yang +3Pose-Conditioned Video Generation Human Avatar Animation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · ShanghaiTech University · Shanghai Jiao Tong University
MultiAnimate: A Unified Framework for Controllable Multi-Character Animation
Jul 15, 2026Zhongyi Zhang, Guangyuan Wang, Li Hu +6Identity-Preserving Video Generation Human Motion Generation
School of Cyber Science and Technology, University of Science and Technology of China, Fuxing Road 100, Hefei, 230026, Anhui, China. · Tongyi Lab, Alibaba Group, Dengcai Road 1008, Hangzhou, 310023, Zhejiang, China. · College of Computing and Data Science, Nanyang Technological University, Jurong West, Singapore, 639798, Singapore, Singapore.
Wan-Animate-2: Pushing the Application Boundaries of Character Animation
Aug 6, 2026Guangyuan Wang, Li Hu, Dechao Meng +11Video Diffusion Models Real-Time Interactive Video Generation
Tongyi Lab, Alibaba Group

Feb 25, 2026J/K move · Enter open · S save

MultiAnimate: Pose-Guided Image Animation Made Extensible

Yingcheng Hu, Haowen Gong, Chuanguang Yang, Zhulin An+2

State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · ShanghaiTech University · Shanghai Jiao Tong University

Pose-Conditioned Video Generation Human Avatar Animation Image-to-Video Generation Video Diffusion Models

Open paper AskPDF arXiv

Pose-guided human image animation aims to synthesize realistic videos of a reference character driven by a sequence of poses. While diffusion-based methods have achieved remarkable success, most existing approaches are limited to single-character animation. We observe that naively extending these methods to multi-character scenarios often leads to identity confusion and implausible occlusions between characters. To address these challenges, in this paper, we propose an extensible multi-character image animation framework built upon modern Diffusion Transformers (DiTs) for video generation. At its core, our framework introduces two novel components-Identifier Assigner and Identifier Adapter - which collaboratively capture per-person positional cues and inter-person spatial relationships. This mask-driven scheme, along with a scalable training strategy, not only enhances flexibility but also enables generalization to scenarios with more characters than those seen during training. Remarkably, trained on only a two-character dataset, our model generalizes to multi-character animation while maintaining compatibility with single-character cases. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in multi-character image animation, surpassing existing diffusion-based baselines.

Abstract

Identity-Preserving Video Generation

Explore similar work

MultiAnimate: Pose-Guided Image Animation Made Extensible

MultiAnimate: A Unified Framework for Controllable Multi-Character Animation

Wan-Animate-2: Pushing the Application Boundaries of Character Animation