RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation

Zhen Xiao; Zhen Shen; Zhaofan Qiu; Ting Yao; Xueliang Liu; Tao Mei

cs.CVAug 28, 2026

RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation

Authors: Zhen Xiao, Zhen Shen, Zhaofan Qiu, Ting Yao, Xueliang Liu, Tao Mei

Organizations: Hefei University of Technology, Hefei, China · HiDream.ai Inc., Hefei, China

Abstract

Cross-identity character animation aims to drive a target identity from a reference image to follow the motion of a source character from a driving video. The core challenge lies in the inherent entanglement of two capabilities: cross-identity spatial mapping (aligning position, scale, and skeletal proportions) and motion control (refining joint articulation, volumetric consistency, and view coherence). We introduce Reference-Aware Structural Alignment (RASA), a framework that disentangles spatial mapping from motion control by injecting structured priors into a Diffusion Transformer (DiT). Our approach has two stages. First, a Spatial Prior Calibrator (SPC) fuses reference identity with driving pose to generate a spatially grounded initial noise latent, ensuring correct positioning, scaling, and alignment with the driving skeleton. Second, an Inherent Motional Guider (IMG) encodes shape-agnostic SMPL articulation parameters into a semantic motion vector beyond appearance-biased 2D keypoints. Injected into intermediate DiT layers, this vector complements the base pose condition for anatomically consistent articulation and view-aware volumetric refinement. We curate CIM-Bench, a high-quality benchmark with rigorous curation, for evaluation. Extensive experiments show RASA significantly outperforms state-of-the-art methods in motion fidelity and visual quality. Our work establishes a new paradigm showing disentangled spatial and motional priors are key to robust character animation. Project page: https://hidream.ai.github.io/RASA/

Figures & tables

Figure 1: RASA enables high-fidelity character animation under diverse and structurally misaligned driving poses, including large motion variations, stylized identities, and multi-character animation.

Figure 2: The overview of RASA. Given a reference image and its extracted pose, a driving video and its corresponding pose sequence, as well as SMPL-based motion parameters, we first simulate structural misalignment by applying pose transformations to the driving poses. The motion parameters of the first frame are replicated three times to align temporally with the video latents. Through the Spatial Prior Calibrator module and the Inherent Motional Guider module, motion information is accurately transferred to the reference image while preserving structural consistency.

Figure 3: Illustration of CIM-Bench. Different identities perform the same motion sequence, creating authentic cross-identity structural discrepancies.

Method	Image Metrics				Video Metrics
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	FVD $\downarrow$	FID-VID $\downarrow$
DreamPose [ 24 ]	12.82	0.511	0.442	72.62	551.02	78.77
MagicAnimate [ 46 ]	-	0.714	0.239	32.09	179.07	21.75
Champ [ 57 ]	-	0.802	0.234	-	160.82	21.07
DisCo [ 42 ]	16.55	0.668	0.292	30.75	292.80	59.90
Animate Anyone [ 21 ]	17.40	0.734	0.287	60.70	453.00	60.00

Table 1: Quantitative results on the TikTok [ 22 ] dataset.

Method	Image Metrics				Video Metrics		Identity Metrics		Efficiency
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	FVD $\downarrow$	FID-VID $\downarrow$	Sim-Arc $\uparrow$	Face-FID $\downarrow$	Latency (s)	VRAM (MiB)
Animate-X [ 37 ]	14.05	0.426	0.474	105.14	1312.60	75.47	0.51	146.55	109	14,263
MTVCrafter [ 9 ]	14.19	0.431	0.457	85.21	1723.89	85.21	0.54	154.77	170	20,680
Wan-Animate [ 7 ]	11.29	0.372	0.585	178.58	1920.88	137.05	0.35	225.29	327	42,291
One-to-All-1.3B [ 31 ]	13.75	0.383	0.471	96.70	1324.74	76.79	0.51	163.99	91	21,880
Ours	15.93	0.472	0.372	60.96	667.96	17.63	0.72	90.63	61	11,951

Table 2: Quantitative results on the CIM-Bench.

Figure 4: Qualitative comparison for cross-identity character animation task under structurally misaligned scenarios.

SMS	FCA	IMG	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	FVD $\downarrow$	FID-VID $\downarrow$
✓			18.90/14.60	0.799/0.453	0.209/0.393	23.04/80.96	316.45/737.48	10.77/30.93
		✓	15.47/13.89	0.707/0.416	0.356/0.445	41.57/91.17	1567.03/1625.69	15.90/37.46
✓		✓	18.51/14.14	0.811/0.470	0.238/0.399	28.83/77.34	487.66/706.22	22.06/34.51
✓	✓		20.86/15.26	0.820/0.471	0.202/0.391	22.01/71.16	289.16/686.79	6.56/27.05
✓	✓	✓	22.03/15.93	0.830/0.472	0.189/0.372	18.93/60.96	190.82/667.96	3.32/17.63

Table 3: Ablation study on each key component of our model on TikTok [ 22 ] /CIM-Bench. SMS denotes performing random spatially transformation on driving poses. FCA denotes performing frame-wise cross attention on reference and driving pose. IMG denotes performing inherent motional guider.

Figure 5: Qualitative results of ablation study.

Figure 6: Ablation study of the FCA attention design in SPC module. Using the driving latent as the query preserves motion transfer, while the reference-query variant weakens motion dynamics.

Figure 7: Representative results under extreme proportions and non-humanoid topologies. The green boxes indicate successful cases under extreme body proportions, while the red boxes highlight failure cases involving non-humanoid topologies.

Human Avatar Animation

Human avatar animation builds digital people that can be reconstructed, driven by motion or speech, and rendered consistently across poses and views.

See all 91 papers

AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation
Mar 16, 2026Zhenyu Xie, Ji Xia, Michael Kampffmeyer +9Identity-Preserving Video Generation Human Motion Generation
Mohamed bin Zayed University of Artificial Intelligence, UAE · University of Tromsø (UiT) – The Arctic University of Norway, Norway · Shenzhen campus of Sun Yet-sen University, China
Wan-Animate-2: Pushing the Application Boundaries of Character Animation
Aug 6, 2026Guangyuan Wang, Li Hu, Dechao Meng +11Video Diffusion Models Real-Time Interactive Video Generation
Tongyi Lab, Alibaba Group
Kling-MotionControl Technical Report
Mar 3, 2026Kling Team, Jialu Chen, Yikang Ding +21Human Avatar Animation Controllable Video Generation
Kuaishou Technology

Mar 16, 2026J/K move · Enter open · S save

AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation

Zhenyu Xie, Ji Xia, Michael Kampffmeyer, Panwen Hu+8

Mohamed bin Zayed University of Artificial Intelligence, UAE · University of Tromsø (UiT) – The Arctic University of Norway, Norway · Shenzhen campus of Sun Yet-sen University, China

Identity-Preserving Video Generation Human Motion Generation Video Diffusion Models Pose-Conditioned Video Generation

Open paper AskPDF arXiv

Controllable character animation has advanced rapidly in recent years, yet multi-character animation remains underexplored. As the number of characters grows, multi-character reference encoding becomes more susceptible to latent identity entanglement, resulting in identity bleeding and reduced controllability. Moreover, learning precise and spatio-temporally consistent correspondences between reference identities and driving pose sequences becomes increasingly challenging, often leading to identity-pose mis-binding and inconsistency in generated videos. To address these challenges, we propose AnyCrowd, a Diffusion Transformer (DiT)-based video generation framework capable of scaling to an arbitrary number of characters. Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement. Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global foreground-background coordination. Furthermore, to mitigate token ambiguity in overlapping regions, an Adaptive Gated Fusion (AGF) module is integrated within TSDA to predict identity-aware weights, effectively fusing competing token groups into identity-consistent representations...

Abstract

Human Avatar Animation

Explore similar work

AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation

Wan-Animate-2: Pushing the Application Boundaries of Character Animation

Kling-MotionControl Technical Report