Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image

Joohyun Kwon; Geonhee Sim; Gyeongsik Moon

cs.CVMar 16, 2026

Zero-Shot Reconstruction of Animatable 3D Avatars with Cloth Dynamics from a Single Image

Authors: Joohyun Kwon, Geonhee Sim, Gyeongsik Moon

Organizations: Korea University

Abstract

Existing single-image 3D human avatar methods primarily rely on rigid joint transformations, limiting their ability to model realistic cloth dynamics. We present DynaAvatar, a zero-shot framework that reconstructs animatable 3D human avatars with motion-dependent cloth dynamics from a single image. Trained on large-scale multi-person motion datasets, DynaAvatar employs a Transformer-based feed-forward architecture that directly predicts dynamic 3D Gaussian deformations without subject-specific optimization. To overcome the scarcity of dynamic captures, we introduce a static-to-dynamic knowledge transfer strategy: a Transformer pretrained on large-scale static captures provides strong geometric and appearance priors, which are efficiently adapted to motion-dependent deformations through lightweight LoRA fine-tuning on dynamic captures. We further propose the DynaFlow loss, an optical flow-guided objective that provides reliable motion-direction geometric cues for cloth dynamics in rendered space. Finally, we reannotate the missing or noisy SMPL-X fittings in existing dynamic capture datasets, as most public dynamic capture datasets contain incomplete or unreliable fittings that are unsuitable for training high-quality 3D avatar reconstruction models. Experiments demonstrate that DynaAvatar produces visually rich and generalizable animations, outperforming prior methods.

Figures & tables

Methods	Zero shot	Cloth dynamics	Single image
ExAvatar [ 30 ]	✗	✗	✗
IDOL [ 61 ]	✓	✗	✓
AniGS [ 40 ]	✗	✗	✓
LHM [ 38 ]	✓	✗	✓
PERSONA [ 44 ]	✗	✓	✓
MPMAvatar [ 23 ]	✗	✓	✗

Table 1: Comparison of existing avatar reconstruction methods and the proposed DynaAvatar. Each column indicates whether the method reconstructs avatars in a zero-shot manner ( i.e. , without subject-specific optimization), whether it supports cloth dynamics, and whether it operates from a single input image.

Figure 2: Overall pipeline of the proposed DynaAvatar. We first extract detailed geometry and appearance without cloth dynamics using a Static Transformer. Next, cloth dynamics are incorporated from motion history through a Dynamic Transformer. The final 3D avatar in canonical space is reconstructed using a Gaussian decoder and then animated and rendered with LBS and a 3DGS renderer. Since the canonical avatar already encodes motion-dependent cloth dynamics, the animation produced by LBS faithfully maintains these dynamics.

Figure 3: Visualization of the proposed DynaFlow loss. Our DynaFlow loss encourages the Gaussians at source locations (black-outlined white circles) to move toward the endpoints of the estimated flow vectors.

Figure 4: Comparison between (b) the original annotations and (c) our reannotations for the DNA-Rendering [ 5 ] (top) and Actors-HQ [ 19 ] (bottom) datasets.

Figure 5: Effectiveness of our Dynamic Transformer.

Figure 6: Rendered avatars in similar poses but different motion histories. (a) and (b) share similar poses, but (a) is falling in mid-air while (b) is jumping up. (c) and (d) also exhibit similar poses, with (c) walking backward and (d) falling in mid-air. Despite having nearly identical poses, their clothing appears clearly different because our Dynamic Transformer effectively handles different motion information.

Settings	PSNR↑	SSIM↑	LPIPS↓
Wo. Dynamic Transformer	22.57	0.952	0.068
DynaAvatar (Ours)	23.62	0.958	0.062

Table 2: Effectiveness of our Dynamic Transformer on 4D-Dress.

Figure 7: Effectiveness of our static-to-dynamic knowledge transfer with LoRA.

Figure 8: Effectiveness of our DynaFlow loss function.

Figure 9: Comparison between DynaAvatar and previous single-image–based state-of-the-art methods [ 61 , 38 ] on in-the-wild images.

Methods	DNA-Rendering [ 5 ]			4D-Dress [ 48 ]			Actors-HQ [ 19 ]
	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
IDOL [ 61 ]	17.84	0.902	0.155	21.31	0.948	0.077	20.93	0.910	0.138
PERSONA [ 44 ]	14.91	0.883	0.207	19.46	0.943	0.098	19.08	0.904	0.171
LHM [ 38 ]	17.42	0.901	0.169	21.03	0.950	0.085	20.29	0.908	0.151
DynaAvatar (Ours)	19.45	0.916	0.136	23.74	0.960	0.064	21.38	0.916	0.128

Table 3: Comparison between DynaAvatar and previous single-image–based state-of-the-art methods.

Figure S1 : Comparison between DynaAvatar and previous single-image–based state-of-the-art methods [ 38 , 61 ] on in-the-wild images.

Methods	FC $\uparrow$
IDOL [ 60 ]	0.625
LHM [ 37 ]	0.697
DynaAvatar (Ours)	0.712

Table S1 : Comparison of face consistency (FC) on DNA-Rendering.

Methods	Zero-	Cloth	Time	# of
Methods	shot	dynamics	Time	params.
PERSONA [ 43 ]	✗	✓	3.11h	4M
LHM-500M [ 37 ]	✓	✗	1.08s	356M
LHM-1B [ 37 ]	✓	✗	3.00s	1.1B
DynaAvatar (Ours)	✓	✓	1.82s	719M

Table S2 : Comparison of computational costs.

Figure S2 : Comparison between physics-based method [ 8 ] and DynaAvatar.

Figure S3 : Comparison between diffusion-based method [ 57 ] and DynaAvatar.

Figure S4 : Comparison between (b) the original annotations and (c) our reannotations. The bounding box colors indicate the source datasets: Red denotes DNA-Rendering [ 5 ] , and Yellow denotes Actors-HQ [ 19 ] .

Settings	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
(1)	20.72	0.952	0.085
(2)	20.91	0.953	0.084
(3) (Ours)	23.74	0.960	0.064

Table S3 : Effectiveness of our dataset reannotations on 4D-Dress.

Explore similar work

AvatarDynamizer: From Static to Dynamic Human Avatars via Generative Dynamic Textures
Aug 20, 2026Guoxing Sun, Heming Zhu, Linjie Lyu +3Human Avatar Animation 3D Avatar Generation
Max Planck Institute for Informatics, Saarland Informatics Campus · EPFL · VIA Research Center
MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstruction
Apr 23, 2026Yisheng He, Steven Hoi3D Reconstruction 3D Avatar Generation
Tongyi Lab, Alibaba Group
Latent Dynamics for Full Body Avatar Animation
May 20, 2026Shichong Peng, Chengxiang Yin, Fei Jiang +7Latent Dynamics Modeling Human Motion Generation
Simon Fraser University, Canada · Codec Avatars Lab, Meta, USA

Aug 20, 2026J/K move · Enter open · S save

AvatarDynamizer: From Static to Dynamic Human Avatars via Generative Dynamic Textures

Guoxing Sun, Heming Zhu, Linjie Lyu, Pascal Fua+2

Max Planck Institute for Informatics, Saarland Informatics Campus · EPFL · VIA Research Center

Human Avatar Animation 3D Avatar Generation Texture Synthesis

Open paper AskPDF arXiv

For full-body avatars, modeling surface dynamics is crucial for overcoming the uncanny valley and achieving perceptual realism. Person-agnostic methods recover static 3D avatars from monocular images, videos, or text prompts, but their skeleton-driven animations lack realistic surface dynamics such as clothing wrinkles. In contrast, person-specific methods achieve high-quality rendering and realistic dynamics, but require expensive multi-view captures for each individual. Recent generalizable dynamic avatar methods struggle to embed surface dynamics, leading to either limited multi-view consistency or dynamic expressiveness. To this end, we propose AvatarDynamizer, a generative method that transforms an off-the-shelf static 3D avatar into a controllable, realistic, and multi-view-consistent 4D avatar. We introduce a novel texture-space surface-dynamics embedding and formulate avatar dynamics modeling as conditional texture generation. Our encoder--decoder representation embeds pose-dependent dynamics into dynamic texture maps, enabling compatibility with pre-trained video diffusion models while decoding them into 3D Gaussians for multi-view consistent rendering. Since existing datasets are limited in scale, sequence length, or motion diversity, we collect a large-scale multi-view dataset with long sequences covering diverse skeletal motions and surface dynamics. Experiments show that our method effectively animates static avatars with faithful surface dynamics and outperforms competing generalizable methods in visual fidelity, especially under limited dynamic training data.

Abstract

Explore similar work

AvatarDynamizer: From Static to Dynamic Human Avatars via Generative Dynamic Textures

MeshLAM: Feed-Forward One-Shot Animatable Textured Mesh Avatar Reconstruction

Latent Dynamics for Full Body Avatar Animation