AniGen: Unified $S^3$ Fields for Animatable 3D Asset Generation

Yi-Hua Huang; Zi-Xin Zou; Yuting He; Chirui Chang; Cheng-Feng Pu; Ziyi Yang; Yuan-Chen Guo; Yan-Pei Cao

cs.GRApr 9, 2026

AniGen: Unified $S^3$ Fields for Animatable 3D Asset Generation

Authors: Yi-Hua Huang, Zi-Xin Zou, Yuting He, Chirui Chang, Cheng-Feng Pu, Ziyi Yang, Yuan-Chen Guo, Yan-Pei Cao, +1 more

Organizations: The University of Hong Kong, China · VAST, China · The Chinese University of Hong Kong, China · Tsinghua University, China

Abstract

Animatable 3D assets, defined as geometry equipped with an articulated skeleton and skinning weights, are fundamental to interactive graphics, embodied agents, and animation production. While recent 3D generative models can synthesize visually plausible shapes from images, the results are typically static. Obtaining usable rigs via post-hoc auto-rigging is brittle and often produces skeletons that are topologically inconsistent with the generated geometry. We present AniGen, a unified framework that directly generates animate-ready 3D assets conditioned on a single image. Our key insight is to represent shape, skeleton, and skinning as mutually consistent $S^3$ Fields (Shape, Skeleton, Skin) defined over a shared spatial domain. To enable the robust learning of these fields, we introduce two technical innovations: (i) a confidence-decaying skeleton field that explicitly handles the geometric ambiguity of bone prediction at Voronoi boundaries, and (ii) a dual skin feature field that decouples skinning weights from specific joint counts, allowing a fixed-architecture network to predict rigs of arbitrary complexity. Built upon a two-stage flow-matching pipeline, AniGen first synthesizes a sparse structural scaffold and then generates dense geometry and articulation in a structured latent space. Extensive experiments demonstrate that AniGen substantially outperforms state-of-the-art sequential baselines in rig validity and animation quality, generalizing effectively to in-the-wild images across diverse categories including animals, humanoids, and machinery. Homepage: https://yihua7.github.io/AniGen-web/

Figures & tables

Figure 1. Given a single conditional image as input, AniGen generates a 3D shape along with its skeleton and skinning weights, supporting a wide range of 3D assets, including organic entities such as animals, cartoon characters, humans, and articulated man-made objects.

Figure 2. We present a case study combining TRELLIS ( Xiang et al., 2025b ) with state-of-the-art auto-rigging baselines for animatable asset generation. Existing methods produce skeletons with missing bones (UniRig ( Zhang et al., 2025 ) ), unstructured rigs (Anymate ( Deng et al., 2025 ) ), or poor skinning (Puppeteer ( Song et al., 2025a ) , RigAnything ( Liu et al., 2025 ) ). The resulting animations expose practical failure modes of this pipeline, whereas AniGen generates plausible skeletons and skinning that support stable character animation.

Figure 3. Overview of the AniGen framework. The top panel illustrates the encoding and decoding of the $S^{3}$ fields using the structured latent autoencoder $\mathcal{E}_{L}$ and $\mathcal{D}_{L}$ . From top to bottom, the three rows correspond to the shape, skin, and skeleton branches, respectively. The bottom panel depicts the generation pipeline for animatable 3D assets. Given an input image, the sparse structure flow model $\mathcal{G}_{S}$ predicts voxel scaffolds that serve as supports for the shape and skeleton. Conditioned on these supports, a structured latent flow $\mathcal{G}_{L}$ synthesizes the corresponding $S^{3}$ fields. Finally, the decoded outputs yield a 3D shape equipped with skeleton rigging and associated skinning, producing an animatable 3D asset.

Figure 4. Illustration of the (b) shape voxels $\mathcal{V}$ , (c) skeleton voxels $\mathcal{V}_{sk}$ , and the (d) confidence-decaying nearest-neighbor joint field.

Figure 5. Illustration of the conversion from a continuous field to a discrete skeleton. The voting results of the joint field in (a) are clustered to obtain discrete joints in (b) and parent nodes in (c). The skeletal topology is then determined by assigning each parent to its nearest joint as in (d).

Figure 6. SkinAE encodes skeleton joints

{j_{i}}

and parents

{p_{i}}

into joint skin features

{W^{j}_{i}}

, which are averaged by GT skin weights

{w_{k}^{gt}}

to obtain vertex skin features

{W^{v}_{k}}

. The decoder processes

{W^{j}_{i}}

and

{W^{v}_{k}}

via MLPs, producing

{\tilde{W}_{i}^{j}}

{\tilde{W}_{v}^{k}}

, and vertex temperatures

{\tilde{T}_{v}^{k}}

. Final skin weights are obtained through cross-attention and Softmax with temperature adjustment.

Figure 7. BVH-based skin transfer vs. nearest-vertex (NN) transfer. We implement a CUDA BVH supporting multi-device deployment and save/restore.

Figure 8. The architectures of the AniGen flow models,

\mathcal{G}S

and

\mathcal{G}L

. The input to

\mathcal{G}S

comprises noisy volumetric features encoding geometry,

z^{\prime s}

, and skeleton information,

z^{\prime s}{\mathrm{skl}}

\mathcal{G}L

processes noisy structured latents representing geometry,

z^{\prime}{\mathrm{s}}

, skin,

z^{\prime}{\mathrm{skin}}

, and skeleton,

z^{\prime}{\mathrm{skl}}

. These flow models predict the velocities of the noisy features and iteratively denoise them using the Euler method.

Method	Joint-to-Joint $\downarrow$	Joint-to-Bone $\downarrow$	Bone-to-Bone $\downarrow$	Wasserstein $\downarrow$	Gromov–Wasserstein $\downarrow$	Skin $\ell_{1}$ $\downarrow$	Skin $\ell_{2}$ $\downarrow$	Skin KL $\downarrow$
GT-Mesh + UniRig	0.190	0.180	0.166	0.260	0.389	0.0973	0.224	6.117
GT-Mesh + Anymate	0.169	0.161	0.145	0.232	0.368	0.0918	0.204	4.388
GT-Mesh + Puppeteer	0.235	0.228	0.212	0.239	0.322	0.0823	0.197	4.133
GT-Mesh + RigAnything	0.193	0.181	0.165	0.268	0.392	0.1018	0.223	6.161
TRELLIS ∗ + UniRig	0.205	0.192	0.179	0.269	0.397	0.0966	0.221	5.903
TRELLIS ∗ + Anymate	0.179	0.172	0.157	0.232	0.349	0.0919	0.203	4.221

Table 1. Quantitative evaluation of skeleton accuracy and skin metrics for various methods. Results include Chamfer distances, mm-space distances, Wasserstein-based distances, and skin metrics (

\ell_{1}

\ell_{2}

, KL divergence). The best scores are highlighted in bold, and the second-best scores are underlined. Note that TRELLIS ∗ refers to the finetuned TRELLIS model trained on the split train set, while GT-Mesh combined with rigging methods serves as a reference.

Figure 9. Qualitative comparison of skeleton and skin results across different methods. We visualize the predicted skeletons, corresponding skins, and demonstrate animations to evaluate the practical usability of the rigged assets. The dragon case represents a relatively simple example with a clear identity and four-limbed structure, while the flower case poses a more complex challenge due to its intricate and non-standard structure.

Figure 10. Qualitative results on in-the-wild images, including examples from real-world photographs, web content, and AI-generated imagery. The showcased objects span a wide range of categories, such as sea animals, household items, cartoon characters, pets, humans, birds, plants, and machinery. Edited poses are demonstrated to illustrate animation capabilities, with examples such as a whale swimming, a lamp adjusting its angle, a dog running, and a robotic arm picking up an apple. These results highlight the versatility and practical usability of AniGen for applications in animation, games, image editing, VR, and more.

Method	Chamfer $\downarrow$	F-Score $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$
TRELLIS	0.0475	0.77	24.25	0.126
TRELLIS ∗	0.0394	0.89	25.23	0.104
AniGen	0.0409	0.88	25.12	0.108

Table 2. Geometry evaluation on the rigged-domain test set. We report surface Chamfer distance, F-score, and image-space PSNR & LPIPS.

Method	TRELLIS ∗	TRELLIS ∗ +UniRig	TRELLIS ∗ +Anymate
Time (s) $\downarrow$	15	146	19
Method	TRELLIS ∗ +Puppeteer	TRELLIS ∗ +RigAnything	AniGen
Time (s) $\downarrow$	36	127	19

Table 3. Inference cost runtime comparison.

Figure 11. Skeleton generation with different joint density levels. Higher joint density enables flexible, human-like motions, while medium and sparse densities result in reduced flexibility, resembling real robots. This demonstrates the adaptability of the method to varying motion requirements.

Method	Bayesian	w/o-Conf	w/o-SkinAE	Ours
Joint-GW $\downarrow$	0.310	0.337	0.383	0.286
Skin-KL $\downarrow$	3.174	3.187	5.138	2.919

Table 4. Quantitative ablation on confidence modeling and SkinAE pretraining. Lower is better for both metrics.

Figure 12. Ablation study on confidence design and SkinAE pretraining. Without confidence learning or with Bayesian confidence, skeleton predictions are noisy and redundant (column 2,3). Without pretraining SkinAE, skin quality and skeleton structure degrade significantly (column 4). Our method ensures clean skeletons and accurate skin generation (column 5).

Explore similar work

Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation
May 13, 2026Nikitas Chatzis, Marios Loizou, Evangelos Kalogerakis3D Mesh Generation 3D Asset Generation
1Technical University of Crete · 2CYENS Center of Excellence · University of Massachusetts Amherst
UniMate: One Unified Model to Animate Diverse Skeletons
Sep 4, 2026Linzhan Mou, Jiahui Lei, Zhiyang Dou +4Text-to-Motion Generation Motion Diffusion Models
Princeton University, USA · University of California, Berkeley, USA · Massachusetts Institute of Technology, USA +1
SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapes
Jun 1, 2026Ye Tao, Yuxin Yao, Kendong Liu +23D Asset Generation
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China

May 13, 2026J/K move · Enter open · S save

Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation

Nikitas Chatzis, Marios Loizou, Evangelos Kalogerakis

1Technical University of Crete · 2CYENS Center of Excellence · University of Massachusetts Amherst

3D Mesh Generation 3D Asset Generation

Open paper AskPDF arXiv

Recent 3D generative models can synthesize high-quality assets, but their outputs are typically static: they lack the skeletal rigs, joint hierarchies, and skinning weights required for animation. This limits their use in games, film, simulation, virtual agents, and embodied AI, where assets must not only look plausible but also move plausibly. We introduce Rigel3D, a generative method for animation-ready 3D assets represented as rigged meshes. Unlike post-hoc auto-rigging methods that attach rigs to completed shapes, our method jointly models geometry and rig structure through coupled surface and skeleton structured latent representations. A rig-aware autoencoder decodes these representations into mesh geometry, skeleton topology, joint coordinates, and skinning weights, while a two-stage latent generative model synthesizes both surface and skeleton representations for image-conditioned generation. To support downstream animation workflows, we further introduce an open-vocabulary joint labeling module that embeds generated joints into a shared vision-language space, enabling correspondence to arbitrary retargeting templates. Experiments on large-scale rigged asset datasets demonstrate that our method generates diverse, high-quality animation-ready assets and outperforms existing rigging baselines across multiple metrics.

Abstract

Explore similar work

Rigel3D: Rig-aware Latents for Animation-Ready 3D Asset Generation

UniMate: One Unified Model to Animate Diverse Skeletons

SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapes