Organizations: BNRist, Department of Computer Science and Technology, Tsinghua University, Beijing, China · Zhili College, Tsinghua University, Beijing, China · VAST, Beijing, China
The rapid proliferation of generative 3D models has created a critical bottleneck in animation pipelines: rigging. Existing automated methods are fundamentally limited by their approach to skinning, treating it as an ill-posed, high-dimensional regression task that is inefficient to optimize and is typically decoupled from skeleton generation. We posit this is a representation problem and introduce SkinTokens: a learned, compact, and discrete representation for skinning weights. By leveraging an FSQ-CVAE to capture the intrinsic sparsity of skinning, we reframe the task from continuous regression to a more tractable token sequence prediction problem. This representation enables TokenRig, a unified autoregressive framework that models the entire rig as a single sequence of skeletal parameters and SkinTokens, learning the complicated dependencies between skeletons and skin deformations. The unified model is then amenable to a reinforcement learning stage, where tailored geometric and semantic rewards improve generalization to complex, out-of-distribution assets. Quantitatively, the SkinTokens representation leads to a 98%-133% percents improvement in skinning accuracy over state-of-the-art methods, while the full TokenRig framework, refined with RL, enhances bone prediction by 17%-22%. Our work presents a unified, generative approach to rigging that yields higher fidelity and robustness, offering a scalable solution to a long-standing challenge in 3D content creation.
Figures & tables
Figure 1. Automated rigging with TokenRig. We present TokenRig, a unified generative framework that produces high-quality rigs for diverse 3D assets. By leveraging SkinTokens, i.e., our novel, discrete representation for skinning weights, our method robustly generates high-fidelity skeletons and precise skinning maps (visualized as heatmaps) for complex, real-world geometries, ranging from stylized anime characters to quadrupeds and fantasy creatures.
Figure 2. Overview of the TokenRig Framework. Our method consists of three key stages: (1) Learning SkinTokens (Section 4.2.2 ): We first train a FSQ-CVAE ( Kingma and Welling, 2013 ; Sohn et al., 2015 ; Mentzer et al., 2024 ) to compress sparse skinning weights into a compact, discrete representation. Mesh geometry and skinning weights are processed by VecSet ( Zhang et al., 2023 ) encoders, and the resulting features are discretized into SkinTokens via Finite Scalar Quantization (FSQ) ( Mentzer et al., 2024 ) . We employ nested dropout ( Bachmann et al., 2025 ; Rippel et al., 2014 ) and importance sampling to ensure robust reconstruction of active deformation regions. (2) Unified Autoregressive Modeling (Section 4.3 ): We formulate rigging as a sequence generation task. A Transformer generates a single, unified sequence comprising the complete skeleton followed by the learned SkinTokens (from Stage 1), conditioned on global shape embeddings to capture structural dependencies. (3) RL Refinement via GRPO (Section 4.4 ): To improve generalization to in-the-wild assets, we fine-tune the model using Group Relative Policy Optimization (GRPO) ( Liu et al., 2024 ) . We introduce four specific rewards: Volumetric Joint Coverage (ensuring bone distribution), Bone-Mesh Containment (preventing protrusion), Skinning Coverage and Sparsity (ensuring valid weighting), and Deformation Smoothness (preventing artifacts during animation).
ModelsResource
VRoid Hub
Articulation 2.0
avg. N
1297.69
16929.78
6247.05
avg. J
19.87
95.56
34.46
avg. ∑I[w>0]
1700.18
28392.74
12655.45
avg. sparsity
7.40%
2.43%
9.38%
Table 1. Sparsity Analysis of Skinning Weights. Statistics across three major datasets ( Models-Resource, 2019 ; Isozaki et al., 2021 ; Song et al., 2025b ) reveal that skinning matrices are extremely sparse ( <10% active weights), motivating our design of the compressed SkinTokens representation.
Figure 3. Gradient Analysis of Loss Functions. A comparison of Binary Cross Entropy (BCE) and Dice loss ( Sudre et al., 2017 ) landscapes for a target weight w=0.2 . While both minimize at the correct value, Dice loss provides significantly larger gradients for non-zero targets ( wpred∈[0,1] ), effectively counteracting the extreme sparsity of skinning matrices where BCE gradients tend to vanish.
Figure 4. SkinTokens Reconstruction Fidelity. We evaluate the reconstruction quality (IoU and L1 Error) of the FSQ-CVAE across varying codebook sizes C and token sequence lengths TD . The results demonstrate that SkinTokens achieve high fidelity with as few as 4 tokens, validating the compressibility of skinning data. The configuration C=[8,8,8,6,5]=15,360 (lines with circles) is selected for our final model for its superior balance of compression and accuracy. The figure reports the IoU scores at ε=10−2 and corresponding L1 reconstruction errors on the Articulation 2.0 ( Song et al., 2025b ) test dataset
Codebook Size
Total Entries
Utilization
Compression Ratio∗
[8,8,8,5,6]
15360
99.7%
208.23×
[8,8,8,8,8]
32768
98.5%
195.22×
[8,8,8,5,5,5]
64000
97.3%
183.74×
Table 2. Codebook Utilization and Compression Analysis. We compare different FSQ codebook configurations ( C ). ∗ The compression ratio is computed against a raw FP16 baseline using the Articulation 2.0 ( Song et al., 2025b ) dataset. For example, storing skinning weights for an average mesh ( 6,247 vertices) typically requires 12,494 bytes; our representation ( TD=32 tokens) reduces this to just 64 ( 32×2 ) bytes, achieving a substantial reduction in storage requirements.
Figure 5. Learned Semantics of SkinTokens. A t-SNE visualization of the continuous latent vectors LW prior to quantization, sampled from 300 instances in the VRoid dataset ( Isozaki et al., 2021 ) . Points are colored by bone category (e.g., Head, Hips). The clear emergence of anatomical clusters indicates that the encoder captures a semantic structural prior, learning to represent “body part concepts” invariant to specific mesh geometries.
Figure 6. Qualitative Comparison of Skeleton Generation. We compare TokenRig (Ours) against state-of-the-art baselines. While baseline methods exhibit partial structures, missing details, or redundant joints, our method synthesizes structurally coherent and semantically faithful skeletons across diverse character types.
Figure 7. Qualitative Comparison of Skinning Prediction. We visualize predicted skinning weights and the corresponding average L1 error maps. Baseline methods often suffer from “bleeding” artifacts, where weights spill onto unconnected mesh parts (see UniRig/Puppeteer columns). TokenRig (Ours) produces clean, locally coherent influence maps that closely match the Ground Truth, particularly in fine-grained regions like fingers.
Stage 1: SkinTokens
Stage 2: TokenRig
(FSQ-CVAE)
(Autoregressive)
Architecture (structure only, trained from scratch)
Network
3DShape2Vecset
Qwen3-0.6B
Parameters
110 M
∼0.6 B
Layers
2 / 10 (enc./dec.)
0/28 (decoder-only)
Hidden width
768
1024
Table 3. Two-Stage Model and Training Configurations. We summarize the architecture and training recipe of the SkinTokens FSQ-CVAE (Stage 1) and the TokenRig autoregressive model (Stage 2). For both stages we adopt only the network structure of the referenced designs and train from scratch. Stage 2 is further refined via GRPO post-training ( 800 steps, learning rate 10−6 , group size G=24 , ϵ=0.2 , β=0.1 ; see Sec. 4.4 ).
Metric
FSQ (Ours)
VQ
CVAE (cont.)
Reconstruction fidelity
IoU@ 10−2↑
88.0%
87.6%
94.3%
L1 error ( ×10−2 ) ↓
1.00
1.03
0.45
Code usage & token repetition
Codebook utilization ↑
97.29%
65.90%
—
Token repetition rate @32 ↓
2.29%
8.92%
—
Table 4. Comparison of Tokenizer Choices. We compare FSQ (ours), a learnable VQ-VAE codebook, and a continuous-latent CVAE under an identical backbone and token budget. Both FSQ and VQ are evaluated with the same codebook size of 64,000 , with all other settings kept consistent with the configuration described above. The L1 error follows the same skinning reconstruction metric as Figure 4 (in units of 10−2 ).
ModelsResource
Articulation 2.0
J2J ↓
J2B ↓
B2B ↓
J2J ↓
J2B ↓
B2B ↓
RigNet ( Xu et al., 2020 )
3.901
2.412
2.213
7.376
5.841
4.802
MagicArticulate ( Song et al., 2025b )
3.024
2.260
1.915
4.003
3.026
2.586
Puppeteer ( Song et al., 2025a )
3.841
2.881
2.475
3.033
2.300
1.923
UniRig ( Zhang et al., 2025b )
3.390
2.592
1.890
3.115
2.211
1.926
TokenRig (4 skin tokens)
2.857
2.025
1.568
2.515
1.694
1.469
Table 5. Quantitative Comparison of Skeletal Generation. We evaluate skeletal structure accuracy using Chamfer Distance metrics: Joint-to-Joint (J2J), Joint-to-Bone (J2B), and Bone-to-Bone (B2B) on the ModelsResource ( Models-Resource, 2019 ) and Articulation 2.0 ( Song et al., 2025b ) datasets. Lower values ( ↓ ) indicate better performance. TokenRig consistently outperforms state-of-the-art baselines across all metrics, demonstrating superior fidelity in joint placement and bone connectivity.
ModelsResource ( Xu et al., 2020 )
Articulation 2.0 ( Song et al., 2025b )
Skin L1 ↓
L1 Var. ↓
Precision ↑
Recall ↑
Motion ↓
Skin L1 ↓
L1 Var. ↓
Precision ↑
Recall ↑
Motion ↓
RigNet ( Xu et al., 2020 )
0.0573
0.0464
62.4
59.9
0.0789
0.0431
0.0395
67.8
54.6
0.0915
Puppeteer ( Song et al., 2025a )
0.0321
0.0173
64.4
87.2
0.0279
0.0278
0.0144
76.7
75.1
0.0314
UniRig ( Zhang et al., 2025b )
0.0381
0.0212
65.8
86.7
0.0312
0.0297
0.0165
72.6
73.5
0.0419
TokenRig (4 skin tokens)
0.0168
0.0072
78.9
88.1
0.0184
0.0178
0.0077
78.1
87.7
0.0236
TokenRig (6 skin tokens)
0.0166
0.0069
79.1
89.1
0.0176
0.0153
0.0061
78.8
88.8
0.0228
Table 6. Quantitative Evaluation of Skinning Prediction. We compare skinning fidelity using L1 Error, Precision/Recall, and Motion Loss. A threshold of ε=10−2 is applied to filter negligible weights. TokenRig achieves superior performance across all categories, demonstrating significantly lower reconstruction error and higher precision/recall than baseline regression approaches.
Figure 8. Impact of GRPO on Skeletal Topology. While supervised training alone often misses non-standard anatomy, the GRPO-refined model effectively synthesizes auxiliary bones for secondary structures, such as tails, horns, wings, and clothing accessories.
Figure 9. Impact of GRPO on Skinning Precision. In addition to topology, the reinforcement learning stage tightens skinning predictions. The model produces precise, locally distinct influence maps that minimize artifacts, yielding valid rigs suitable for high-quality animation.
Method
Skeleton
Skinning
Total
RigNet ( Xu et al., 2020 )
1 s – 20 min (joint-count dependent)
UniRig ( Zhang et al., 2025b )
∼4 s
∼1 s
∼5 s
Puppeteer ( Song et al., 2025a )
∼4 s
∼30 s (marching cubes)
∼34 s
TokenRig (Ours)
∼4 s
10 – 12 s
∼15 s
Table 7. Inference Time Comparison. We report the average wall-clock time for skeleton and skinning prediction. RigNet’s runtime is highly sensitive to the predicted joint count. Puppeteer’s skinning stage is dominated by a marching-cubes volumetric step. TokenRig achieves a full rig in ∼15 s, with skinning decoded directly from SkinTokens.
VRoid Hub
ModelResource
Articulation 2.0
L1 Error ↓
IoU ↑
Mask ∗↑
L1 Error ↓
IoU ↑
Mask ∗↑
L1 Error ↓
IoU ↑
Mask ∗↑
TokenRig
5.41×10−3
87.1 %
92.5 %
1.38×10−2
91.1 %
83.9 %
1.40×10−2
84.0 %
82.2 %
- w/o Dice loss
5.98×10−3
82.2 %
91.6 %
1.35×10−2
88.2 %
82.7 %
1.36×10−2
80.9 %
80.8 %
Table 8. Ablation Study on Loss Function Design. We evaluate the impact of the Dice Loss term on reconstruction quality, using a baseline configuration with codebook size C=[8,8,8,5,6]=15,360 and TD=32 skin tokens. The results show that removing Dice supervision significantly degrades IoU performance across all datasets. ∗ Mask Accuracy quantifies the proportion of samples where the predicted non-zero support fully covers the ground truth active region (tolerance ε=10−2 ), highlighting the loss’s role in preventing under-segmentation.
ModelsResource
Articulation 2.0
J2J ↓
J2B ↓
B2B ↓
J2J ↓
J2B ↓
B2B ↓
TokenRig
2.857
2.025
1.568
2.515
1.694
1.469
- w/o non-uniform scaling
3.030
2.171
1.689
2.679
1.844
1.618
- w/o sub-tree dropping
3.019
2.179
1.704
2.648
1.806
1.582
- w/o joints deleting
3.077
2.220
1.742
2.818
1.977
1.713
Table 9. Ablation Study on Data Augmentation Strategies. We analyze the contribution of each robustness-oriented augmentation module to skeletal prediction accuracy (J2J, J2B, B2B). Experiments are conducted using a consistent lightweight configuration ( TD=4 ). The results indicate that removing any augmentation component (particularly random joint deletion) leads to higher error rates, confirming that simulating imperfections is essential for achieving robust generalization.
Table 10. Held-out OOD Evaluation of GRPO Post-training (Truebones Zoo). We report skeletal Chamfer distances (J2J/J2B/B2B), the skinning L1 error, and the skinning IoU at thresholds ε∈{10−3,10−1} on the Truebones Zoo ( Truebones, 2022 ) dataset, which is entirely held out from training and orthogonal to our training corpus. GRPO post-training improves all metrics; the relative improvement over the base model is annotated in red ( (+x%) ), providing direct quantitative evidence that GRPO enhances out-of-distribution extrapolation rather than overfitting the training distribution.
FSQ Recon.
Autoregressive
Dataset
TD=4
TD=8
w/o GRPO
w/ GRPO
GT
Truebones Zoo
0.5748 ( 96.4% )
0.5816 ( 97.6% )
0.5473 ( 91.8% )
0.5684 ( 95.3% )
0.5962 ( 100% )
Articulation 2.0
0.2711 ( 87.3% )
0.2872 ( 92.4% )
0.2289 ( 73.7% )
0.2505 ( 80.6% )
0.3107 ( 100% )
Table 11. Isolating Tokenization from RL Refinement on Deformation Smoothness. We report the deformation-smoothness reward Rmo (higher is better; Section 4.4 ); each cell lists the raw Rmo together with its percentage relative to the Ground Truth skinning, shown in red parentheses. FSQ Reconstruction evaluates the tokenizer alone (no autoregressive generation, no RL) at TD∈{4,8} tokens per bone; Autoregressive evaluates the full generation ( TD=4 ) before and after GRPO. Truebones Zoo ( Truebones, 2022 ) is fully held out (OOD), whereas Articulation 2.0 ( Song et al., 2025b ) is in-distribution. Tokenization alone retains most of the Ground Truth smoothness (and more so with a larger token budget), while the larger drop from autoregressive sampling is substantially recovered by GRPO.
Figure 10. Failure Cases. Left: on a thin, looped handle-like structure, the predicted skeleton deviates from the intended articulation (highlighted). Right: a slender claw exhibits a residual skinning spike, where a few vertices are erroneously bound to a distant bone, producing a localized high-weight artifact (highlighted). These failures mainly occur on out-of-distribution geometries with extremely thin or ambiguous topology.
Figure 11. Diverse Generation Results. We demonstrate the generalization capacity of TokenRig on a wide range of inputs, including unseen test-set samples and complex in-the-wild assets. The model robustly synthesizes fully articulated skeletons and accurate skinning weights.
Figure 12. Additional Qualitative Results. An extended gallery of rigs produced by TokenRig across diverse characters, including birds, insects, dragons, and other non-humanoid creatures. Each triplet shows the input mesh, the generated skeleton, and the predicted skinning weights (heatmap), illustrating robust generalization to varied morphologies and topologies.
Recent 3D generative models can synthesize high-quality assets, but their outputs are typically static: they lack the skeletal rigs, joint hierarchies, and skinning weights required for animation. This limits their use in games, film, simulation, virtual agents, and embodied AI, where assets must not only look plausible but also move plausibly. We introduce Rigel3D, a generative method for animation-ready 3D assets represented as rigged meshes. Unlike post-hoc auto-rigging methods that attach rigs to completed shapes, our method jointly models geometry and rig structure through coupled surface and skeleton structured latent representations. A rig-aware autoencoder decodes these representations into mesh geometry, skeleton topology, joint coordinates, and skinning weights, while a two-stage latent generative model synthesizes both surface and skeleton representations for image-conditioned generation. To support downstream animation workflows, we further introduce an open-vocabulary joint labeling module that embeds generated joints into a shared vision-language space, enabling correspondence to arbitrary retargeting templates. Experiments on large-scale rigged asset datasets demonstrate that our method generates diverse, high-quality animation-ready assets and outperforms existing rigging baselines across multiple metrics.
3D Gaussian Splatting has achieved remarkable success in photorealistic and efficient rendering, leading to a rapid increase in 3D assets represented by 3D Gaussian primitives. Directly rigging these assets with arbitrary skeleton topologies is highly desirable. However, training a feed-forward skinning framework is infeasible due to the lack of high-quality 3D Gaussian rigging datasets. An alternative solution is to transfer mesh-based techniques to 3D Gaussian-based representation, but 3D Gaussian primitives are not restricted to the surface and lack explicit topological connectivity. Moreover, this kind of method suffers from poor generalization to unseen data due to its strong dependence on training data, while acquiring high-quality rigging data is prohibitively expensive. To address this challenging problem, we propose G-Skin, a novel generative skinning framework designed for expressive and high-fidelity animation with 3D Gaussian representation. To overcome this 3D data scarcity, we introduce a skeleton-controllable image generation model leveraging 2D vision foundation models to distill powerful motion priors into pseudo-guidance. Guided by these priors, we formulate an optimization pipeline incorporating geometry-aware regularizations, which stabilizes the learning process and ensures smooth, structurally coherent skinning weights. G-Skin also generalizes flexibly to the augmented variants of 3D Gaussian representation designed to mitigate animation-induced rendering artifacts. Extensive experiments validate the effectiveness of our approach, demonstrating clear advantages over state-of-the-art methods. Project page: https://yaoyx689.github.io/GSkin.html.
Yuxin Yao, Kendong Liu, Shiqi Zhou +2
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · Central Media Technology Institute, Huawei, China · School of Computer Science and Engineering, Central South University, Changsha, China
Recent advances in automatic rigging now deliver animation-ready 3D assets at scale, yet generating the motion to drive them remains a bottleneck. Existing learned animators are topology-constrained: they rely on category-specific templates or require per-skeleton fine-tuning and reference motions at inference. We present UniMate, a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test-time optimization or per-skeleton retraining. UniMate introduces a topology-aware diffusion transformer, which integrates skeletal topology into attention via three mechanisms: (1) a graph-aware attention bias from pairwise joint relations and geodesic distances; (2) a spectral rotary position embedding generalizing RoPE to arbitrary kinematic trees via the graph Laplacian; and (3) a global topological conditioner attention-pooled from the rest-pose skeleton. We also curate UniML3D, 13,006 motion sequences spanning bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid objects with unified canonicalization and text pairing. Trained on this dataset, UniMate outperforms state-of-the-art baselines in quality, generalization, and efficiency, and supports zero-shot cross-topology transfer, in-betweening, expansion, and text-guided editing. Our project page is available at https://linzhanmou.com/unimate/.
Linzhan Mou, Jiahui Lei, Zhiyang Dou +4
Princeton University, USA · University of California, Berkeley, USA · Massachusetts Institute of Technology, USA +1