cs.GRFeb 4, 2026
SaveSkin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging
Organizations: BNRist, Department of Computer Science and Technology, Tsinghua University, Beijing, China · Zhili College, Tsinghua University, Beijing, China · VAST, Beijing, China
Abstract
The rapid proliferation of generative 3D models has created a critical bottleneck in animation pipelines: rigging. Existing automated methods are fundamentally limited by their approach to skinning, treating it as an ill-posed, high-dimensional regression task that is inefficient to optimize and is typically decoupled from skeleton generation. We posit this is a representation problem and introduce SkinTokens: a learned, compact, and discrete representation for skinning weights. By leveraging an FSQ-CVAE to capture the intrinsic sparsity of skinning, we reframe the task from continuous regression to a more tractable token sequence prediction problem. This representation enables TokenRig, a unified autoregressive framework that models the entire rig as a single sequence of skeletal parameters and SkinTokens, learning the complicated dependencies between skeletons and skin deformations. The unified model is then amenable to a reinforcement learning stage, where tailored geometric and semantic rewards improve generalization to complex, out-of-distribution assets. Quantitatively, the SkinTokens representation leads to a 98%-133% percents improvement in skinning accuracy over state-of-the-art methods, while the full TokenRig framework, refined with RL, enhances bone prediction by 17%-22%. Our work presents a unified, generative approach to rigging that yields higher fidelity and robustness, offering a scalable solution to a long-standing challenge in 3D content creation.
Figures & tables
Figure 1. Automated rigging with TokenRig. We present TokenRig, a unified generative framework that produces high-quality rigs for diverse 3D assets. By leveraging SkinTokens, i.e., our novel, discrete representation for skinning weights, our method robustly generates high-fidelity skeletons and precise skinning maps (visualized as heatmaps) for complex, real-world geometries, ranging from stylized anime characters to quadrupeds and fantasy creatures.
Figure 2. Overview of the TokenRig Framework. Our method consists of three key stages: (1) Learning SkinTokens (Section 4.2.2 ): We first train a FSQ-CVAE ( Kingma and Welling, 2013 ; Sohn et al., 2015 ; Mentzer et al., 2024 ) to compress sparse skinning weights into a compact, discrete representation. Mesh geometry and skinning weights are processed by VecSet ( Zhang et al., 2023 ) encoders, and the resulting features are discretized into SkinTokens via Finite Scalar Quantization (FSQ) ( Mentzer et al., 2024 ) . We employ nested dropout ( Bachmann et al., 2025 ; Rippel et al., 2014 ) and importance sampling to ensure robust reconstruction of active deformation regions. (2) Unified Autoregressive Modeling (Section 4.3 ): We formulate rigging as a sequence generation task. A Transformer generates a single, unified sequence comprising the complete skeleton followed by the learned SkinTokens (from Stage 1), conditioned on global shape embeddings to capture structural dependencies. (3) RL Refinement via GRPO (Section 4.4 ): To improve generalization to in-the-wild assets, we fine-tune the model using Group Relative Policy Optimization (GRPO) ( Liu et al., 2024 ) . We introduce four specific rewards: Volumetric Joint Coverage (ensuring bone distribution), Bone-Mesh Containment (preventing protrusion), Skinning Coverage and Sparsity (ensuring valid weighting), and Deformation Smoothness (preventing artifacts during animation).
| ModelsResource | VRoid Hub | Articulation 2.0 | |
| avg. | 1297.69 | 16929.78 | 6247.05 |
| avg. | 19.87 | 95.56 | 34.46 |
| avg. | 1700.18 | 28392.74 | 12655.45 |
| avg. sparsity | 7.40% | 2.43% | 9.38% |
Table 1. Sparsity Analysis of Skinning Weights. Statistics across three major datasets ( Models-Resource, 2019 ; Isozaki et al., 2021 ; Song et al., 2025b ) reveal that skinning matrices are extremely sparse ( active weights), motivating our design of the compressed SkinTokens representation.
Figure 3. Gradient Analysis of Loss Functions. A comparison of Binary Cross Entropy (BCE) and Dice loss ( Sudre et al., 2017 ) landscapes for a target weight . While both minimize at the correct value, Dice loss provides significantly larger gradients for non-zero targets ( ), effectively counteracting the extreme sparsity of skinning matrices where BCE gradients tend to vanish.
Figure 4. SkinTokens Reconstruction Fidelity. We evaluate the reconstruction quality (IoU and L1 Error) of the FSQ-CVAE across varying codebook sizes and token sequence lengths . The results demonstrate that SkinTokens achieve high fidelity with as few as tokens, validating the compressibility of skinning data. The configuration (lines with circles) is selected for our final model for its superior balance of compression and accuracy. The figure reports the IoU scores at and corresponding reconstruction errors on the Articulation 2.0 ( Song et al., 2025b ) test dataset
| Codebook Size | Total Entries | Utilization | |
Table 2. Codebook Utilization and Compression Analysis. We compare different FSQ codebook configurations ( ). ∗ The compression ratio is computed against a raw FP16 baseline using the Articulation 2.0 ( Song et al., 2025b ) dataset. For example, storing skinning weights for an average mesh ( vertices) typically requires bytes; our representation ( tokens) reduces this to just ( ) bytes, achieving a substantial reduction in storage requirements.
Figure 5. Learned Semantics of SkinTokens. A t-SNE visualization of the continuous latent vectors prior to quantization, sampled from instances in the VRoid dataset ( Isozaki et al., 2021 ) . Points are colored by bone category (e.g., Head, Hips). The clear emergence of anatomical clusters indicates that the encoder captures a semantic structural prior, learning to represent “body part concepts” invariant to specific mesh geometries.
Figure 6. Qualitative Comparison of Skeleton Generation. We compare TokenRig (Ours) against state-of-the-art baselines. While baseline methods exhibit partial structures, missing details, or redundant joints, our method synthesizes structurally coherent and semantically faithful skeletons across diverse character types.
Figure 7. Qualitative Comparison of Skinning Prediction. We visualize predicted skinning weights and the corresponding average L1 error maps. Baseline methods often suffer from “bleeding” artifacts, where weights spill onto unconnected mesh parts (see UniRig/Puppeteer columns). TokenRig (Ours) produces clean, locally coherent influence maps that closely match the Ground Truth, particularly in fine-grained regions like fingers.
| Stage 1: SkinTokens | Stage 2: TokenRig | |
| (FSQ-CVAE) | (Autoregressive) | |
| Architecture (structure only, trained from scratch) | ||
| Network | 3DShape2Vecset | Qwen3-0.6B |
| Parameters | M | B |
| Layers | / (enc./dec.) | (decoder-only) |
| Hidden width | ||
Table 3. Two-Stage Model and Training Configurations. We summarize the architecture and training recipe of the SkinTokens FSQ-CVAE (Stage 1) and the TokenRig autoregressive model (Stage 2). For both stages we adopt only the network structure of the referenced designs and train from scratch. Stage 2 is further refined via GRPO post-training ( steps, learning rate , group size , , ; see Sec. 4.4 ).
| Metric | FSQ (Ours) | VQ | CVAE (cont.) |
| Reconstruction fidelity | |||
| IoU@ | |||
| error ( ) | |||
| Code usage & token repetition | |||
| Codebook utilization | — | ||
| Token repetition rate @32 | — | ||
Table 4. Comparison of Tokenizer Choices. We compare FSQ (ours), a learnable VQ-VAE codebook, and a continuous-latent CVAE under an identical backbone and token budget. Both FSQ and VQ are evaluated with the same codebook size of , with all other settings kept consistent with the configuration described above. The error follows the same skinning reconstruction metric as Figure 4 (in units of ).
| ModelsResource | Articulation 2.0 | |||||
| J2J | J2B | B2B | J2J | J2B | B2B | |
| RigNet ( Xu et al., 2020 ) | 3.901 | 2.412 | 2.213 | 7.376 | 5.841 | 4.802 |
| MagicArticulate ( Song et al., 2025b ) | 3.024 | 2.260 | 1.915 | 4.003 | 3.026 | 2.586 |
| Puppeteer ( Song et al., 2025a ) | 3.841 | 2.881 | 2.475 | 3.033 | 2.300 | 1.923 |
| UniRig ( Zhang et al., 2025b ) | 3.390 | 2.592 | 1.890 | 3.115 | 2.211 | 1.926 |
| TokenRig (4 skin tokens) | 2.857 | 2.025 | 1.568 | 2.515 | 1.694 | 1.469 |
Table 5. Quantitative Comparison of Skeletal Generation. We evaluate skeletal structure accuracy using Chamfer Distance metrics: Joint-to-Joint (J2J), Joint-to-Bone (J2B), and Bone-to-Bone (B2B) on the ModelsResource ( Models-Resource, 2019 ) and Articulation 2.0 ( Song et al., 2025b ) datasets. Lower values ( ) indicate better performance. TokenRig consistently outperforms state-of-the-art baselines across all metrics, demonstrating superior fidelity in joint placement and bone connectivity.
| ModelsResource ( Xu et al., 2020 ) | Articulation 2.0 ( Song et al., 2025b ) | |||||||||
| Skin L1 | L1 Var. | Precision | Recall | Motion | Skin L1 | L1 Var. | Precision | Recall | Motion | |
| RigNet ( Xu et al., 2020 ) | 0.0573 | 0.0464 | 62.4 | 59.9 | 0.0789 | 0.0431 | 0.0395 | 67.8 | 54.6 | 0.0915 |
| Puppeteer ( Song et al., 2025a ) | 0.0321 | 0.0173 | 64.4 | 87.2 | 0.0279 | 0.0278 | 0.0144 | 76.7 | 75.1 | 0.0314 |
| UniRig ( Zhang et al., 2025b ) | 0.0381 | 0.0212 | 65.8 | 86.7 | 0.0312 | 0.0297 | 0.0165 | 72.6 | 73.5 | 0.0419 |
| TokenRig (4 skin tokens) | 0.0168 | 0.0072 | 78.9 | 88.1 | 0.0184 | 0.0178 | 0.0077 | 78.1 | 87.7 | 0.0236 |
| TokenRig (6 skin tokens) | 0.0166 | 0.0069 | 79.1 | 89.1 | 0.0176 | 0.0153 | 0.0061 | 78.8 | 88.8 | 0.0228 |
Table 6. Quantitative Evaluation of Skinning Prediction. We compare skinning fidelity using L1 Error, Precision/Recall, and Motion Loss. A threshold of is applied to filter negligible weights. TokenRig achieves superior performance across all categories, demonstrating significantly lower reconstruction error and higher precision/recall than baseline regression approaches.
Figure 8. Impact of GRPO on Skeletal Topology. While supervised training alone often misses non-standard anatomy, the GRPO-refined model effectively synthesizes auxiliary bones for secondary structures, such as tails, horns, wings, and clothing accessories.
Figure 9. Impact of GRPO on Skinning Precision. In addition to topology, the reinforcement learning stage tightens skinning predictions. The model produces precise, locally distinct influence maps that minimize artifacts, yielding valid rigs suitable for high-quality animation.
| Method | Skeleton | Skinning | Total |
| RigNet ( Xu et al., 2020 ) | s – min (joint-count dependent) | ||
| UniRig ( Zhang et al., 2025b ) | s | s | s |
| Puppeteer ( Song et al., 2025a ) | s | s (marching cubes) | s |
| TokenRig (Ours) | s | – s | s |
Table 7. Inference Time Comparison. We report the average wall-clock time for skeleton and skinning prediction. RigNet’s runtime is highly sensitive to the predicted joint count. Puppeteer’s skinning stage is dominated by a marching-cubes volumetric step. TokenRig achieves a full rig in s, with skinning decoded directly from SkinTokens.
| VRoid Hub | ModelResource | Articulation 2.0 | |||||||
| L1 Error | IoU | Mask | L1 Error | IoU | Mask | L1 Error | IoU | Mask | |
| TokenRig | 87.1 % | 92.5 % | 91.1 % | 83.9 % | 84.0 % | 82.2 % | |||
| - w/o Dice loss | 82.2 % | 91.6 % | 88.2 % | 82.7 % | 80.9 % | 80.8 % | |||
Table 8. Ablation Study on Loss Function Design. We evaluate the impact of the Dice Loss term on reconstruction quality, using a baseline configuration with codebook size and skin tokens. The results show that removing Dice supervision significantly degrades IoU performance across all datasets. ∗ Mask Accuracy quantifies the proportion of samples where the predicted non-zero support fully covers the ground truth active region (tolerance ), highlighting the loss’s role in preventing under-segmentation.
| ModelsResource | Articulation 2.0 | |||||
| J2J | J2B | B2B | J2J | J2B | B2B | |
| TokenRig | 2.857 | 2.025 | 1.568 | 2.515 | 1.694 | 1.469 |
| - w/o non-uniform scaling | 3.030 | 2.171 | 1.689 | 2.679 | 1.844 | 1.618 |
| - w/o sub-tree dropping | 3.019 | 2.179 | 1.704 | 2.648 | 1.806 | 1.582 |
| - w/o joints deleting | 3.077 | 2.220 | 1.742 | 2.818 | 1.977 | 1.713 |
Table 9. Ablation Study on Data Augmentation Strategies. We analyze the contribution of each robustness-oriented augmentation module to skeletal prediction accuracy (J2J, J2B, B2B). Experiments are conducted using a consistent lightweight configuration ( ). The results indicate that removing any augmentation component (particularly random joint deletion) leads to higher error rates, confirming that simulating imperfections is essential for achieving robust generalization.
| Skeleton (Chamfer Distance) | Skinning | |||||
| J2J | J2B | B2B | IoU@ | IoU@ | ||
| TokenRig (w/o GRPO) | ||||||
| TokenRig (w/ GRPO) | {2.388}_{\,\text{\color[rgb]{1,0,0}(+2.9\%)}} | {1.717}_{\,\text{\color[rgb]{1,0,0}(+2.4\%)}} | {1.394}_{\,\text{\color[rgb]{1,0,0}(+1.3\%)}} | {9.57\!\times\!10^{-3}}_{\,\text{\color[rgb]{1,0,0}(+2.7\%)}} | {0.571}_{\,\text{\color[rgb]{1,0,0}(+1.6\%)}} | {0.714}_{\,\text{\color[rgb]{1,0,0}(+0.5\%)}} |
Table 10. Held-out OOD Evaluation of GRPO Post-training (Truebones Zoo). We report skeletal Chamfer distances (J2J/J2B/B2B), the skinning error, and the skinning IoU at thresholds on the Truebones Zoo ( Truebones, 2022 ) dataset, which is entirely held out from training and orthogonal to our training corpus. GRPO post-training improves all metrics; the relative improvement over the base model is annotated in red ( (+x%) ), providing direct quantitative evidence that GRPO enhances out-of-distribution extrapolation rather than overfitting the training distribution.
| FSQ Recon. | Autoregressive | ||||
| Dataset | w/o GRPO | w/ GRPO | GT | ||
| Truebones Zoo | ( ) | ( ) | ( ) | ( ) | ( ) |
| Articulation 2.0 | ( ) | ( ) | ( ) | ( ) | ( ) |
Table 11. Isolating Tokenization from RL Refinement on Deformation Smoothness. We report the deformation-smoothness reward (higher is better; Section 4.4 ); each cell lists the raw together with its percentage relative to the Ground Truth skinning, shown in red parentheses. FSQ Reconstruction evaluates the tokenizer alone (no autoregressive generation, no RL) at tokens per bone; Autoregressive evaluates the full generation ( ) before and after GRPO. Truebones Zoo ( Truebones, 2022 ) is fully held out (OOD), whereas Articulation 2.0 ( Song et al., 2025b ) is in-distribution. Tokenization alone retains most of the Ground Truth smoothness (and more so with a larger token budget), while the larger drop from autoregressive sampling is substantially recovered by GRPO.
Figure 10. Failure Cases. Left: on a thin, looped handle-like structure, the predicted skeleton deviates from the intended articulation (highlighted). Right: a slender claw exhibits a residual skinning spike, where a few vertices are erroneously bound to a distant bone, producing a localized high-weight artifact (highlighted). These failures mainly occur on out-of-distribution geometries with extremely thin or ambiguous topology.
Figure 11. Diverse Generation Results. We demonstrate the generalization capacity of TokenRig on a wide range of inputs, including unseen test-set samples and complex in-the-wild assets. The model robustly synthesizes fully articulated skeletons and accurate skinning weights.
Figure 12. Additional Qualitative Results. An extended gallery of rigs produced by TokenRig across diverse characters, including birds, insects, dragons, and other non-humanoid creatures. Each triplet shows the input mesh, the generated skeleton, and the predicted skinning weights (heatmap), illustrating robust generalization to varied morphologies and topologies.