Branch-Centric Tokenization and Test-Time Augmentation for Skeleton Generation

Zhengyuan Li; Chuanyu Pan; Yuanming Hu; Raymond A. Yeh

cs.CVSep 5, 2026

Branch-Centric Tokenization and Test-Time Augmentation for Skeleton Generation

Authors: Zhengyuan Li, Chuanyu Pan, Yuanming Hu, Raymond A. Yeh

Organizations: Purdue University · Meshy AI

Abstract

Automatic skeleton generation involves predicting both joint positions and skeletal connectivity. However, existing approaches struggle to encode branch structures into token sequences and do not use test-time computation effectively. We study these choices within a unified autoregressive framework. First, we introduce branch-centric tokenization, a branch-aware representation that places structurally related elements next to each other and encodes connectivity directly in the sequence. Compared with standard BFS-style serialization, this representation yields more compact sequences. Second, we introduce view-augmented generation, a test-time augmentation procedure that applies axis-aligned rotations to the input mesh, maps all predictions back to a common frame, and selects the final skeleton based on mesh coverage and consistency among predictions from different views. Experiments show that our method achieves better skeleton prediction accuracy than state-of-the-art methods. In particular, our method reduces the CD-J2B error by 16.9% on the Articulation-XL2.0 dataset compared to the strongest directly comparable baseline, Auto-Connect. Qualitative results on in-the-wild meshes further demonstrate generalization across diverse inputs.

Figures & tables

Figure 1: Our method predicts skeletons accurately from in-the-wild meshes, including humans, animals, and non-existent fantasy creatures.

Figure 2: Overview of our method. Top: network inference with branch-centric tokenization. Following the standard mesh-conditioned autoregressive formulation reviewed in Sec. 3 , the input mesh ${\mathcal{M}}$ is first encoded into shape-conditioned features, and the autoregressive transformer predicts a token sequence from [BOS] to [EOS] . In Sec. 4.1 , we design branch-centric tokenization (BCT), which organizes the skeleton by branch structure and is used to detokenize the predicted sequence $\widehat{\bm{\tau}}$ into the skeleton tree $\widehat{{\mathcal{T}}}$ . Bottom: test-time augmentation. In Sec. 4.2 , we apply the rotation set $\mathcal{A}_{r}$ to the input mesh, decode one skeleton hypothesis for each rotated view, map all predictions back to the canonical frame with the inverse rotations $R_{k}^{-1}$ , and form the hypothesis pool $\mathcal{H}$ . Coverage-consensus aggregation $g(\mathcal{H})$ then selects the final prediction ${\mathcal{T}}^{*}$ by favoring hypotheses that both better cover the mesh geometry and agree with the other candidates.

Figure 3: Comparison of node ordering with Auto-Connect. Left: illustration of branch-centric tokenization (BCT). Starting from the input skeleton tree, we first construct its series-reduced tree and run BFS on this reduced tree to order the branches. We then reinsert the smoothed connection nodes along each visited branch, which yields the final node order used by BCT. For clarity, we omit delimiter insertion in this figure. We use purple arrows to indicate the node traversal order. Right: the node order of Auto-Connect. As indicated by the arrows, Auto-Connect interleaves nodes from one branch with nodes from other branches in the traversal sequence. In contrast, our branch-level ordering keeps nodes from the same branch adjacent in the serialized sequence.

Figure 4: Token sequence length distributions on Articulation-XL2.0. Our branch-centric tokenization on average produces shorter token sequences than Auto-Connect, with mean lengths of

127.6

vs.

146.3

tokens (indicated by the dashed lines). Auto-Connect produces longer token sequences than ours in

94.4\%

of samples.

Method	Articulation-XL2.0			ModelsResource			Diverse-pose
Method	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$
Pinocchio ( Baran and Popović, 2007 )	8.324	6.612	5.485	6.852	4.824	4.089	7.967	6.411	5.149
RigNet ( Xu et al., 2020 )	7.618	6.076	5.279	7.223	5.987	4.329	7.751	6.392	5.713
UniRig ( Zhang et al., 2025 )	3.305	2.611	2.180	3.964	3.021	2.570	3.252	2.569	2.077
MagicArti. ( Song et al., 2025b )	3.264	2.503	2.123	4.114	3.137	2.693	4.376	3.456	2.955
Puppeteer ( Song et al., 2025a )	3.109	2.370	1.983	3.766	2.804	2.405	2.514	1.986	1.598

Table 1: Quantitative comparison with baselines across three test sets. We report results on Articulation-XL2.0, ModelsResource, and the diverse-pose split using CD-J2J, CD-J2B, and CD-B2B. Our method achieves the best results among directly comparable methods. Bold numbers denote the best result among directly comparable methods in each column. *Gray results are reported from TokenRig’s paper and are included for reference only, as TokenRig uses additional training data, including the ModelsResource training set.

Figure 5: Qualitative comparison on challenging examples from Articulation-XL2.0. We compare our method, our method without TTA, Puppeteer ( Song et al., 2025a ) , and UniRig ( Zhang et al., 2025 ) against the ground-truth skeletons. Our method better matches the target connectivity and joint positions, especially around challenging part attachments and elongated structures, while TTA further improves over the variant without TTA. The dashed boxes highlight failure regions.

BCT	DAT	Articulation-XL2.0			Diverse-pose
BCT	DAT	J2J	J2B	B2B	J2J	J2B	B2B
		2.543	1.864	1.582	2.216	1.695	1.471
$\checkmark$		2.479	1.798	1.528	2.145	1.585	1.367
$\checkmark$	$\checkmark$	2.438	1.770	1.517	2.088	1.557	1.338

Table 2: Ablation on tokenization design choices.

Aug.	Sampling	# Hypo.	Aggregation	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$
$\mathcal{A}_{\mathrm{id}}$	Greedy	1	-	2.438	1.770	1.517
$\mathcal{A}_{r}$	Greedy	6	Likelihood	2.593	1.901	1.631
$\mathcal{A}_{r}$	Greedy	6	Length-normalized likelihood	2.474	1.809	1.563
$\mathcal{A}_{r}$	Greedy	6	Ours	2.364	1.687	1.447
$\mathcal{A}_{\mathrm{id}}$	Beam search	6	Likelihood	2.381	1.739	1.504
$\mathcal{A}_{\mathrm{id}}$	Beam search	6	Length-normalized likelihood	2.377	1.735	1.501

Table 3: Ablation on test-time augmentation on Articulation-XL2.0. We compare two ways to generate candidate skeletons under a fixed six-hypothesis budget: greedy decoding with

\mathcal{A}_{r}

(Ours) and beam search under

\mathcal{A}_{\mathrm{id}}

. We then compare different selection rules over the resulting candidates. The best results are shown in bold .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Method	TED $\downarrow$	Graph2Vec distance $\downarrow$
MagicArti.	6.837	0.254
Puppeteer	6.736	0.245
TokenRig	7.120	0.235
Ours	5.984	0.222

Appendix

Table 4: Quantitative comparison of skeleton prediction methods on Articulation-XL2.0. We report TED and Graph2Vec distance on the Articulation-XL2.0 test set. Bold numbers denote the best result in each column.

Aug.	Sampling	# Hypo.	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$
$\mathcal{A}_{r}$	Greedy	6	2.364	1.687	1.447
$\mathcal{A}_{\mathrm{cube}}$	Greedy	24	2.339	1.673	1.438

Appendix

Table 5: Ablation on a larger rotation group for view-augmented generation on Articulation-XL2.0. We compare the six-view rotation set

\mathcal{A}_{r}

used in the main paper with the larger 24-view cube rotation set

\mathcal{A}_{\mathrm{cube}}

. The larger set gives a small accuracy gain at the cost of four times more decoded hypotheses.

Aug.	Sampling	# Hypo.	Aggregation	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$
$\mathcal{A}_{r}$	Greedy	6	Coverage-consensus aggregation	2.364	1.687	1.447
$\mathcal{A}_{\mathrm{id}}$	Stochastic	6	Likelihood	2.550	1.884	1.612
$\mathcal{A}_{\mathrm{id}}$	Stochastic	6	Length-normalized likelihood	2.441	1.779	1.524
$\mathcal{A}_{\mathrm{id}}$	Stochastic	6	Coverage-consensus aggregation	2.408	1.734	1.487

Appendix

Table 6: Comparison of selection strategies for stochastic samples without view-augmented generation on Articulation-XL2.0. The first row reports greedy decoding under

\mathcal{A}_{r}

for reference. For stochastic samples under

\mathcal{A}_{\mathrm{id}}

, coverage-consensus aggregation outperforms likelihood-based selection.

Augmentation	J2J $\downarrow$	J2B $\downarrow$	B2B $\downarrow$
Rotation set ( $\mathcal{A}_{r}$ )	2.364	1.687	1.447
Random translation	4.752	3.128	2.202
Random scaling	8.291	5.324	3.113

Appendix

Table 7: Additional ablation on test-time augmentation. Each augmentation is evaluated individually using our coverage-consensus aggregation.

$\alpha_{r}$	Joint-consensus weight $\lambda_{\tt joint}$
$\alpha_{r}$	0.00	0.25	0.50	1.00	2.00	4.00
5.0	2.516	2.402	2.395	2.366	2.364	2.381
10.0	2.481	2.393	2.377	2.364	2.367	2.381
15.0	2.478	2.378	2.371	2.364	2.368	2.385
20.0	2.472	2.377	2.372	2.363	2.374	2.383
25.0	2.469	2.375	2.374	2.364	2.377	2.387

Appendix

Table 8: Hyperparameter sensitivity of coverage-consensus aggregation on Articulation-XL2.0. We report CD-J2J over the spatial tolerance radius

\alpha_{r}

and the joint-consensus weight

\lambda_{\tt joint}

under the 6-hypothesis rotation TTA setting with

\mathcal{A}_{r}

. Bold indicates the best result.

Method	CD $\downarrow$	EMD $\downarrow$
Pinocchio ( Baran and Popović, 2007 )	0.198	0.659
RigNet ( Xu et al., 2020 )	0.089	0.127
Anymate ( Deng et al., 2025 )	0.077	0.098
UniRig ( Zhang et al., 2025 )	0.096	0.135
Ours	0.038	0.046

Appendix

Table 9: Quantitative results on the Anymate dataset ( Deng et al., 2025 ) . Our method achieves the best CD and EMD.

Method	UniRig	Puppeteer	TokenRig	Ours w/o TTA	Ours
Inference time	1.8	1.3	7.5	1.0	5.8

Appendix

Table 10: Inference time comparison on Articulation-XL2.0 test set. Measured in seconds.

Explore similar work

Less is More: Compact-Token Masked Feature Prediction for Skeleton Representation Learning
Mar 11, 2026Jeonghyeok Do, Yun Chen, Geunhyuk Youk +1Representation Learning Motion Representation Learning
Information & Electronics Research Institute, KAIST · School of Electrical Engineering, KAIST
SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology
Jul 2, 2026Yuhao Zhang, Gerard Pons-Moll, Tolga BirdalGenerative Modeling Motion Representation Learning
Imperial College London · University of Tübingen, Tübingen AI Center
Generative Data Augmentation for Skeleton Action Recognition
Apr 16, 2026Xu Dong, Wanqing Li, Anthony Adeyemi-Ejeye +1Skeleton-Based Action Recognition Synthetic Data Augmentation
Innovative Media Lab, University of Surrey, Guildford, UK · Advanced Multimedia Research Lab, University of Wollongong, Wollongong, Australia

Mar 11, 2026J/K move · Enter open · S save

Less is More: Compact-Token Masked Feature Prediction for Skeleton Representation Learning

Jeonghyeok Do, Yun Chen, Geunhyuk Youk, Munchurl Kim

Information & Electronics Research Institute, KAIST · School of Electrical Engineering, KAIST

Representation Learning Motion Representation Learning Masked Autoencoders Contrastive Learning

Open paper AskPDF arXiv

Current skeleton representation learning paradigms face distinct limitations: Contrastive Learning (CL) often overlooks fine-grained motion details, while Masked Auto-Encoders (MAE) rely on coordinate-level reconstruction. This reconstruction inherently demands dense token sequences and heavy decoders, wasting pre-training computation on discarded components and forcing downstream inference to process dense token grids. To resolve these bottlenecks, we propose SLiM (Skeleton Less is More), a compact-token framework that unifies masked feature prediction and contrastive learning via a shared encoder. By shifting the objective from raw coordinate reconstruction to decoder-free, teacher-guided feature prediction, SLiM breaks the reliance on dense tokenization and enables effective learning with a highly compact token grid. Crucially, to prevent trivial shortcut learning arising from strong inter-joint dependencies of human, we introduce Semantic Tube Masking together with Skeleton-Aware Augmentations to enforce deep skeletal-temporal reasoning and anatomical consistency. Extensive experiments across multiple downstream protocols demonstrate that SLiM achieves state-of-the-art performance while structurally reducing inference computation by 7.89

\times

compared to dense-token MAE baselines.

Abstract

Explore similar work

Less is More: Compact-Token Masked Feature Prediction for Skeleton Representation Learning

SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology

Generative Data Augmentation for Skeleton Action Recognition