DragonMoCap: Monocular Video-based Motion Capture for Diverse Dragon Topologies

Zhongfei Qing; Xiaoxiang Zhao; Ling-Hao Chen; Zhongang Cai; Wenjia Wang; Yifan Wu; Qianying Liu; Wei Zeng

cs.CVOct 18, 2026

DragonMoCap: Monocular Video-based Motion Capture for Diverse Dragon Topologies

Authors: Zhongfei Qing, Xiaoxiang Zhao, Ling-Hao Chen, Zhongang Cai, Wenjia Wang, Yifan Wu, Qianying Liu, Wei Zeng,

Organizations: Tencent, Shenzhen, China · Tsinghua University, Shenzhen, China · Nanyang Technological University, Singapore, Singapore · The University of Hong Kong, Shenzhen, China ·

Abstract

This work explores monocular motion capture for dragon-type characters, which currently relies on manual keyframing due to the lack of high-quality datasets and parametric models. The primary obstacle lies in the extreme topological diversity across dragon rigs, where varying joint counts and naming conventions render traditional fixed-template regressors inapplicable. To address this, we introduce DragonMoCap, a novel framework that treats the target rig as a dynamic input rather than a fixed prior. By deliberately scoping the framework to the most prevalent production topologies, we ensure high-fidelity results across diverse morphologies. The core of our approach is the design of language-grounded joint query tokens, which establish a dual-property representation: ensuring intra-rig discriminability while maintaining cross-rig similarity. We further propose a non-parametric query-token retrieval mechanism to enable zero-shot adaptation to unseen rigs without retraining. Extensive evaluations demonstrate that DragonMoCap consistently outperforms HMR-style baselines across diverse dragon topologies. The practical utility of our framework is further evidenced by its integration into professional animation workflows, highlighting its potential for real-time industrial applications.

Figures & tables

Figure 1. DragonMocap captures a dragon motion from a monocular video and retargets it to diverse dragon rigs (Bipedal or quadrupedal, with or without wings).

Figure 2. Method overview. DragonMoCap is an image-based, per-frame predictor. For each input frame, a frozen image backbone produces image tokens. For each bone of the target rig, we build a language-grounded query token by combining a CLIP bone-name embedding with a topological descriptor. The transformer decoder uses these query tokens to cross-attend to the image tokens and predicts per-joint rotations. For unseen rigs, missing per-bone residuals are synthesized by retrieving and interpolating the learned residuals of the nearest bones in a bone embedding bank. A method diagram showing image tokens extracted by a frozen image backbone, language-grounded bone queries combining semantic and topological information, a transformer decoder for per-joint rotation prediction, and residual retrieval from a bone embedding bank for unseen dragon rigs.

	Seen rigs		Unseen rigs
Method	MPJPE $\downarrow$	PA-MPJPE $\downarrow$	MPJPE $\downarrow$	PA-MPJPE $\downarrow$
SMPLest-X ( 2026 )	10.57	6.53	—	—
OSX ( 2023 )	11.20	7.12	—	—
Learned embedding	10.24	6.58	14.62	9.32
CLIP only	10.51	7.20	15.32	9.59
Topo only	10.84	6.67	14.82	9.63

Table 1. Quantitative comparison and query-design ablation on seen and unseen dragon rigs (MPJPE / PA-MPJPE in cm, lower is better). All methods are trained on our dragon data. External baselines (“—”) cannot be evaluated on unseen rigs because their output heads are defined over the training-rig joints. All variants use the same residual retrieval on unseen rigs. A table comparing external baselines and query designs using MPJPE and PA-MPJPE on seen and unseen dragon rigs.

	Mean score $\uparrow$			Win rate $\uparrow$
Criterion	SMPLest-X	Ours	$p$	Ours vs. SMPLest-X
Motion fidelity	3.7	4.4	$<\!0.001$	72.5%
Anatomical plausibility	3.2	4.5	$<\!0.001$	83.6%
Overall visual quality	4.1	4.6	$<\!0.001$	74.8%

Table 2. User study on AI-generated inputs (

40

participants

\times

30

clips

=1{,}200

paired trials per criterion). Left: mean

5

-point Likert score. Right: win rate of ours over SMPLest-X, with ties counted as half a win. A user study table comparing motion fidelity, anatomical plausibility, and visual quality between SMPLest-X and DragonMoCap.

Figure 3. Qualitative results. We take a video as input, and the captured dragon is performed by SMPLest-X (A), OSX (B), and Ours (C). As baselines cannot be generalized to novel dragons, we only provide results on one target rig. Moreover, we provide another capture result for unseen rig at the bottom of (C).

Figure 4. One-video-to-many-rigs. A single input dragon video (left) simultaneously drives 8 dragon rigs of different topologies (right), producing coordinated yet rig-appropriate motion. Especially, the left 4 rigs are seen during training, and the right 4 rigs are unseen for our model.

Figure 5. t-SNE visualization of query embeddings. In contrast to baselines that suffer from a lack of global coherence or intermingled features, our integrated approach effectively resolves ambiguities between anatomically similar parts to achieve superior, semantic-aware clustering.

Markerless Motion Capture

Markerless motion capture estimates 3D body movement from video or other sensors without attached tracking markers.

See all 67 papers

TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation
Jun 10, 2026Cheng-Feng Pu, Jia-Peng Zhang, Meng-Hao Guo +2Markerless Motion Capture Motion Retargeting
Zhili College, Tsinghua University, China · BNRist, Department of Computer Science and Technology, Tsinghua University, China · VAST, China
SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapes
Jun 1, 2026Ye Tao, Yuxin Yao, Kendong Liu +23D Asset Generation
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China
CAST: Real-Time Motion Capture for Any Skeleton Topology
Oct 10, 2026Pu Pang, Zhongfei Qing, Ling-Hao Chen +3Markerless Motion Capture 3D Human Pose Estimation
Xi’an Jiaotong University, China · Zhongguancun Academy, China · Tencent, China +2

Jun 10, 2026J/K move · Enter open · S save

TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation

Cheng-Feng Pu, Jia-Peng Zhang, Meng-Hao Guo, Yan-Pei Cao+1

Zhili College, Tsinghua University, China · BNRist, Department of Computer Science and Technology, Tsinghua University, China · VAST, China

Markerless Motion Capture Motion Retargeting Motion Representation Learning

Open paper AskPDF arXiv

The explosion of generative 3D assets has created a massive demand for animation, yet current motion capture methods remain brittle, restricted to species-specific templates (e.g., SMPL) or requiring labor-intensive manual rigging. We introduce TopoCap, the first unified framework capable of extracting motion from monocular video and retargeting it onto characters with arbitrary, unseen skeletal topologies, i.e., from bipeds to hexapods and inanimate objects, without test-time optimization. Our key insight is that while skeletal structures are combinatorial and discrete, the underlying physics of motion occupy a continuous, low-dimensional manifold. We materialize this insight via a two-stage generative pipeline. First, we learn a Universal Motion Manifold using a Graph CVAE that compresses heterogeneous kinematic chains into a shared, fixed-length latent code. By explicitly conditioning the decoder on a structural embedding of the target rig, we disentangle motion dynamics from skeletal topology. Second, we treat video-to-animation as a conditional flow matching problem, predicting these topology-agnostic codes from visual features. To learn this generalized prior, we introduce Mobjaverse, a massive-scale dataset curated from Objaverse-XL. Comprising over 5,000 unique skeletal topologies and 2 million frames, it exceeds the structural diversity of existing datasets by two orders of magnitude. Extensive experiments demonstrate that \MethodMotion outperforms specialist models on human and quadruped benchmarks while enabling zero-shot retargeting for the long tail of 3D creatures. Dataset is publicly available at https://huggingface.co/datasets/duckduckplz/Mobjaverse.

Abstract

Markerless Motion Capture

Explore similar work

TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation

SkelMo: Universal Skeletal Motion Generation for 3D Rigged Shapes

CAST: Real-Time Motion Capture for Any Skeleton Topology