Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model

Yunjiao Zhou; Junlang Qian; Lihua Xie; Jianfei Yang

cs.CVOct 2, 2026

Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model

Authors: Yunjiao Zhou, Junlang Qian, Lihua Xie, Jianfei Yang

Organizations: Nanyang Technological University, Singapore

Abstract

Despite never being supervised on explicit 3D motion, large-scale text-to-video diffusion models synthesize realistic human motion in their generated videos. We ask whether this implicit knowledge can be turned into explicit 3D motion generation, without training a separate motion model. Probing a frozen Wan2.1 reveals that a recoverable motion signal is present in its intermediate states across the entire denoising schedule, not confined to the clean output. Motivated by this, we introduce parasitic co-denoising, a paradigm in which motion is decoded from the host model along its denoising schedule rather than produced by an independent generator. We instantiate it as the Parasitic Motion Decoder (PMD), an efficient flow-matching decoder that shares the host's noise schedule and reads its intermediate features through a $σ$ -adaptive multi-layer fusion, leaving the host unmodified. Drawing its coverage from the host rather than from motion data, PMD leads dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters, while producing paired video and motion in a single pass that motion-only baselines cannot match.

Figures & tables

Figure 1: Motion is decodable from a frozen video diffusion model at every step of its denoising trajectory. Cross-modal retrieval top-1 accuracy between the Wan2.1 video velocity and paired 3D motion at each noise level $\sigma$ , against a held-out pool of 400 candidates (chance $=0.25\%$ ), compared with HY-Motion, a large-scale flow-matching model trained explicitly for 3D motion. (a) Frozen Wan2.1 retrieves motion at 24–31% across the schedule, two orders of magnitude above chance. (b) At every $\sigma$ , Wan2.1 reaches 54–81% of HY-Motion’s accuracy, peaking at $81\%$ at $\sigma=0.9$ where the input is nearly pure Gaussian noise.

Figure 2: The structure of Parasitic Motion Decoder (PMD).

Method	Trainable Params	Semantic		Motion Quality
Method	Trainable Params	Motion Cond. Consistency $\uparrow$	Motion Generalizability $\uparrow$	Jitter Degree $\downarrow$	Dynamic Degree $\uparrow$	Foot Floating $\downarrow$	Foot Sliding $\downarrow$
Protocol A
MDM ( Tevet et al. 2022 )	23M	0.34	0.68	0.015	0.027	0.37	0.006
T2M-GPT ( Zhang et al. 2023a )	228M	0.34	0.74	0.035	0.034	0.33	0.005
FineMoGen ( Zhang et al. 2023b )	55M	0.29	0.63	0.031	0.033	0.39	0.007
MotionLCM ( Dai et al. 2024 )	30M	0.41	0.86	0.025	0.038	0.27	0.009

Table 1: Quantitative comparison on MBench under two protocols.

Method	R-P Top-1 $\uparrow$	R-P Top-2 $\uparrow$	R-P Top-3 $\uparrow$	FID $\downarrow$	MM-Dist $\downarrow$	MModality $\uparrow$
MotionDiffuse ( Zhang et al. 2024 )	$0.491^{\pm.001}$	$0.681^{\pm.001}$	$0.782^{\pm.001}$	$0.630^{\pm.001}$	$3.113^{\pm.001}$	$1.553^{\pm.042}$
T2M-GPT ( Zhang et al. 2023a )	$0.492^{\pm.003}$	$0.679^{\pm.002}$	$0.775^{\pm.002}$	$0.141^{\pm.004}$	$3.121^{\pm.008}$	$1.831^{\pm.048}$
FineMoGen ( Zhang et al. 2023b )	$0.504^{\pm.002}$	$0.690^{\pm.002}$	$0.784^{\pm.002}$	$0.151^{\pm.008}$	$2.998^{\pm.008}$	$\mathbf{2.696}^{\pm.079}$
MoMask ( Guo et al. 2024 )	$0.521^{\pm.002}$	$0.713^{\pm.002}$	$0.807^{\pm.002}$	$0.045^{\pm.002}$	$2.958^{\pm.008}$	$1.241^{\pm.040}$
DisCoRD ( Cho et al. 2025 )	$0.524^{\pm.003}$	$0.715^{\pm.003}$	$0.809^{\pm.002}$	$\mathbf{0.032}^{\pm.002}$	$2.938^{\pm.010}$	$1.288^{\pm.043}$
GenM3 ( Shi et al. 2025 )	$0.511^{\pm.002}$	$0.705^{\pm.002}$	$0.804^{\pm.002}$	$0.046^{\pm.002}$	$\mathbf{2.852}^{\pm.009}$	–

Table 2: Text-to-motion results on HumanML3D.

Figure 3: Qualitative comparison on MBench prompts.

Variant	MCC $\uparrow$	MG $\uparrow$	Jitter $\downarrow$	FS $\downarrow$
Group 1: Paradigm-critical components
w/o host signal	0.29	0.46	0.037	0.011
w/o text condition	0.42	0.74	0.022	0.007
w/o Stage 1 align	0.21	0.38	0.045	0.015
Group 2: Fusion strategy
w/o fusion (final layer)	0.43	0.69	0.026	0.007

Table 3: Ablation on Protocol A. Groups vary paradigm-critical components 1, fusion strategy 2, and hooked layers 3.

Figure 4: Learned $\sigma$ -conditioned fusion weights across $5$ hooked DiT layers (out of $40$ ).

Figure 5: Video and motion latents along the denoising trajectory. (a) Centered cosine similarity between $P_{v}(\mathbf{x^{\prime}}^{v}_{\sigma})$ and $\mathbf{x}^{m}_{\sigma}$ , averaged over $184$ evaluation prompts. Shaded band shows $\pm 1$ std. (b) Per-sample PCA projections of paired video (blue) and motion (red) latents from $\sigma=1$ ( $\bigstar$ ) to $\sigma=0$ ( $\times$ ).

Figure 6: Per-video diversity of PMD-generated motions.

Explore similar work

World2Motion: Turning Video World Models into 3D Human Motion Generators
Sep 29, 2026Fangyuan Tu, Xiangyue Zhang, Yiyi Cai +8Human Motion Generation Diffusion Models for Motion Generation
Japan Advanced Institute of Science and Technology · The University of Tokyo · Institute of Science Tokyo +1
Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
Jan 21, 2026Yifei Liu, Changxing Ding, Ling Guo +2Diffusion Model Guidance Text-to-Motion Generation
South China University of Technology · Joy Future Academy
VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
Jun 11, 2026Amir Mann, Gal Michael Harari, Merav Keidar +1Human Motion Generation Weakly Supervised Learning
Technion · NVIDIA

Sep 29, 2026J/K move · Enter open · S save

World2Motion: Turning Video World Models into 3D Human Motion Generators

Fangyuan Tu, Xiangyue Zhang, Yiyi Cai, Yichen Peng+7

Japan Advanced Institute of Science and Technology · The University of Tokyo · Institute of Science Tokyo+1

Human Motion Generation Diffusion Models for Motion Generation Video World Models Multimodal Diffusion Models

Open paper AskPDF arXiv

We present World2Motion, a framework that generates scene-aware 3D human motion and corresponding video from a single image and a text prompt. While existing 3D motion generators learn from motion datasets, their generalization is constrained by limited coverage of environments. In contrast, video world models such as Cosmos 3 offer broader environmental priors but are not designed for full-body motion generation; recovering motion from their generated videos requires costly two-stage inference. To address these, we turn Cosmos 3 into a single-stage 3D motion generator. This adaptation has two challenges: the scarcity of paired video--motion data and temporal instability in the generated motion. First, we construct a training dataset combining synthetic video--motion pairs with real videos paired with estimated 3D motion. Second, we propose a shift-decoupled noise schedule that assigns different noise levels to video and motion through shared denoising progress. This design accommodates the different denoising requirements of the two modalities, reducing motion jitter. Experiments on a multi-source interaction benchmark show that World2Motion has better motion--text alignment and scene interaction compared with the evaluated 3D motion generators. It also matches the interaction success rate of the two-stage baseline while achieving approximately 3.3

\times

faster inference. Our project page is available at https://fyantu.github.io/World2Motion/.

Abstract

Explore similar work

World2Motion: Turning Video World Models into 3D Human Motion Generators

Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation

VideoMDM: Towards 3D Human Motion Generation From 2D Supervision