PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

Chuxuan Huang; Zhanxiang Cao; Tianyue Zhan; Haoyang Li; Yang Zhang; Liyun Yan; Chenlin Ming; Keqi Zhu

cs.ROOct 9, 2026

PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

Authors: Chuxuan Huang, Zhanxiang Cao, Tianyue Zhan, Haoyang Li, Yang Zhang, Liyun Yan, Chenlin Ming, Keqi Zhu, +9 more

Organizations: Fudan University · Shanghai Innovation Institute · Shanghai Jiaotong University · Zhejiang University · Tongji University · The Chinese University of Hong Kong · Humanoid Robot (Shanghai) Co., Ltd. · Beihang University

Abstract

Translating language into humanoid motion requires coordinating body parts, determining motion duration, and producing references suitable for robot execution. We introduce PartiGen, a part-aware latent diffusion framework that incorporates anatomical structure into a shared whole-body motion representation. Part-specific attention processes anatomically masked frame features, and its outputs are fused to combine local limb information with whole-body context. Conditioned on text and motion history, the model generates motion primitives autoregressively. Forward-kinematics supervision encourages kinematic consistency, while a learned end-of-sequence predictor selects a frame-level endpoint after generation. On HumanML3D retargeted to the Unitree G1, PartiGen achieves a segment-level FID of 3.165, a 16.8% reduction over TextOp, and improves R@1 by 1.4 percentage points. In the duration study with 16-frame primitives, endpoint prediction reduces mean absolute duration error from 12.471 to 3.080 frames compared with a fixed-length cutoff. We further evaluate robot execution using a pretrained, frozen GRIT tracker: 409 of 500 generated references (81.8%) pass the posture-based stability screen in MuJoCo. Following simulation screening and manual selection, 26 of 27 attempted motions are judged successful on the real G1.

Figures & tables

Figure 1: The PartiGen system: from language to motion and execution. Our motion generator produces references; the execution component uses a pretrained, frozen GRIT controller ( mrzuang, 2026 ) for tracking in MuJoCo and on the Unitree G1. Hardware examples are selected after simulation screening.

Figure 2: Architecture and training of the PartiGen motion generator. GRIT is the pretrained, frozen execution component described in Section 3.2 . (a) Stage 1: BP-MVAE Training. Given history motion $h$ and future motion $f$ , the BP-MVAE encodes motion into a shared latent vector $\mathbf{z}\in\mathbb{R}^{128}$ . Part-aware features are fused into this vector for whole-body reconstruction. (b) Stage 2: DA-LDM Training. Building on latent diffusion ( Rombach et al., 2022 ) , DA-LDM directly predicts the clean shared motion code ( $x_{0}$ parameterization), conditioned on the noisy latent, CLIP text embeddings ( Radford et al., 2021 ) , history motion $h$ , and diffusion timestep. The predicted clean latent is decoded to generate future motion $\hat{f}$ . (c) Endpoint Selection. EOS selects an endpoint after complete candidate generation. (d) Multi-Part Instructions. Composition is evaluated as a diagnostic stress test, not an established zero-shot benchmark.

Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	mm-dist $\downarrow$
Dataset	$0.391\!\pm\!0.004$	$0.475\!\pm\!0.005$	$0.703\!\pm\!0.009$	$0\!\pm\!0.000$	$8.157\!\pm\!0.0035$	$3.814\!\pm\!0.013$
DART+GMR	$0.230\!\pm\!0.007$	$0.372\!\pm\!0.020$	$0.486\!\pm\!0.018$	$4.837\!\pm\!0.103$	$\mathbf{8.012}\!\pm\!0.069$	$5.746\!\pm\!0.026$
MDM+GMR	$0.186\!\pm\!0.004$	$0.357\!\pm\!0.017$	$0.473\!\pm\!0.018$	$8.038\!\pm\!0.082$	$7.883\!\pm\!0.046$	$6.025\!\pm\!0.014$
AttT2M+GMR	$0.157\!\pm\!0.003$	$0.302\!\pm\!0.009$	$0.438\!\pm\!0.013$	$6.712\!\pm\!0.135$	$\underline{8.323}\!\pm\!0.071$	$5.561\!\pm\!0.023$
TextOp	$\underline{0.287}\!\pm\!0.005$	$\mathbf{0.398}\!\pm\!0.009$	$\underline{0.587}\!\pm\!0.012$	$\underline{3.802}\!\pm\!0.145$	$8.764\!\pm\!0.087$	$\underline{4.464}\!\pm\!0.059$
Ours	$\mathbf{0.301}\!\pm\!0.007$	$\underline{0.386}\!\pm\!0.014$	$\mathbf{0.611}\!\pm\!0.015$	$\mathbf{3.165}\!\pm\!0.068$	$7.736\!\pm\!0.129$	$\mathbf{4.308}\!\pm\!0.086$

Table 1: HumanML3D-G1 reference quality: segment (top) and transition (bottom).

Figure 3: Qualitative comparison. (a) For “walk while waving,” the illustrated DART output shows a less pronounced arm gesture than PartiGen. (b) For “throw then take two steps,” PartiGen produces 148 frames after EOS-based endpoint selection, while the variant without duration adaptation uses a fixed length of 160 frames. These examples illustrate generated reference motions.

Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$
Compositional Dataset	$0.331\!\pm\!0.006$	$0.439\!\pm\!0.009$	$0.634\!\pm\!0.013$	$0.000\!\pm\!0.000$	$6.102\!\pm\!0.127$
DART+GMR	$0.102\!\pm\!0.005$	$0.148\!\pm\!0.012$	$0.220\!\pm\!0.009$	$9.340\!\pm\!0.125$	$15.120\!\pm\!0.392$
TextOp	$0.129\!\pm\!0.007$	$0.143\!\pm\!0.018$	$0.241\!\pm\!0.011$	$9.951\!\pm\!0.174$	$9.121\!\pm\!0.298$
Ours	$\mathbf{0.211}\!\pm\!0.008$	$\mathbf{0.297}\!\pm\!0.016$	$\mathbf{0.413}\!\pm\!0.021$	$\mathbf{6.531}\!\pm\!0.119$	$\mathbf{8.693}\!\pm\!0.167$

Table 2: Generated-reference results on the compositional stress test (100 multi-part instructions).

Method	Requests	Valid	Excluded	Stability passes	Stability rate ( $\uparrow$ )	Global MPJPE (mm, $\downarrow$ )	Root-relative MPJPE (mm, $\downarrow$ )
PartiGen (Ours)	500	496	4	409	81.8%	252.402 / 108.871	57.995 / 39.765
GT	500	499	1	427	85.4%	399.78 / 183.22	56.24 / 37.94
DART+GMR	500	494	6	395	79.0%	288.35 / 241.03	58.32 / 40.07
Random-mask	500	494	6	401	80.2%	283.90 / 157.29	56.38 / 38.25

Table 3: Frozen-GRIT tracking in MuJoCo. MPJPE: mean / median (mm), including valid unstable runs. Stability proxy: passes / requests.

Figure 4: G1 executions selected after simulation screening and manual review.

Variant	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	PJ $\downarrow$	Contact Acc $\uparrow$
w/o Part Attention	$0.274\!\pm\!0.005$	$0.356\!\pm\!0.009$	$0.575\!\pm\!0.012$	$3.811\!\pm\!0.094$	$7.791\!\pm\!0.108$	$0.035\!\pm\!0.001$	$0.861\!\pm\!0.007$
Random-mask	$0.281\!\pm\!0.017$	$0.362\!\pm\!0.009$	$0.586\!\pm\!0.016$	$3.533\!\pm\!0.218$	$7.771\!\pm\!0.043$	$0.028\!\pm\!0.009$	$0.880\!\pm\!0.016$
Full	$\mathbf{0.301}\!\pm\!0.007$	$\mathbf{0.386}\!\pm\!0.014$	$\mathbf{0.611}\!\pm\!0.015$	$\mathbf{3.165}\!\pm\!0.068$	$7.736\!\pm\!0.129$	$\mathbf{0.019}\!\pm\!0.001$	$\mathbf{0.914}\!\pm\!0.005$

Table 4: Generator ablations on HumanML3D-G1. Random-mask summarizes three different partition/run combinations (mean

\pm

sample SD). Contact Acc: foot-contact agreement.

Variant	Duration Error $\downarrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$
Oracle Length	$\mathbf{0.000}\!\pm\!0.000$	$\mathbf{0.618}\!\pm\!0.002$	$\mathbf{2.980}\!\pm\!0.015$	6.900 $\!\pm\!$ 0.045
Fixed-Length	12.471 $\!\pm\!$ 3.596	0.584 $\!\pm\!$ 0.005	4.317 $\!\pm\!$ 0.021	6.760 $\!\pm\!$ 0.038
Full (16-frame primitives)	3.080 $\!\pm\!$ 0.385	0.599 $\!\pm\!$ 0.013	3.207 $\!\pm\!$ 0.033	$\mathbf{7.836}\!\pm\!0.048$

Table 5: Effect of duration modeling on generated references on HumanML3D-G1.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Body-part grouping of the G1 robot. Colors identify the left leg (DoFs 0–5), right leg (6–11), torso (12–14), and both arms (15–22, labeled “Upper Body”). The skeleton is schematic; its nodes do not correspond one-to-one to individual DoFs. Grouping applies only to the joint-angle channels: root orientation, foot contacts, local root displacement, and root height (channels 0–10), together with all joint-angle increments (34–56), are shared across all heads.

Head Index	Body Part	DOF Indices	DOF Count
Head 0	Left Leg	[0, 1, 2, 3, 4, 5]	6
Head 1	Right Leg	[6, 7, 8, 9, 10, 11]	6
Head 2	Torso	[12, 13, 14]	3
Head 3	Upper Body	[15, 16, 17, 18, 19, 20, 21, 22]	8

Appendix

Table S1: Mapping of Attention Heads to Semantic Body Parts and DOF Indices.

Partition	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	PJ $\downarrow$	Contact Acc $\uparrow$
A	0.279	0.363	0.588	3.701	7.726	0.038	0.870
B	0.299	0.370	0.601	3.287	7.812	0.021	0.898
C	0.265	0.353	0.569	3.612	7.776	0.024	0.871

Appendix

Table S2: Individual Random-mask results. A, B, and C use different fixed partitions and separate training runs; each row is a point estimate.

Channels	Dimension	Feature
0	1	$\sin(\mathrm{roll})$
1	1	$\cos(\mathrm{roll})-1$
2	1	$\sin(\mathrm{pitch})$
3	1	$\cos(\mathrm{pitch})-1$
4	1	$\Delta\mathrm{yaw}$ : inter-frame yaw increment
5–6	2	Left and right foot contact indicators

Appendix

Table S3: Ordered layout of the 57-dimensional motion feature vector. Indices are zero-based and inclusive.

Aggregate Term (Main Paper)	Internal Component	Description	Sub-weight ( $\omega$ )
$\mathcal{L}_{\text{rec}}$	rec	Primary feature reconstruction	1.0
$\beta\mathcal{L}_{\text{KL}}$	kl	Variational bottleneck regularization	$1.0\times 10^{-4}$
$\lambda_{\text{kin}}\mathcal{L}_{\text{kin}}$ ( $\lambda_{\text{kin}}=0.8$ )	body_trans	FK-based body position error	0.05
	body_rot	FK-based body rotation error	0.01
	dof_pos	Joint angle precision	0.03
	dof_vel	Temporal joint velocity smoothness	$1.0\times 10^{-5}$

Appendix

Table S4: Comprehensive Decomposition of Loss Weights for BP-MVAE Training.

Category	Parameter	Value
Transformer Backbone	Number of Layers	8
	Hidden Dimension ( $h_{\text{dim}}$ )	512
	FFN Intermediate Dimension ( $ff_{\text{size}}$ )	1024
	Activation Function	GELU
Attention Mechanism	Number of Heads ( $N_{\text{heads}}$ )	4
	Dimension per Head ( $d_{k}$ )	128

Appendix

Table S5: Detailed Specifications for the DA-LDM Denoiser.

Kinematic Category	Instruction Prompt
Continuous Locomotion	A person is walking and pointing forward.
	A person is walking sideways and crossing arms.
	A person is walking backward and clapping hands.
	A person is jogging and putting hands on hips.
	A person is running and waving hands.
	A person is running and touching the head.

Appendix

Table S6: Representative Prompts from the Compositional Stress Test. The instructions are systematically categorized based on the primary lower-body/root dynamics.

Region	MuJoCo body names
Root	pelvis
Left leg	left_hip_pitch_link , left_knee_link , left_ankle_roll_link , left_toe_link
Right leg	right_hip_pitch_link , right_knee_link , right_ankle_roll_link , right_toe_link
Torso/head	torso_link , head_mimic
Left arm	left_shoulder_pitch_link , left_elbow_link , left_wrist_roll_link , left_hand_mimic
Right arm	right_shoulder_pitch_link , right_elbow_link , right_wrist_roll_link , right_hand_mimic

Appendix

Table S7: The 19 body origins used for tracking metrics. Concatenating rows from top to bottom gives the point order. The pelvis is included in both metrics.

Method	Requests	Valid	Excluded	Posture passes	Pass rate ( $\uparrow$ )	Global MPJPE (mm, $\downarrow$ )	Root-relative MPJPE (mm, $\downarrow$ )
Full	500	496	4	409	81.8%	252.402 / 108.871	57.995 / 39.765
w/o FK	500	495	5	464	92.8%	275.61 / 112.37	60.01 / 41.82

Appendix

Table S8: FK execution ablation under frozen GRIT. MPJPE: mean / median (mm), including valid unstable runs. The posture-screen rate is passes divided by all requests, rather than a measure of semantic completion.

Method	Stable motions	Global MPJPE	Root-relative MPJPE
PartiGen	409	198.00 / 97.08	42.84 / 37.98
GT	427	297.81 / 161.34	44.26 / 36.04
DART+GMR	395	247.33 / 145.29	44.37 / 36.64
w/o FK	464	252.47 / 140.92	45.20 / 37.53

Appendix

Table S9: Raw MPJPE within the simulation-stability-passing subsets. Values are mean / median (mm); no initial-translation correction or upper-tail trimming is applied.

Category	Selected	Attempted	Successful	Failed	Unattempted
Upper-body	10	10	10	0	0
Lower-body	10	7	7	0	3
Long-sequence	5	5	5	0	0
Composite	5	5	4	1	0
Total	30	27	26	1	3

Appendix

Table S10: Outcomes for manually selected hardware motions. Counts refer to distinct motions, not repeated executions. Success is assessed manually by the authors for instruction fulfillment and qualitative agreement with simulation. Unattempted motions are separate from execution failures.

Explore similar work

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
Dec 15, 2025Yannan He, Garvita Tiwari, Xiaohan Zhang +4Latent Diffusion Models Text-to-Motion Generation
University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany +2
ParTY: Part-Guidance for Expressive Text-to-Motion Synthesis
Mar 10, 2026KunHo Heo, SuYeon Kim, Yonghyun Gwon +2Text-to-Motion Generation Human Motion Generation
Kyung Hee University
TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation
Jun 22, 2026Jianuo Cao, Yuxin Chen, Yuzhen Song +3Humanoid Robot Control Test-Time Scaling
Nanjing University · University of California, Berkeley · Southern University of Science and Technology

Dec 15, 2025J/K move · Enter open · S save

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

Yannan He, Garvita Tiwari, Xiaohan Zhang, Pankaj Bora+3

University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany+2

Latent Diffusion Models Text-to-Motion Generation Cross-Modal Alignment Motion Diffusion Models Human Motion Generation Diffusion Models for Motion Generation

Open paper AskPDF arXiv

We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diffusion, either on the whole latent at once or auto-regressively over multiple latents. In this paper, we study how to make diffusion on continuous motion latents work best. We focus on two questions: (1) how to build a semantically aligned latent space so diffusion becomes more effective, and (2) how to best inject text conditioning so the motion follows the description closely. We propose a semantic-aligned motion encoder trained with frame-level text labels so that latents with similar text meaning stay close, which makes the latent space more diffusion-friendly. We also compare single-token conditioning with a multi-token cross-attention scheme and find that cross-attention gives better motion realism and text-motion alignment. With semantically aligned latents, auto-regressive generation, and cross-attention text conditioning, our model sets a new state of the art in human motion generation on standard metrics and in a user study. We will release our code and models for further research and downstream usage.

Abstract

Explore similar work

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

ParTY: Part-Guidance for Expressive Text-to-Motion Synthesis

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation