PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

Chuxuan Huang; Zhanxiang Cao; Tianyue Zhan; Haoyang Li; Yang Zhang; Liyun Yan; Chenlin Ming; Keqi Zhu

cs.ROOct 9, 2026

PartiGen: Part-Aware Representations for Language-Conditioned Humanoid Motion Generation

Authors: Chuxuan Huang, Zhanxiang Cao, Tianyue Zhan, Haoyang Li, Yang Zhang, Liyun Yan, Chenlin Ming, Keqi Zhu, +9 more

Organizations: Fudan University · Shanghai Innovation Institute · Shanghai Jiaotong University · Zhejiang University · Tongji University · The Chinese University of Hong Kong · Humanoid Robot (Shanghai) Co., Ltd. · Beihang University

Abstract

Translating language into humanoid motion requires coordinating body parts, determining motion duration, and producing references suitable for robot execution. We introduce PartiGen, a part-aware latent diffusion framework that incorporates anatomical structure into a shared whole-body motion representation. Part-specific attention processes anatomically masked frame features, and its outputs are fused to combine local limb information with whole-body context. Conditioned on text and motion history, the model generates motion primitives autoregressively. Forward-kinematics supervision encourages kinematic consistency, while a learned end-of-sequence predictor selects a frame-level endpoint after generation. On HumanML3D retargeted to the Unitree G1, PartiGen achieves a segment-level FID of 3.165, a 16.8% reduction over TextOp, and improves R@1 by 1.4 percentage points. In the duration study with 16-frame primitives, endpoint prediction reduces mean absolute duration error from 12.471 to 3.080 frames compared with a fixed-length cutoff. We further evaluate robot execution using a pretrained, frozen GRIT tracker: 409 of 500 generated references (81.8%) pass the posture-based stability screen in MuJoCo. Following simulation screening and manual selection, 26 of 27 attempted motions are judged successful on the real G1.

Figures & tables

Figure 1: The PartiGen system: from language to motion and execution. Our motion generator produces references; the execution component uses a pretrained, frozen GRIT controller ( mrzuang, 2026 ) for tracking in MuJoCo and on the Unitree G1. Hardware examples are selected after simulation screening.

Figure 2: Architecture and training of the PartiGen motion generator. GRIT is the pretrained, frozen execution component described in Section 3.2 . (a) Stage 1: BP-MVAE Training. Given history motion $h$ and future motion $f$ , the BP-MVAE encodes motion into a shared latent vector $\mathbf{z}\in\mathbb{R}^{128}$ . Part-aware features are fused into this vector for whole-body reconstruction. (b) Stage 2: DA-LDM Training. Building on latent diffusion ( Rombach et al., 2022 ) , DA-LDM directly predicts the clean shared motion code ( $x_{0}$ parameterization), conditioned on the noisy latent, CLIP text embeddings ( Radford et al., 2021 ) , history motion $h$ , and diffusion timestep. The predicted clean latent is decoded to generate future motion $\hat{f}$ . (c) Endpoint Selection. EOS selects an endpoint after complete candidate generation. (d) Multi-Part Instructions. Composition is evaluated as a diagnostic stress test, not an established zero-shot benchmark.

Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	mm-dist $\downarrow$
Dataset	$0.391\!\pm\!0.004$	$0.475\!\pm\!0.005$	$0.703\!\pm\!0.009$	$0\!\pm\!0.000$	$8.157\!\pm\!0.0035$	$3.814\!\pm\!0.013$
DART+GMR	$0.230\!\pm\!0.007$	$0.372\!\pm\!0.020$	$0.486\!\pm\!0.018$	$4.837\!\pm\!0.103$	$\mathbf{8.012}\!\pm\!0.069$	$5.746\!\pm\!0.026$
MDM+GMR	$0.186\!\pm\!0.004$	$0.357\!\pm\!0.017$	$0.473\!\pm\!0.018$	$8.038\!\pm\!0.082$	$7.883\!\pm\!0.046$	$6.025\!\pm\!0.014$
AttT2M+GMR	$0.157\!\pm\!0.003$	$0.302\!\pm\!0.009$	$0.438\!\pm\!0.013$	$6.712\!\pm\!0.135$	$\underline{8.323}\!\pm\!0.071$	$5.561\!\pm\!0.023$
TextOp	$\underline{0.287}\!\pm\!0.005$	$\mathbf{0.398}\!\pm\!0.009$	$\underline{0.587}\!\pm\!0.012$	$\underline{3.802}\!\pm\!0.145$	$8.764\!\pm\!0.087$	$\underline{4.464}\!\pm\!0.059$
Ours	$\mathbf{0.301}\!\pm\!0.007$	$\underline{0.386}\!\pm\!0.014$	$\mathbf{0.611}\!\pm\!0.015$	$\mathbf{3.165}\!\pm\!0.068$	$7.736\!\pm\!0.129$	$\mathbf{4.308}\!\pm\!0.086$

Table 1: HumanML3D-G1 reference quality: segment (top) and transition (bottom).

Figure 3: Qualitative comparison. (a) For “walk while waving,” the illustrated DART output shows a less pronounced arm gesture than PartiGen. (b) For “throw then take two steps,” PartiGen produces 148 frames after EOS-based endpoint selection, while the variant without duration adaptation uses a fixed length of 160 frames. These examples illustrate generated reference motions.

Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$
Compositional Dataset	$0.331\!\pm\!0.006$	$0.439\!\pm\!0.009$	$0.634\!\pm\!0.013$	$0.000\!\pm\!0.000$	$6.102\!\pm\!0.127$
DART+GMR	$0.102\!\pm\!0.005$	$0.148\!\pm\!0.012$	$0.220\!\pm\!0.009$	$9.340\!\pm\!0.125$	$15.120\!\pm\!0.392$
TextOp	$0.129\!\pm\!0.007$	$0.143\!\pm\!0.018$	$0.241\!\pm\!0.011$	$9.951\!\pm\!0.174$	$9.121\!\pm\!0.298$
Ours	$\mathbf{0.211}\!\pm\!0.008$	$\mathbf{0.297}\!\pm\!0.016$	$\mathbf{0.413}\!\pm\!0.021$	$\mathbf{6.531}\!\pm\!0.119$	$\mathbf{8.693}\!\pm\!0.167$

Table 2: Generated-reference results on the compositional stress test (100 multi-part instructions).

Method	Requests	Valid	Excluded	Stability passes	Stability rate ( $\uparrow$ )	Global MPJPE (mm, $\downarrow$ )	Root-relative MPJPE (mm, $\downarrow$ )
PartiGen (Ours)	500	496	4	409	81.8%	252.402 / 108.871	57.995 / 39.765
GT	500	499	1	427	85.4%	399.78 / 183.22	56.24 / 37.94
DART+GMR	500	494	6	395	79.0%	288.35 / 241.03	58.32 / 40.07
Random-mask	500	494	6	401	80.2%	283.90 / 157.29	56.38 / 38.25

Table 3: Frozen-GRIT tracking in MuJoCo. MPJPE: mean / median (mm), including valid unstable runs. Stability proxy: passes / requests.

Figure 4: G1 executions selected after simulation screening and manual review.

Variant	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	PJ $\downarrow$	Contact Acc $\uparrow$
w/o Part Attention	$0.274\!\pm\!0.005$	$0.356\!\pm\!0.009$	$0.575\!\pm\!0.012$	$3.811\!\pm\!0.094$	$7.791\!\pm\!0.108$	$0.035\!\pm\!0.001$	$0.861\!\pm\!0.007$
Random-mask	$0.281\!\pm\!0.017$	$0.362\!\pm\!0.009$	$0.586\!\pm\!0.016$	$3.533\!\pm\!0.218$	$7.771\!\pm\!0.043$	$0.028\!\pm\!0.009$	$0.880\!\pm\!0.016$
Full	$\mathbf{0.301}\!\pm\!0.007$	$\mathbf{0.386}\!\pm\!0.014$	$\mathbf{0.611}\!\pm\!0.015$	$\mathbf{3.165}\!\pm\!0.068$	$7.736\!\pm\!0.129$	$\mathbf{0.019}\!\pm\!0.001$	$\mathbf{0.914}\!\pm\!0.005$

Table 4: Generator ablations on HumanML3D-G1. Random-mask summarizes three different partition/run combinations (mean

\pm

sample SD). Contact Acc: foot-contact agreement.

Variant	Duration Error $\downarrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$
Oracle Length	$\mathbf{0.000}\!\pm\!0.000$	$\mathbf{0.618}\!\pm\!0.002$	$\mathbf{2.980}\!\pm\!0.015$	6.900 $\!\pm\!$ 0.045
Fixed-Length	12.471 $\!\pm\!$ 3.596	0.584 $\!\pm\!$ 0.005	4.317 $\!\pm\!$ 0.021	6.760 $\!\pm\!$ 0.038
Full (16-frame primitives)	3.080 $\!\pm\!$ 0.385	0.599 $\!\pm\!$ 0.013	3.207 $\!\pm\!$ 0.033	$\mathbf{7.836}\!\pm\!0.048$

Table 5: Effect of duration modeling on generated references on HumanML3D-G1.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Body-part grouping of the G1 robot. Colors identify the left leg (DoFs 0–5), right leg (6–11), torso (12–14), and both arms (15–22, labeled “Upper Body”). The skeleton is schematic; its nodes do not correspond one-to-one to individual DoFs. Grouping applies only to the joint-angle channels: root orientation, foot contacts, local root displacement, and root height (channels 0–10), together with all joint-angle increments (34–56), are shared across all heads.

Head Index	Body Part	DOF Indices	DOF Count
Head 0	Left Leg	[0, 1, 2, 3, 4, 5]	6
Head 1	Right Leg	[6, 7, 8, 9, 10, 11]	6
Head 2	Torso	[12, 13, 14]	3
Head 3	Upper Body	[15, 16, 17, 18, 19, 20, 21, 22]	8

Appendix

Table S1: Mapping of Attention Heads to Semantic Body Parts and DOF Indices.

Partition	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	FID $\downarrow$	Div $\rightarrow$	PJ $\downarrow$	Contact Acc $\uparrow$
A	0.279	0.363	0.588	3.701	7.726	0.038	0.870
B	0.299	0.370	0.601	3.287	7.812	0.021	0.898
C	0.265	0.353	0.569	3.612	7.776	0.024	0.871

Appendix

Table S2: Individual Random-mask results. A, B, and C use different fixed partitions and separate training runs; each row is a point estimate.

Channels	Dimension	Feature
0	1	$\sin(\mathrm{roll})$
1	1	$\cos(\mathrm{roll})-1$
2	1	$\sin(\mathrm{pitch})$
3	1	$\cos(\mathrm{pitch})-1$
4	1	$\Delta\mathrm{yaw}$ : inter-frame yaw increment
5–6	2	Left and right foot contact indicators

Appendix

Table S3: Ordered layout of the 57-dimensional motion feature vector. Indices are zero-based and inclusive.

Aggregate Term (Main Paper)	Internal Component	Description	Sub-weight ( $\omega$ )
$\mathcal{L}_{\text{rec}}$	rec	Primary feature reconstruction	1.0
$\beta\mathcal{L}_{\text{KL}}$	kl	Variational bottleneck regularization	$1.0\times 10^{-4}$
$\lambda_{\text{kin}}\mathcal{L}_{\text{kin}}$ ( $\lambda_{\text{kin}}=0.8$ )	body_trans	FK-based body position error	0.05
	body_rot	FK-based body rotation error	0.01
	dof_pos	Joint angle precision	0.03
	dof_vel	Temporal joint velocity smoothness	$1.0\times 10^{-5}$

Appendix

Table S4: Comprehensive Decomposition of Loss Weights for BP-MVAE Training.

Category	Parameter	Value
Transformer Backbone	Number of Layers	8
	Hidden Dimension ( $h_{\text{dim}}$ )	512
	FFN Intermediate Dimension ( $ff_{\text{size}}$ )	1024
	Activation Function	GELU
Attention Mechanism	Number of Heads ( $N_{\text{heads}}$ )	4
	Dimension per Head ( $d_{k}$ )	128

Appendix

Table S5: Detailed Specifications for the DA-LDM Denoiser.

Kinematic Category	Instruction Prompt
Continuous Locomotion	A person is walking and pointing forward.
	A person is walking sideways and crossing arms.
	A person is walking backward and clapping hands.
	A person is jogging and putting hands on hips.
	A person is running and waving hands.
	A person is running and touching the head.

Appendix

Table S6: Representative Prompts from the Compositional Stress Test. The instructions are systematically categorized based on the primary lower-body/root dynamics.

Region	MuJoCo body names
Root	pelvis
Left leg	left_hip_pitch_link , left_knee_link , left_ankle_roll_link , left_toe_link
Right leg	right_hip_pitch_link , right_knee_link , right_ankle_roll_link , right_toe_link
Torso/head	torso_link , head_mimic
Left arm	left_shoulder_pitch_link , left_elbow_link , left_wrist_roll_link , left_hand_mimic
Right arm	right_shoulder_pitch_link , right_elbow_link , right_wrist_roll_link , right_hand_mimic

Appendix

Table S7: The 19 body origins used for tracking metrics. Concatenating rows from top to bottom gives the point order. The pelvis is included in both metrics.

Method	Requests	Valid	Excluded	Posture passes	Pass rate ( $\uparrow$ )	Global MPJPE (mm, $\downarrow$ )	Root-relative MPJPE (mm, $\downarrow$ )
Full	500	496	4	409	81.8%	252.402 / 108.871	57.995 / 39.765
w/o FK	500	495	5	464	92.8%	275.61 / 112.37	60.01 / 41.82

Appendix

Table S8: FK execution ablation under frozen GRIT. MPJPE: mean / median (mm), including valid unstable runs. The posture-screen rate is passes divided by all requests, rather than a measure of semantic completion.

Method	Stable motions	Global MPJPE	Root-relative MPJPE
PartiGen	409	198.00 / 97.08	42.84 / 37.98
GT	427	297.81 / 161.34	44.26 / 36.04
DART+GMR	395	247.33 / 145.29	44.37 / 36.64
w/o FK	464	252.47 / 140.92	45.20 / 37.53

Appendix

Table S9: Raw MPJPE within the simulation-stability-passing subsets. Values are mean / median (mm); no initial-translation correction or upper-tail trimming is applied.

Category	Selected	Attempted	Successful	Failed	Unattempted
Upper-body	10	10	10	0	0
Lower-body	10	7	7	0	3
Long-sequence	5	5	5	0	0
Composite	5	5	4	1	0
Total	30	27	26	1	3

Appendix

Table S10: Outcomes for manually selected hardware motions. Counts refer to distinct motions, not repeated executions. Success is assessed manually by the authors for instruction fulfillment and qualitative agreement with simulation. Unattempted motions are separate from execution failures.

Explore similar work

Dec 15, 2025

MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation

We introduce MoLingo, a text-to-motion (T2M) model that generates realistic, lifelike human motion by denoising in a continuous latent space. Recent works perform latent space diffusion, either on the whole latent at once or auto-regressively over multiple latents. In this paper, we study how to make diffusion on continuous motion latents work best. We focus on two questions: (1) how to build a semantically aligned latent space so diffusion becomes more effective, and (2) how to best inject text conditioning so the motion follows the description closely. We propose a semantic-aligned motion encoder trained with frame-level text labels so that latents with similar text meaning stay close, which makes the latent space more diffusion-friendly. We also compare single-token conditioning with a multi-token cross-attention scheme and find that cross-attention gives better motion realism and text-motion alignment. With semantically aligned latents, auto-regressive generation, and cross-attention text conditioning, our model sets a new state of the art in human motion generation on standard metrics and in a user study. We will release our code and models for further research and downstream usage.

Yannan He, Garvita Tiwari, Xiaohan Zhang +4

University of Tübingen, Germany · Tübingen AI Center, Germany · Max Planck Institute for Informatics, Saarland Informatics Campus, Germany +2

Latent Diffusion Models Text-to-Motion Generation Cross-Modal Alignment Motion Diffusion Models Human Motion Generation Diffusion Models for Motion Generation

Ask PDF

Mar 10, 2026

ParTY: Part-Guidance for Expressive Text-to-Motion Synthesis

Text-to-motion synthesis aims to generate natural and expressive human motions from textual descriptions. While existing approaches primarily focus on generating holistic motions from text descriptions, they struggle to accurately reflect actions involving specific body parts. Recent part-wise motion generation methods attempt to resolve this but face two critical limitations: (i) they lack explicit mechanisms for aligning textual semantics with individual body parts, and (ii) they often generate incoherent full-body motions due to integrating independently generated part motions. To overcome these issues and resolve the fundamental trade-off in existing methods, we propose ParTY, a novel framework that enhances part expressiveness while generating coherent full-body motions. ParTY comprises: (1) Part-Guided Network, which first generates part motions to obtain part guidance, then uses it to generate holistic motions; (2) Part-aware Text Grounding, which diversely transforms text embeddings and appropriately aligns them with each body part; and (3) Holistic-Part Fusion, which adaptively fuses holistic motions and part motions. Extensive experiments, including part-level and coherence-level evaluations, demonstrate that ParTY achieves substantial improvements over previous methods.

KunHo Heo, SuYeon Kim, Yonghyun Gwon +2

Kyung Hee University

Text-to-Motion Generation Human Motion Generation

Ask PDF

Jun 22, 2026

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation

Text-conditioned motion generation is a promising interface for programming humanoid robots, yet current generators are often trained on human motion datasets retargeted to robot morphologies. Although such data provides rich semantic and kinematic priors, it fails to capture the nuances of whole-body tracking controllers, including balance, contact dynamics, actuation limits, and controller-specific failure modes. As a result, generated motions can be semantically plausible but difficult or impossible for the robot to execute. We introduce TEXEDO, a test-time scaling framework for humanoid motion generation that improves motion quality without requiring a stronger underlying generator. Given a text prompt, TEXEDO samples multiple candidate motions from a pretrained text-conditioned generator and selects the best motion that is both executable and task-aligned. The reward model combines a dynamic feasibility verifier, distilled from whole-body tracking rollouts to predict physical executability, with a semantic alignment verifier that measures text-motion alignment in a learned co-embedding space. Our pipeline treats dynamic feasibility as a hard constraint and semantic alignment as the selection objective within the feasible set. Through large-scale simulation studies and real-world deployment on a Unitree G1 humanoid robot, we show that TEXEDO consistently improves both tracking fidelity and text alignment. These results demonstrate that grounded verification is an effective path toward deployable language-guided humanoid motion generation. Project website: https://jianuocao.github.io/TEXEDO/

Jianuo Cao, Yuxin Chen, Yuzhen Song +3

Nanjing University · University of California, Berkeley · Southern University of Science and Technology

Humanoid Robot Control Test-Time Scaling Human Motion Generation Robot Motion Generation

Ask PDF