TRACE: Typed Reference-Aware Conversational Editing for Humanoid Motion Diffusion

Zejian Kang; Yifu Luo; Boyang Shen; Jingxuan Li; Zekai Chen; Zekun Shi; Yongxue Xu; Zhenyu Hou

cs.ROOct 15, 2026

TRACE: Typed Reference-Aware Conversational Editing for Humanoid Motion Diffusion

Authors: Zejian Kang, Yifu Luo, Boyang Shen, Jingxuan Li, Zekai Chen, Zekun Shi, Yongxue Xu, Zhenyu Hou,

Organizations: Zhejiang University · Westlake University · Technical University of Munich · Huazhong University of Science and Technology ·

Abstract

Language-conditioned humanoid motion generation enables users to create and refine robot movements through natural language. The central challenge is to connect an underspecified request to concrete motion structure that can subsequently be revised relative to an accepted reference. We present TRACE, a generation and editing framework for the Unitree G1 humanoid robot built around typed, measurable kinematic quantities. Our insight is to ground generation conditions in paired training motions and editing goals in the current motion. A language model learns to predict support, root, and hand events from motion-derived supervision; their time-aligned encodings complement the original text in a diffusion generator. Gravity-referenced states and trajectory--contact objectives improve kinematic consistency. For editing, typed requests define body--time scopes and source-relative goals. A separate reference-conditioned diffusion editor uses these goals and revised events to realize changes, while sampling-time projection preserves protected state coordinates. Quantitative evaluation and human assessment show improved semantic alignment and motion quality over the compared methods. Multi-turn editing evaluations demonstrate more reliable execution of successive relative requests under shared preservation constraints. Code is available at https://github.com/Multimodal-Perception-Action/TRACE.

Figures & tables

Figure 1: From text generation to reference-aware refinement. (a) The request “Do a wide jumping jack, arms overhead to sides” produces an initial motion reference. (b) “Jump higher, rest unchanged” specifies a relative height change; the schematic highlights the pelvis and legs involved in the revision. The robot sequence shows a deeper preparatory crouch and a higher jump. Orange and blue height profiles compare the initial and edited motions over normalized time.

Figure 2: TRACE with a motion-grounded sparse-event plan. The upper path maps a text instruction to continuous G1 motion. A frozen text encoder supplies prompt features, while an LLM predicts duration and typed support, root, and hand events. The event projector maps their numerical, time-aligned encoding to plan features that condition the diffusion Transformer (DiT) alongside the prompt. The lower-left path constructs training supervision: gravity-gauged states and forward kinematics (FK) provide measurable signals for event extraction, linked to the plan by the dashed path. Test-time plans are predicted from text. Snowflake and flame symbols denote frozen and trainable modules in their respective training stages; the trained LLM is frozen for diffusion training. Generated references can be passed to a downstream controller for robot execution, illustrated at lower right. The edit loop denotes follow-up requests; Section 3.4 describes the separate reference-conditioned editor.

	Automatic test-set metrics						User study
Method	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	MM-Dist $\downarrow$	FID $\downarrow$	Diversity	Quality $\uparrow$	Text match $\uparrow$	Overall $\uparrow$
TextOP ( Xie et al., 2026 )	$10.54$ [-1pt] $\pm 0.07$	$19.02$ [-1pt] $\pm 0.09$	$25.18$ [-1pt] $\pm 0.11$	$1.2950$ [-1pt]	$0.3985$ [-1pt]	$1.2437$ [-1pt]	$1.331$ [-1pt] $\pm 0.199$	$1.281$ [-1pt] $\pm 0.152$	$1.306$ [-1pt] $\pm 0.154$
TeXedo ( Cao et al., 2026 )	$10.78$ [-1pt] $\pm 0.10$	$17.80$ [-1pt] $\pm 0.09$	$22.60$ [-1pt] $\pm 0.07$	$1.3150$ [-1pt]	$0.4060$ [-1pt]	$1.2505$ [-1pt]	$1.275$ [-1pt] $\pm 0.191$	$1.106$ [-1pt] $\pm 0.106$	$1.191$ [-1pt] $\pm 0.125$
ECHO ( Jia et al., 2026 )	$24.08$ [-1pt] $\pm 0.12$	$36.90$ [-1pt] $\pm 0.13$	$44.05$ [-1pt] $\pm 0.13$	$1.2008$ [-1pt]	$0.2577$ [-1pt]	$1.3288$ [-1pt]	$1.469$ [-1pt] $\pm 0.218$	$1.469$ [-1pt] $\pm 0.196$	$1.469$ [-1pt] $\pm 0.195$
MoMask ( Guo et al., 2024 )	$31.57$ [-1pt] $\pm 0.08$	$41.88$ [-1pt] $\pm 0.10$	$46.80$ [-1pt] $\pm 0.07$	$1.1801$ [-1pt]	$0.3057$ [-1pt]	$1.2716$ [-1pt]	$1.463$ [-1pt] $\pm 0.258$	$1.250$ [-1pt] $\pm 0.216$	$1.356$ [-1pt] $\pm 0.188$
MDM ( Tevet et al., 2023 )	$58.84$ [-1pt] $\pm 0.09$	$73.51$ [-1pt] $\pm 0.11$	$79.31$ [-1pt] $\pm 0.06$	$0.9716$ [-1pt]	$0.0897$ [-1pt]	$1.3769$ [-1pt]	$1.313$ [-1pt] $\pm 0.225$	$1.313$ [-1pt] $\pm 0.193$	$1.313$ [-1pt] $\pm 0.182$

Table 1: BONES-SEED comparison: automatic metrics on test motions (R@k in percent) and a user study on 100 test prompts (scores 1–3). The upper block uses official checkpoints; the lower block contains G1-adapted baselines and TRACE trained on the processed BONES-SEED split. Cells show means above and uncertainty below where reported: evaluator-resampling 95% CI for R-Precision and SD across 16 participant means for the study. Automatic metrics and user-study scores follow the protocols defined in Appendix E.4 .

Figure 3: Qualitative comparison of Ours, MotionGPT3, and MDM. The third column illustrates generation from a free-form text prompt.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Real-robot demonstrations. Panel (a) corresponds to the in-the-wild request in Figure 3 . Panels (b,c) sample 0/25/50/75/100% of each recording. Fixed spatial crops preserve poses and temporal order; clips are not time-synchronized.

Channels	Quantity	Fixed states / rules
0–1	Left/right support	Enter at height $\leq 0.035$ and speed $\leq 0.20$ m/s; leave at height $>0.055$ or speed $>0.30$ m/s.
2–4	Root vertical speed	Descending / level / ascending, split at $-0.08,+0.08$ m/s.
5–7	Root height	Low / middle / high, split at $0.60,0.85$ .
8–11	Left hand height	H0 / H1 / H2 / H3, split at $0,0.25,0.45$ relative to pelvis.
12–15	Right hand height	Same four bins as the left hand.

Appendix

Table 2: Sparse-event schema. Channels are zero-indexed. The four categorical groups use softmax; the feet are independent sigmoid probabilities. All distances are in meters.

Patch	L foot	R foot	Root speed	Root height	L hand	R hand
0	1.00/1.00	1.00/1.00	level/1.00	middle/1.00	H0/1.00	H0/1.00
4	0.00/0.50	0.00/0.50	up/1.00	high/0.75	H0/1.00	H0/1.00
87	0.67/0.75	0.50/1.00	up/1.00	middle/0.75	H0/1.00	H0/0.50
127	1.00/1.00	1.00/1.00	level/1.00	middle/1.00	H0/1.00	H0/1.00

Appendix

Table 3: Actual cached training Plan example. Caption: a simple jump in place .

N=512

frames (25.6 s),

L=128

valid patches. The first non-mirrored jump UID in cache order is used. Patches show endpoints and first extrema of support, ascent, and confidence, selected solely from annotations, independently of model performance. Entries show support probability or categorical mode followed by confidence

W

Model	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	MM-Dist $\downarrow$	FID $\downarrow$	Diversity
Text Only	73.10	85.72	89.81	0.8341	0.0464	1.3817
Global Events	73.65	86.17	90.26	0.8310	0.0448	1.3809
Aligned Plan (Ours)	73.84	86.24	90.31	0.8165	0.0434	1.3766

Appendix

Table 4: Matched LLM Plan controls on test. Aligned Plan is the main-table model. R@k is in percent.

Supervision	R@1 $\uparrow$	R@2 $\uparrow$	R@3 $\uparrow$	MM-Dist $\downarrow$	FID $\downarrow$	Diversity
Caption-derived	73.26	85.89	90.00	0.8331	0.0458	1.3817
Motion-derived (Ours)	73.84	86.24	90.31	0.8165	0.0434	1.3766

Appendix

Table 5: Matched Plan-supervision source control on test. Both rows use the same predicted Plan at inference; only the training target source changes. R@k is in percent.

Variant	Sliding	Drift	Height error	Penetration
Heading-local	$28.152\pm 0.090$	$31.756\pm 0.074$	$3.385\pm 0.018$	$0.2659\pm 0.0004$
Gravity-Gauged	$26.047\pm 0.010$	$29.558\pm 0.023$	$2.596\pm 0.003$	$0.1575\pm 0.0021$
+ Geometry	$24.456\pm 0.033$	$27.817\pm 0.055$	$2.533\pm 0.008$	$0.1970\pm 0.0017$
+ Contact	$23.047\pm 0.062$	$26.383\pm 0.068$	$2.366\pm 0.007$	$0.1333\pm 0.0017$

Appendix

Table 6: Absolute physical errors underlying Figure 5 . Mean

\pm

SD over repeated sampling, in cm; lower is better. Sliding and Drift measure planar endpoint displacement over reference contact runs, using ankle and sole-point geometry, respectively.

Adapter	Supervised output	Saved inference artifact
Generation	duration and typed support/root/hand intervals	adapter weights, tokenizer/template identifier, schema, and event vocabulary
Editing	attribute, effector, signed change, and interval	adapter weights, tokenizer/template identifier, schema, and compiler vocabulary

Appendix

Table 7: Serialized language interfaces. Both outputs are schema-validated before being consumed by a motion model.

Stage	Updates	Supervision / added condition
Original editing	8k	Original retargeted motion pairs
Mixed editing	8k	Local/original source+scope initialization
Relative goal	4k	Relative offsets and wrist goal loss
Event-aware	4k	Source-derived event residual and consistency

Appendix

Table 8: TRACE editor training stages. Mixed batches contain equal numbers of local and original MotionFix pairs.

Method	Success (%)	XY RMSE (cm)	$\Delta v$ (deg/s)	$\Delta a$ (deg/s 2 )	New limits (%)
SimMotionEdit-G1	2.74	2.238	34.273	602.284	0.033
TMED-G1	6.48	2.206	52.730	967.980	0.043
TRACE	30.82	2.047	26.304	520.346	0.017

Appendix

Table 9: Current-cohort editing comparison. Success requires all four turns to pass. Quality diagnostics concern the fourth edit relative to its incoming motion. Lower is better for all quality columns.

Split	Originals	Mirrors	Full clips	Packed / eligible windows	Hours
Train	42,156	42,131	84,287	86,451 / 86,385	170.54
Validation	6,650	6,643	13,293	13,803 / 13,799	26.95
Test	6,653	6,649	13,302	13,722 / 13,706	26.91

Appendix

Table 10: Processed BONES-SEED counts. Full clips, packed windows, and eligible model examples are different units. Eligible windows have at least ten frames; durations count full clips at 20 FPS.

Setting	Raw-text backbone	Sparse-event continuation
Updates	32k	8k
Global batch	256	128
Learning rate	$10^{-4}$	$2\times 10^{-5}$

Appendix

Table 11: Generation training schedule. Continuations start from the same raw-text EMA initialization and retain equal residual capacity. Physical from-scratch controls are specified in Appendix C.3 .

Robot Motion Generation

Robot motion generation creates feasible, task-directed trajectories for robots to move, manipulate objects, and interact with people and environments.

See all 198 papers

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation
Jun 22, 2026Jianuo Cao, Yuxin Chen, Yuzhen Song +3Humanoid Robot Control Test-Time Scaling
Nanjing University · University of California, Berkeley · Southern University of Science and Technology
ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control
Mar 17, 2026Haozhe Jia, Jianfei Song, Yuan Zhang +5Language-Conditioned Robot Control Robot Motion Generation
The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. +1
ECHO-G: Embodied Co-speech Humanoid mOtion Generation
Sep 30, 2026Yizhao Li, Pusen Gao, Ming Wang +3Text-to-Motion Generation Human Motion Generation
Beihang University · Mondo Robotics · The Hong Kong University of Science and Technology +1

Jun 22, 2026J/K move · Enter open · S save

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation

Jianuo Cao, Yuxin Chen, Yuzhen Song, Masayoshi Tomizuka+2

Nanjing University · University of California, Berkeley · Southern University of Science and Technology

Humanoid Robot Control Test-Time Scaling Human Motion Generation Robot Motion Generation

Open paper AskPDF arXiv

Text-conditioned motion generation is a promising interface for programming humanoid robots, yet current generators are often trained on human motion datasets retargeted to robot morphologies. Although such data provides rich semantic and kinematic priors, it fails to capture the nuances of whole-body tracking controllers, including balance, contact dynamics, actuation limits, and controller-specific failure modes. As a result, generated motions can be semantically plausible but difficult or impossible for the robot to execute. We introduce TEXEDO, a test-time scaling framework for humanoid motion generation that improves motion quality without requiring a stronger underlying generator. Given a text prompt, TEXEDO samples multiple candidate motions from a pretrained text-conditioned generator and selects the best motion that is both executable and task-aligned. The reward model combines a dynamic feasibility verifier, distilled from whole-body tracking rollouts to predict physical executability, with a semantic alignment verifier that measures text-motion alignment in a learned co-embedding space. Our pipeline treats dynamic feasibility as a hard constraint and semantic alignment as the selection objective within the feasible set. Through large-scale simulation studies and real-world deployment on a Unitree G1 humanoid robot, we show that TEXEDO consistently improves both tracking fidelity and text alignment. These results demonstrate that grounded verification is an effective path toward deployable language-guided humanoid motion generation. Project website: https://jianuocao.github.io/TEXEDO/

Abstract

Robot Motion Generation

Explore similar work

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation

ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control

ECHO-G: Embodied Co-speech Humanoid mOtion Generation