ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control

Haozhe Jia; Jianfei Song; Yuan Zhang; Honglei Jin; Youcheng Fan; Wenshuo Chen; Wei Zhang; Yutao Yue

cs.CVMar 17, 2026

ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control

Authors: Haozhe Jia, Jianfei Song, Yuan Zhang, Honglei Jin, Youcheng Fan, Wenshuo Chen, Wei Zhang, Yutao Yue

Organizations: The Hong Kong University of Science and Technology (Guangzhou). · LimX Dynamics Technology Co., Ltd. · Shandong University. · Institute of Deep Perception Technology, Jiangsu Industrial Technology Research Institute (JITRI).

Abstract

We present ECHO, an edge--cloud framework for language-driven whole-body control of humanoid robots. A cloud-hosted diffusion-based text-to-motion generator synthesizes motion references from natural language instructions, while an edge-deployed reinforcement-learning tracker executes them in closed loop on the robot. The two modules are bridged by a compact, robot-native 38-dimensional motion representation that encodes joint angles, root planar velocity, root height, and a continuous 6D root orientation per frame, eliminating inference-time retargeting from human body models and remaining directly compatible with low-level PD control. The generator adopts a 1D convolutional UNet with cross-attention conditioned on CLIP-encoded text features; at inference, DDIM sampling with 10 denoising steps and classifier-free guidance produces motion sequences in approximately one second on a cloud GPU. The tracker follows a Teacher--Student paradigm: a privileged teacher policy is distilled into a lightweight student equipped with an evidential adaptation module for sim-to-real transfer, further strengthened by morphological symmetry constraints and domain randomization. An autonomous fall recovery mechanism detects falls via onboard IMU readings and retrieves recovery trajectories from a pre-built motion library. We evaluate ECHO on a retargeted HumanML3D benchmark, where it achieves strong generation quality (FID 0.029, R-Precision Top-1 0.686) under a unified robot-domain evaluator, while maintaining high motion safety and trajectory consistency. Real-world experiments on a Unitree G1 humanoid demonstrate stable execution of diverse text commands with zero hardware fine-tuning.

Figures & tables

Fig. 1: Overview of the proposed framework: The system features a Cloud-Edge decoupled deployment. The Cloud module utilizes a Diffusion Generator to synthesize motion from text instructions via CLIP encoding. The Edge module employs an RL-trained Student Policy that tracks targets using estimated privileged information. The resulting actions are executed via a PD controller for stable humanoid motion in the real world.

Panel of Figure 2 — Fig. 2: Sim-to-Real Results: Validation of robust tracking performance from simple gestures to dynamic maneuvers.

Method	R-Precision $\uparrow$			FID $\downarrow$	Div. $\uparrow$	MM Dist. $\downarrow$	MSS $\uparrow$	RTC $\uparrow$
Method	Top 1	Top 2	Top 3	FID $\downarrow$	Div. $\uparrow$	MM Dist. $\downarrow$	MSS $\uparrow$	RTC $\uparrow$
Ground Truth	$0.774^{\pm.004}$	$0.902^{\pm.002}$	$0.943^{\pm.001}$	$0.002$	$1.375^{\pm.003}$	-	$0.546^{\pm.000}$	-
MDM [ 19 ]	$0.578^{\pm.004}$	$0.755^{\pm.004}$	$0.832^{\pm.004}$	$0.073^{\pm.001}$	$1.362^{\pm.004}$	$0.651^{\pm.017}$	$0.527^{\pm.001}$	$\underline{0.442^{\pm.002}}$
TM2T [ 37 ]	$0.461^{\pm.003}$	$0.612^{\pm.004}$	$0.695^{\pm.004}$	$0.183^{\pm.001}$	$1.300^{\pm.004}$	$0.561^{\pm.017}$	$\mathbf{\underline{0.542^{\pm.001}}}$	$0.413^{\pm.002}$
MotionDiffuse [ 20 ]	$0.269^{\pm.003}$	$0.477^{\pm.003}$	$0.696^{\pm.002}$	$0.213^{\pm.004}$	$1.310^{\pm.006}$	$1.226^{\pm.082}$	$0.323^{\pm.002}$	$0.195^{\pm.008}$
StableMofusion [ 38 ]	$\underline{0.683^{\pm.003}}$	$\underline{0.843^{\pm.003}}$	$\underline{0.905^{\pm.002}}$	$\underline{0.037^{\pm.000}}$	$\underline{1.368^{\pm.004}}$	$\underline{0.414^{\pm.013}}$	$0.405^{\pm.001}$	$0.421^{\pm.002}$

TABLE I: Quantitative comparison of text-to-motion generation methods on the Retargeted HumanML3D. All baselines are re-evaluated under the same MoCLIP on the retargeted test split.

\pm

denotes 95% confidence intervals. Bold : best overall; underline : best among prior works.

Scheduler
Variant	R1 $\uparrow$	FID $\downarrow$	MSS $\uparrow$	RTC $\uparrow$	Time (s) $\downarrow$
DPM-Solver	0.643	0.040	0.466	0.431	1.02
DDPM	0.686	0.030	0.492	0.498	1.02
DDIM	0.686	0.029	0.484	0.493	1.02
Denoising Steps (DDIM)
5 steps	0.697	0.029	0.481	0.508	0.50

TABLE II: Ablation study on inference configurations (scheduler, denoising steps, and CFG scale). All variants use the ECHO-UNet backbone. Bold : best result within each group.

Text Command	Succ $\uparrow$	$E_{\text{g-mpjpe}}$ (mm) $\downarrow$	$E_{\text{mpjpe}}$ (mm) $\downarrow$
“punch”	$20/20$	$287.420^{\pm 50.200}$	$32.274^{\pm 1.764}$
“wave right hand”	$20/20$	$89.681^{\pm 6.345}$	$22.780^{\pm 0.619}$
“strum guitar with left hand”	$20/20$	$72.392^{\pm 6.687}$	$24.810^{\pm 0.573}$
“play the violin”	$20/20$	$111.449^{\pm 20.439}$	$23.933^{\pm 1.169}$

TABLE III: Real-world tracking performance over 20 trials.

E_{\text{g-mpjpe}}

and

E_{\text{mpjpe}}

are in mm;

\pm

denotes standard deviation.

Reward Term	Weight
Motion Tracking
Root Position Tracking	0.5
Root Rotation Tracking	0.5
Root Lin / Ang Velocity	1.0 / 1.0
Keypoint Tracking	1.0
Upper / Lower Keypoint Tracking	0.5 / 0.5

TABLE IV: Reward Function Configurations

Parameter	Randomization Range
Pelvis & Torso Mass	$[0.9\times,1.1\times]$
Pelvis & Torso COM Offset	$[-0.02,0.02]$ m
Ankle Static Friction	$[0.5\times,1.5\times]$
Ankle Solref Time Constant	$[0.015,0.03]$ s
Ankle Solref Damping Ratio	$[0.5,2.0]$
Joint Offset Error	$[-0.01,0.01]$ rad

TABLE V: Domain Randomization Parameters

Parameter	Value
Total Training Frames	$8\times 10^{9}$
PPO Epochs / Minibatches	5 / 8
Learning Rate	$5\times 10^{-4}$
Target KL Divergence	0.01
Discount Factor ( $\gamma$ ) / GAE ( $\lambda$ )	0.99 / 0.95
Clip Range	0.2

TABLE VI: PPO Hyperparameters & Details

Language-Conditioned Robot Control

Language-conditioned robot control connects natural-language instructions to grounded, safe actions across tasks and robot embodiments.

See all 74 papers

ECHO-G: Embodied Co-speech Humanoid mOtion Generation
Sep 30, 2026Yizhao Li, Pusen Gao, Ming Wang +3Text-to-Motion Generation Human Motion Generation
Beihang University · Mondo Robotics · The Hong Kong University of Science and Technology +1
TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation
Jun 22, 2026Jianuo Cao, Yuxin Chen, Yuzhen Song +3Humanoid Robot Control Test-Time Scaling
Nanjing University · University of California, Berkeley · Southern University of Science and Technology
RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids
Mar 18, 2026Xichen Yuan, Zhe Li, Bofan Lyu +4Humanoid Robot Locomotion Human Motion Generation
MARS Lab, Nanyang Technological University

Sep 30, 2026J/K move · Enter open · S save

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

Yizhao Li, Pusen Gao, Ming Wang, Shaojie Shen+2

Beihang University · Mondo Robotics · The Hong Kong University of Science and Technology+1

Text-to-Motion Generation Human Motion Generation Robot Motion Generation Co-Speech Gesture Generation Diffusion Models

Open paper AskPDF arXiv

Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To this end, we present ECHO-G, a framework jointly conditioned on speech audio and timed transcripts. Its Speech-Grounded Diffusion Transformer (SGDiT) combines frame-aligned acoustic features with token-level linguistic context, preserving their distinct granularities. Trained with rectified flow matching, it models one-to-many utterance-motion relationships directly in robot space. To support training and evaluation, we introduce a BEAT2-derived audio-text-robot dataset and a benchmark covering co-speech characteristics, robot-motion quality, and runtime efficiency. Comparative evaluation supports direct robot-space generation over the evaluated human-motion generation and retargeting pipelines, while modality ablations highlight the benefits of joint audio-text conditioning. We further demonstrate deployment on a physical humanoid robot. A complementary video-rating study also favors joint conditioning over the alternatives. The dataset and training, inference, and evaluation code are available through our project page.

Abstract

Language-Conditioned Robot Control

Explore similar work

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

TEXEDO : Test Time Scaling for Controller-aware Language-conditioned Humanoid Motion Generation

RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids