Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control

Zelin Tao; Zeran Su; Peiran Liu; Jingkai Sun; Wenqiang Que; Jiahao Ma; Jialin Yu; Jiahang Cao

cs.ROMar 29, 2026

Heracles: Bridging Precise Tracking and Generative Synthesis for General Humanoid Control

Authors: Zelin Tao, Zeran Su, Peiran Liu, Jingkai Sun, Wenqiang Que, Jiahao Ma, Jialin Yu, Jiahang Cao,

Abstract

Achieving general-purpose humanoid control requires a delicate balance between the precise execution of commanded motions and the flexible, anthropomorphic adaptability needed to recover from unpredictable environmental perturbations. Current general controllers predominantly formulate motion control as a rigid reference-tracking problem. While effective in nominal conditions, these trackers often exhibit brittle, non-anthropomorphic failure modes under severe disturbances, lacking the generative adaptability inherent to human motor control. To overcome this limitation, we propose Heracles, a novel state-conditioned diffusion middleware that bridges precise motion tracking and generative synthesis. Rather than relying on rigid tracking paradigms or complex explicit mode-switching, Heracles operates as an intermediary layer between high-level reference motions and low-level physics trackers. By conditioning on the robot's real-time state, the diffusion model implicitly adapts its behavior: it approximates an identity map when the state closely aligns with the reference, preserving zero-shot tracking fidelity. Conversely, when encountering significant state deviations, it seamlessly transitions into a generative synthesizer to produce natural, anthropomorphic recovery trajectories. Our framework demonstrates that integrating generative priors into the control loop not only significantly enhances robustness against extreme perturbations but also elevates humanoid control from a rigid tracking paradigm to an open-ended, generative general-purpose architecture.

Figures & tables

Figure 1: Heracles synthesizes diverse, anthropomorphic recovery motions via state-conditioned diffusion. In contrast to recovery policies trained with termination tricks that converge to a limited set of stereotypical maneuvers, Heracles leverages its generative middleware to produce a rich repertoire of agile, human-like recovery behaviors, enabling more general and robust responses across a wide range of extreme perturbations.

Figure 2: Overview of the Heracles framework. (a) A flow matching model $\hat{D}_{\theta}$ learns to synthesize feasible keyframe trajectories conditioned on the current state. (b) Reference motions are quantized into discrete tokens via FSQ, shared by reconstruction and action prediction heads. (c) At inference, the middleware generates trajectories through closed-loop replanning for the motion tracker to execute.

Panel of Table 1 — Table 1: Training configuration for the general-purpose physics tracker. Left: reward function with exponential tracking terms and regularization penalties. Right: domain randomization ranges for command-level, physical, and external disturbance parameters.

Tracker Hyperparameter	Value
Parallel environments	16,384
Rollout horizon	24 steps
Discount factor $\gamma$	0.99
GAE $\lambda$	0.95
PPO epochs / mini-batches	5 / 4
Clipping ratio $\epsilon$	0.2

Table 2: Training hyperparameters for the physics tracker and the trajectory generator. Left: PPO-based tracker training configuration. Right: flow matching trajectory generator architecture, optimization, and inference settings.

Method	Component	Configuration
MLP	Actor	$[4096,2048,1024,512,256]$ MLP
	Critic	$[4096,2048,1024,512,256]$ MLP
Transformer	Tokenizer	$[512,512]$ MLP $\times\,2$ modalities $\to$ 3 tokens (512-dim)
	Backbone	3-layer, 4-head, 512-dim Transformer
	Action head	$[2048,1024,256]$ MLP
	Critic	$[3072,1536,768,512]$ MLP

Table 3: Network architecture specifications for all evaluated methods.

[\cdot]

denotes MLP hidden-layer widths. iFSQ ∗ covers all iFSQ variants (iFSQ

{}_{\text{BM}}

, iFSQ

{}_{\text{+H}}

, iFSQ), which share the same network but differ in observation design.

Method	Obs.	Tokenizer	Policy	Traj. Gen.
MLP	BM	–	MLP	✗
Transformer	BM	–	Trans.	✗
VQ-VAE	BM	VQ	Enc-Dec	✗
SONIC	SONIC	–	SONIC	✗
iFSQ ${}_{\text{BM}}$	BM+H	iFSQ	Enc-Dec	✗
iFSQ ${}_{\text{+H}}$	$\dagger$ +H	iFSQ	Enc-Dec	✗

Table 4: Component configuration and quantitative comparison of all evaluated methods. (a) Obs. column: BM = BeyondMimic-style Liao et al. (2025) ;

\dagger

= proposed ( Sec. 3.3 ); +H = with height features. ✓/✗ indicates presence or absence of the trajectory generator. (b) Completion rate measures the fraction of reference frames for which the root height error remains below 0.3 m and the root orientation error stays below 1.2 rad; tracking errors are reported for joint positions, root height, root orientation (excl. yaw), and root linear velocity. Blue cells mark the best result per metric. Colored percentages show relative change from MLP: teal = better, red = worse.

Figure 3: Qualitative sim-to-sim comparison on an out-of-distribution martial arts sequence. Each row shows a different method tracking the same reference motion on a Unitree G1 humanoid alongside a reference ghost in MuJoCo. MLP, Transformer, and SONIC collapse early; VQ-VAE barely tracks the motion. Among our ablations, iFSQ BM and iFSQ survive without falling, while iFSQ +H falls but later recovers. Heracles (Ours) tracks the full sequence most accurately, demonstrating the strongest robustness to OOD motions.

Figure 4: Real-world motion tracking across diverse and dynamic behaviors. Real-world experiments demonstrate that our model generalizes to a broad spectrum of motions, from everyday locomotion (walk, run) to highly dynamic skills (kick, 360° kick) and human-object interaction.

Method	CR (%) $\uparrow$	Joint Err (rad) $\downarrow$	Height Err (m) $\downarrow$	Ori Err (rad) $\downarrow$	LinVel Err (m/s) $\downarrow$
MLP	$44.0$	$2.1720$	$0.3586$	$1.0157$	$0.2710$
Transformer	$40.6$ ( $-7.7\%$ )	$2.8309$ ( $-30.3\%$ )	$0.3706$ ( $-3.3\%$ )	$0.8355$ ( $+17.7\%$ )	$0.2708$ ( $+0.1\%$ )
VQ-VAE	$69.8$ ( $+58.6\%$ )	$2.5700$ ( $-18.3\%$ )	$0.1898$ ( $+47.1\%$ )	$0.4307$ ( $+57.6\%$ )	$0.2719$ ( $-0.3\%$ )
SONIC	$42.8$ ( $-2.7\%$ )	$2.9897$ ( $-37.6\%$ )	$0.3342$ ( $+6.8\%$ )	$0.8629$ ( $+15.0\%$ )	$0.2895$ ( $-6.8\%$ )
iFSQ ${}_{\text{BM}}$	$52.4$ ( $+19.1\%$ )	$2.4482$ ( $-12.7\%$ )	$0.2880$ ( $+19.7\%$ )	$0.9109$ ( $+10.3\%$ )	$0.3134$ ( $-15.6\%$ )
iFSQ ${}_{\text{+H}}$	$52.7$ ( $+19.8\%$ )	$1.7660$ ( $+18.7\%$ )	$\mathbf{0.0419}$ ( $+88.3\%$ )	$0.2744$ ( $+73.0\%$ )	$0.2793$ ( $-3.1\%$ )

Table 5: Fall-and-recovery evaluation on challenging lie-to-stand, prone-to-stand, and stand-to-lie sequences. CR denotes completion rate. Blue marks the best result per metric. Colored percentages show relative change from MLP: teal = better, red = worse.

Variant	CR (%) $\uparrow$	Joint Err (rad) $\downarrow$	Height Err (m) $\downarrow$	Ori Err (rad) $\downarrow$	LinVel Err (m/s) $\downarrow$
Heracles (full)	$\mathbf{90.6}$	$\mathbf{1.3272}$	$\mathbf{0.0764}$	$\mathbf{0.2728}$	$0.2325$
w/o directional warm start	$87.2$ (-3.8%)	$1.6236$ (-22.3%)	$0.0962$ (-25.9%)	$0.3393$ (-24.4%)	$0.2423$ (-4.2%)
w/o noisy-state augmentation	$78.6$ (-13.2%)	$1.8896$ (-42.4%)	$0.1463$ (-91.5%)	$0.4318$ (-58.3%)	$\mathbf{0.2182}$ (+6.1%)
w/o kinematics-aware weighting	$82.1$ (-9.4%)	$1.6931$ (-27.6%)	$0.1200$ (-57.1%)	$0.4055$ (-48.6%)	$0.2394$ (-3.0%)

Table 6: Ablation study on the trajectory generator’s key design components. All variants are evaluated on the full 101-sequence test set using the same iFSQ tracker. Blue marks the best result per metric. Colored percentages show relative change from Heracles (full): teal = better, red = worse.

Figure 5: Qualitative sim-to-sim comparison on an OOD lie-to-stand sequence. Same setup as Fig. 3 . MLP, Transformer, SONIC, and iFSQ BM fail to stand up; VQ-VAE, iFSQ +H , and iFSQ partially track the motion. Heracles (Ours) completes the full lie-to-stand transition and most accurately tracks the root position.

Figure 6: Emergent semantic clustering in the learned iFSQ codebook. Each point represents a quantized motion token projected into 3D via PCA; colors denote motion categories. Despite receiving no category labels during training, the codebook self-organizes into semantically coherent clusters corresponding to distinct motor skills.

Figure 7: Omnidirectional fall recovery motion tracking. Real-world experiments demonstrate that our model generalizes to arbitrary lie-to-stand recovery motions, successfully handling varied initial fallen configurations and recovery directions without task-specific engineering.

Humanoid Robot Control

Humanoid robot control develops methods for coordinating bipedal balance, locomotion, and whole-body interaction with people and objects.

See all 202 papers

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control
Feb 25, 2026Weisheng Xu, Qiwei Wu, Jiaxi Zhang +7Humanoid Robot Control Robot Motion Generation
Hong Kong University of Science and Technology (Guangzhou) · University of Oxford
OmniTrack: General Motion Tracking via Physics-Consistent Reference
Feb 27, 2026Yuhan Li, Peiyuan Zhi, Yunshen Wang +6Humanoid Motion Tracking Humanoid Robot Control
Huazhong University of Science and Technology · State Key Lab of General AI, Beijing Institute for General Artificial Intelligence (BIGAI) · Shanghai Jiao Tong University
GenTrack: Physical Alignment for Robot-Native Motion Generation and Zero-Shot Humanoid Tracking
Aug 2, 2026Zeyu Ling, Xinyao Yu, Renye Yan +4Zero-Shot Learning Humanoid Robot Control
Zhejiang University · Peking University · Tencent +1

Feb 25, 2026J/K move · Enter open · S save

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control

Weisheng Xu, Qiwei Wu, Jiaxi Zhang, Tan Jing+6

Hong Kong University of Science and Technology (Guangzhou) · University of Oxford

Humanoid Robot Control Robot Motion Generation Synthetic Data Generation for Robotics Humanoid Motion Tracking

Open paper AskPDF arXiv

Physics-based humanoid control relies on training with motion datasets that have diverse data distributions. However, the fixed difficulty distribution of datasets limits the performance ceiling of the trained control policies. Additionally, the method of acquiring high-quality data through professional motion capture systems is constrained by costs, making it difficult to achieve large-scale scalability. To address these issues, we propose a closed-loop automated motion data generation and iterative framework. It can generate high-quality motion data with rich action semantics, including martial arts, dance, combat, sports, gymnastics, and more. Furthermore, our framework enables difficulty iteration of policies and data through physical metrics and objective evaluations, allowing the trained tracker to break through its original difficulty limits. On the PHC single-primitive tracker, using only approximately 1/10 of the AMASS dataset size, the average failure rate on the test set (2201 clips) is reduced by 45% compared to the baseline. Finally, we conduct comprehensive ablation and comparative experiments to highlight the rationality and advantages of our framework.

Abstract

Humanoid Robot Control

Explore similar work

Iterative Closed-Loop Motion Synthesis for Scaling the Capabilities of Humanoid Control

OmniTrack: General Motion Tracking via Physics-Consistent Reference

GenTrack: Physical Alignment for Robot-Native Motion Generation and Zero-Shot Humanoid Tracking