We present MaskAdapt, a framework for flexible motion adaptation in physics-based humanoid control. The framework follows a two-stage residual learning paradigm. In the first stage, we train a mask-invariant base policy using stochastic body-part masking and a regularization term that enforces consistent action distributions across masking conditions. This yields a robust motion prior that remains stable under missing observations, anticipating later adaptation in those regions. In the second stage, a residual policy is trained atop the frozen base controller to modify only the targeted body parts while preserving the original behaviors elsewhere. We demonstrate the versatility of this design through two applications: (i) motion composition, where varying masks enable multi-part adaptation within a single sequence, and (ii) text-driven partial goal tracking, where designated body parts follow kinematic targets provided by a pre-trained text-conditioned autoregressive motion generator. Through experiments, MaskAdapt demonstrates strong robustness and adaptability, producing diverse behaviors under masked observations and delivering superior targeted motion adaptation compared to prior work.
Figures & tables
Figure 2: Overview of the base policy training process. At each step, a mask mt specifies which body parts are hidden from observation. The full state st is masked to form sˉt , which is passed to the base policy π to generate the action at . The mask-invariant loss enforces consistency between masked and unmasked action distributions, promoting robustness to partial observations.
Figure 3: Overview of training the residual policy for motion composition. At each step, a mask mt specifies which body parts are subject to adaptation, and ψ learns to generate residual actions for those regions while maintaining stability elsewhere.
Figure 4: Overview of text-guided partial motion tracking during inference. The trajectory generator G continuously produces kinematic trajectories in response to dynamic text inputs, which serve as tracking targets for the residual policy.
Figure 5: Examples of motion adaptation achieved by the residual policy. (a) Jump + Alternating Kick : Two behaviors are combined via alternating masks on lower and upper body. Kicks first emerge in the legs, then swapping the mask restores jumping while shifting the adapted behavior to the upper body. (b) Locomotion + Rotate Arms : With walking as the base motion, both arms are initially masked to perform rotating-arm motion. Unmasking the left arm yields fine-grained control, where only the right arm continues the adapted behavior, producing smooth, localized transitions. (c) Aim + Sneak : When no region is masked, the controller performs only aiming. Masking the lower body induces sneaking in that region, creating a natural blend between base and adapted behaviors. (d) Text-driven partial motion tracking: The character begins with walking and responds to sequential text commands by producing semantically aligned arm motions.
Figure 6: Goal-driven tasks (a–c) and complex scenarios (d–e) enabled by dynamic motion composition.
Base Policy Model
Masking
Hnorm↑
AMP [ 20 ]
×
0.9124
− Ours w/o LMI
✓
0.7448
+ Ours w/ LMI ( λMI=0.1 )
✓
0.8968
+ Ours w/ LMI ( λMI=1.0 )
✓
0.9025
Table 1: Normalized entropy ( Hnorm ) comparison between the AMP baseline and ablated variants of our base policy trained with and without the MI loss.
Figure 7: Comparison of root trajectories on the XY plane when both legs (left) and both arms (right) are masked during locomotion, with and without the mask-invariant loss. All sequences start from the same initial state and random seed. The \medstar symbol denotes the starting point, and × marks the end of each episode.
Method
A/B
Hnorm↑
Case (a)
Case (b)
Case (c)
CML [ 32 ]
A
0.7835
0.9193
0.8461
B
0.8788
0.7478
0.8460
Ours
A
0.8580
0.9268
0.8980
B
0.8935
0.8710
0.9214
Table 2: Comparison of Hnorm across three motion composition cases: Case definitions: (a) Jump + Alternating Kick ( Lower Body ), (b) Locomotion + Rotate Arms ( Arms ), (c) Aim + Sneak ( Lower Body ). Each case is denoted as “ A+B (P) ,” with A as the base motion, B as the new motion, and P as the adapted body parts.
Method
Success Rate ↑ (%)
Location
Strike
Heading
AdaptNet [ 34 ]
94.3
1.0
82.1
CML [ 32 ]
100.0
46.5
75.2
Ours
99.5
64.2
76.9
Table 3: Comparison of performance on goal-driven tasks
Method
Tracking Error ↓ (m)
Pose Error ↓ (m)
From Scratch
0.076 ± 0.005
0.057 ± 0.008
CML [ 32 ]
0.069 ± 0.001
0.079 ± 0.008
Ours
0.055 ± 0.007
0.047 ± 0.001
Table 4: Comparison of tracking and imitation errors.
Method
Tracking Error ↓ (m)
Success Rate ↑ (%)
MaskedMimic [ 25 ] (Arms)
0.0849 ± 0.003
95.5
MaskedMimic [ 25 ] (Full)
0.0699 ± 0.007
100.0
Ours (Arms)
0.0423 ± 0.006
100.0
Table 5: Comparison of tracking error against MaskedMimic.
Figure 8: Learning curves of residual policies for tracking (left) and imitation (right) tasks, showing normalized returns averaged over three random seeds.
Parameters
Value
horizon length
32
number of environments
4096
minibatch size
4096
actor learning rate
1e-5
critic learning rate
1e-4
discriminator learning rate
1e-4
Table 1: Hyperparameters for learning base policy.
Parameters
Value
horizon length
32
number of environments
4096 / 2048
minibatch size
4096
actor learning rate
1e-5
critic learning rate
1e-4
discriminator learning rate
1e-4
Table 2: Hyperparameters for learning residual policy.
Motion
Dataset
Duration (s)
Stage
Task
Jump
LAFAN1
24.4
Base
C
Locomotion
LAFAN1
1000
Base
C
Aim
LAFAN1
305
Base
C
Walk
LAFAN1
5.6
Base
T
Walk (G1)
LAFAN1
33.3
Base
C
Alternating kick
AMASS
9.9
Residual
C
Table 3: Reference motions used in our experiments. Task type ”C” denotes motions used for motion composition, and ”T” indicates motions used for partial tracking.
Method
Masking
MI Weights ( λMI )
Avg. Hnorm↑
Random Init.
Same Init.
AMP [ 20 ]
×
–
0.9124
0.8890
− Ours w/o LMI
✓
–
0.7448
0.7019
+ Ours w/ LMI
✓
0.1
0.8968
0.8387
✓
0.5
0.8987
0.8441
✓
1.0 (default)
0.9025
0.8833
Table 4: Ablation study on the mask-invariant loss weights.
Trunk
Left Arm
Right Arm
Left Leg
Right Leg
Trunk
0.909
0.896
0.906
0.908
0.906
Left Arm
–
0.895
0.900
0.898
0.896
Right Arm
–
–
0.907
0.902
0.903
Left Leg
–
–
–
0.904
0.905
Right Leg
–
–
–
–
0.902
Table 5: Normalized entropy ( Hnorm ) across all masking conditions. Values are arranged in an upper-triangular matrix, with lower entries omitted (“–”) due to symmetry. Diagonal entries correspond to single body-part masking, while off-diagonal entries represent the average Hnorm when two body parts are masked simultaneously. Darker shades indicate higher entropy.
Trunk
Left Arm
Right Arm
Left Leg
Right Leg
Trunk
0.798
0.830
0.790
0.749
0.629
Left Arm
–
0.814
0.736
0.796
0.702
Right Arm
–
–
0.845
0.681
0.664
Left Leg
–
–
–
0.754
0.602
Right Leg
–
–
–
–
0.783
Table 6: Normalized entropy ( Hnorm ) across all masking conditions when the MI loss is not applied. The Hnorm values decrease substantially across all combinations, with particularly severe degradation when two body parts are masked simultaneously.
Figure 1: Frame visitation frequencies for locomotion with both legs masked. The policy without MI loss (blue) shows a skewed visitation pattern, whereas applying the MI loss (yellow) produces a markedly more uniform distribution. Our mask-invariant base achieves a level of uniformity comparable to AMP (green), indicating that the MI loss preserves diverse state visitation even under heavy masking.
Figure 2: Qualitative comparison of composition methods, each motion generated from the same initial state. In (a), the kick motion in CML is delayed; in (b), CML exhibits severe artifacts and fails to complete full arm rotations; and in (c), CML produces limited motion combinations compared to ours.
Figure 3: Qualitative comparison of our method, CML, and a from-scratch model for the partial tracking task given the text commands on the left. The red region highlights deviations from the base behavior.
We present ADAPT, an end-to-end framework for interactive, text-conditioned humanoid whole-body control. Unlike dominant text-to-motion pipelines that generate kinematic motions for a separate tracker, ADAPT solves language control with an end-to-end closed-loop control framework, where the robot must continuously respond to changing commands while maintaining balance, natural motion, and smooth transitions. ADAPT learns a diffusion-based action prior from text-labeled humanoid state-action trajectories, enabling diverse motion skills to be directly executed from language commands. To improve long-horizon robustness and smooth prompt switching, we train a lightweight residual reinforcement learning policy on top of the frozen diffusion controller. We further show that the same diffusion policy can be reused as a steerable text-conditioned motion prior for downstream task adaptation. Experiments demonstrate robust language-grounded skill execution, smooth interactive transitions, and style-preserving downstream control.
Humanoid robots are promising to acquire various skills by imitating human behaviors. However, existing algorithms are only capable of tracking smooth, low-speed human motions, even with delicate reward and curriculum design. This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing through multi-steps motion processing and adaptive motion tracking. For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints to the maximum extent. For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance based on the current tracking error, creating an adaptive curriculum mechanism. We further construct an asymmetric actor-critic framework for policy training. In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions. Our method achieves significantly lower tracking errors than existing approaches and is successfully deployed on the Unitree G1 robot, demonstrating stable and expressive behaviors. The project page is https://kungfubot.github.io.
Weiji Xie, Jinrui Han, Jiakun Zheng +6
Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · East China University of Science and Technology +2
Achieving precise, versatile whole-body character control in physics-based animation remains challenging. Recent diffusion-based policies generate rich and expressive motions but typically rely on gradient-based test-time guidance to satisfy task objectives, which is slow and can reduce robustness. We introduce NaP-Control (Navigating Diffusion Prior for Versatile and Fast Character Control), abbreviated as NaP. Our method uses reinforcement learning to manipulate the latent noise of a task-agnostic diffusion policy prior, steering it toward task-specific behaviors for fast, robust control with high motion fidelity. In contrast to methods that rely solely on offline training, NaP interacts with the environment during training to correct motions and optimize task rewards, improving success rates and enabling adaptation to challenging scenarios. By directly predicting task-optimized diffusion noise, NaP eliminates iterative guidance during denoising and enables efficient inference. Experiments show that NaP attains higher success rates and faster inference while preserving natural motion across diverse tasks.