MaskAdapt: Learning Flexible Motion Adaptation via Mask-Invariant Prior for Physics-Based Characters

Soomin Park; Eunseong Lee; Kwang Bin Lee; Sung-Hee Lee

cs.CVMar 31, 2026

MaskAdapt: Learning Flexible Motion Adaptation via Mask-Invariant Prior for Physics-Based Characters

Authors: Soomin Park, Eunseong Lee, Kwang Bin Lee, Sung-Hee Lee

Organizations: KAIST

Abstract

We present MaskAdapt, a framework for flexible motion adaptation in physics-based humanoid control. The framework follows a two-stage residual learning paradigm. In the first stage, we train a mask-invariant base policy using stochastic body-part masking and a regularization term that enforces consistent action distributions across masking conditions. This yields a robust motion prior that remains stable under missing observations, anticipating later adaptation in those regions. In the second stage, a residual policy is trained atop the frozen base controller to modify only the targeted body parts while preserving the original behaviors elsewhere. We demonstrate the versatility of this design through two applications: (i) motion composition, where varying masks enable multi-part adaptation within a single sequence, and (ii) text-driven partial goal tracking, where designated body parts follow kinematic targets provided by a pre-trained text-conditioned autoregressive motion generator. Through experiments, MaskAdapt demonstrates strong robustness and adaptability, producing diverse behaviors under masked observations and delivering superior targeted motion adaptation compared to prior work.

Figures & tables

Figure 2: Overview of the base policy training process. At each step, a mask $m_{t}$ specifies which body parts are hidden from observation. The full state $s_{t}$ is masked to form $\bar{s}_{t}$ , which is passed to the base policy $\pi$ to generate the action $a_{t}$ . The mask-invariant loss enforces consistency between masked and unmasked action distributions, promoting robustness to partial observations.

Figure 3: Overview of training the residual policy for motion composition. At each step, a mask $m_{t}$ specifies which body parts are subject to adaptation, and $\psi$ learns to generate residual actions for those regions while maintaining stability elsewhere.

Figure 4: Overview of text-guided partial motion tracking during inference. The trajectory generator $G$ continuously produces kinematic trajectories in response to dynamic text inputs, which serve as tracking targets for the residual policy.

Figure 5: Examples of motion adaptation achieved by the residual policy. (a) Jump + Alternating Kick : Two behaviors are combined via alternating masks on lower and upper body. Kicks first emerge in the legs, then swapping the mask restores jumping while shifting the adapted behavior to the upper body. (b) Locomotion + Rotate Arms : With walking as the base motion, both arms are initially masked to perform rotating-arm motion. Unmasking the left arm yields fine-grained control, where only the right arm continues the adapted behavior, producing smooth, localized transitions. (c) Aim + Sneak : When no region is masked, the controller performs only aiming. Masking the lower body induces sneaking in that region, creating a natural blend between base and adapted behaviors. (d) Text-driven partial motion tracking: The character begins with walking and responds to sequential text commands by producing semantically aligned arm motions.

Figure 6: Goal-driven tasks (a–c) and complex scenarios (d–e) enabled by dynamic motion composition.

Base Policy Model	Masking	$H_{\text{norm}}\uparrow$
AMP [ 20 ]	$\times$	0.9124
$-$ Ours w/o $\mathcal{L}_{\text{MI}}$	$\checkmark$	0.7448
$+$ Ours w/ $\mathcal{L}_{\text{MI}}$ ( $\lambda_{\text{MI}}=0.1$ )	$\checkmark$	0.8968
$+$ Ours w/ $\mathcal{L}_{\text{MI}}$ ( $\lambda_{\text{MI}}=1.0$ )	$\checkmark$	0.9025

Table 1: Normalized entropy (

H_{\text{norm}}

) comparison between the AMP baseline and ablated variants of our base policy trained with and without the MI loss.

Figure 7: Comparison of root trajectories on the XY plane when both legs (left) and both arms (right) are masked during locomotion, with and without the mask-invariant loss. All sequences start from the same initial state and random seed. The $\medstar$ symbol denotes the starting point, and $\times$ marks the end of each episode.

Method	A/B	$H_{\text{norm}}\uparrow$
		Case (a)	Case (b)	Case (c)
CML [ 32 ]	A	0.7835	0.9193	0.8461
	B	0.8788	0.7478	0.8460
Ours	A	0.8580	0.9268	0.8980
	B	0.8935	0.8710	0.9214

Table 2: Comparison of

H_{\text{norm}}

across three motion composition cases: Case definitions: (a) Jump + Alternating Kick ( Lower Body ), (b) Locomotion + Rotate Arms ( Arms ), (c) Aim + Sneak ( Lower Body ). Each case is denoted as “ A+B (P) ,” with A as the base motion, B as the new motion, and P as the adapted body parts.

Method	Success Rate $\uparrow$ (%)
	Location	Strike	Heading
AdaptNet [ 34 ]	94.3	1.0	82.1
CML [ 32 ]	100.0	46.5	75.2
Ours	99.5	64.2	76.9

Table 3: Comparison of performance on goal-driven tasks

Method	Tracking Error $\downarrow$ (m)	Pose Error $\downarrow$ (m)
From Scratch	0.076 $\pm$ 0.005	0.057 $\pm$ 0.008
CML [ 32 ]	0.069 $\pm$ 0.001	0.079 $\pm$ 0.008
Ours	0.055 $\pm$ 0.007	0.047 $\pm$ 0.001

Table 4: Comparison of tracking and imitation errors.

Method	Tracking Error $\downarrow$ (m)	Success Rate $\uparrow$ (%)
MaskedMimic [ 25 ] (Arms)	0.0849 $\pm$ 0.003	95.5
MaskedMimic [ 25 ] (Full)	0.0699 $\pm$ 0.007	100.0
Ours (Arms)	0.0423 $\pm$ 0.006	100.0

Table 5: Comparison of tracking error against MaskedMimic.

Figure 8: Learning curves of residual policies for tracking (left) and imitation (right) tasks, showing normalized returns averaged over three random seeds.

Parameters	Value
horizon length	32
number of environments	4096
minibatch size	4096
actor learning rate	1e-5
critic learning rate	1e-4
discriminator learning rate	1e-4

Table 1: Hyperparameters for learning base policy.

Parameters	Value
horizon length	32
number of environments	4096 / 2048
minibatch size	4096
actor learning rate	1e-5
critic learning rate	1e-4
discriminator learning rate	1e-4

Table 2: Hyperparameters for learning residual policy.

Motion	Dataset	Duration (s)	Stage	Task
Jump	LAFAN1	24.4	Base	C
Locomotion	LAFAN1	1000	Base	C
Aim	LAFAN1	305	Base	C
Walk	LAFAN1	5.6	Base	T
Walk (G1)	LAFAN1	33.3	Base	C
Alternating kick	AMASS	9.9	Residual	C

Table 3: Reference motions used in our experiments. Task type ”C” denotes motions used for motion composition, and ”T” indicates motions used for partial tracking.

Method	Masking	MI Weights ( ${\lambda_{\text{MI}}}$ )	Avg. $H_{\text{norm}}\uparrow$
			Random Init.	Same Init.
AMP [ 20 ]	$\times$	–	0.9124	0.8890
$-$ Ours w/o $\mathcal{L}_{\text{MI}}$	$\checkmark$	–	0.7448	0.7019
$+$ Ours w/ $\mathcal{L}_{\text{MI}}$	$\checkmark$	0.1	0.8968	0.8387
	$\checkmark$	0.5	0.8987	0.8441
	$\checkmark$	1.0 (default)	0.9025	0.8833

Table 4: Ablation study on the mask-invariant loss weights.

	Trunk	Left Arm	Right Arm	Left Leg	Right Leg
Trunk	0.909	0.896	0.906	0.908	0.906
Left Arm	–	0.895	0.900	0.898	0.896
Right Arm	–	–	0.907	0.902	0.903
Left Leg	–	–	–	0.904	0.905
Right Leg	–	–	–	–	0.902

Table 5: Normalized entropy (

H_{\text{norm}}

) across all masking conditions. Values are arranged in an upper-triangular matrix, with lower entries omitted (“–”) due to symmetry. Diagonal entries correspond to single body-part masking, while off-diagonal entries represent the average

H_{\text{norm}}

when two body parts are masked simultaneously. Darker shades indicate higher entropy.

	Trunk	Left Arm	Right Arm	Left Leg	Right Leg
Trunk	0.798	0.830	0.790	0.749	0.629
Left Arm	–	0.814	0.736	0.796	0.702
Right Arm	–	–	0.845	0.681	0.664
Left Leg	–	–	–	0.754	0.602
Right Leg	–	–	–	–	0.783

Table 6: Normalized entropy (

H_{\text{norm}}

) across all masking conditions when the MI loss is not applied. The

H_{\text{norm}}

values decrease substantially across all combinations, with particularly severe degradation when two body parts are masked simultaneously.

Figure 1: Frame visitation frequencies for locomotion with both legs masked. The policy without MI loss (blue) shows a skewed visitation pattern, whereas applying the MI loss (yellow) produces a markedly more uniform distribution. Our mask-invariant base achieves a level of uniformity comparable to AMP (green), indicating that the MI loss preserves diverse state visitation even under heavy masking.

Figure 2: Qualitative comparison of composition methods, each motion generated from the same initial state. In (a), the kick motion in CML is delayed; in (b), CML exhibits severe artifacts and fails to complete full arm rotations; and in (c), CML produces limited motion combinations compared to ours.

Figure 3: Qualitative comparison of our method, CML, and a from-scratch model for the partial tracking task given the text commands on the left. The red region highlights deviations from the base behavior.

Human Motion Generation

Human motion generation creates articulated movement from language, audio, video, spatial constraints, or interaction goals.

See all 291 papers

ADAPT: Agile Diffusion Action Priors for Robust and Steerable Online Text-Driven Humanoid Control
Sep 1, 2026Yan Wu, Chenhao Li, Kaifeng Zhao +3Humanoid Robot Control Diffusion Policy
ETH Zurich, Switzerland
KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills
Jun 15, 2025Weiji Xie, Jinrui Han, Jiakun Zheng +6Humanoid Robot Control Robot Imitation Learning
Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Jiao Tong University · East China University of Science and Technology +2
NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control
Apr 15, 2026Chia-Wen Chen, Yan Wu, Korrawe Karunratanakul +1Diffusion Policy Diffusion Models for Motion Generation
ETH Zurich

Sep 1, 2026J/K move · Enter open · S save

ADAPT: Agile Diffusion Action Priors for Robust and Steerable Online Text-Driven Humanoid Control

Yan Wu, Chenhao Li, Kaifeng Zhao, Gen Li+2

ETH Zurich, Switzerland

Humanoid Robot Control Diffusion Policy Language-Conditioned Robot Control Robot Learning

Open paper AskPDF arXiv

We present ADAPT, an end-to-end framework for interactive, text-conditioned humanoid whole-body control. Unlike dominant text-to-motion pipelines that generate kinematic motions for a separate tracker, ADAPT solves language control with an end-to-end closed-loop control framework, where the robot must continuously respond to changing commands while maintaining balance, natural motion, and smooth transitions. ADAPT learns a diffusion-based action prior from text-labeled humanoid state-action trajectories, enabling diverse motion skills to be directly executed from language commands. To improve long-horizon robustness and smooth prompt switching, we train a lightweight residual reinforcement learning policy on top of the frozen diffusion controller. We further show that the same diffusion policy can be reused as a steerable text-conditioned motion prior for downstream task adaptation. Experiments demonstrate robust language-grounded skill execution, smooth interactive transitions, and style-preserving downstream control.

Abstract

Human Motion Generation

Explore similar work

ADAPT: Agile Diffusion Action Priors for Robust and Steerable Online Text-Driven Humanoid Control

KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

NaP-Control: Navigating Diffusion Prior for Versatile and Fast Character Control