BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning

Yitang Li; Zhengyi Luo; Tonghe Zhang; Cunxi Dai; Anssi Kanervisto; Andrea Tirinzoni; Haoyang Weng; Kris Kitani

cs.RONov 6, 2025

BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning

Authors: Yitang Li, Zhengyi Luo, Tonghe Zhang, Cunxi Dai, Anssi Kanervisto, Andrea Tirinzoni, Haoyang Weng, Kris Kitani,

Organizations: 1Carnegie Mellon University · 2Meta

Abstract

Building Behavioral Foundation Models (BFMs) for humanoid robots has the potential to unify diverse control tasks under a single, promptable generalist policy. However, existing approaches are either exclusively deployed on simulated humanoid characters, or specialized to specific tasks such as tracking. We propose BFM-Zero, a framework that learns an effective shared latent representation that embeds motions, goals, and rewards into a common space, enabling a single policy to be prompted for multiple downstream tasks without retraining. This well-structured latent space in BFM-Zero enables versatile and robust whole-body skills on a Unitree G1 humanoid in the real world, via diverse inference methods, including zero-shot motion tracking, goal reaching, and reward optimization, and few-shot optimization-based adaptation. Unlike prior on-policy reinforcement learning (RL) frameworks, BFM-Zero builds upon recent advancements in unsupervised RL and Forward-Backward (FB) models, which offer an objective-centric, explainable, and smooth latent representation of whole-body motions. We further extend BFM-Zero with critical reward shaping, domain randomization, and history-dependent asymmetric learning to bridge the sim-to-real gap. Those key design choices are quantitatively ablated in simulation. A first-of-its-kind model, BFM-Zero establishes a step toward scalable, promptable behavioral foundation models for whole-body humanoid control.

Figures & tables

Figure 1: BFM-Zero enables versatile and robust whole-body skills. (A-C) Diverse zero-shot inference methods. (D) Natural recovery from large perturbation. (E) Few-shot adaptation.

Figure 2: An overview of the BFM-Zero framework. After the pre-training stage, BFM-Zero forms a latent space that can be used for zero-shot reward optimization, single-frame goal reaching, and tracking. It can also be adapted in a few-shot fashion to reach more challenging poses.

Figure 3: Tracking, reward, and goal-reaching performance across models for different testing configurations (left), and example distributions of reward evaluation scores for BFM-Zero in Isaac (DR) (right). Each metric is averaged over tasks. We consider the average return over episodes lasting 500 steps for reward, the average joint position error

E_{\mathrm{mpjpe}}

averaged over the whole motion for tracking, and the error

E_{\mathrm{mpjpe}}

averaged over the episode for goal-reaching.

Figure 4: Real-World Validation of Tracking . Left: Highly dynamic dancing. Middle: Frequently turning during walking. Right: Naturally recover to continue track the motion.

Figure 5: Real-World Validation of Goal Reaching . (a) Continuously goal-reaching: the blue/yellow pose denotes the goal pose, while black marks the real robot pose, and gray visualizes the transition between each pose. (b) Transition from any pose to T-pose.

Figure 6: Real-World Reward Optimization . The red arrow represents the base velocity tracking target. (a) sitting ; (b) crouch-0.25 ; (c) move-low0.6-ego-0-0.7 ; (d) Diverse behaviors from one reward raisearm-m-l ; (e) combing raisearm-m-l with move-ego-180-0.3 and move-ego--90-0.7.

Figure 7: Disturbance Rejection: (a) Keeps steady when kicked in the leg. (b) Absorbs a hard push with one smooth rear step. (c) Naturally stands up and returns to T-pose after being yanked down.

Figure 8: Few-Shot Adaptation: (a) Single-pose adaptation improving single-leg standing under an additional payload. (b) Trajectory adaptation reduces tracking error.

Panel of Figure 9 — Figure 9: Latent space visualization and analysis.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
Environment and Training Setup
History Length $H$	4
Episode Length $T$	500
$N_{\mathrm{env}}$	1024
$N_{\mathrm{batch}}$	1024
$N_{\mathrm{ups}}$	16

Appendix

Table 1: Train parameters

Domain Randomization Parameter Range COM Offset [m]

\mathcal{U}([-0.02,0.02])

Link Mass

\mathcal{U}([0.95,1.05])

Friction

\mathcal{U}([-0.5,1.25])

Default Joint Pos [m]

\mathcal{U}([-0.02,0.02])

Push Robots [m/s]

\mathcal{U}([0,0.5])

Additive Observation Noise Observation Range

q_{t}-\bar{q}

\mathcal{U}([-0.01,0.01])

\dot{q}_{t}

\mathcal{U}([-0.5,0.5])

\mathrm{grav}_{t}

\mathcal{U}([-0.05,0.05])

\dot{\omega}^{\mathrm{root}}_{t}/4

\mathcal{U}([-0.05,0.05])

Regularization Rewards Name Weight DoF Limit

-10

Action Rate

-0.1

Self Contact

-1

Feet Orientation

-0.4

Ankle Roll

-4

Feet Slip

-2

Appendix

Table 4: Additional training configurations.

Panel of Figure 10 — Figure 10: Goal poses selected from frames of the LAFAN1 dataset ( Harvey et al., 2020 ) (top) and AMASS dataset ( Mahmood et al., 2019 ) (bottom).

Panel of Figure 11 — Figure 11: Applications of BFM-Zero on Booster T1.

Figure 12: Tracking and reward performance on the test set for different models and datasets. The lower the better for tracking and the higher the better for reward. Reward inference is done using

600,000

samples from the LAFAN1 expert buffer.

Dataset	Frames @30FPS	Duration
LAFAN1	258600	2.4h
AMASS	698437	6.5h
12.5%	119620	1.1h
25%	234982	2.2h
50%	475793	4.4h
75%	721720	6.7h

Appendix

Table 5: Number of frames per dataset

		Number of Parameters
Architecture	Model	$\pi$	$Q_{R}$	$B$	$Q_{D}$	D	F	Total
ResNet	3-block, 2048dim	19.3M	59.2M	201k	59.2M	2.9M	60.3M	201.1M
ResNet ⋆	6-block, 2048dim	31.9M	134.8M	201k	134.8M	2.9M	135.9M	440.5M
ResNet	9-block, 2048dim	44.5M	210.4M	201k	210.4M	2.9M	211.5M	679.9M
ResNet	3-block, 1024dim	5.5M	17.0M	201k	17.0M	2.9M	17.6M	60.2M
ResNet	6-block, 1024dim	8.6M	36.0M	201k	36.0M	2.9M	36.5M	120.1M

Appendix

Table 6: Configurations of the architectures and total number of parameters.

\star

denotes the configuration used in the main paper.

Figure 13: Reward inference performance when using the experience generated by the agent (i.e., online replay buffer) or the motion dataset used for training. We get better reward performance when using the motion dataset, in particular when using LAFAN1 (see Fig. 12 ).

Figure 14: Return distributions for different tasks when using LAFAN1 motions or the agent self-generated samples (replay buffer) for reward inference. Reward inference was repeated 10 times leading to evaluating 10 different policies for each tasks. Each policy was evaluated over 20 episodes.

Figure 15: Examples of training losses as a function of the number of environment steps when training with and without auxiliary rewards (top). Bottom table shows the evaluation score of the models on tracking, goal reaching and reward inference. The model is the ResNet model with 6 blocks and 2048 hidden dimension.

Figure 16: Auxiliary reward metrics as a function of the number of steps in training. We can clearly see that the model trained with auxiliary rewards optimizes the provided auxiliary tasks.

Figure 17: Evaluation curves for tracking during training as a function of the steps in the environment for different latent space dimension (i.e.,

d_{z}

). These metrics are computed on the motion dataset used for training (i.e., LAFAN1). Results are averaged over 2 seeds.

Figure 18: Reward evaluation score split by individual tasks for models with different latent space dimension

d_{z}

, along with the score increase after CEM adaptation on selected tasks.

Z dimension	Tracking ( $\downarrow$ )	Goal ( $\downarrow$ )	Reward (LAFAN1) ( $\uparrow$ )
32	1.3963	1.6870	213.8224
64	1.2455	1.3141	240.4211
128	1.0512	1.0830	263.9501
256	0.9806	0.9673	273.0826
384	0.9266	0.9704	290.9922
512	0.9232	0.9758	307.3982

Appendix

Table 7: Test metrics for the different inference modalities for models trained with different z dimensions. The base configuration is has the one in the main paper (see Table 3 and Table 1 ). Results are averaged over 2 seeds.

Motions	Start Frame	End Frame	$E^{\text{real}}_{\text{mpjpe}}$ ( $\downarrow$ )	$E^{\texttt{sim-nodr}}_{\text{mpjpe}}$ ( $\downarrow$ )	$E^{\texttt{sim-dr}}_{\text{mpjpe}}$ ( $\downarrow$ )
Dance-1	1	2400	1.1945	0.7631	0.7736
Dance-2	1	2000	1.0003	0.9047	0.9235
Walk slowly	1	320	0.8662	0.5746	0.6561
Walk fast	4061	4285	1.2920	0.7421	0.7931
Walk and turn	2136	2870	1.1528	0.7481	0.7560
Marching	10931	11710	1.3989	0.9258	0.9220

Appendix

Table 8: Comparison of real-world and simulation performance

Methods	Tracking Error ( $E_{\text{mpjpe}}$ ) ( $\downarrow$ )		Number of Samples ( $\downarrow$ )
Methods	LAFAN1	AMASS	Number of Samples ( $\downarrow$ )
GMT	2.2425	1.9064	6800M
BFM-Zero	1.0789	1.0342	200M

Appendix

Table 9: Comparison of general tracking methods and BFM-Zero

Figure 19: Distribution of EMD distances between two rollouts from two randomly sampled latents.

Figure 20: Coverage of the dataset by randomly sampling behavior latents, rolling them out and matching the trajectories with the closest motion (left), and the distribution of distances to the closest motions (right).

Figure 21: UMAP dimensionality reduction of LAFAN1 motions embedded into BFM-Zero’s latent space.

Humanoid Robot Control

Humanoid robot control develops methods for coordinating bipedal balance, locomotion, and whole-body interaction with people and objects.

See all 192 papers

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning
Oct 5, 2026Ziqi Han, Yitang Li, Junhan Sun +8Representation Learning Reinforcement Learning
Tongji University · Tsinghua University · Zhejiang University +3
P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning
Oct 11, 2026Xue Wang, Yitang Li, Yiming Zhang +7Unsupervised RL Behavioral Foundation Models
University of Shanghai for Science and Technology · RoboPartyLab · Tsinghua University +2
Scaling Behavior Foundation Model for Humanoid Robots
Jul 16, 2026Weishuai Zeng, Kangning Yin, Xiaojie Niu +15Robot Foundation Models Humanoid Robot Control
The Chinese University of Hong Kong · Galbot · Shanghai Artificial Intelligence Laboratory +4

Oct 5, 2026J/K move · Enter open · S save

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

Ziqi Han, Yitang Li, Junhan Sun, Fanrong Dong+7

Tongji University · Tsinghua University · Zhejiang University+3

Representation Learning Reinforcement Learning Humanoid Robot Control Unsupervised RL Robust RL Human-Object Interaction

Open paper AskPDF arXiv

Behavioral foundation models (BFMs) have recently shown that a single humanoid policy can support diverse whole-body control, but extending such generality to physical interaction remains challenging. We introduce I-BFM, to our knowledge the first BFM for humanoid-object interaction. Rather than relying on task-specific policies or reference tracking, I-BFM learns a shared representation of the coupled dynamics among the humanoid, objects, and their contacts using forward-backward representations and unsupervised reinforcement learning. Given a downstream task reward, the same policy can be directly conditioned on a latent command to execute closed-loop interaction without task-specific policy optimization. To improve interaction control over different time scales, we further train the policy with both short-horizon interaction targets and longer-horizon goal targets. A single I-BFM policy performs carrying, pushing, and kicking, while also supporting goal reaching, motion tracking, stylistic control, and long-horizon task chaining. More importantly, it remains effective after large deviations from nominal execution: on Carry, I-BFM achieves 94.3% nominal success and retains 89.3% success after robot falls, compared with 1.3% for a planning-based baseline. Real-world experiments on a Unitree G1 further demonstrate diverse loco-manipulation behaviors, rapid recovery from interaction failures and external disturbances, and task chaining without task-specific retraining.

Abstract

Humanoid Robot Control

Explore similar work

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning

Scaling Behavior Foundation Model for Humanoid Robots