I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

Ziqi Han; Yitang Li; Junhan Sun; Fanrong Dong; Yaojie Shen; Lei Ye; Zetong Jing; Yongqi Zhang

cs.ROOct 5, 2026

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning

Authors: Ziqi Han, Yitang Li, Junhan Sun, Fanrong Dong, Yaojie Shen, Lei Ye, Zetong Jing, Yongqi Zhang, +3 more

Organizations: Tongji University · Tsinghua University · Zhejiang University · RoboParty Lab · Harbin Institute of Technology · ShanghaiTech University

Abstract

Behavioral foundation models (BFMs) have recently shown that a single humanoid policy can support diverse whole-body control, but extending such generality to physical interaction remains challenging. We introduce I-BFM, to our knowledge the first BFM for humanoid-object interaction. Rather than relying on task-specific policies or reference tracking, I-BFM learns a shared representation of the coupled dynamics among the humanoid, objects, and their contacts using forward-backward representations and unsupervised reinforcement learning. Given a downstream task reward, the same policy can be directly conditioned on a latent command to execute closed-loop interaction without task-specific policy optimization. To improve interaction control over different time scales, we further train the policy with both short-horizon interaction targets and longer-horizon goal targets. A single I-BFM policy performs carrying, pushing, and kicking, while also supporting goal reaching, motion tracking, stylistic control, and long-horizon task chaining. More importantly, it remains effective after large deviations from nominal execution: on Carry, I-BFM achieves 94.3% nominal success and retains 89.3% success after robot falls, compared with 1.3% for a planning-based baseline. Real-world experiments on a Unitree G1 further demonstrate diverse loco-manipulation behaviors, rapid recovery from interaction failures and external disturbances, and task chaining without task-specific retraining.

Figures & tables

Figure 2 : Overview of I-BFM . During pre-training, I-BFM learns an interaction-aware behavioral embedding and a shared policy in an interactive environment, jointly optimized with forward-backward and discriminator critics. At inference, task rewards retrieve a corresponding latent from the learned embedding space, enabling the same policy to perform robust carrying, pushing, kicking, and compositional loco-manipulation behaviors.

Figure 3 : LOGO metric conditioning. The current latent is

z_{t}

;

z_{\mathrm{goal}}

defines the long-horizon geodesic and

z_{\mathrm{local}}

the immediate successor target. Their log-map vectors are evaluated in the same tangent plane and passed to the shared actor.

Method	Carry Box				Push Box				Kick Box
Method	SR $\uparrow$	$E_{\mathrm{obj}}$ $\downarrow$	SR-O $\uparrow$	SR-R $\uparrow$	SR $\uparrow$	$E_{\mathrm{obj}}$ $\downarrow$	SR-O $\uparrow$	SR-R $\uparrow$	SR $\uparrow$	$E_{\mathrm{obj}}$ $\downarrow$	SR-O $\uparrow$	SR-R $\uparrow$
PhysHSI [ 18 ]	$37.7$ $\pm 2.3$ %	$0.63$ $\pm 0.03$	$1.0$ $\pm 0.0$ %	$0.0$ $\pm 0.0$ %	–	–	–	–	–	–	–	–
LessMimic [ 19 ]	$26.0$ $\pm 1.7$ %	$0.79$ $\pm 0.01$	$7.0$ $\pm 2.6$ %	$2.7$ $\pm 2.1$ %	$1.7$ $\pm 1.5$ %	$0.95$ $\pm 0.01$	$1.0$ $\pm 1.0$ %	$0.7$ $\pm 0.6$ %	–	–	–	–
OmniContact [ 7 ]	$92.7$ $\pm 0.6$ %	$0.19$ $\pm 0.01$	$60.0$ $\pm 7.0$ %	$1.3$ $\pm 2.3$ %	$42.0$ $\pm 2.6$ %	$0.24$ $\pm 0.01$	$40.0$ $\pm 10.5$ %	$2.7$ $\pm 1.5$ %	$1.3$ $\pm 0.6$ %	$0.96$ $\pm 0.02$	$0.0$ $\pm 0.0$ %	$0.0$ $\pm 0.0$ %
I-BFM (Ours)	$94.3$ $\pm 3.1$ %	$0.15$ $\pm 0.02$	$91.0$ $\pm 1.0$ %	$89.3$ $\pm 1.5$ %	$75.0$ $\pm 6.6$ %	$0.31$ $\pm 0.05$	$46.0$ $\pm 4.4$ %	$74.3$ $\pm 5.8$ %	$70.0$ $\pm 3.6$ %	$0.28$ $\pm 0.03$	$57.3$ $\pm 7.6$ %	$60.0$ $\pm 8.7$ %

Table I : Quantitative comparison across three loco-manipulation tasks. SR is the success rate,

E_{\mathrm{obj}}

is the final object-to-goal error in meters (clipped at

1

before aggregation), and SR-O and SR-R denote success rates under object-fall and robot-fall perturbations, respectively. A dash (

-

) denotes that the corresponding method does not support the task.

Figure 4 : Qualitative Results. I-BFM reliably completes box transport, while baselines are less consistent.

Figure 5 : Recovery from Perturbations. During object perturbation, OmniContact fails to follow the replanned trajectory (translucent blue), while I-BFM autonomously recovers and completes the task even under robot perturbation.

Method	SR $\uparrow$	$\boldsymbol{E_{\mathrm{obj}}}\downarrow$	$\boldsymbol{E_{\mathrm{jpos}}}\downarrow$	$\boldsymbol{E_{\mathrm{vel}}}\downarrow$
I-BFM w/o LOGO	$51.0\%$	$0.33$	$1.06$	$129.7$
I-BFM	$94.3\%$	$0.15$	$1.02$	$126.9$

Table II: Ablation study of I-BFM .

Figure 6 : Goal Reaching from diverse initial states. From diverse robot poses and box configurations (green), a single I-BFM policy recovers and transports the box to the shared goal pose (red). Translucent blue figures show the goal pose.

Figure 7 : Motion tracking with a unified policy. I-BFM tracks generated whole-body motions for both box carrying (top) and dancing (bottom). Translucent blue figures denote the reference motion.

Figure 8 : Timely Adaptation to Box Displacement. When the box is moved during execution, I-BFM promptly updates its motion, re-approaches the new box pose, and resumes transportation in closed loop.

Figure 9 : Compositional box manipulation. A single I-BFM policy sequentially approaches, lifts, carries, raises, and places the box in response to changing task commands.

Figure 10 : Task Chaining. The robot sequentially pushes, carries, and places the box.

Explore similar work

CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments
Sep 29, 2026Tan-Dzung Do, Tuan Dat Phuong, Nico Bohlinger +6Cross-Embodiment Transfer Full-Body Humanoid Control
VinRobotics · National University of Singapore · TU Darmstadt +3
FAME: Force-Adaptive RL for Expanding the Manipulation Envelope of a Full-Scale Humanoid
Mar 9, 2026Niraj Pudasaini, Yutong Zhang, Jensen Lavering +3Dexterous Manipulation Wrist
Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309.
Scaling Behavior Foundation Model for Humanoid Robots
Jul 16, 2026Weishuai Zeng, Kangning Yin, Xiaojie Niu +15Full-Body Humanoid Control Behavioral Foundation Models
The Chinese University of Hong Kong · Galbot · Shanghai Artificial Intelligence Laboratory +4

Sep 29, 2026, cs.ROJ/K move · Enter open · S save

CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments

Tan-Dzung Do, Tuan Dat Phuong, Nico Bohlinger, Cuc T. Trinh+5

VinRobotics · National University of Singapore · TU Darmstadt+3

Cross-Embodiment Transfer Full-Body Humanoid Control Embodiment Goal-Conditioned Value Function Video Latents Humanoid

Open paper AskPDF arXiv

Behavior Foundation Models (BFMs) give humanoids a promptable policy over a latent behavior space, enabling one single vector to represent a motion to imitate, a pose to reach, or a reward to maximize. Forward-Backward representations successfully produce such spaces, but at the cost of hundreds of GPU-hours for a single robot. Moreover, when the training process is repeated for a second robot, it produces a second space unrelated to the first, resulting in embodiment-specific latents that do not unify or transfer. We address these problems with CrossBFM, treating the latent space as the transferable asset for various embodiments. As retargeting provides frame-level cross-embodiment correspondence, we propose a unified encoder architecture with no robot-specific parameters for distilling the behavior space to address all training embodiments simultaneously in less than a GPU-hour. Following this encoder, latent-conditioned trackers turn the distilled latent into whole-body control in a conventional PPO training manner in just 10 more GPU-hours. On three distilled humanoids, all three prompting modes transfer: motion tracking with latent-conditioned policy losing only

0.025

rad to its joint-conditioned counterpart, smooth goal reaching between poses with no falls, and reward optimization for all

41

reward prompts. Our experiments further reveal that 1) regressing the encoder on a quarter of the motion corpus costs only

5\%

of tracking performance and 2) training the encoder on a subset of robots and evaluating on an unseen one recovers up to

89\%

of the tracking performance of seen robots, demonstrating cross-embodiment generalization to morphologically similar robots. We also verify the pipeline on real robots across all three prompting modes and with flow-based generated latents. Project website: https://dotandung.github.io/crossbfm/

Abstract

Explore similar work

CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments

FAME: Force-Adaptive RL for Expanding the Manipulation Envelope of a Full-Scale Humanoid

Scaling Behavior Foundation Model for Humanoid Robots