P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning

Xue Wang; Yitang Li; Yiming Zhang; Siqin Wang; Junhan Sun; Zhihao Liu; Yaojie Shen; Jingbo Yuan

cs.ROOct 11, 2026

P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning

Authors: Xue Wang, Yitang Li, Yiming Zhang, Siqin Wang, Junhan Sun, Zhihao Liu, Yaojie Shen, Jingbo Yuan,

Organizations: University of Shanghai for Science and Technology · RoboPartyLab · Tsinghua University · Zhejiang University ·

Abstract

Behavioral foundation models (BFMs) provide humanoids with reusable repertoires of whole-body behaviors; in real-world humanoid control, the same behavioral intent may require fundamentally different motions under different terrain constraints. Covering such variation with terrain-matched demonstrations requires enumerating a combinatorial space of behaviors and terrain geometries. We introduce P-BFM, a perceptive behavioral foundation model that factorizes humanoid control into terrain-invariant intent and terrain-conditioned realization. A forward-backward representation maps motion references, target states, and reward functions into a shared intent space without terrain information, while a perceptive policy learns how to realize each intent under the surrounding terrain geometry. We further identify a complementary limitation of purely latent control: behavioral latents can preserve local motion patterns while losing precise global progress. We therefore introduce Explicit Root Instructions (ERI), which separates global transport (e.g., heading and planar velocity) from behavioral intent, while leaving local whole-body adaptation to the perceptive policy. This decomposition leads to a characteristic capability absent from conventional BFMs: one fixed intent induces different, terrain-appropriate behaviors, including changes in step height, stride, posture, foothold placement, and contacts, without terrain-specific demonstrations. Across challenging terrains, P-BFM improves motion fidelity, root-control precision, and perturbation recovery over strong baselines. A single frozen policy further supports motion tracking, goal reaching, reward inference, teleoperation, and terrain-aware recovery, and transfers zero-shot to a Unitree G1 using onboard depth sensing.

Figures & tables

Figure 1: Unified intent representation in P-BFM . Motion references, teleoperation inputs, target states, and reward functions are mapped into a shared terrain-agnostic intent space. A single perceptive policy realizes the same intent through distinct behaviors adapted to different terrains. Videos: https://roboparty.github.io/P-BFM/

Figure 2: Overview of P-BFM . During pretraining, we learn a terrain-agnostic intent space through FB learning and a perceptive actor that interprets the intent latent into terrain-adaptive behavior. At inference, different task specifications are encoded into latents, and the same actor executes them across terrains without task-specific training. Orange arrows denote inputs to the actor.

Method	Seen motions					Unseen motions
Method	$E_{\mathrm{jpos}}\;(\mathrm{rad})\downarrow$	$E_{\mathrm{vel}}^{\mathrm{root}}\;(\mathrm{m/s})\downarrow$	$E_{\mathrm{head}}^{\mathrm{root}}\;({}^{\circ})\downarrow$	$\mathrm{FS}\;(\mathrm{cm})\downarrow$	$\mathrm{RR}\;(\%)\uparrow$	$E_{\mathrm{jpos}}\;(\mathrm{rad})\downarrow$	$E_{\mathrm{vel}}^{\mathrm{root}}\;(\mathrm{m/s})\downarrow$	$E_{\mathrm{head}}^{\mathrm{root}}\;({}^{\circ})\downarrow$	$\mathrm{FS}\;(\mathrm{cm})\downarrow$	$\mathrm{RR}\;(\%)\uparrow$
UFO	$0.15{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$0.65{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.46}$	$49.6{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,29.3}$	$0.092{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.050}$	$3.4$	$0.16{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$0.37{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.27}$	$28.9{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,23.9}$	$0.066{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.047}$	$16.6$
PMT	$0.21{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.04}$	$\mathbf{0.20}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.08}$	$8.8{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,7.7}$	$0.078{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.029}$	$2.4$	$0.46{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.09}$	$0.86{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.27}$	$84.4{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,22.5}$	$0.205{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.103}$	$0.5$
P-BFM	$\mathbf{0.11}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.04}$	${0.25}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.12}$	$\mathbf{6.9}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,5.4}$	$\mathbf{0.043}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.026}$	$\mathbf{41.2}$	$\mathbf{0.13}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$\mathbf{0.21}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.12}$	$\mathbf{6.7}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,4.7}$	$\mathbf{0.026}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.016}$	$\mathbf{61.4}$

Table 1: Comparison with existing perception-based general-purpose humanoid controllers on seen motions and a subset of 994 unseen SOMA motions.

Figure 3: Terrain-adaptive behavior from a single forward-walking intent. Given the same reference-derived intent, P-BFM produces distinct locomotion patterns across terrains, including stair ascent/descent, slope traversal, discrete foothold selection, and rough-terrain stepping.

Method	Seen motions					Unseen motions
Method	$E_{\mathrm{jpos}}\;(\mathrm{rad})\downarrow$	$E_{\mathrm{vel}}^{\mathrm{root}}\;(\mathrm{m/s})\downarrow$	$E_{\mathrm{head}}^{\mathrm{root}}\;({}^{\circ})\downarrow$	$\mathrm{FS}\;(\mathrm{cm})\downarrow$	$\mathrm{RR}\;(\%)\uparrow$	$E_{\mathrm{jpos}}\;(\mathrm{rad})\downarrow$	$E_{\mathrm{vel}}^{\mathrm{root}}\;(\mathrm{m/s})\downarrow$	$E_{\mathrm{head}}^{\mathrm{root}}\;({}^{\circ})\downarrow$	$\mathrm{FS}\;(\mathrm{cm})\downarrow$	$\mathrm{RR}\;(\%)\uparrow$
w/o head	$0.12{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.04}$	$0.28{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.14}$	$19.3{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,14.2}$	$0.045{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.026}$	$20.7$	$\mathbf{0.13}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$0.22{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.12}$	$14.6{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,14.3}$	$0.026{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.018}$	$39.5$
w/o vel	$0.12{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.04}$	$0.41{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.26}$	$7.7{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,6.9}$	$\mathbf{0.032}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.018}$	$22.4$	$\mathbf{0.13}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$0.32{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.19}$	$\mathbf{6.3}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,4.9}$	$\mathbf{0.020}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.012}$	$42.9$
P-BFM	$\mathbf{0.11}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.04}$	$\mathbf{0.25}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.12}$	$\mathbf{6.9}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,5.4}$	$0.043{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.026}$	$\mathbf{41.2}$	$\mathbf{0.13}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.06}$	$\mathbf{0.21}{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.12}$	$6.7{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,4.7}$	$0.026{\color[rgb]{0.5,0.5,0.5}\scriptstyle\,\pm\,0.016}$	$\mathbf{61.4}$

Table 2: Ablation study of global instructions on seen and unseen motions. “w/o head” removes the global heading instruction, while “w/o vel” removes the global base-velocity instruction.

Figure 4: Qualitative ablation. Without heading instructions, the robot exhibits heading drift; without velocity instructions, it steps in place. P-BFM maintains the intended heading and progress.

Figure 5: Real-world teleoperation with perception-driven terrain adaptation.

Figure 6: Real-world demonstrations of goal reaching and terrain-aware recovery.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Observation group	Dimension
Proprioceptive state	64
Previous action	29
Four-step proprioception–action history	372
Heading instruction	2
Horizontal-velocity instruction and validity flag	3
Estimated current root velocity	3

Appendix

Table 3: Observation groups used by the full model. Privileged inputs are used during training and are excluded from the deployed actor.

Network	Width $w$	Branch blocks $k$	Trunk blocks $r$	Output $m$	Members
$F$	2048	4	6	256	2
$Q_{D}$	1024	2	4	1	2
$Q_{A}$	2048	2	4	1	2
$Q_{H}$	1024	1	3	1	2
$Q_{V}$	1024	1	3	1	2

Appendix

Table 4: Forward and critic architectures. Each branch includes two affine projections in addition to its listed residual blocks. Each trunk includes an additional output projection. Each ensemble member is a complete, independently parameterized network.

Layer	Parameters	Output shape
Input	Normalized depth	$1\times 18\times 32$
Conv + ReLU	$1\to 16,\ k=3,\ s=1,\ p=1$	$16\times 18\times 32$
Conv + ReLU	$16\to 32,\ k=3,\ s=2,\ p=1$	$32\times 9\times 16$
Conv + ReLU	$32\to 64,\ k=3,\ s=1,\ p=1$	$64\times 9\times 16$
Adaptive average pooling	Output size $2\times 4$	$64\times 2\times 4$
Flatten	—	$512$

Appendix

Table 5: Shared single-frame depth encoder. Shapes are channels

\times

height

\times

width before flattening.

Value term	Role	Base coefficient
$Q_{\mathrm{FB}}$	Behavioral intent	1
$Q_{D}$	Conditional motion prior	0.01
$Q_{A}$	Physical auxiliary objectives	0.02
$Q_{H}$	Heading tracking	0.002
$Q_{V}$	Horizontal-velocity tracking	0.002

Appendix

Table 6: Coefficients in the actor objective. The four auxiliary value terms are additionally multiplied by the detached scale

\alpha

Cost	Coefficient
Action change	$-0.1$
Joint-position limit violation	$-10$
Body-impact severity	$-1$
Loaded-foot tangential speed	$-4$
Ankle-roll magnitude	$-1$
Foot stumbling	$-2$

Appendix

Table 7: Physical costs and their coefficients inside

r_{A}^{\mathrm{raw}}

. These coefficients are separate from the actor coefficient

0.02

for

Q_{A}

Parameter	Value
Latent dimension	256
Discount factor	0.98
Minibatch size per GPU	1,024
Effective minibatch size across eight GPUs	8,192
Forward-network learning rate	$3\times 10^{-4}$
Backward-network learning rate	$1\times 10^{-5}$

Appendix

Table 8: Optimization settings of the evaluated full model.

ID	Terrain family	Geometry parameters
0	Flat terrain	Planar surface; geometry is independent of difficulty.
1	Perlin-noise rough terrain	Target height range $0$ to $H_{\mathrm{rough}}(d)=0.02+0.04d\,\mathrm{m}$ before border resampling.
2	Pyramid stairs (outward descent)	Riser height $h(d)=0.10+0.10d\,\mathrm{m}$ ; tread depth $0.30\,\mathrm{m}$ ; central platform $1.6\times 1.6\,\mathrm{m}$ .
3	Inverted pyramid stairs (outward ascent)	Same riser heights, tread depth, and central-platform dimensions as the pyramid stairs, with inverted elevation.
4	Low platforms	Grid-cell width $0.80\,\mathrm{m}$ ; cell heights sampled from $[-H_{\mathrm{plat}}(d),H_{\mathrm{plat}}(d)]$ , where $H_{\mathrm{plat}}(d)=0.03+0.07d\,\mathrm{m}$ , before height quantization.
5	Inverted pyramid slope	Configured slope parameter $s(d)=0.20d$ ; nominal angle envelope $0$ to $\arctan(0.20)\approx 11.31^{\circ}$ .

Appendix

Table 9: Terrain families and difficulty-dependent geometry. IDs follow the terrain generator’s column order. Here

d

denotes normalized tile difficulty. Stair traversal directions are defined for motion from the central platform toward the tile boundary.

Evaluation	Continuous metrics		Recovery
	Motions/segments	Episodes	Segments	Episodes
LaFAN comparison	902	315,700	862	241,360
LaFAN ablation	862	301,700	862	241,360
SOMA	994	347,900	281	78,680
PMT references	297	103,950	297	83,160

Appendix

Table 10: Main evaluation coverage per model. Continuous metrics use five trials per motion–terrain–row condition. Recovery uses one trial at each of four force magnitudes.

Three-method comparison
Method	RR@3s (%)	RR@5s (%)	Pre-push violation (%)
UFO	22.50	28.38	31.70
PMT	0.87	1.07	98.98
P-BFM	56.99	69.50	20.30
Ablation
w/o Heading Instr.	35.15	45.44	21.52

Appendix

Table 11: Category-balanced SOMA recovery results. Rates are recording- and category-macro averages. Each condition uses one trial per force magnitude. Pre-push violation denotes failure to remain within the tracking tolerances throughout the one-second pre-push interval.

Parameter	Setting
Automatic exposure	Disabled
Exposure time	$1600\,\mu\mathrm{s}$
Gain	16
Infrared emitter power	240
Depth-stream resolution	$480\times 270$ pixels
Depth-stream frame rate	$60\,\mathrm{Hz}$

Appendix

Table 12: Recorded camera and depth-processing settings for the final real-world deployment. Gain and infrared emitter power are reported as device setting values.

Unsupervised RL

See all 13 papers

Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain
Jun 6, 2026Zifan Wang, Yizhao Li, Teli Ma +5Humanoid Robot Locomotion Robot Foundation Models
Mondo Robotics · The Hong Kong University of Science and Technology (Guangzhou) · Artificial General Intelligence Institute, University of Science and Technology of China +2
BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning
Nov 6, 2025Yitang Li, Zhengyi Luo, Tonghe Zhang +10Humanoid Robot Control Behavioral Foundation Models
1Carnegie Mellon University · 2Meta
I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning
Oct 5, 2026Ziqi Han, Yitang Li, Junhan Sun +8Representation Learning Reinforcement Learning
Tongji University · Tsinghua University · Zhejiang University +3

Jun 6, 2026J/K move · Enter open · S save

Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain

Zifan Wang, Yizhao Li, Teli Ma, Qiang Zhang+4

Mondo Robotics · The Hong Kong University of Science and Technology (Guangzhou) · Artificial General Intelligence Institute, University of Science and Technology of China+2

Humanoid Robot Locomotion Robot Foundation Models Humanoid Robot Control Motion Retargeting Robot Motion Generation Terrain-Aware Robot Locomotion

Open paper AskPDF arXiv

Humanoid behavior foundation models aim to acquire reusable whole-body control policies from broad human motion priors, enabling a single controller to produce diverse and expressive behaviors. However, existing motion-centric foundation policies largely assume that the reference motion is already physically compatible with the robot's surroundings. This assumption breaks when the demonstrator, operator, and robot inhabit different environments: a human motion may specify the intended behavior, but not the footholds, clearance, body height, or contact timing required by the robot's local terrain. We introduce Perceptive Behavior Foundation Model (Perceptive BFM), a terrain-aware humanoid control framework that grounds human motion priors in robot-centric perception. The model preserves raw kinematic motion references as the behavioral interface, while using local terrain observations to adapt contacts, posture, and timing. To provide scalable terrain supervision, we develop terrain-conformal reference synthesis (TCRS), which converts locomotion-oriented human motion clips into terrain-consistent references through contact-aware foothold construction, foot-geometry-aware swing optimization, support-aware root reconstruction, collision repair, and multi-point inverse kinematics. We then train a blind adapted-reference teacher and transfer its terrain-conformal behavior to a deployed raw-reference student through target-frame action alignment. The student is an identity-gated Transformer tracker whose terrain features enter through residual pathways initialized to preserve the motion-tracking prior and trained to produce local corrections only when needed.

Abstract

Unsupervised RL

Explore similar work

Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain

BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learning

I-BFM: Reward-Conditioned Robust Humanoid Interaction via Unsupervised Reinforcement Learning