HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

Langzhe Gu; Chengkai Hou; Meng Li; Xinhua Wang; Jiaming Liu; Xinyuan Lv; Bowei Zhang; Shuanghao Bai

cs.ROAug 17, 2026

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

Authors: Langzhe Gu, Chengkai Hou, Meng Li, Xinhua Wang, Jiaming Liu, Xinyuan Lv, Bowei Zhang, Shuanghao Bai,

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · Nankai University · Xi’an Jiaotong University

Abstract

Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .

Figures & tables

Figure 1: Overview of HAF. We demonstrate seven humanoid loco-manipulation tasks in real-world settings. HAF consists of two complementary components: HAF-VLA is a hierarchical action-flow vision-language-action model that generates whole-body motions stage by stage according to the kinematic dependencies among locomotion, torso adjustment, and dual-arm manipulation; HAF-Steer performs latent-space reinforcement learning on discrete cosine transform compressed temporal noise from frozen flow-based VLA policies. Together, they improve task performance in real-world settings.

Figure 2: Overview of HAF-VLA. A shared action expert progressively expands the active action space from locomotion and head control to waist posture and bimanual manipulation. Clean action caches from earlier stages condition subsequent generation, and only the final full-body action chunk is executed.

Figure 3: Overview of HAF-Steer. (a) A conventional flow-matching VLA samples full-dimensional Gaussian noise and decodes it into an action trajectory through a frozen flow generator. (b) HAF-Steer instead constructs a low-dimensional spectral action space from expert demonstrations: demonstrated trajectories are mapped back to their initial flow noise through numerical reverse integration, compressed by retaining the first 8 temporal DCT coefficients, and normalized to form expert spectral actions in the offline replay buffer. An RL policy is trained with mixed offline and online experience to predict spectral actions, which are de-normalized, reconstructed into full temporal noise by inverse DCT, and decoded by the frozen VLA to produce executable robot trajectories.

Figure 4: Robot teleoperation data collection setup. The teleoperation pipeline uses an IMU for head control, kinematically matched master arms for dual-arm manipulation, and a joystick to command leg locomotion and waist posture.

Figure 5: Seven real-world humanoid loco-manipulation tasks. Each row depicts a representative execution sequence requiring navigation, whole-body posture adjustment, and physical object interaction.

Task	HAF-VLA	$\boldsymbol{\pi_{0.5}}$	GR00T N1.7	Cosmos Policy	ACT
Laundry Loading	66.7	53.3	40.0	0.0	10.0
Clothes Retrieval	53.3	53.3	33.3	26.7	23.3
Table Tidy	80.0	70.0	40.0	16.7	23.3
Basket Transfer	63.3	50.0	43.3	33.3	16.7
Toy Storage	80.0	53.3	30.0	40.0	23.3
Ball Tossing	56.7	33.3	36.7	3.3	30.0

Table 1: Main HAF-VLA results on seven long-horizon humanoid loco–manipulation tasks. We report the average normalized task score (%) over 10 rollout trials. Higher values indicate better performance.

Method / Variant	Stage Design	Total Steps	Norm. Score (%)
HAF-VLA	Locomotion/Head $\rightarrow$ + Waist $\rightarrow$ + Manip.	30	66.7
All-Joint Denoising	All joints at every stage	30	53.3
Arm-First Hierarchy	Manip. $\rightarrow$ + Waist $\rightarrow$ + Locomotion/Head	30	50.0
$\pi_{0.5}$ , 30 steps	Full-body action	30	20.0

Table 2: Ablation study of HAF-VLA’s hierarchical action-flow design. We report the average normalized task score (%) over 10 rollout trials on the Laundry Loading task. Higher scores are better.

Figure 6: Generalization experiments. Left: object-distraction test during Laundry Loading . Right: position-disturbance test during Clothes Retrieval .

Task	Disturbance	HAF-VLA	$\boldsymbol{\pi_{0.5}}$
Laundry Loading	Unseen chair near the path	40.0	26.7
Clothes Retrieval	20 cm backward start shift	43.3	36.7

Table 3: Generalization evaluation under controlled disturbances. We report the average normalized task score (%) over 10 rollout trials. Higher scores indicate stronger robustness.

Figure 7: In-distribution and out-of-distribution evaluation setups for HAF-Steer. Blue markers indicate destination-table locations covered during demonstration collection, while red markers indicate the OOD evaluation locations. For Toy Storage , the destination table is placed 30 cm beyond the demonstrated spatial range. For Basket Transfer , the destination table is shifted by 30 cm from its demonstrated position.

Figure 8: HAF-Steer performance under in-distribution and out-of-distribution settings. We report successful trials out of 10 rollouts on Toy Storage and Basket Transfer using both

\pi_{0.5}

and HAF-VLA backbones. Noise BC denotes the spectral actor after behavior-cloning initialization without SAC post-training. † DSRL failed to complete training because unsafe exploratory motions triggered early termination during real-robot rollouts.

Humanoid Loco-Manipulation

Humanoid loco-manipulation studies how robots coordinate walking, balance, and whole-body interaction to move and handle objects.

See all 82 papers

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
Jun 20, 2026Yingdong Hu, Haodong Zhu, Boyuan Zheng +6Humanoid Loco-Manipulation Long-Horizon Robotic Manipulation
Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI
Humanoid Loco-Manipulation With Discrete VLA Model
Sep 28, 2026Wenxin Shao, Siqi Chai, Kun Li +4Humanoid Loco-Manipulation Vision-Language-Action Models
Horizon Robotics
WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning
Jun 24, 2026Melya Boukheddimi, Omar Adjali, Daniel Sontag +1Robotic Control Humanoid Robot Locomotion
Robotics Innovation Center, DFKI, 28359 Bremen, Germany · Interactive Machine Learning, DFKI, 26129 Oldenburg, Germany · University of Oldenburg, 26129 Oldenburg, Germany +1

Jun 20, 2026J/K move · Enter open · S save

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Yingdong Hu, Haodong Zhu, Boyuan Zheng, Yihang Hu+5

Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI

Humanoid Loco-Manipulation Long-Horizon Robotic Manipulation Robot Teleoperation Cross-Embodiment Robot Learning Whole-Body Robotic Manipulation Vision-Language-Action Models

Open paper AskPDF arXiv

Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and

Ψ_0

) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

Abstract

Humanoid Loco-Manipulation

Explore similar work

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Humanoid Loco-Manipulation With Discrete VLA Model

WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning