HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control

Langzhe Gu; Chengkai Hou; Meng Li; Xinhua Wang; Xinyuan Lv; Shuanghao Bai; Guangrun Li; Jingyang He

cs.ROOct 19, 2026

HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control

Authors: Langzhe Gu, Chengkai Hou, Meng Li, Xinhua Wang, Xinyuan Lv, Shuanghao Bai, Guangrun Li, Jingyang He,

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · Nankai University · Xi’an Jiaotong University ·

Abstract

Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet adapting generalist vision-language-action (VLA) policies to real-world humanoid loco-manipulation remains challenging. Unlike fixed-base or wheeled manipulators, humanoids must coordinate locomotion, posture, gaze, and dual-arm manipulation over a high-dimensional whole-body action space, where inaccurate base or waist motions can easily induce unstable and aggressive upper-body compensation. We propose HAF-VLA, a hierarchical action-flow framework for humanoid VLA control. Built upon a pretrained flow-matching VLA, HAF-VLA inherits the vision-language prefix encoder and action expert weights, and adapts them to humanoid whole-body loco-manipulation with learnable stage embeddings and cross-stage KV-cache conditioning. Instead of generating the full action chunk in a single pass, HAF-VLA progressively denoises a shared whole-body action space: first locomotion and head commands, then waist refinement, and finally full-body manipulation. To preserve structural dependencies across stages, we introduce a cross-stage KV cache conditioning mechanism, enabling subsequent action generation stages to refine and augment their outputs based on the action representations from preceding stages. Experimental results across six real-world tasks demonstrate that our framework significantly enhances the performance of existing models in humanoid loco-manipulation tasks, demonstrating an effective pathway for adapting strong mobile-manipulation VLAs to humanoid whole-body loco-manipulation.

Figures & tables

Figure 1: Overview of HAF-VLA. (a) We demonstrate seven loco-manipulation tasks tested on the humanoid robot in real-world scenarios. (b) The pipeline illustrating our hierarchical approach to generating actions across different robot stages. (c) Average Real-World Normalized Score of different baselines and HAF-VLA on the six real-world tasks.

Figure 2: Overall framework of HAF-VLA. Given visual observations, robot states, and language instructions, the model progressively generates whole-body action trajectories through hierarchical action-flow stages. The final action chunk is executed by the robot-specific low-level control interface.

Figure 3: Robot teleoperation data collection setup. An IMU controls the head, kinematically matched master arms control the arms and hands, and a handheld joystick controls locomotion and waist motion.

Figure 4: Real-world evaluation tasks. The first six tasks constitute the main benchmark used in Q1, while Box Carry is used for the controlled ablation study in Q2. Each row shows a representative execution sequence requiring locomotion, whole-body posture adjustment, and object interaction.

Task	HAF-VLA (Ours)	$\boldsymbol{\pi_{0.5}}$	GR00T N1.7	Cosmos Policy	ACT
Laundry Loading	66.7	53.3	40.0	0.0	10.0
Clothes Retrieval	53.3	53.3	33.3	26.7	23.3
Table Tidy	80.0	70.0	40.0	16.7	23.3
Plant Watering	60.0	50.0	33.3	30.0	16.7
Ball Tossing	56.7	33.3	36.7	3.3	30.0
Box Transfer	93.3	60.0	43.3	73.3	50.0

Table 1: Main results on six long-horizon humanoid loco–manipulation tasks. We report the average normalized task score (%) over 10 real-robot rollouts. Higher is better.

Component	Variant	Design	Score
Stage design	HAF-VLA (Ours)	Loco. $\rightarrow$ Waist $\rightarrow$ Arms	86.7
	Waist-First	Waist $\rightarrow$ Arms $\rightarrow$ Loco.	70.0
	Arm-First	Arms $\rightarrow$ Waist $\rightarrow$ Loco.	60.0
	Loss-Calibrated	Reweight action groups by $1/3,\;1/2,\;1$	76.7
	All-Joint	Denoise all joints at every stage	73.3
	$\pi_{0.5}$ (10 steps)	Single-stage full-body denoising	70.0

Table 2: Ablation study of HAF-VLA on Box Carry. We report the average normalized task score (%) over 10 real-robot rollouts. “Unsafe” denotes variants terminated due to unsafe motions.

Figure 5: Cross-stage consistency and refinement. Forward-velocity predictions from three stages recorded during a real-world Box Carry rollout. The stages maintain a consistent overall trajectory, while later stages make localized refinements.

Figure 6: Generalization experiments under distribution shifts. (a) Unseen chair, (b) 20 cm initial-position shift, (c) 30 cm target-table shift, (d) dimmed lighting, and (e) smaller object. For each case, the unperturbed setting is shown on the left, with the corresponding distribution-shifted setting shown on the right.

Task	Setting	HAF-VLA (Ours)		$\boldsymbol{\pi_{0.5}}$
Task	Setting	Score	Drop	Score	Drop
Laundry Loading	Unseen Chair	40.0	$\downarrow$ 26.7	26.7	$\downarrow$ 26.7
Clothes Retrieval	20 cm Start Shift	43.3	$\downarrow$ 10.0	36.7	$\downarrow$ 16.7
Box Carry	Dim Lighting	83.3	$\downarrow$ 3.3	60.0	$\downarrow$ 10.0
	Smaller Object	76.7	$\downarrow$ 10.0	50.0	$\downarrow$ 20.0
	30 cm Table Shift	73.3	$\downarrow$ 13.3	40.0	$\downarrow$ 30.0

Table 3: Generalization under controlled distribution shifts. We report the average normalized task score (%) and its absolute drop (

\downarrow

, percentage points) from the corresponding unperturbed setting.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1: Robot Platform. We use Tian Gong 3.0 and Tian Gong 2.0 humanoid robots as our robot platforms.

Task	HAF-VLA (Ours)	$\boldsymbol{\pi_{0.5}}$	GR00T N1.7	Cosmos Policy	ACT
Laundry Loading	30 [11, 60]	20 [6, 51]	10 [2, 40]	0 [0, 28]	10 [2, 40]
Clothes Retrieval	40 [17, 69]	30 [11, 60]	20 [6, 51]	0 [0, 28]	0 [0, 28]
Table Tidy	60 [31, 83]	40 [17, 69]	20 [6, 51]	0 [0, 28]	10 [2, 40]
Plant Watering	50 [24, 76]	40 [17, 69]	20 [6, 51]	20 [6, 51]	0 [0, 28]
Ball Tossing	40 [17, 69]	20 [6, 51]	20 [6, 51]	0 [0, 28]	10 [2, 40]
Box Transfer	80 [49, 94]	30 [11, 60]	20 [6, 51]	30 [11, 60]	30 [11, 60]

Appendix

Table S1: Full-task success rates with statistical uncertainty. We report success rates (%) with 95% Wilson confidence intervals over 10 real-robot rollouts per method and task. Each entry is formatted as success rate [lower, upper].

Humanoid Robot Control

Humanoid robot control develops methods for coordinating bipedal balance, locomotion, and whole-body interaction with people and objects.

See all 192 papers

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL
Aug 17, 2026Langzhe Gu, Chengkai Hou, Meng Li +14Humanoid Loco-Manipulation Whole-Body Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · Nankai University +1
OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
Jun 20, 2026Yingdong Hu, Haodong Zhu, Boyuan Zheng +6Humanoid Loco-Manipulation Long-Horizon Robotic Manipulation
Tsinghua University · Shanghai Qi Zhi Institute · Spirit AI
WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning
Jun 24, 2026Melya Boukheddimi, Omar Adjali, Daniel Sontag +1Robotic Control Humanoid Robot Locomotion
Robotics Innovation Center, DFKI, 28359 Bremen, Germany · Interactive Machine Learning, DFKI, 26129 Oldenburg, Germany · University of Oldenburg, 26129 Oldenburg, Germany +1

Aug 17, 2026J/K move · Enter open · S save

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

Langzhe Gu, Chengkai Hou, Meng Li, Xinhua Wang+13

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · Nankai University+1

Humanoid Loco-Manipulation Whole-Body Robotic Manipulation Vision-Language-Action Models Reinforcement Learning Offline-To-Online RL

Open paper AskPDF arXiv

Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .

Abstract

Humanoid Robot Control

Explore similar work

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning