PhysiFlow: Physics-Aware Humanoid Whole-Body VLA via Multi-Brain Latent Flow Matching and Robust Tracking

Weikai Qin; Sichen Wu; Ci Chen; Mengfan Liu; Linxi Feng; Xinru Cui; Haoqi Han; Hesheng Wang

cs.ROMar 5, 2026

PhysiFlow: Physics-Aware Humanoid Whole-Body VLA via Multi-Brain Latent Flow Matching and Robust Tracking

Authors: Weikai Qin, Sichen Wu, Ci Chen, Mengfan Liu, Linxi Feng, Xinru Cui, Haoqi Han, Hesheng Wang

Organizations: department of Automation Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China · Department of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management, Shanghai Jiao Tong University, Shanghai, 200240, China

Abstract

In the domain of humanoid robot control, the fusion of Vision-Language-Action (VLA) with whole-body control is essential for semantically guided execution of real-world tasks. However, existing methods encounter challenges in terms of low VLA inference efficiency or an absence of effective semantic guidance for whole-body control, resulting in instability in dynamic limb-coordinated tasks. To bridge this gap, we present a semantic-motion intent guided, physics-aware multi-brain VLA framework for humanoid whole-body control. A series of experiments was conducted to evaluate the performance of the proposed framework. The experimental results demonstrated that the framework enabled reliable vision-language-guided full-body coordination for humanoid robots.

Figures & tables

Fig. 1: Introducing PhysiFlow , a multi-brain VLA humanoid system that operates on Unitree G1 robots and performs end-to-end VLA humanoid whole body control in large spaces. The proposed system achieves consecutive tasks autonomously, including (a-c) walking to the designated item, sitting on the proposed item, and raising arm; (d-f) circling the designated item, standing up from the specific item and turning right.

Fig. 2: The overall pipeline of PhysiFlow. This bio-inspired architecture decouples semantic reasoning from physics-aware execution. (a) Neocortical Brain: A curriculum-based CVAE processes vision and language to synthesize a $10\text{\,}\mathrm{Hz}$ latent vector $z_{vl}$ , aligning task semantics with motion intent. (b) Basal Ganglionic Brain: Conditioned on $z_{vl}$ and robot states, a flow-matching model generates $50\text{\,}\mathrm{Hz}$ motion sequence $m_{t}$ for continuity. (c) Cerebellar Brain: A robust motion tracker enforces physical constraints, translating these chunks into stable motor commands for closed-loop whole-body control.

Fig. 3: Visualization of the VLA dataset. (a) Diverse visuals with various Scenes and Items. (b) Diverse camera angles with ego and exo views. (c) Diverse task from turning around to standing up

Metric Name	Full	PC Off	Disc Off	VL Off	Lora Off	Textcond0	Curr. Off
Recon. Post	0.017	0.020	0.019	0.020	0.020	0.041	0.081
Recon. Prior	0.023	0.027	0.026	0.026	0.029	0.045	0.081
Future Shuffle Gap	1.134	0.679	0.852	0.940	0.958	0.516	0.001
KL CVAE	36.095	36.369	29.632	26.835	29.016	38.783	9.100
Retrieval (Cross Ep.)	0.859	0.879	0.872	0.037	0.869	0.846	0.846
Retrieval (Same Task)	0.357	0.364	0.365	0.086	0.362	0.349	0.352

TABLE I: Comparison of core metrics across different ablation settings.Best results are viewed in bold black .

Fig. 4: Performance benchmarking of the Basal Ganglionic Brain. The proposed flow-matching (FM) paradigm is evaluated against autoregressive (AR) and Denoising Diffusion Probabilistic Model (DDPM) baselines.

Fig. 5: Real-world execution of semantically guided whole-body tasks by the Unitree G1 humanoid robot. Top: Complex VLA maneuvers requiring continuous spatial navigation and dynamic multi-limb coordination. Bottom: Basic VLA tasks demonstrating responsive semantic execution and robust postural stability. These results validate the system’s capacity to maintain physical compliance and dynamic consistency during unconstrained deployment.

Task Type	Task Name	LeVERB [ 20 ]	PhysiFlow
Complex	Nav. (Long)	31.2	63.6
	Nav. (Short)	78.5	71.7
	Nav. & Sit	5.8	18.1
	Nav. & Circle	54.5	69.2
Regular	Stand up	88.6	90.9
	Locomotion	97.2	100.0

TABLE II: Comparison of Success Rates (%) between LeVERB and PhysiFlow. Best results are viewed in bold black .

Fig. 6: Performance metrics across different latent dimensions including 256, 512 and 1024. The left axis tracks the posterior and prior reconstruction errors, while the right axis displays the Top-1 retrieval accuracy.

Explore similar work

Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation
Sep 21, 2026Fukang Liu, Yipu Chen, Jaehwi Jang +3Contact-Rich Robotic Manipulation Humanoid Robot Control
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA, USA
PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation
Jun 11, 2026Namai Chandra, Shriram Damodaran, Lin WangRobotic Control Robotic Manipulation
Electronic Systems IIT Madras, India · EmPACT Lab Nanyang Technological University Singapore
WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning
Jun 24, 2026Melya Boukheddimi, Omar Adjali, Daniel Sontag +1Robotic Control Humanoid Robot Locomotion
Robotics Innovation Center, DFKI, 28359 Bremen, Germany · Interactive Machine Learning, DFKI, 26129 Oldenburg, Germany · University of Oldenburg, 26129 Oldenburg, Germany +1

Sep 21, 2026J/K move · Enter open · S save

Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation

Fukang Liu, Yipu Chen, Jaehwi Jang, Danfei Xu+2

Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA, USA

Contact-Rich Robotic Manipulation Humanoid Robot Control Force-Controlled Robotic Manipulation Vision-Language-Action Models

Open paper AskPDF arXiv

Humanoid robots are expected to perform diverse human-level tasks in daily environments, many of which require precise regulation of interaction forces. While recent vision-language-action (VLA) models have shown promise for semantic planning and visuomotor control, existing humanoid systems primarily represent actions through geometric motion goals and rely on whole-body controllers focused on motion tracking, with limited explicit reasoning or control of interaction forces. This limitation is particularly relevant in contact-rich tasks, where geometrically similar motions may require different force regimes depending on the task context and where visual observations may become unreliable after contact. In this work, we present Opt2VLA, a force-aware VLA framework that introduces explicit force commands at the VLA-to-control interface for humanoid whole-body manipulation. A single multi-task VLA policy jointly predicts both geometric motion goals and continuous contact-force references, which are tracked by task-specific reinforcement learning (RL)-based whole-body controllers. To provide scalable and physically grounded supervision, we generate dynamically feasible and contact-consistent training data via whole-body trajectory optimization (TO) with explicit force references. We evaluate Opt2VLA on three contact-rich humanoid tasks and show that explicit force conditioning enables more accurate and consistent force regulation than motion-only control, while physically grounded torque supervision from TO further improves force tracking accuracy and stability. Closed-loop evaluations further demonstrate language-conditioned force modulation with Opt2VLA in simulation and on humanoid hardware.

Abstract

Explore similar work

Opt2VLA: Force-Aware Vision-Language-Action for Contact-Rich Humanoid Whole-Body Manipulation

PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation

WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning