cs.RONov 5, 2026

PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion

Authors: Yunwan Fu, Sicheng He, Jiageng Mao, Yue Wang

Organizations: Shanghai Jiao Tong University · University of Southern California

Abstract

Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation

    Jun 11, 2026Namai Chandra, Shriram Damodaran, Lin WangRobotic ControlRobotic Manipulation

  2. SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization

    Mar 14, 2026Jiashun Li, Xiaoyu Shi, Hong Xie +2VLMs for RoboticsRL for VLMs

  3. Smoothness as a Constraint for Stable Humanoid Locomotion

    Sep 21, 2026Utsav Panchal, Denis Kleyko, Unal Artan +1Robotic ControlHumanoid Robot Locomotion