Reinforcement learning (RL) has demonstrated substantial potential for humanoid bipedal locomotion and the control of complex motions. To cope with oscillations and impacts induced by environmental interactions, compliant control is widely regarded as an effective remedy. However, the model-free nature of RL makes it difficult to impose task-specified and quantitatively verifiable compliance objectives, and classical model-based stiffness designs are not directly applicable. Lipschitz-Constrained Policies (LCP), which regularize the local sensitivity of a policy via gradient penalties, have recently been used to smooth humanoid motions. Nevertheless, existing LCP-based methods typically employ a single scalar Lipschitz budget and lack an explicit connection to physically meaningful compliance specifications in real-world systems. In this study, we propose an anisotropic Lipschitz-constrained policy (ALCP) that maps a task-space stiffness upper bound to a state-dependent Lipschitz-style constraint on the policy Jacobian. The resulting constraint is enforced during RL training via a hinge-squared spectral-norm penalty, preserving physical interpretability while enabling direction-dependent compliance. Experiments on humanoid robots show that ALCP improves locomotion stability and impact robustness, while reducing oscillations and energy usage.
Figures & tables
Fig. 1: Task-specific compliance. (a) Hand-pulling experiment under the SILC method. (b) Hand impact experiment. A 2.5 kg weight is attached to the robot’s right hand via a rope and released from a certain height to generate an impact. (c) Stepping motion experiment. The robot’s one foot intermittently steps onto a foam pad placed on the ground, thereby introducing disturbances from uneven terrain.
Fig. 2: Control framework in reinforcement learning pipeline.
Fig. 3: Stiffness upper-bound derivation diagram. \mbox{\boldmathK}_{x}^{\max} denotes the task-space stiffness upper-bound ellipsoid, illustrated as the pink ellipsoid; \mbox{\boldmathK}_{q}^{\max} denotes the joint-space stiffness upper-bound super-ellipsoid, where individual joint stiffnesses are depicted as torsional springs; \mbox{\boldmathK}_{LCP} represents the Lipschitz upper-bound constraint.
Mode
CoM Stiffness
Null-space Stiffness
High-stiff
[1000,1000,20000]
100.0
Compliance
[200,200,20000]
50.0
Hard-nullspace
[200,200,20000]
500.0
TABLE I: CoM task and null-space stiffness setting in SILC method.
Fig. 4: The x -direction CoM trajectory receiving an −x -direction impact (150 N in 0.05 s) on pelvis link. The red triangle indicates the time instant at which the impact is applied. Compared with the high-stiff mode—in which the CoM returns to its original state via a damped oscillatory response—the compliance mode exhibits more pronounced compliance, as evidenced by larger oscillation amplitudes and the lack of convergence back to the original state. Meanwhile, scalar LCP also exhibits similarly high stiffness to high-stiff mode.
Fig. 5: Hand displacement under different task-space compliance settings of SILC when a constant load (-40 N) is applied to the left hand. It can be observed that, in both cases, the hand displacement responds within a very short time ( < 0.1 s). After the response settles, we compare the steady-state position with the initial position and, together with the applied external force, approximately estimate the equivalent task-space stiffness to be 400 (soft) and 1000 N/m (hard), respectively.
Fig. 6: According to the definition of the Directional Quadratic-form Budget, at each control step we plot the left-hand side (LHS) and right-hand side (RHS) of ( 34 ), respectively (unit of quadratic form: N ⋅ m ⋅ rad). The yellow curve corresponds to the RHS of the equation, i.e., the quadratic form under the target \mbox{\boldmathK}_{q}^{\max} , whereas the blue curve corresponds to the LHS, i.e., the current quadratic form measured \mbox{\boldmathK}_{q}(\mbox{\boldmatho}) . The red markers indicate instances where the measured value exceeds the target. As shown, under the SILC method, the measured value is strictly bounded by the target constraint; in contrast, under scalar LCP and baseline, the measured quadratic form periodically exceeds the target during the stepping gait.
Action Jitter ( ×104 ) ↓
DoF Velocity Jitter [rad/s] ( ×105 ) ↓
Torque Jitter [Nm] ( ×105 ) ↓
Energy [J] ↓
High-stiff SILC
3.08±0.19
1.97±0.17
4.89±0.34
4.77±3.07
Compliance SILC
3.19±0.28
2.03±0.28
5.04±0.38
4.99±5.17
Hard-nullspace SILC
3.03±0.17
1.84±0.16
4.80±0.22
4.86±4.47
Scalar LCP ( λ=0.5 )
3.52±7.00
1.74±1.96
4.47±5.01
5.09±4.76
Baseline
4.13±1.94
1.68±0.51
5.35±1.13
7.58±10.74
TABLE II: Comparison of walking motion under random velocity command. (environment number: 1,024)
Settling time [s] ↓
High-frequency energy [J] ↓
SILC (soft)
1.2
1.01×10−8
Baseline
2.4
10.0×10−8
TABLE III: Metrics of the right hand z -direction trajectory oscillations under external impacts.
RMS ↓
Peak-to-Peak (P95-P5) ↓
Compliance
Gyro.
0.247
0.726
SILC
Acc. ( x and y )
3.885
12.66
Baseline
Gyro.
0.472
1.433
Acc. ( x and y )
4.859
13.80
TABLE IV: IMU measurement in the stepping experiment.
Embodied AI systems, particularly humanoid robots deployed in real world scenarios require whole-body control policies that are both task-responsive and physically smooth. However, smoothness is not uniform across the body: lower body must remain sufficiently reactive, while the upper body must be tightly regulated to preserve stability. Existing reinforcement learning approaches typically impose smoothness through auxiliary terms in the reward function, which compete with task objectives, treating the body as uniform and provide no direct control over the physical quantities responsible for smooth behavior. We introduce DeCap (Decoupled Constraint-aware policy), a constrained reinforcement learning algorithm that decouples whole-body smoothness into separate upper- and lower-body constraint groups, each formulates smoothness as explicit constraints on physical motion limits. To improve constraint satisfaction near feasibility boundaries, DeCap incorporates a bounded barrier penalty that activates proactively as limits are approached and remains bounded at the constraint limit. On real-world humanoid whole-body control task, DeCap reduces upper-body action rate by 2.50x and acceleration by 2.18x relative to reward-based smoothness policies, while also improving lower-body smoothness and reducing transient motion. We demonstrate that a fixed set of smoothness constraints transfers across diverse terrains, alleviating the need of extensive reward tuning.
Utsav Panchal, Denis Kleyko, Unal Artan +1
AI, Robotics and Cybersecurity Center (ARC) and Department of Computer Science, Örebro University, Sweden
Real-world humanoid tasks involve physical interaction with objects and humans, yet current controllers either reject external forces as disturbances or restrict compliance to limited body links while ignoring angular effects. We present LAC, a general whole-body controller that simultaneously realizes commanded Linear and Angular Compliance for wrenches applied to the upper body. First, we synthesize whole-body compliant responses into a large-scale augmented dataset. Sampled force and couple events are imposed on contact frames extracted from human interaction data. At each contact link, the external force and a virtual torque from the passively yielding kinematic chain drive a virtual admittance under the commanded stiffness. Subsequently, teacher-student reinforcement learning trains a single policy to track the compliant motions under external wrenches. Finally, extensive simulation and real-world experiments demonstrate whole-body compliant responses to wrenches across the upper body, monotonic modulation over the full range of both stiffness commands, and applicability to teleoperated loco-manipulation tasks. Project website: https://lac-humanoid.github.io/
Yang Liu, Zhongkai Gu, Wei Zhu +1
Neuro-Robotics Laboratory, Tohoku University, Japan
Whole-body compliant control is essential for deploying heavy humanoids under high payload in human-centric environments. Most prior force-aware learning-based pipelines focus on end-effector resistance, per-link upper-body springs, or end-effector stiffness modulation, leaving arbitrary-site perturbations on heavy platforms with lower-body engagement largely unaddressed. We close this gap with CompliantWBC comprising: (1) A base policy trained with RL to maximize compliance-fidelity reward, guided by a multi-site whole-body impedance reference controller, extending classical Cartesian impedance to any controlled link; (2) A bounded residual policy that edits the per-link impedance equilibrium over a frozen base, correcting the coarse but structured wrench estimate supplied by a force encoder co-trained behind a gradient barrier; (3) A Phong-weighted force-origin sampler with an axis-decoupled pelvis anchor induces lower-body-inclusive compliance curriculum training via two interpretable parameters. We evaluate CompliantWBC in simulation against both compliant and stiff baselines, achieving best compliant fidelity of 2.58cm deviation from analytical solutions, and demonstrate it on a real heavy humanoid across static/dynamic force reaction, board wiping, squat under payload, and cooperative payload transport. Project website: https://dotandung.github.io/compliantwbc/
Tan-Dzung Do, Cuc T. Trinh, Tuan Dat Phuong +4
VinRobotics · National University of Singapore · VinUniversity +1