Organizations: Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan.
Brachiation is a form of locomotion in which primates move primarily using their arms, enabling traversal in environments without footholds. However, this motion requires highly coordinated whole-body movement and precise timing control for bar grasping and release. As a result, achieving robust behavior on life-sized robotic platforms remains challenging. In this study, we present a reinforcement learning-based method to realize brachiation on a life-sized dual-arm robot. The core of the proposed approach is Waypoint-Guided Reinforcement Learning (WGRL), a learning framework for inducing non-linear and complex motions. For high-difficulty tasks where imitation learning data are unavailable, WGRL guides behavior acquisition by sparsely specifying waypoints for the end-effector trajectory, while whole-body motion is generated through reinforcement learning. In addition, by integrating the waypoint-following guidance with rewards based on task success and mechanical energy, and training in an environment designed for Sim-to-Real transfer, the proposed method achieves both forward progression and motion stability. The acquired behavior is evaluated through Sim-to-Sim experiments under monkey-bar environments with geometric variations and hardware experiments, confirming robust brachiation including failure recovery behavior. This study provides effective learning design guidelines for realizing arm-based locomotion on life-sized robotic hardware and expanding the traversable workspace of robots.
Figures & tables
Fig. 1: Brachiation motion of a life-sized dual-arm robot traversing four cylindrical bars (30 mm in diameter) arranged at 0.4 m intervals.
Fig. 2: Mechanical design of the life-sized dual-arm robot. Left: Overview of the robot, modeled after the upper body of an adult human. Right: Hook-shaped end-effector designed for secure bar suspension and robust brachiation.
Name
Symbol
Base (torso link) angular velocity
ωb∈R3
Gravity direction (in torso local frame)
gb∈R3
All joint positions
q∈R10
All joint velocities
q˙∈R10
Motion phase indicator
sphase∈{0,1}
TABLE I: Observation space used for the policies
Name
Symbol
Joint position history
qt−2:t∈R3×10
Joint velocity history
q˙t−2:t∈R3×10
Joint position tracking error history
et−2:t∈R3×10
IMU history
ot−7:tIMU∈R8×6
End-effector hooking contact state
c∈{0,1}2
End-effector contact forces
fc∈R6
TABLE II: Privileged observation space used for the critic
Fig. 3: Waypoint design for inducing non-linear hooking trajectories in WGRL. The reaching hand is guided to detach, move forward beneath the bars while avoiding the support-hand bar, and hook onto the target bar from above.
Name
Equation*
Weight †
1st
2nd
WGRL (Position)
rpos
0.64
0.8
WGRL (Progress)
rprog
4.0
5.0
WGRL (Success)
rsucc
10
–
Support-Hand Grasp
I{support-hand grasp}
0.5
0.5
Bars Reached
Nbar
2
0.3
TABLE III: Reward Definitions for Brachiation
Fig. 4: Reward field visualization for different definitions of rpos . (a) distance-only, (b) direction-only, (c) multiplicative distance–direction coupling, and (d) additive distance–direction formulation. The formulation (c) and (d) encourage the end-effector to approach the target from the direction of the previous waypoint.
Fig. 5: Learning curves for rpos definition ablation. The vertical axis plots the number of bars reached since the start of each episode, averaged over all parallel environments. Each curve represents the mean of three independent training runs. The right panel shows the mean and standard deviation at 8000 iterations. Reward formulations incorporating both distance and directional alignment yield higher performance and more consistent convergence result.
Fig. 6: Learning curves for reward component ablation in WGRL. The horizontal axis indicates training iterations, and the vertical axis shows the number of bars reached since the start of each episode, averaged over all parallel environments. Brachiation behavior emerges only when all three reward components are combined, or when paired with the success reward.
Fig. 7: Sim-to-Sim evaluation in MuJoCo under diverse monkey bars configurations. The learned policy is tested in three unseen environments: ( α ) geometrically perturbed bars, ( β ) a 5° inclined up–down course, and ( γ ) a circular course. The lower plots show the base link center position over time, demonstrating sustained forward progression despite environmental variations.
Fig. 8: Zero-shot Sim-to-Real brachiation on the life-sized dual-arm robot. Sequential snapshots show alternating reaching and hooking phases across four bars spaced at 400 mm intervals.
Fig. 9: Hardware experiment logs during zero-shot Sim-to-Real brachiation. The top plot shows the forward base link position and pitch angle, indicating forward progression through repeated swinging motions. The bottom plot presents the base energy. During hooking failures, swing momentum increases, resulting in progressively higher kinetic energy peaks that enable reattempts. Once stable dual-hand support is established, the mechanical energy decays and converges toward the target value.
Fig. 10: Failure recovery behavior during hardware brachiation. After an initial hooking failure, the robot increases swing amplitude to build momentum and successfully reattempts bar engagement, preventing fall and enabling continued progression.
Brachiation enables primates to move across overhead supports when ground paths are blocked, suggesting a complementary locomotion mode for robots operating in cluttered or hazardous environments. Bringing this capability to high-DoF humanoid robots is difficult because the controller must discover a long-horizon release-swing-capture sequence, coordinate alternating contacts with whole-body momentum, and act without reliable measurements of segment-relative displacement or hook-contact state. We present SwingBot, a learning framework for continuous humanoid brachiation with passive wrist hooks. SwingBot makes the task trainable by organizing learning around the structure of brachiation: biomimetic keyframes make rare release-swing-capture transitions reachable during early exploration, and recurrent privileged-state estimation provides compact position and contact latents for deployment. Hardware experiments demonstrate continuous bar traversal and robustness to payload, external disturbances and different bar spacings, showing that this formulation offers a practical route to whole-body robotic brachiation.
Yujie Xiong, Peng Zhai, Taixian Hou +6
College of Intelligent Robotics and Advanced Manufacturing, Fudan University
A walking robotic hand must use the same fingers to move its body, support its weight, and interact with the environment. We show how an anthropomorphic hand can learn these skills while retaining its finger design and position controller. Onboard power and computation make the platform self-contained. Our reinforcement learning approach accounts for the hand's unequal fingers, with training in a simulator calibrated from hardware measurements. In simulation, the hand moves faster with our reward formulation than with tuned rewards originally designed for quadrupeds. On hardware, task-specific policies enable untethered crawling, steering, and fall recovery. While supporting its own weight, the hand also executes successive keyboard commands without vision and pushes an object to targets using overhead visual feedback. These results demonstrate a compact mobile manipulator that reuses its fingers for locomotion and interaction, without a separate locomotion mechanism.
Amirhossein Kazemipour, Hehui Zheng, Robert Katzschmann
Soft Robotics Lab, ETH Zurich, Zurich, Switzerland
Learning-based approaches to locomotion have risen in popularity in recent years, showing the capability for complex legged locomotion and whole-body control. Reinforcement learning (RL), the primary learning-based approach for locomotion, often utilizes a high-performance simulation tool, providing a controlled and efficient training and development environment. However, policies that perform well in simulation frequently encounter unexpected challenges when deployed on a physical system, known as the sim-to-real gap. This work presents a robust RL locomotion framework capable of whole-body control. The proposed RL framework utilizes Nvidia's new set of simulation tools, Isaac Sim, and its companion RL framework, Isaac Lab, for training, achieving a zero-shot sim-to-real policy. The performance of our policy is validated on physical hardware using the Unitree Go1, with experimental results showing similar velocity tracking performance to the quadruped's integrated controller, with a greater ability to recover from large disturbances, and achieve linear velocities of 2.0 m/s and angular velocities of 1.8 rad/s.
Jordan Dowdy, Jean Chagas Vaz
Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA