Robust Brachiation on a Life-Sized Dual-Arm Robot Using Waypoint-Guided Reinforcement Learning

Ayumu Iwata; Kento Kawaharazuka; Keita Yoneda; Takahiro Hattori; Kei Okada

cs.ROAug 18, 2026

Robust Brachiation on a Life-Sized Dual-Arm Robot Using Waypoint-Guided Reinforcement Learning

Authors: Ayumu Iwata, Kento Kawaharazuka, Keita Yoneda, Takahiro Hattori, Kei Okada

Organizations: Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan.

Abstract

Brachiation is a form of locomotion in which primates move primarily using their arms, enabling traversal in environments without footholds. However, this motion requires highly coordinated whole-body movement and precise timing control for bar grasping and release. As a result, achieving robust behavior on life-sized robotic platforms remains challenging. In this study, we present a reinforcement learning-based method to realize brachiation on a life-sized dual-arm robot. The core of the proposed approach is Waypoint-Guided Reinforcement Learning (WGRL), a learning framework for inducing non-linear and complex motions. For high-difficulty tasks where imitation learning data are unavailable, WGRL guides behavior acquisition by sparsely specifying waypoints for the end-effector trajectory, while whole-body motion is generated through reinforcement learning. In addition, by integrating the waypoint-following guidance with rewards based on task success and mechanical energy, and training in an environment designed for Sim-to-Real transfer, the proposed method achieves both forward progression and motion stability. The acquired behavior is evaluated through Sim-to-Sim experiments under monkey-bar environments with geometric variations and hardware experiments, confirming robust brachiation including failure recovery behavior. This study provides effective learning design guidelines for realizing arm-based locomotion on life-sized robotic hardware and expanding the traversable workspace of robots.

Figures & tables

Fig. 1: Brachiation motion of a life-sized dual-arm robot traversing four cylindrical bars (30 mm in diameter) arranged at 0.4 m intervals.

Fig. 2: Mechanical design of the life-sized dual-arm robot. Left: Overview of the robot, modeled after the upper body of an adult human. Right: Hook-shaped end-effector designed for secure bar suspension and robust brachiation.

Name	Symbol
Base (torso link) angular velocity	$\boldsymbol{\omega}_{b}\in\mathbb{R}^{3}$
Gravity direction (in torso local frame)	$\boldsymbol{g}_{b}\in\mathbb{R}^{3}$
All joint positions	$\boldsymbol{q}\in\mathbb{R}^{10}$
All joint velocities	$\dot{\boldsymbol{q}}\in\mathbb{R}^{10}$
Motion phase indicator	$s_{\mathrm{phase}}\in\{0,1\}$

TABLE I: Observation space used for the policies

Name	Symbol
Joint position history	$\boldsymbol{q}_{t-2:t}\in\mathbb{R}^{3\times 10}$
Joint velocity history	$\dot{\boldsymbol{q}}_{t-2:t}\in\mathbb{R}^{3\times 10}$
Joint position tracking error history	$\boldsymbol{e}_{t-2:t}\in\mathbb{R}^{3\times 10}$
IMU history	$\boldsymbol{o}^{\mathrm{IMU}}_{t-7:t}\in\mathbb{R}^{8\times 6}$
End-effector hooking contact state	$\boldsymbol{c}\in\{0,1\}^{2}$
End-effector contact forces	$\boldsymbol{f}_{\mathrm{c}}\in\mathbb{R}^{6}$

TABLE II: Privileged observation space used for the critic

Fig. 3: Waypoint design for inducing non-linear hooking trajectories in WGRL. The reaching hand is guided to detach, move forward beneath the bars while avoiding the support-hand bar, and hook onto the target bar from above.

Name	Equation*	Weight †
		1st	2nd
WGRL (Position)	$r_{\mathrm{pos}}$	0.64	0.8
WGRL (Progress)	$r_{\mathrm{prog}}$	4.0	5.0
WGRL (Success)	$r_{\mathrm{succ}}$	10	–
Support-Hand Grasp	$\mathbb{I}\{\text{support-hand grasp}\}$	0.5	0.5
Bars Reached	$N_{\mathrm{bar}}$	2	0.3

TABLE III: Reward Definitions for Brachiation

Fig. 4: Reward field visualization for different definitions of $r_{\mathrm{pos}}$ . (a) distance-only, (b) direction-only, (c) multiplicative distance–direction coupling, and (d) additive distance–direction formulation. The formulation (c) and (d) encourage the end-effector to approach the target from the direction of the previous waypoint.

Fig. 5: Learning curves for $r_{\mathrm{pos}}$ definition ablation. The vertical axis plots the number of bars reached since the start of each episode, averaged over all parallel environments. Each curve represents the mean of three independent training runs. The right panel shows the mean and standard deviation at 8000 iterations. Reward formulations incorporating both distance and directional alignment yield higher performance and more consistent convergence result.

Fig. 6: Learning curves for reward component ablation in WGRL. The horizontal axis indicates training iterations, and the vertical axis shows the number of bars reached since the start of each episode, averaged over all parallel environments. Brachiation behavior emerges only when all three reward components are combined, or when paired with the success reward.

Fig. 7: Sim-to-Sim evaluation in MuJoCo under diverse monkey bars configurations. The learned policy is tested in three unseen environments: ( $\alpha$ ) geometrically perturbed bars, ( $\beta$ ) a 5° inclined up–down course, and ( $\gamma$ ) a circular course. The lower plots show the base link center position over time, demonstrating sustained forward progression despite environmental variations.

Fig. 8: Zero-shot Sim-to-Real brachiation on the life-sized dual-arm robot. Sequential snapshots show alternating reaching and hooking phases across four bars spaced at 400 mm intervals.

Fig. 9: Hardware experiment logs during zero-shot Sim-to-Real brachiation. The top plot shows the forward base link position and pitch angle, indicating forward progression through repeated swinging motions. The bottom plot presents the base energy. During hooking failures, swing momentum increases, resulting in progressively higher kinetic energy peaks that enable reattempts. Once stable dual-hand support is established, the mechanical energy decays and converges toward the target value.

Fig. 10: Failure recovery behavior during hardware brachiation. After an initial hooking failure, the robot increases swing amplitude to build momentum and successfully reattempts bar engagement, preventing fall and enabling continued progression.

Robot Locomotion

Robot locomotion studies how robots move efficiently and reliably across varied terrains, environments, and movement modes.

See all 233 papers

SwingBot: Learning Whole-Body Brachiation for Humanoid Robots
Sep 9, 2026Yujie Xiong, Peng Zhai, Taixian Hou +6Humanoid Robot Locomotion Robot Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University
Fingers as Legs: Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand
Sep 15, 2026Amirhossein Kazemipour, Hehui Zheng, Robert KatzschmannRL for Robotics Mobile Manipulation
Soft Robotics Lab, ETH Zurich, Zurich, Switzerland
Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds
Jul 20, 2026Jordan Dowdy, Jean Chagas VazSim-to-Real Transfer Whole-Body Control
Department of Electrical and Computer Engineering, University of Louisville · Faculty of Electrical and Computer Engineering, University of Louisville, Louisville, KY, 40208, USA

Sep 9, 2026J/K move · Enter open · S save

SwingBot: Learning Whole-Body Brachiation for Humanoid Robots

Yujie Xiong, Peng Zhai, Taixian Hou, Quancheng Qian+5

College of Intelligent Robotics and Advanced Manufacturing, Fudan University

Humanoid Robot Locomotion Robot Learning

Open paper AskPDF arXiv

Brachiation enables primates to move across overhead supports when ground paths are blocked, suggesting a complementary locomotion mode for robots operating in cluttered or hazardous environments. Bringing this capability to high-DoF humanoid robots is difficult because the controller must discover a long-horizon release-swing-capture sequence, coordinate alternating contacts with whole-body momentum, and act without reliable measurements of segment-relative displacement or hook-contact state. We present SwingBot, a learning framework for continuous humanoid brachiation with passive wrist hooks. SwingBot makes the task trainable by organizing learning around the structure of brachiation: biomimetic keyframes make rare release-swing-capture transitions reachable during early exploration, and recurrent privileged-state estimation provides compact position and contact latents for deployment. Hardware experiments demonstrate continuous bar traversal and robustness to payload, external disturbances and different bar spacings, showing that this formulation offers a practical route to whole-body robotic brachiation.

Abstract

Robot Locomotion

Explore similar work

SwingBot: Learning Whole-Body Brachiation for Humanoid Robots

Fingers as Legs: Learning Self-Supported Locomotion and Manipulation with an Anthropomorphic Hand

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds