Dual-Quadruped Collaborative Transportation in Narrow Environments via Safe Reinforcement Learning
Authors: Zhezhi Lei, Zhihai Bi, Wenxin Wang, Jun Ma
Organizations: Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Advanced Remanufacturing & Technology Centre (ARTC), Agency for Science, Technology and Research (A*STAR), Singapore 637143 · Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology, Hong Kong SAR, China
Collaborative transportation, where multiple robots collaboratively transport a payload, has garnered significant attention in recent years. While ensuring safe and high-performance inter-robot collaboration is critical for effective task execution, it is difficult to pursue in narrow environments where the feasible region is extremely limited. To address this challenge, we propose a novel approach for dual-quadruped collaborative transportation via safe reinforcement learning (RL). Specifically, we model the task as a fully cooperative constrained Markov game, where collision avoidance is formulated as constraints. We introduce a cost-advantage decomposition method that enforces the sum of team constraints to remain below an upper bound, thereby guaranteeing task safety within an RL framework. Furthermore, we propose a constraint allocation method that assigns shared constraints to individual robots to maximize the overall task reward, encouraging autonomous task-assignment among robots, thereby improving collaborative task performance. Simulation and real-time experimental results demonstrate that the proposed approach achieves superior performance and a higher success rate in dual-quadruped collaborative transportation compared to existing methods.
Figures & tables
Fig. 1 : An overview of our method. 1) Task Environment: The robot team is required to collaboratively transport an object from the initial region to the target region while avoiding collisions. 2) Proposed Framework: We formulate the task as a constrained Markov decision process. For the total reward of the team, we adopt a safe RL approach to sequentially estimate the surrogate returns of R1 and R2, thereby determining the contribution of each member to the overall reward. For the total cost of the team, we employ a constraint allocation method to assign individual cost budgets to each team member. Robots R1 and R2 execute independent policies to accomplish the task in a fully distributed manner. 3) Training Strategy: We use two separate critics to estimate the expected reward and cost, respectively. We adopt a Lagrangian approach in which the total advantage is computed by combining the reward and cost advantages. Then, the residual between the cost and the budget is used to update the Lagrange multiplier. Lastly, the updated policy is subsequently deployed in the simulation environment to collect data for the next iteration.
Method
Explicit Constraint
Constraint Allocation
Independent Policies
Lagrangian
MAPPO
✗
–
✗
–
HAPPO
✗
–
✓
–
UCA
✓
✗
✓
✓
MACPO
✓
✓
✗
✓
Ours
✓
✓
✓
✓
TABLE I : Comparison of Baseline Methods
Fig. 2 : Average episode reward (left) and cost (right) during training. Our method (red curve) achieves high reward while maintaining a low cost, suggesting that it effectively balances safety and efficiency.
Method
Collision Rate [%]
Arrival Rate [%]
Trajectory Straightness
Time Consumption [s]
MAPPO
90.0
10.0
–
–
HAPPO
83.3
16.7
–
–
UCA
6.7
53.3
0.68
26.8
MACPO
0.0
6.7
0.74
32.0
Ours
0.0
100.0
0.95
18.3
TABLE II : Safety and Collaboration Efficiency of Different Methods in the Gate Scenario over 30 Trials ( Trajectory Straightness=Trajectory LengthDisplacement Along Goal Direction )
Fig. 3 : (a): Parameters of the gate scenario. The robot team is required to traverse the environment from the initial region to the target region without collisions. (b): The robot team successfully reaches the target region within the specified time horizon. The red line denotes the trajectory of the payload. (c) & (d): The robot team fails to reach the target region within the specified time horizon. The payload trajectory is distorted. (e) & (f): The robots collide with obstacles during navigation.
Fig. 4 : Robot team collaboration for navigating the gate scenario. From t1 to t3 , the team collaboratively adjusts its formation, and then passes through the narrow region in a front–rear formation.
Fig. 5 : Robot team collaboration in a long and narrow corridor scenario. Robot R1 continuously follows Robot R2 to maintain the formation. From t2 to t4 , the team changes its formation to pass through the corner.
Fig. 6 : Robot team collaboration in a forest scenario. From t1 to t2 , when encountering a junction, the two robots make identical action decisions to avoid conflicts.
Robotic collaborative carrying could greatly benefit human activities like warehouse and construction site management. However, coordinating the simultaneous motion of multiple robots represents a significant challenge. Existing works primarily focus on obstacle-free environments, making them unsuitable for most real-world applications. Works that account for obstacles, either overfit to a specific terrain configuration or rely on pre-recorded maps combined with path planners to compute collision-free trajectories. This work focuses on two quadrupedal robots mechanically connected to a carried object. We propose a Reinforcement Learning (RL)-based policy that enables tracking a commanded velocity direction while avoiding collisions with nearby obstacles using only onboard sensing, eliminating the need for precomputed trajectories and complete map knowledge. Our work presents a hierarchical architecture, where a perceptive high-level object-centric policy commands two pretrained locomotion policies. Additionally, we employ a game-inspired curriculum to increase the complexity of obstacles in the terrain progressively. We validate our approach on two quadrupedal robots connected to a bar via spherical joints, benchmarking it against optimization-based and decentralized RL baselines. Our hardware experiments demonstrate the ability of our system to locomote in unknown environments without the need for a map or a path planner. The video of our work is available in the multimedia material.
Francesca Bray, Simone Tolomei, Andrei Cramariuc +2
Robotic Systems Lab, ETH Zurich. · Centro di Ricerca “Enrico Piaggio”, and Dipartimento di Ingegneria dell’Informazione, Universit`a di Pisa, Pisa, Italy.
This paper presents a hierarchical density-based model predictive control framework for safe collaborative manipulation by multiple quadrupedal robots. The framework enables a team of robots to push a shared object to a desired pose using only the initial and goal poses, without requiring a precomputed reference trajectory. A centralized box-level MPC optimizes contact forces while enforcing a control-density constraint for goal convergence and obstacle avoidance. Each robot then solves its own distributed robot-level whole-body MPC, under a stated shared-information assumption, to track its moving contact location while accounting for static obstacles and the time-varying positions of neighboring robots. The approach is evaluated in MuJoCo using whole-body contact dynamics for two and three Unitree Go2 quadrupeds collaboratively pushing rigid objects through narrow passages. Comparisons with matched Control Barrier Function and RRT* based tracking baselines demonstrate the effectiveness of the proposed density-based formulation for push-only, force- and torque-coupled manipulation tasks. Implementation videos are available at https://jaggu2606.github.io/go2-density-mpc-pushing/
Jagannath Prasad Sahoo, Sriram S. K. S. Narayanan, Umesh Vaidya
Center for Artificial Intelligence and Robotics, Indian Institute of Technology Mandi, India · Department of Mechanical Engineering, Clemson University, Clemson, SC 29630, USA
We introduce HCLM, a hierarchical framework for general-purpose cooperative loco-manipulation with dual quadrupedal systems. Coordinating multi-robot collaborative manipulation across floating bases is highly challenging due to the conflicting demands of spatial coordination, robust locomotion, and closed-chain physical interactions. To resolve this, our architecture systematically decouples high-level collaborative reasoning from low-level robust motion execution. At the high level, a centralized Joint Diffusion Policy leverages an SE(3)-invariant task-space representation to learn coordinate-agnostic spatial coordination patterns. To translate these frame-agnostic references into physical motion, a task-centric hybrid Whole-Body Controller synergizes a proactive kinematic Model Predictive Control for collision-free velocity distribution with a reactive execution layer. Crucially, this reactive layer guarantees rapid responsiveness for precise end-effector tracking, while concurrently integrating active force regulation via a cooperative admittance scheme to safely resolve kinematic conflicts and strictly regulate internal stresses during closed-chain interactions. We validate the framework across progressively challenging simulated scenarios, including cooperative carrying, packing and handovers, and successfully deploy the latter in the real world. The results demonstrate reliable task execution, strict configuration agnosticism, and exceptional resilience against severe physical perturbations, offering a highly robust pathway for multi-robot embodied coordination.
Qixuan Li, Chen Le, Jincheng Yu +1
Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · Department of Electronic Engineering, and the Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing, China.