cs.ROMar 24, 2026

Path Planning and Reinforcement Learning-Driven Control of On-Orbit Free-Flying Multi-Arm Robots

Authors: Álvaro Belmonte-Baeza, José Luis Ramón, Leonard Felicetti, Miguel Cazorla, Jorge Pomares

Organizations: University of Alicante, Spain · Cranfield University, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents a hybrid approach that integrates trajectory optimization (TO) and reinforcement learning (RL) for motion planning and control of free-flying multi-arm robots in on-orbit servicing scenarios. The proposed system integrates TO for generating feasible, efficient paths while accounting for dynamic and kinematic constraints, and RL for adaptive trajectory tracking under uncertainties. The multi-arm robot design, equipped with thrusters for precise body control, enables redundancy and stability in complex space operations. TO optimizes arm motions and thruster forces, reducing reliance on the arms for stabilization and enhancing maneuverability. RL further refines this by leveraging model-free control to adapt to dynamic interactions and disturbances. The experimental results validated through comprehensive simulations demonstrate the effectiveness and robustness of the proposed hybrid approach. Two case studies are explored: surface motion with initial contact and a free-floating scenario requiring surface approximation. In both cases, the hybrid method outperforms traditional strategies. In particular, the thrusters notably enhance motion smoothness, safety, and operational efficiency. The RL policy effectively tracks TO-generated trajectories, handling high-dimensional action spaces and dynamic mismatches. This integration of TO and RL combines the strengths of precise, task-specific planning with robust adaptability, ensuring high performance in the uncertain and dynamic conditions characteristic of space environments. By addressing challenges such as motion coupling, environmental disturbances, and dynamic control requirements, this framework establishes a strong foundation for advancing the autonomy and effectiveness of space robotic systems.

Figures & tables

Figure 1: Overview of the hybrid approach proposed in this paper. TO-based motion planning provides a reference motion for the robot with constraints specific to an on-orbit application, while the RL-driven motion control robustly and adaptively tracks this reference motion.
Figure 2: Visualization of the described system architecture and on-orbit operational environment.
ConstraintValue
Starting robot pose[db0,ϕb0]\left[\boldsymbol{d}_{b0},\boldsymbol{\phi}_{b0}\right]
Target robot pose[dbd,ϕbd]\left[\boldsymbol{d}_{bd},\boldsymbol{\phi}_{bd}\right]
Maneuver duration∑j=1NiΔTi,j=T\sum_{j=1}^{N_{i}}\Delta T_{i,j}=T
Dynamics and Kinematicseqs. 6 , 3 and 4
Contact phase (t∈Ci)(t\in C_{i}) 
End-effector is fixedp˙i(t∈Ci)=0\boldsymbol{\dot{p}}_{i}(t\in C_{i})=\boldsymbol{0}
Table 1: System Contraints for the OCP
Figure 3: Overview of the Reinforement Learning-Driven control methodology . The policy network receives as input an observed robot state, and outputs desired joint positions and thruster forces for the robot. These are fed to the physics simulation, yielding a new state. The transition is used to update the network parameters via an RL algorithm of choice, and the loop is repeated until convergence.
Observation nameExpression
Base pose and velocities[db,ϕb,vb,ωb]\left[\boldsymbol{d}_{b},\boldsymbol{\phi}_{b},\boldsymbol{v}_{b},\boldsymbol{\omega}_{b}\right]
Desired base pose[db∗,ϕb∗]\left[\boldsymbol{d}_{b}^{\ast},\boldsymbol{\phi}_{b}^{\ast}\right]
Joint positions and velocities[q,q˙]\left[\boldsymbol{q},\boldsymbol{\dot{q}}\right]
Previous actionsat−1=[qt−1∗,uth,t−1]\boldsymbol{a}_{t-1}=[\boldsymbol{q}_{t-1}^{\ast},\boldsymbol{u}_{th,t-1}]
Current end-effector posepi\boldsymbol{p}_{i}
Desired end-effector posepi∗\boldsymbol{p}_{i}^{\ast}
Table 2: Policy Observations
Figure 4: Multi-arm robot used in our experiments.
Figure 5: Position and orientation of the robot base during the trajectory
Figure 6: Robot base thruster force and acceleration with and without thrusters
Figure 7: End-effector interaction forces
Figure 8: End-effector position during the maneuver
Figure 9: Case 1.1: Tracking performance using the RL-driven control policy
Figure 10: Case 1.2: Tracking performance using the RL-driven control policy
Figure 11: End-effector position and interaction forces considering 2, 3, 4 a 5 polynomials per segment.
Figure 12: End-effector position and interaction forces considering 4 and 6 contacts (three polynomials per segment).
Figure 13: Position and orientation of the robot base during the trajectory for the second scenario.
Figure 14: Robot base thruster forces during the second case trajectory.
Figure 15: End-effector positions and forces during the maneuver
Figure 16: Case 2: Tracking performance using RL-driven control policy
Body IDCase 1 Error (m)Case 2 Error (m)
Base1.5×10−21.5\times 10^{-2}2.5×10−22.5\times 10^{-2}
LF arm6.7×10−36.7\times 10^{-3}6.2×10−36.2\times 10^{-3}
LH arm1.1×10−21.1\times 10^{-2}1.3×10−21.3\times 10^{-2}
RF arm7.3×10−37.3\times 10^{-3}7.1×10−37.1\times 10^{-3}
RH arm1.2×10−21.2\times 10^{-2}1.5×10−21.5\times 10^{-2}
Table 3: Mean tracking errors
ControllerMean error (m)
PD Controller2.7×10−22.7\times{10^{-2}}
Velocity-based2.3×10−22.3\times{10^{-2}}
Acceleration-based1.2×10−21.2\times{10^{-2}}
Force-based controller1.4×10−21.4\times{10^{-2}}
Optimal control approach9.3×10−39.3\times{10^{-3}}
RL-driven (ours)6.7×10−3\mathbf{6.7\times{10^{-3}}}
Table 4: Control performance comparative
LinkMassLengthIxx / IyyIzz
Base4.00.0900.00440.0072
Shoulder3.70.1620.01030.0067
Upper Arm8.3930.4250.22690.0151
Forearm2.2750.3920.04940.0041
Wrist 11.2190.1330.11120.2194
Wrist 21.2190.1330.11120.2194
Table 5: Main dynamic parameters of the robot arms. Mass is in kg, length in meters, and all inertia terms are expressed in kg⋅m2\boldsymbol{kg\cdot m^{2}}
Reward term weightValue
wbodyw_{body}20.020.0
weew_{ee}15.015.0
ϵ\epsilon10−510^{-5}
wpoww_{pow}−2.5×10−2-2.5\times{10^{-2}}
waccw_{acc}−1×10−6-1\times{10^{-6}}
wbaccw_{bacc}−1×10−2-1\times{10^{-2}}
Table 6: Reward weights and regularization terms for training our RL policy
ParameterValue
Initial body positionU(−0.2,0.2)m\mathcal{U}(-0.2,0.2)m
Initial joint positionsU(−0.1,0.1)rad\mathcal{U}(-0.1,0.1)rad
System massU(−0.1m,0.1m)kg\mathcal{U}(-0.1m,0.1m)kg
Table 7: Domain randomization values. All values represent the randomization added to the default value of the parameter (e.g. A random initialization of −0.1m-0.1m kg means that the mass is 90%90\% of the default mass).
ObservationNoise
db\boldsymbol{d}_{b}U(−0.025,0.025)m\mathcal{U}(-0.025,0.025)m
ϕb\boldsymbol{\phi}_{b}U(−0.02,0.02)rad\mathcal{U}(-0.02,0.02)rad
vb\boldsymbol{v}_{b}U(−0.05,0.05)m/s\mathcal{U}(-0.05,0.05)m/s
ωb\boldsymbol{\omega}_{b}U(−0.05,0.05)rad/s\mathcal{U}(-0.05,0.05)rad/s
q\boldsymbol{q}U(−0.02,0.02)rad\mathcal{U}(-0.02,0.02)rad
q˙\boldsymbol{\dot{q}}U(−0.1,0.1)rad/s\mathcal{U}(-0.1,0.1)rad/s
Table 8: Observation noise injection. All values represent the nosie added to the default value of the parameter
ParameterValue
Batch size196608 (8192x24)
Mini-batch size49152 (8192x6)
Number of epochs8
Clip range0.2
Entropy coefficient0.01
Discount factor γ\gamma0.99
Table 9: PPO Hyperparamenters
More in this topic

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Explore similar work

CardsList
Sep 17, 2026

Resilient Motion Planning for Free-Flying Space Robots under Actuator Failures

Free-flying robots rely on multiple thrusters to maneuver in space. If one or more of these thrusters fail, the robot may lose control authority and risk mission failure. At the same time, their free-flying nature implies that, even in the absence of actuation, they continue along (locally) straight-line trajectories. In this work we present a probabilistic, proactive, motion planning framework that explicitly accounts for actuator failures in space. We model actuator failure modes as a Markov chain and propagate the probability of successfully reaching the goal along the planning horizon. Precomputed reachable sets evaluate the robot's capabilities of reaching waypoints under potential failures and an RRT∗^*-based planner concatenates these waypoints. The resulting algorithm maximizes the overall target-reaching probability, providing maximally resilient motion plans utilizing free-flying properties. We validate our approach experimentally on a physical free-flyer platform with injected actuator failures.

Nicolas de Maddalena, Joris Verhagen, Jana Tumova

Institute for Dynamic Systems and Control, ETH Zürich, 8092 Zürich, ZH, Switzerland · Division of Robotics, Perception and Learning, KTH Royal Institute of Technology, Stockholm, Sweden, and also affiliated with Digital Futures.

Space RoboticsRoboticsMotion PlanningRobot Motion PlanningSampling-Based Motion Planning
AskPDF
May 26, 2026

TCBiRRT: Rapid Motion Planning for Tightly Coupled Dual-arm Space Manipulator Using Task-space Random Expansion

Planning the motion path for a tightly coupled dual-arm space manipulator under closed-chain constraints is a fundamental yet challenging problem in on-orbit assembly of large-scale space structures. The closed-chain constraints significantly reduce the feasible configuration space, making it difficult for existing planners to efficiently generate collision-free motions, especially in cluttered environments. To address this issue, this paper proposes a task-space constrained bidirectional rapidly-exploring random tree algorithm, termed TCBiRRT. Unlike conventional methods that operate in the high-dimensional configuration space, the proposed approach performs random sampling and node expansion directly in the task space defined by the manipulated object pose. A task-space node expansion strategy is developed to generate candidate object motions, which are then mapped to continuous joint paths using a path inverse kinematics algorithm. The method is further integrated with a bidirectional RRT framework and a regrasp mechanism to efficiently connect two random trees. Extensive simulations are conducted in representative on-orbit assembly scenarios with varying levels of environmental complexity. The results demonstrate that TCBiRRT achieves significantly higher success rates and orders-of-magnitude improvements in planning time compared to state-of-the-art planners. The proposed method provides an efficient and robust solution for motion planning of tightly coupled dual-arm space manipulators.

Jiawei Zhang, Xinhao Miao, Jifeng Guo +2

Harbin Institute of Technology, Xidazhi Street 92, 150001, Harbin, China

Space RoboticsBimanual Robotic ManipulationRoboticsMotion PlanningSampling-Based Motion Planning
AskPDF
Mar 29, 2026

Learning Smooth and Robust Space Robotic Manipulation of Dynamic Target via Inter-frame Correlation

On-orbit servicing represents a critical frontier in future aerospace engineering, with the manipulation of dynamic non-cooperative targets serving as a key technology. In microgravity environments, objects are typically free-floating, lacking the support and frictional constraints found on Earth, which significantly escalates the complexity of tasks involving space robotic manipulation. Conventional planning and control-based methods are primarily limited to known, static scenarios and lack real-time responsiveness. To achieve precise robotic manipulation of dynamic targets in unknown and unstructured space environments, this letter proposes a data-driven space robotic manipulation approach that integrates historical temporal information and inter-frame correlation mechanisms. By exploiting the temporal correlation between historical and current frames, the system can effectively capture motion features within the scene, thereby producing stable and smooth manipulation trajectories for dynamic targets. To validate the effectiveness of the proposed method, we developed a ground-based experimental platform consisting of a PIPER X robotic arm and a dual-axis linear stage, which accurately simulates micro-gravity free-floating motion in a 2D plane.

Siyi Lang, Hongyi Gao, Yingxin Zhang +5

Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University, Xi’an 710072, China · School of Automation, Northwestern Polytechnical University, Xi’an 710072, China · School of Aeronautics and Astronautics, Sichuan University, Chengdu 610065, China

Space RoboticsDynamic Robotic ManipulationRobotic ControlVision-Based Robot Control
AskPDF