cs.ROOct 11, 2026

Learning Safety-Constrained Whole-Body Loco-Manipulation for Lunar Surface Operations

Authors: Alvaro Belmonte-Baeza, Miguel Cazorla, Gabriel J. García, Carlos J. Pérez-Del-Pulgar, Jorge Pomares

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Robotic systems for lunar surface operations must act autonomously and preserve their own integrity in an environment where failures are unrecoverable and continuous supervision is impossible. This paper presents a Constrained Reinforcement Learning framework for quadrupedal mobile manipulators that learns a single whole-body policy for tracking a 6D end-effector pose specified in task space, with operational limits enforced during learning through a constrained learning mechanism. We introduce a reward formulation in which the transition between locomotion and manipulation emerges from the reward geometry rather than from an explicit gating mechanism, a stability term derived from the Gravito-Inertial Inclination Margin and motivated by the reduced-gravity dynamics of the lunar surface, and a design rule for reward regularization that follows from the multiplicative structure of the termination-based objective. The framework is evaluated in a lunar-analog simulation spanning three terrain morphologies at five difficulty levels under lunar gravity, with regolith contact properties calibrated to the JSC-1a simulant and varied from loose to dense. The learned policy attains mean position and orientation errors of 3.22 cm and 6.83 deg. under nominal conditions, retains this precision across regolith densities and over the operationally representative range of terrain difficulty, and reduces soft-constraint violation time by up to an order of magnitude relative to an unconstrained baseline while incurring no hard-constraint violations.

Figures & tables

Figure 1: Legged Mobile Manipulator in Lunar environment used in our work. The proposed system combines quadrupedal locomotion with a 6-DoF manipulator for whole-body control under low-gravity and irregular terrain conditions.
WorkTaskUnified6D task-ExplicitReducedRegolith
  WBCspaceconstraintsgravity 
Valsecchi et al. [36]Locomotion————✓
Kolvenbach et al. [16]Locomotion————✓
Fu et al. [9]Loco-manipulationPartial————
Ma et al. [20]Loco-manipulation—————
Ha et al. [10]Loco-manipulationPartial✓———
Table 1: Positioning of this work with respect to representative learning-based loco-manipulation and constrained locomotion methods. Unified WBC indicates a single policy controlling base and arm without a separate manipulation module; 6D task-space indicates end-effector pose commands expressed in an inertial frame rather than the robot body frame; Explicit constraints indicates safety limits enforced through a constrained formulation rather than reward shaping; Reduced gravity and Regolith indicate evaluation under lunar gravity and on compliant granular substrates, respectively.
Figure 2: Problem statement: The robot starts with its desired EE pose out of reach. It must locomote towards a vicinity of the target pose, and then coordinate both arm and legged platform to reach the commanded pose.
Figure 3: Overview of the proposed methodology.
ObservationExpressionDim.
Proprioceptive
Projected gravity vectorϕb\boldsymbol{\phi}_{b}3
Body linear velocityvb\boldsymbol{v}_{b}3
Body angular velocityωb\boldsymbol{\omega}_{b}3
Leg joint positionsqlegs\boldsymbol{q}_{legs}12
Arm joint positionsqarm\boldsymbol{q}_{arm}6
Table 2: Policy observations. Dimensions correspond to the κ=4\kappa=4 -legged platform with ζlegs=3\zeta_{legs}=3 and ζarm=6\zeta_{arm}=6 used in this work.
Figure 4: Terrains used in simulation to mimic lunar surface challenges.
Figure 5: Overview of the terrain levels used for experimentation.
Figure 6: Distribution of the position and orientation errors for the 5000 evaluated samples
Figure 7: Mean position and orientation errors across terrain-difficulty pairs.
Soil typeμs\mu_{s} / μd\mu_{d}Stiffness ( N/mN/m )Damping ( Ns/mNs/m )Challenge for Controller
Loose0.8 / 0.610,0002,000Slip & Sinkage. Leg may get trapped or slip away due to loose regolith.
Nominal (JSC-1a)1.1 / 0.950,0004,000Standard. The baseline soil properties
Dense1.35 / 1.01,000,00010,000Impact Shock. Hard contacts and impact forces. High friction might make the legs stumble.
Table 3: Soil type properties considered in our experimentation
Figure 8: Mean tracking error for different soil types across terrains and difficulties.
ConstraintBaselineNo ConstraintsNo cvc_{v}No cdevc_{dev}
cτ,hipc_{\tau,hip}0.0030.0030.0010.010
cτ,kneec_{\tau,knee}0.00613.920.0010.001
cq˙c_{\dot{q}}0.04.890.00.0
cvc_{v}11.5822.6621.7912.78
cdevc_{dev}6.4538.607.9452.91
cfstumblec_{f_{stumble}}0.03.420.00.0
Table 4: Episode violation time percentage of soft constraints for easy rough terrain for different policies.
ConstraintEasyAverageHard
cτ,hipc_{\tau,hip}0.0010.0030.005
cτ,kneec_{\tau,knee}0.0030.0070.022
cq˙c_{\dot{q}}0.00.00.0
cvc_{v}11.6613.7915.14
cdevc_{dev}5.225.958.16
cfstumblec_{f_{stumble}}0.00.00.0
Table 5: Episode violation time percentage of soft constraints for baseline policy across different difficulty Hills terrains.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

DescriptionSymbolValue
Term weights
Pose trackingωpose\omega_{pose}5.0
Stabilityωs\omega_{s}0.4
Leg powerωlegs\omega_{legs}0.5
Arm powerωarm\omega_{arm}0.2
Leg action smoothnessωact,legs\omega_{act,legs}0.2
Appendix
Table A.1 : Reward weights and hyperparameters
PositionOrientation
epose_{\text{pos}} [m]σpos\sigma_{\text{pos}}erote_{\text{rot}} [rad]σrot\sigma_{\text{rot}}
[0.75,∞)[0.75,\infty)0.500[1.00,∞)[1.00,\infty)0.50
[0.50,0.75)[0.50,0.75)0.400[0.50,1.00)[0.50,1.00)0.40
[0.25,0.50)[0.25,0.50)0.250[0.25,0.50)[0.25,0.50)0.25
[0.10,0.25)[0.10,0.25)0.150[0.10,0.25)[0.10,0.25)0.15
[0.05,0.10)[0.05,0.10)0.050[0.00,0.10)[0.00,0.10)0.05
Appendix
Table A.2 : Error-adaptive kernel width schedules
Command ranges
ComponentInitialFinal
xx [m][−0.30, 0.30][-0.30,\ 0.30][−0.30, 2.50][-0.30,\ 2.50]
yy [m][−0.30, 0.30][-0.30,\ 0.30][−1.00, 1.00][-1.00,\ 1.00]
zz [m][−0.10, 0.10][-0.10,\ 0.10][−0.35, 0.20][-0.35,\ 0.20]
Roll [rad]00
Pitch [rad][−π/4, π/2][-\pi/4,\ \pi/2]
Appendix
Table B.1 : Command sampling ranges and terrain generation parameters
ConstraintSymbolLimit
Soft
Hip and thigh joint torquecτ,hipc_{\tau,hip}23.5 N ⋅\cdot m
Knee joint torquecτ,kneec_{\tau,knee}35.0 N ⋅\cdot m
Leg joint velocitycq˙c_{\dot{q}}30.0 rad/s
Base linear velocitycvc_{v}0.25 m/s
Leg joint deviationcdevc_{dev}0.4 rad
Appendix
Table C.1 : Constraint limits. Soft constraints use pimax⁡=0.25p_{i}^{\max}=0.25 ; hard constraints use pimax⁡=1p_{i}^{\max}=1 .
ParameterValue
Policy and control
Hidden layers512, 256, 128
ActivationELU
Policy rate50 Hz
PD controller rate200 Hz
Episode duration10 s
Appendix
Table D.1 : Training configuration
More in this topic

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 735 papers

Explore similar work

CardsList
Jun 9, 2026

Locomotion analysis of a quadruped interacting with the lunar granular surface

Deploying legged robots in extra-terrestrial environments includes many challenges due to complex terrain interactions, energy, and thermal constraints. For effective mechanical design of a lunar exploration quadrupedal robot, careful consideration of motor torques, energy expenditure, and cost of transport is required. The lunar surface is composed of granular regolith, which impacts the locomotion of legged robots and their performance. Locomotion algorithms trained with rigid contact assumptions are also ineffective when applied to environments with soft contacts, such as granular surfaces, which can result in instability and poor tracking. In this report, the physical modelling of the granular lunar surface-robot foot contacts is applied to a simulation environment with locomotion trained using Reinforcement Learning. A comparison is conducted between the policy trained on rigid contact and soft contact environments, analysing the gait and locomotion performance metrics. The analysis demonstrates that soft contacts simulating regolith surfaces pose additional challenges for Reinforcement Learning based training, result in a qualitatively different gait, and increase the overall energy expenditure.

Yash J Vyas

Department of Industrial Engineering, University of Padua

Granular Terrain LocomotionRobotics SimulationGait AnalysisRoboticsTerrain-Aware Robot LocomotionRobot Locomotion+2
AskPDF
Oct 16, 2026

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator

Legged robots equipped with manipulators offer promising capabilities for loco-manipulation tasks in unstructured environments. However, achieving task-consistent end-effector behavior during locomotion remains challenging due to terrain-induced disturbances and the strong coupling between locomotion and manipulation, particularly for small-scale quadruped platforms. A reinforcement-learning based whole-body control framework is proposed for quadruped loco-manipulation on complex terrain. The proposed framework combines a hybrid task representation with task-prioritized whole-body learning to coordinate locomotion and manipulation while preserving task-consistent end-effector behavior. A lightweight quadruped manipulator platform is further developed to validate the proposed framework on a small-scale quadruped system. Real-world experiments demonstrate successful deployment on complex terrains such as slopes up to 30 degrees, stairs, and deformable terrain. These results demonstrate the feasibility of deploying learned whole-body control policy for quadruped loco-manipulation in real-world environments and highlight the importance of task-consistent whole-body coordination for terrain-adaptive loco-manipulation.

Yanzhou Jin, Fei Suo, Chang Liu +5

Graduate School of Engineering, The University of Tokyo, Japan. · Department of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya, 464-8603, Japan. · Department of Mathematics and Informatics, Ecole Centrale de Lyon, France.

Legged RoboticsWhole-Body ControlWhole-Body Robotic ManipulationTerrain-Aware Robot LocomotionRL for Legged Locomotion
AskPDF
May 23, 2026

Towards Low-Gravity Planetary Exploration using Reinforcement Learning for Walking, Jumping, and In-flight Attitude Control

This paper presents reinforcement learning (RL) policies for dynamic quadrupedal locomotion in planetary exploration scenarios. Building on a taskoptimized quadruped with a 5-bar leg design, we develop RL policies for walking, vertical jumping, forward jumping, and in-flight attitude control, explicitly tailored to the reduced gravity on Mars. These policies jointly enable such robots to overcome obstacles larger than themselves through coordinated jumping and precise in-flight reorientation for safe landings. We demonstrate Sim2Real transfer of the attitude control policy on the Olympus quadruped through single-axis reorientation tests, while all locomotion policies are validated in simulation. A complete Mars exploration mission scenario demonstrates coordinated policy deployment across challenging terrain. Experimental results show 90° attitude reorientation in 2.6 seconds, with simulations demonstrating 3.1 meter vertical jumps and 3.9 meter forward jumps under Martian gravity conditions. - Supplementary video: https://www.youtube.com/watch?v=qlSJ3P87A4A

Jørgen Anker Olsen, Kostas Alexis

The authors are with the Autonomous Robots Lab, NTNU, O.S. Bragstads Plass 2D, 7034, Trondheim, Norway.

Robotic ControlSpace RoboticsRobot LocomotionRL for Legged LocomotionQuadruped Locomotion
AskPDF