Learning Safety-Constrained Whole-Body Loco-Manipulation for Lunar Surface Operations

Alvaro Belmonte-Baeza; Miguel Cazorla; Gabriel J. García; Carlos J. Pérez-Del-Pulgar; Jorge Pomares

cs.ROOct 11, 2026

Learning Safety-Constrained Whole-Body Loco-Manipulation for Lunar Surface Operations

Authors: Alvaro Belmonte-Baeza, Miguel Cazorla, Gabriel J. García, Carlos J. Pérez-Del-Pulgar, Jorge Pomares

Abstract

Robotic systems for lunar surface operations must act autonomously and preserve their own integrity in an environment where failures are unrecoverable and continuous supervision is impossible. This paper presents a Constrained Reinforcement Learning framework for quadrupedal mobile manipulators that learns a single whole-body policy for tracking a 6D end-effector pose specified in task space, with operational limits enforced during learning through a constrained learning mechanism. We introduce a reward formulation in which the transition between locomotion and manipulation emerges from the reward geometry rather than from an explicit gating mechanism, a stability term derived from the Gravito-Inertial Inclination Margin and motivated by the reduced-gravity dynamics of the lunar surface, and a design rule for reward regularization that follows from the multiplicative structure of the termination-based objective. The framework is evaluated in a lunar-analog simulation spanning three terrain morphologies at five difficulty levels under lunar gravity, with regolith contact properties calibrated to the JSC-1a simulant and varied from loose to dense. The learned policy attains mean position and orientation errors of 3.22 cm and 6.83 deg. under nominal conditions, retains this precision across regolith densities and over the operationally representative range of terrain difficulty, and reduces soft-constraint violation time by up to an order of magnitude relative to an unconstrained baseline while incurring no hard-constraint violations.

Figures & tables

Figure 1: Legged Mobile Manipulator in Lunar environment used in our work. The proposed system combines quadrupedal locomotion with a 6-DoF manipulator for whole-body control under low-gravity and irregular terrain conditions.

Work	Task	Unified	6D task-	Explicit	Reduced	Regolith
		WBC	space	constraints	gravity
Valsecchi et al. [36]	Locomotion	—	—	—	—	✓
Kolvenbach et al. [16]	Locomotion	—	—	—	—	✓
Fu et al. [9]	Loco-manipulation	Partial	—	—	—	—
Ma et al. [20]	Loco-manipulation	—	—	—	—	—
Ha et al. [10]	Loco-manipulation	Partial	✓	—	—	—

Table 1: Positioning of this work with respect to representative learning-based loco-manipulation and constrained locomotion methods. Unified WBC indicates a single policy controlling base and arm without a separate manipulation module; 6D task-space indicates end-effector pose commands expressed in an inertial frame rather than the robot body frame; Explicit constraints indicates safety limits enforced through a constrained formulation rather than reward shaping; Reduced gravity and Regolith indicate evaluation under lunar gravity and on compliant granular substrates, respectively.

Figure 2: Problem statement: The robot starts with its desired EE pose out of reach. It must locomote towards a vicinity of the target pose, and then coordinate both arm and legged platform to reach the commanded pose.

Figure 3: Overview of the proposed methodology.

Observation	Expression	Dim.
Proprioceptive
Projected gravity vector	$\boldsymbol{\phi}_{b}$	3
Body linear velocity	$\boldsymbol{v}_{b}$	3
Body angular velocity	$\boldsymbol{\omega}_{b}$	3
Leg joint positions	$\boldsymbol{q}_{legs}$	12
Arm joint positions	$\boldsymbol{q}_{arm}$	6

Table 2: Policy observations. Dimensions correspond to the

\kappa=4

-legged platform with

\zeta_{legs}=3

and

\zeta_{arm}=6

used in this work.

Figure 4: Terrains used in simulation to mimic lunar surface challenges.

Figure 5: Overview of the terrain levels used for experimentation.

Panel of Figure 6 — Figure 6: Distribution of the position and orientation errors for the 5000 evaluated samples

Figure 7: Mean position and orientation errors across terrain-difficulty pairs.

Soil type	$\mu_{s}$ / $\mu_{d}$	Stiffness ( $N/m$ )	Damping ( $Ns/m$ )	Challenge for Controller
Loose	0.8 / 0.6	10,000	2,000	Slip & Sinkage. Leg may get trapped or slip away due to loose regolith.
Nominal (JSC-1a)	1.1 / 0.9	50,000	4,000	Standard. The baseline soil properties
Dense	1.35 / 1.0	1,000,000	10,000	Impact Shock. Hard contacts and impact forces. High friction might make the legs stumble.

Table 3: Soil type properties considered in our experimentation

Figure 8: Mean tracking error for different soil types across terrains and difficulties.

Constraint	Baseline	No Constraints	No $c_{v}$	No $c_{dev}$
$c_{\tau,hip}$	0.003	0.003	0.001	0.010
$c_{\tau,knee}$	0.006	13.92	0.001	0.001
$c_{\dot{q}}$	0.0	4.89	0.0	0.0
$c_{v}$	11.58	22.66	21.79	12.78
$c_{dev}$	6.45	38.60	7.94	52.91
$c_{f_{stumble}}$	0.0	3.42	0.0	0.0

Table 4: Episode violation time percentage of soft constraints for easy rough terrain for different policies.

Constraint	Easy	Average	Hard
$c_{\tau,hip}$	0.001	0.003	0.005
$c_{\tau,knee}$	0.003	0.007	0.022
$c_{\dot{q}}$	0.0	0.0	0.0
$c_{v}$	11.66	13.79	15.14
$c_{dev}$	5.22	5.95	8.16
$c_{f_{stumble}}$	0.0	0.0	0.0

Table 5: Episode violation time percentage of soft constraints for baseline policy across different difficulty Hills terrains.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Description	Symbol	Value
Term weights
Pose tracking	$\omega_{pose}$	5.0
Stability	$\omega_{s}$	0.4
Leg power	$\omega_{legs}$	0.5
Arm power	$\omega_{arm}$	0.2
Leg action smoothness	$\omega_{act,legs}$	0.2

Appendix

Table A.1 : Reward weights and hyperparameters

Position		Orientation
$e_{\text{pos}}$ [m]	$\sigma_{\text{pos}}$	$e_{\text{rot}}$ [rad]	$\sigma_{\text{rot}}$
$[0.75,\infty)$	0.500	$[1.00,\infty)$	0.50
$[0.50,0.75)$	0.400	$[0.50,1.00)$	0.40
$[0.25,0.50)$	0.250	$[0.25,0.50)$	0.25
$[0.10,0.25)$	0.150	$[0.10,0.25)$	0.15
$[0.05,0.10)$	0.050	$[0.00,0.10)$	0.05

Appendix

Table A.2 : Error-adaptive kernel width schedules

Command ranges
Component	Initial	Final
$x$ [m]	$[-0.30,\ 0.30]$	$[-0.30,\ 2.50]$
$y$ [m]	$[-0.30,\ 0.30]$	$[-1.00,\ 1.00]$
$z$ [m]	$[-0.10,\ 0.10]$	$[-0.35,\ 0.20]$
Roll [rad]	$0$
Pitch [rad]	$[-\pi/4,\ \pi/2]$

Appendix

Table B.1 : Command sampling ranges and terrain generation parameters

Constraint	Symbol	Limit
Soft
Hip and thigh joint torque	$c_{\tau,hip}$	23.5 N $\cdot$ m
Knee joint torque	$c_{\tau,knee}$	35.0 N $\cdot$ m
Leg joint velocity	$c_{\dot{q}}$	30.0 rad/s
Base linear velocity	$c_{v}$	0.25 m/s
Leg joint deviation	$c_{dev}$	0.4 rad

Appendix

Table C.1 : Constraint limits. Soft constraints use

p_{i}^{\max}=0.25

; hard constraints use

p_{i}^{\max}=1

Parameter	Value
Policy and control
Hidden layers	512, 256, 128
Activation	ELU
Policy rate	50 Hz
PD controller rate	200 Hz
Episode duration	10 s

Appendix

Table D.1 : Training configuration

Robotics

Robotics studies the design and control of machines that perceive, move, and act in physical environments.

See all 735 papers

Locomotion analysis of a quadruped interacting with the lunar granular surface
Jun 9, 2026Yash J VyasGranular Terrain Locomotion Robotics Simulation
Department of Industrial Engineering, University of Padua
Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator
Oct 16, 2026Yanzhou Jin, Fei Suo, Chang Liu +5Legged Robotics Whole-Body Control
Graduate School of Engineering, The University of Tokyo, Japan. · Department of Micro-Nano Mechanical Science and Engineering, Nagoya University, Nagoya, 464-8603, Japan. · Department of Mathematics and Informatics, Ecole Centrale de Lyon, France.
Towards Low-Gravity Planetary Exploration using Reinforcement Learning for Walking, Jumping, and In-flight Attitude Control
May 23, 2026Jørgen Anker Olsen, Kostas AlexisRobotic Control Space Robotics
The authors are with the Autonomous Robots Lab, NTNU, O.S. Bragstads Plass 2D, 7034, Trondheim, Norway.

Jun 9, 2026J/K move · Enter open · S save

Locomotion analysis of a quadruped interacting with the lunar granular surface

Yash J Vyas

Department of Industrial Engineering, University of Padua

Granular Terrain Locomotion Robotics Simulation Gait Analysis Robotics Terrain-Aware Robot Locomotion Robot Locomotion+2

Open paper AskPDF arXiv

Deploying legged robots in extra-terrestrial environments includes many challenges due to complex terrain interactions, energy, and thermal constraints. For effective mechanical design of a lunar exploration quadrupedal robot, careful consideration of motor torques, energy expenditure, and cost of transport is required. The lunar surface is composed of granular regolith, which impacts the locomotion of legged robots and their performance. Locomotion algorithms trained with rigid contact assumptions are also ineffective when applied to environments with soft contacts, such as granular surfaces, which can result in instability and poor tracking. In this report, the physical modelling of the granular lunar surface-robot foot contacts is applied to a simulation environment with locomotion trained using Reinforcement Learning. A comparison is conducted between the policy trained on rigid contact and soft contact environments, analysing the gait and locomotion performance metrics. The analysis demonstrates that soft contacts simulating regolith surfaces pose additional challenges for Reinforcement Learning based training, result in a qualitatively different gait, and increase the overall energy expenditure.

Abstract

Robotics

Explore similar work

Locomotion analysis of a quadruped interacting with the lunar granular surface

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator

Towards Low-Gravity Planetary Exploration using Reinforcement Learning for Walking, Jumping, and In-flight Attitude Control