Phase-Gated Constrained Reinforcement Learning for End-to-End Capture and Detumbling of Non-Cooperative Targets

Álvaro Belmonte-Baeza; Celia Redondo-Verdú; José L. Ramón; Miguel Cazorla; Leonard Felicetti; Jorge Pomares

cs.ROOct 11, 2026

Phase-Gated Constrained Reinforcement Learning for End-to-End Capture and Detumbling of Non-Cooperative Targets

Authors: Álvaro Belmonte-Baeza, Celia Redondo-Verdú, José L. Ramón, Miguel Cazorla, Leonard Felicetti, Jorge Pomares

Organizations: Department of Computer Science and Artificial Intelligence, University of Alicante, Spain. · Department of Physics, Systems Engineering and Signal Theory, University of Alicante, Spain. · Centre for Space Systems, Cranfield University, Cranfield, UK.

Abstract

Active debris removal requires a servicer spacecraft to approach a non-cooperative tumbling object, capture it, and arrest its rotation. This sequence is conventionally divided among phase-specific controllers, each depending on knowledge of the target's inertia, a dedicated grappling fixture, or a hand-designed switch at the contact-rich seam where such pipelines are most fragile. We show that a single constrained reinforcement learning policy can execute the entire mission, driving a dual-arm free-flying servicer from free-space closing through friction-based fixture-free capture to full detumbling of a target whose mass and inertia are unknown and whose state is observed only through noisy estimates. The enabling mechanism is a smooth proximity gate applied at three loci of a constrained Markov decision process. In the reward it induces the phase structure rather than imposing it; in the action space it forecloses a degenerate grab-and-pull exploit as a hard guarantee; and in the constraint set it resolves a normalization failure of constraint-based learning, in which a constraint spanning phases that load it unequally is rendered nearly inert in the phase where precision matters most. Both the failure mode and the remedy apply to any multi-phase problem of this kind. Over 1000 randomized trials the policy succeeds in 99.5% while holding its contact-force, actuator, base-reaction and collision limits throughout, transfers zero-shot to unseen geometries and centre-of-mass offsets, and extrapolates across a tenfold range of target mass.

Figures & tables

Fig. 1 : The three phases of the ADR mission . The servicer closes the separation under thruster force $\bm{f}_{\mathrm{thr}}$ and reaction torque $\bm{\tau}_{\mathrm{rw}}$ with the arms stowed; the arms are then actuated by joint torques $\bm{\tau}$ to press two disk-shaped end effectors on opposing faces of the tumbling target, and the resulting contact wrenches $\bm{f}_{c,\mathrm{L}}$ , $\bm{f}_{c,\mathrm{R}}$ dissipate its angular momentum $\bm{\omega}_{t}$ . The phases are named for exposition only: no phase index is supplied to the controller and no switching condition is evaluated at run time.

Quantity	Symbol	Value	Reference
Relative position	$\sigma_{p}$	$1\times 10^{-2}~\mathrm{m}$	[ 26 ]
Relative attitude	$\sigma_{\theta}$	$1.75\times 10^{-2}~\mathrm{rad}$	[ 26 , 45 ]
Relative linear vel.	$\sigma_{v}$	$5\times 10^{-3}~\mathrm{m}\,\mathrm{s}^{-1}$	[ 27 ]
Target angular vel.	$\sigma_{\omega}$	$1.7\times 10^{-3}~\mathrm{rad}\,\mathrm{s}^{-1}$	[ 27 ]

TABLE I : Relative-state estimation noise (standard deviations).

Fig. 2 : Method overview . Both gates are computed from the state, not from the policy: the proximity gate $g_{p}$ modulates the reward, the manipulator command and the constraint terms, while the contact gate $g_{c}$ cascades onto the stabilization rewards. The reward and constraint machinery acts only during training; at deployment the phase structure it induced is already internal to the policy, which requires no supervision beyond the action gating of Eq. 15 .

Group	Quantity	Symbol	Dim.
Target (noisy)	Relative position, base frame	$\tilde{\bm{p}}_{t}^{b}$	3
	Relative orientation	$\varphi(\tilde{\bm{R}}_{t}^{b})$	6
	Relative linear velocity	$\tilde{\bm{v}}_{t}^{b}$	3
	Angular velocity	$\tilde{\bm{\omega}}_{t}^{b}$	3
Servicer	Base linear / angular velocity	$\bm{v}_{b}^{b},\,\bm{\omega}_{b}^{b}$	6
	Joint positions	$\bm{q}$	14

TABLE II : Observation space (

93

dimensions). Target quantities are corrupted by the estimation noise of ( 7 ); the remainder are measured onboard.

\varphi(\cdot)

denotes the

6

D rotation encoding [ 46 ] .

Term	Error $\bm{x}$	$\sigma$	Gate	$w$
Approach
Closing distance	$d-d_{\star}$	$0.25$ m	—	5.0
Arm stillness	$\bm{q}-\bm{q}_{0}$	$0.3$ rad	$1-g_{p}$	5.0
Capture
EE pose tracking a	$\bm{e}_{p}$ ; $\bm{e}_{R}$	$0.3$ ; $0.5$	$g_{p}$	10.0
EE centering	$e_{i}$	$0.10$ m	$g_{p}$	2.0

TABLE III : Reward terms . Each term is

\kappa(\bm{x};\sigma)

on the listed error unless marked;

g_{p}

is the proximity gate ( 18 ) and

g_{c}

the contact gate ( 22 ). All terms are scaled by the control timestep.

Constraint	Limit	Gate	$p^{\max}$
Mission
Target linear velocity	$0.01~\mathrm{m/s}$	$\mathbb{1}_{c}$	0.25
Target angular velocity	$0.02~\mathrm{rad/s}$	$\mathbb{1}_{c}$	0.25
Base linear velocity (loose)	$0.08~\mathrm{m/s}$	—	0.25
Base angular velocity (loose)	$0.10~\mathrm{rad/s}$	—	0.25
Base linear velocity (tight)	$0.01~\mathrm{m/s}$	$\mathbb{1}_{c}$	0.50

TABLE IV : Constraint set .

\mathbb{1}_{c}

denotes the end-effector contact indicator.

p^{\max}

is the maximum termination probability of ( 11 ).

Stage	Rate [95% CI] (%)
Approach complete	100.0 [99.6, 100.0]
Grasp established	100.0 [99.6, 100.0]
Detumble complete (kinematic)	99.5 [98.8, 99.8]
Full success (grasp held)	99.5 [98.8, 99.8]

TABLE V : Phase completion rates over

n=1000

randomized trials . Intervals are

95\%

Wilson score intervals.

Quantity	Value
Residual target linear velocity $\lVert\bm{v}_{t}\rVert$	$0.0020\pm 0.0009$ m/s
Residual target angular velocity $\lVert\bm{\omega}_{t}\rVert$	$0.0036\pm 0.0024$ rad/s
Residual base linear velocity $\lVert\bm{v}_{b}\rVert$	$0.0028\pm 0.0012$ m/s
Residual base angular velocity $\lVert\bm{\omega}_{b}\rVert$	$0.0040\pm 0.0014$ rad/s
Time to full stabilization (mean)	$9.7\pm 8.9$ s
Time to full stabilization (median)	$5.9$ s

TABLE VI : Residual state, timing, and actuation effort at episode end , mean

\pm

std over the

n=995

successful trials.

Fig. 3 : Time to full stabilization against initial target angular velocity over

1000

trials ; the five failures are marked at the episode horizon. The side panel shows the distribution of the initial tumbling magnitude.

Constraint	Symbol	Violation (%)
Mission
Target linear velocity	$c_{v_{t}}$	2.25
Target angular velocity	$c_{\omega_{t}}$	3.55
Base linear velocity (loose)	$c_{v_{b}}$	0.00
Base angular velocity (loose)	$c_{\omega_{b}}$	0.01
Base linear velocity (tight)	$c_{v_{b}}^{\,c}$	1.43

TABLE VII : Constraint satisfaction over the

1000

trials of section VI-B , as the percentage of control steps at which each limit is exceeded.

Sample ID	$\lVert\bm{\omega}_{t}\rVert_{0}$	$\min\lVert\bm{\omega}_{t}\rVert$	$t_{\min}$	$\lVert\bm{\omega}_{t}\rVert_{f}$	$m_{t}$
	(rad/s)	(rad/s)	(s)	(rad/s)	(kg)
38	0.329	0.0041	28.9	0.0289	30.9
550	0.314	0.0016	12.7	0.0584	33.4
578	0.358	0.0018	8.2	0.0192	40.5
662	0.305	0.0008	2.9	0.0259	44.4
825	0.331	0.0009	6.4	0.0445	26.6

TABLE VIII : The five failed trials of section VI-B .

\min\lVert\bm{\omega}_{t}\rVert

is the lowest angular velocity reached, at

t_{\min}

;

\lVert\bm{\omega}_{t}\rVert_{f}

is the value at the end of the episode. The tolerance is

\varepsilon_{\omega}=0.03

rad/s.

Fig. 4 : Emergent phase structure in one representative episode . Left column, top to bottom: servicer–target distance with the proximity gate

g_{p}

; target and base angular velocity with the success tolerance

\varepsilon_{\omega}

; manipulator joint torques (solid: left arm; dashed: right arm). Right column, top to bottom: end-effector contact forces with the contact gate

g_{c}

; target and base linear velocity; commanded thruster force and reaction torque magnitudes. Vertical lines mark the behaviorally-defined phase boundaries — the onset of bilateral contact and the onset of sustained tumble reduction — rather than any gate value.

	Performance				Mission constraints						Actuation			Safety
Scenario	Succ.	$\lVert\bm{\omega}_{t}\rVert_{f}$	$\lVert\bm{v}_{t}\rVert_{f}$	$t_{\mathrm{stab}}$	$c_{v_{t}}$	$c_{\omega_{t}}$	$c_{v_{b}}$	$c_{\omega_{b}}$	$c_{v_{b}}^{\,c}$	$c_{\omega_{b}}^{\,c}$	$c_{f}$	$c_{\tau}$	$c_{q}$	$c_{F}$	$c_{\mathrm{col}}$
	(%)	(rad/s)	(m/s)	(s)	(% of steps)
Nominal	99.5	0.0036	0.0020	9.7	2.25	3.55	0.00	0.01	1.43	2.99	0.12	0.06	0.02	0.27	0.00
Unknown geometry (zero-shot)
Cylinder	99.2	0.0036	0.0021	6.5	2.24	3.83	0.13	0.11	3.00	4.51	0.19	0.06	0.03	0.22	0.00
Cuboid	99.7	0.0039	0.0021	7.7	2.22	4.08	0.04	0.04	2.74	5.03	0.22	0.08	0.03	0.25	0.00

TABLE IX : Generalization and robustness . Left: success rate, residual target velocities, and settling time, the latter three as means over successful trials. Right: constraint satisfaction, as the percentage of control steps at which each limit is exceeded; symbols are defined in Table IV . The geometry and inertia scenarios are zero-shot; tumbling rates are per-axis bounds.

Panel of Figure 14 — Fig. 5 : Zero-shot capture of unseen geometries . The policy is unchanged from the cube-trained one.

Fig. 6 : Success rate versus the initial tumbling magnitude . Error bars are

95\%

Wilson intervals; the shaded region marks magnitudes attainable during training.

R	A	C	Success	Grasp loss	$\lVert\bm{\omega}_{t}\rVert_{f}$	$\lVert\bm{\omega}_{b}\rVert_{f}$	$\Delta v$
			(%)	(%)	(rad/s)	(rad/s)	(m/s)
✓	✓	dec.	99.5	0.0	0.0036	0.0040	0.045
✓	–	dec.	97.3	0.0	0.0087	0.0058	0.189
–	✓	dec.	92.1	0.0	0.0154	0.0115	0.727
–	–	dec.	54.6	10.8	0.0433	0.0296	0.707
✓	✓	int.	23.8	3.5	0.0311	0.0149	0.254

TABLE X : Ablations . R and A denote the reward and action-space gating of sections V-D and V-E ; C denotes the constraint formulation of section V-F , either the decomposed loose and tight pair ( dec. ) or the single constraint whose limit is interpolated between the same two values according to the contact signal ( int. ). Grasp loss is the percentage of trials in which contact is established and subsequently lost. Residuals are means over successful trials.

Method	Target state	Inertial params.	Phase structure	Constraints in loop	Fixture- free	Reported outcome
Friction detumbling [ 32 ]	full	required	switched	no	yes	$20^{\circ}$ /s halted in $30$ s, base $<0.15^{\circ}$
Adaptive detumbling [ 33 ]	full	online ID	post-capture	no	yes	inertia uncertainty rejected
Robust SMC [ 13 ]	full	bounded	switched	no	no	chatter-free second-order SMC
Time-optimal [ 11 ]	full	bounded	plan+track	bounds	no	detumble and pose in one action
Optimal concurrent [ 34 ]	full	online ID	plan+track	actuator +motion	yes	finite-time, low chatter
Position/force [ 35 ]	partial	none	switched SMC+RL	joints	yes	PPO force loop, unknown target

TABLE XI : Comparison with representative model-based and hybrid capture/detumbling methods, along the assumptions each requires. Reported outcomes are quoted under each method’s own conditions and are not comparable across rows.

Space Robotics

Space robotics develops robots and autonomy for exploring, assembling, and maintaining spacecraft and planetary environments.

See all 71 papers

Oct 14, 2026

Adversarial Reinforcement Learning for Robust Planetary Landing

Deep reinforcement learning (DRL) policies for planetary landing and other aerospace guidance, navigation, and control tasks are commonly trained under randomized disturbances to improve robustness. However, typical stochastic perturbation sampling assumes independent variations, whereas real mission disturbances often exhibit spatial and temporal correlation and cannot be reproduced through independent random draws. This work investigates adversarial reinforcement learning to discover challenging off-nominal conditions. A secondary agent generates physically constrained perturbations that degrade the performance of a landing controller, forming an asymmetric sequential zero-sum game. Training proceeds in three stages: baseline controller learning, robustness enhancement with stochastic perturbations, and an iterative adversarial phase in which the controller and disturbance generator are alternately optimized. Preliminary experiments in a three-degree-of-freedom landing environment show that adversarial disturbances reveal vulnerabilities missed by conventional randomization and provide early indications that, with a more extensive training campaign, the approach can yield improved robustness and stronger generalization under previously unseen perturbation regimes.

Miguel Leiva-Vélez, Nicolas Gaston-Rozado, Andrea Scorsoglio +3

ETSI Aeronáutica y del Espacio, Universidad Politécnica de Madrid, Madrid, Spain · Indra Sistemas, Madrid, Spain · Department of Systems and Industrial Engineering, University of Arizona, Tucson, USA +2

Adversarial MARL Reinforcement Learning Robust RL Adversarial Perturbations

Ask PDF

Sep 25, 2026

Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learning

This paper presents Grasp2Twist, a bimanual dexterous manipulation system that learns to grasp and twist open jar lids using reinforcement learning. Learning this task raises three challenges: learning a unified policy for a multi-stage task, sustaining lid twisting, and sim-to-real transfer. To address the first challenge, we introduce a continuous enclosure measure to guide grasp formation and a binary enclosure indicator to guide the grasp-to-twist transition for unified policy learning. We derive both from the geometric relationship between the object center and the convex hull formed by the hand's palm and fingertips. Kinematic constraints limit how far the hand can rotate the lid with fixed contacts, so sustained twisting requires finger contact reconfiguration. We use a three-stage curriculum to facilitate exploration of these contact changes and also improve robustness for sim-to-real transfer. With our approach, the learned policy demonstrates finger gaiting, reconfiguring hand-object contacts to sustain lid rotation. It transfers zero-shot to the physical system and achieves an 88% task success rate across six household containers, including peanut-butter, vitamin, and instant-coffee jars. Ablations further validate the roles of the geometric enclosure in grasp formation and the curriculum in contact-reconfiguration exploration.

Mo Xu, Yunfu Deng, Jianuo Wang +2

University of Wisconsin-Madison · University of Wisconsin–Madison, Madison, WI, USA. · Independent Researcher +1

Robotic Manipulation Bimanual Robotic Manipulation Contact-Rich Robotic Manipulation Robotic Grasping RL for Robotics

Ask PDF

Sep 28, 2026

Aerial GRIPPER: A Gradient-based Real-time Inverse-game Predictor and Planner

Accurate capture of non-cooperative targets is critical. In an attempt to tackle this intractable challenge, an aerial gripper system integrated with a Gradient-based Real-time Inverse-game Predictor and PlannER (GRIPPER) framework is proposed. The interaction is formulated as a general-sum pursuit-evasion game under incomplete information. Specifically, underlying cost parameters of the target are inferred online, and the open-loop Nash equilibrium (OLNE) strategy is iteratively refined within a receding-horizon loop. To ensure high-frequency execution, a computationally friendly gradient-based inverse-game solver is developed. Without explicit computation of the Hessian inverse, the optimized solution is updated (> 50 Hz) based on implicit differentiation and fast Hessian-vector products. Meanwhile, an anti-disturbance controller is developed to overcome disturbances of uncertain payload and gripper actuation, enabling precise tracking of the planned trajectory and accurate grasping of the target. Simulations and real-world experiments illustrate the superior computational efficiency and task performance of GRIPPER. The task of capturing and delivering a non-cooperative target is accomplished, highlighting the robustness, adaptability, and real-time performance of the framework in highly adversarial scenarios.

Zeshuai Chen, Meng Wang, Jindou Jia +2

School of Automation Science and Electrical Engineering, Beihang University, 100191, Beijing, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, 639798, Singapore

Robotic Grasping Aerial Manipulation

Ask PDF

Abstract

Space Robotics

Explore similar work

Adversarial Reinforcement Learning for Robust Planetary Landing

Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learning

Aerial GRIPPER: A Gradient-based Real-time Inverse-game Predictor and Planner