Beyond Terminal Constraints: Reliable Robot Control with Learned Safe Sets
Authors: Gianni Lunardi, Elias Fontanari, Matteo Saveriano, Andrea Del Prete
Organizations: Faculty of Engineering, Free University of Bolzano, Bolzano, Italy · Department of Industrial Engineering, University of Trento, Trento, Italy
Ensuring safety in advanced robot control methods such as Model Predictive Control (MPC) and Reinforcement Learning remains an open challenge. A common approach relies on control-invariant sets, yet their exact computation for nonlinear systems is generally intractable, and numerical approximations compromise theoretical guarantees. In this work, we show that safety does not need to rely exclusively on control invariance. We introduce novel classes of safe sets, each broader than classical control-invariant sets, and analyze corresponding MPC schemes that guarantee recursive feasibility by constraining different nodes of the prediction horizon rather than only the terminal state. Furthermore, when the safe set is an inner approximation of a suitable control-invariant set, our methods can ensure safety through a safe-abort strategy, even in the absence of recursive feasibility. We validate our approaches in extensive simulations on a 6-joint robot manipulator, showing fewer constraint violations than state-of-the-art methods while maintaining comparable tracking performance and computational efficiency. Hardware experiments further demonstrate the practicality of our schemes for real-world robotic applications.
Figures & tables
Figure 1: Hierarchy of safe set classes proposed in this work. The innermost set represents the well-known class of Control Invariant Sets (CISs). We introduce broader classes, namely up-to- N -step CISs, within- N -step CISs, back-and-forth- N -step CISs, and back-and-forth-within- N -step CISs.
Figure 2: Example of RC-MPC with N=3 and a 3-step CIS. At MPC iteration 3, the condition x3∣0∈S implies, from ( 7 ), that the state can return to S in 3 steps. The soft terminal constraint x3∣3∈S is therefore feasible and can be satisfied. Consequently, the receding constraint shifts forward to time step 2.
Figure 3: Simulation setup for the setpoint regulation task. The robot starts from a random initial configuration (left) and must reach a target with the EE (right), avoiding collisions with the ball and the ground.
Figure 4: Simulation results of the different MPC controllers for the setpoint regulation task, considering different values of horizons and NN safety margins.
Figure 5: Values of r and NN outputs (left-hand side of 24 ) evaluated at xr∗ and xN∗ , in one particular simulation, with horizon N=45 and safety margin α=10% .
Naive
Zerovel
STC
HTC
RC
RTI [ ms ]
4.68
4.18
7.76
4.15
4.78
Table 1: 99-percentile computation times for the MPC using the RTI scheme.
Mean [\text{,}\mathrm{ms}]
1.71
STD [\text{,}\mathrm{ms}]
0.48
99-quantile [\text{,}\mathrm{ms}]
2.59
Times < 5 ms [%]
99.76
Table 2: RTI safe abort time statistics, for simulation with horizon N=20 , α=40% on 423 samples.
Figure 6: Simulation setup to evaluate disturbance rejection capability in the setpoint task. The robot starts from a fixed initial configuration (left) and must reach the EE’s target (right), avoiding collisions with the ground and the window-shaped obstacle.
Figure 7: Simulation results with modeling errors, considering different values of parameter noise and joint/collision margins. The horizon and NN safety margin α for these tests are respectively N=25 and α=20% .
Set
Failures (%)
CIS
0.91
Up-to-N-step CIS, [K = 3]
0.69
Within-N-step CIS
0.62
Back-and-forth-N-step CIS, [r = 7]
0.031
Back-and-forth-within-N-step CIS
0.02
Table 3: Statistical test results for the 4-DOF set used in Section 8.2 .
Set
Failures (%)
CIS
0.72
Up-to-N-step CIS, [K = 24]
0.56
Within-N-step CIS
0.52
Back-and-forth-N-step CIS, [r = 4]
0.079
Back-and-forth-within-N-step CIS
0.071
Table 4: Statistical test results for the 5-DOF set used in Section 8.4 .
Figure 8: Setup and example of robot motion with the RC-MPC controller (videos can be found in the supplementary material). The manipulator tracks the moving target (the cube with markers in the human’s hand) without colliding with the environment and without violating the joint limits. The frames, ordered in time from left to right, have been taken from a single experiment.
Figure 9: Cumulative percentage of the computation times of the RC-MPC during a real experiment. These plots show the likelihood of the computation time being less than a certain threshold.
Figure 10: Constraint violations during the real experiment with RC-MPC.
Autonomous systems are increasingly deployed in real-world environments, where they must achieve high performance while maintaining safety under state and input constraints. Although Model Predictive Control (MPC) provides a principled framework for constrained optimal control, guaranteeing safety beyond its finite planning horizon remains a fundamental challenge. In this work, we augment MPC with a safety value function-based terminal constraint that enforces membership in a control-invariant safe set at the end of each planning horizon. This formulation enables real-time synthesis of trajectories that are both high-performing and provably safe. We show that, under an exact safety value function and a feasible initialization, the proposed MPC scheme is recursively feasible, thereby ensuring persistent safety. In contrast to traditional terminal set constructions that rely on local linearizations or conservative approximations, our approach incorporates a reachability-based safety value function for terminal constraints, yielding less conservative and more expressive safety guarantees. We validate the proposed framework through simulation and hardware experiments on a Flexiv Rizon 10s manipulator. Results demonstrate improved constraint satisfaction and robustness compared to standard state-constrained MPC and reactive safety filtering, while maintaining competitive task performance. The full implementation and experiments are available on the project website.
Hao Wang, Nam Nguyen, Armand Jordana +2
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California · Department of Aeronautics and Astronautics, Stanford University · Electrical and Computer Engineering Department, New York University
Reinforcement learning (RL) enables the synthesis of control policies directly from data, making it highly appealing for complex cyber-physical systems (CPSs) and robotics. A persistent challenge, however, is ensuring strict, hard safety constraints during the active learning phase. In real-world physical systems, violating mechanical limits can cause irreversible damage, necessitating that exploration remains strictly within safe operational regions. We propose a generalized framework that combines the adaptive, high-performance nature of deep reinforcement learning (DRL) with the formal safety guarantees of model predictive control (MPC). Using a mathematical model of the system dynamics, offline MPC computations define a feasible state-action space, representing all safe combinations of system states and control inputs that guarantee constraint satisfaction. During training and deployment, the RL agent's instantaneous actions are projected onto this globally verified feasible set via a safety filter. We systematically evaluate our generalized approach on a non-linear 1-DoF laboratory testbed, demonstrating successful exploration and stable policy convergence on physical hardware.
Georg Schäfer, Jakob Rehrl, Stefan Huber +1
Josef Ressel Centre for Intelligent and Secure Industrial Automation, Salzburg University of Applied Sciences, Salzburg, Austria · Department of Artificial Intelligence and Human Interfaces, Paris Lodron University of Salzburg, Salzburg, Austria
Sampling-Based Model-Predictive Control (MPC) algorithms are a flexible class of controllers used for navigation on a wide range of robotic systems. Historically, such approaches have lacked hard safety guarantees, a shortcoming which we remedy in this work by computing guaranteed reachable-set overapproximations online with a fast, interval-based pipeline. We show that our method achieves similar performance to a state-of-the-art reachability-based planner without the need for the expensive pre-computation step, and can be scaled to systems that are infeasible using existing approaches. Finally, we demonstrate that our technique reduces safety violations by over 99% in a racing simulation and successfully controls a model racecar on real hardware experiments without crashes.