Enforcing Ellipsoidal Constraints in Neural Networks with a Polynomial Safety Layer
Authors: Rolf Findeisen, Sebastian Hirt, Hoang Hai Nguyen, Grace Li Zhang, Hendrik Alsmeier
Organizations: Control and Cyber-Physical Systems Laboratory, Technical University of Darmstadt, Germany · Hardware for Artificial Intelligence, Technical University of Darmstadt, Germany
Often the output of a learned mapping must satisfy prescribed constraints. In the control of autonomous systems, for example, the chosen action must keep the system within a collision-free region. We consider the case in which a proposed output is corrected by projection onto a safety ellipsoid. Computing this projection amounts to solving a quadratically constrained quadratic program (QCQP). We construct a neural-network safety layer by unrolling a feasibility-preserving polynomial penalty iteration. The unrolled structure allows the loss gradient to propagate from the feasible output through the safety layer to the network parameters during training. We show how to choose the layer parameters so that every iterate is feasible and give a computable bound on its distance to the exact projection at any prescribed depth. We illustrate the method by training a LiDAR-based policy to control an unmanned aerial vehicle. A privileged model predictive control expert uses the map and the measured vehicle state to generate the training data. In the simulated narrow mazes, the filtered policy avoided wall contact at every tested stage of training, including at initialization, when every unfiltered flight hit a wall. A representative build of the learned policy and its safety layer for an Arm Cortex-M4F microcontroller occupies about 36 KiB of flash, with a RAM budget below 5 KiB.
Figures & tables
Figure 1: The neural network hθ proposes uNN . The safety layer starts at the feasible point u(0)=v . Each block Fℓ performs one polynomial penalty step and returns a feasible iterate. The diamond marks the filtered output uS , and the black dot marks the exact projection u⋆ . Pink shows the learning process. During training, y=ydata and the loss compares uS with the paired target udata .
Figure 2: The learned LiDAR policy proposes an acceleration from sensor measurements and route references. The safety layer filters the proposal before it is applied. The MPC supplies the training targets.
Trained with filter
Trained without filter
Examples
Filter active
Bypassed
Filter added
No filter
10,000
102 / 0
78 / 6
60 / 0
90 / 0
20,000
90 / 0
54 / 0
102 / 0
90 / 0
Full dataset
96 / 0
72 / 0
102 / 0
102 / 0
Table 1: Goal arrivals / wall contacts in 108 narrow-maze flights per column and training budget. The remaining flights timed out.
Figure 3: Flight on a held-out maze. The enlargement shows the filter active and bypassed. The square marks where the drone comes to rest when the filter is added only at deployment.
Figure 5: Perturbed starts on a complex maze (left, one policy seed). The enlarged bend shows matched calm and wind-driven flights. The filled circle marks the end of the gust and the open circle marks return to the route tube.
While neural network control policies are powerful, their deployment on safety critical systems depends on ensuring that they obey strict constraints. Existing work often treats safety as a metric to optimize for, which competes with other performance objectives, if training converges at all. Instead, we introduce ShardNet, a neural network architecture that strictly enforces unions of polyhedral constraints by construction, using a differentiable projection layer parameterized by a classification network. The key insight is to embed safety into the neural network's structure, allowing performance to be optimized independently because formal safety guarantees are always given. In contrast with existing neural architectures that can only enforce simple convex constraints, ShardNet enables the first safe-by-construction synthesis of forward-invariant neural network controllers on closed-loop systems where safety constraints are expressed as nonconvex unions of polyhedras or learned value function level sets. To support this, we also introduce a technique to verify and train such value functions correctly as rectified linear unit (ReLU) networks, which has not previously been possible. On double integrator benchmarks drawn from the literature, ShardNet policies maintain 100% safety on verified sets and achieves significantly lower objective loss compared to existing formal methods. Furthermore, our value function training technique also produces safe sets more than 3 times larger than existing verification approaches.
Long Kiu Chung, Shreyas Kousik
Department of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.
We propose an imitation-learning design for neural-network control policies under state and input constraints. Training alternates a standard imitation gradient step with a block of k safety steps that pull the network's actions toward their projection onto the safe set; at run time, the controller is the trained network alone, with no safety filter. We analyze this scheme as inexact projected gradient descent in the space of policy actions. When the projected actions are recomputed at every safety step and each step moves the actions consistently toward the safe set, letting k grow logarithmically yields asymptotic constraint satisfaction on the training states and bounds the distance to the constrained optimum of the imitation loss; with the projected actions held fixed, the same holds only if they are exactly representable by the network. On a nonlinear autonomous racing task, we compare our method with adding a weighted constraint-violation penalty to the imitation loss. With a sufficiently large weight, our method matches the lap time of unconstrained imitation while reducing the fraction of violating episodes from 15% to 1%, about six times fewer than the penalty approach at its best weight. Its lap times are less sensitive to the weight, which instead sets how quickly violations vanish during training. In racing, the safety corrections are sparse and the conditions of the analysis do not hold; the gain arises instead through the data collected during training. These gains come at the cost of additional training computation.
We propose a framework for training neural network controllers with certified robust forward invariant polytopes. First, we parameterize a family of lifted control systems in a higher dimensional space, where the original neural controlled system evolves on an invariant subspace of each lifted system. We use interval analysis and neural network verifiers to further construct a family of lifted embedding systems, carefully capturing the knowledge of this invariant subspace. If the vector field of any lifted embedding system satisfies a sign constraint at a single point, then a certain convex polytope of the original system is robustly forward invariant. Treating the neural network controller and the lifted system parameters as variables, we propose an algorithm to train controllers with certified forward invariant polytopes in the closed-loop control system. Through two examples, we demonstrate how the simplicity of the sign constraint allows our approach to scale with system dimension to over 50 states, and outperform state-of-the-art Lyapunov-based sampling approaches in runtime.
Akash Harapanahalli, Samuel Coogan
School of Electrical and Computer Engineering, Georgia Institute of Technology