Joint Learning of Control Lyapunov Functions and Controllers via Stable-by-Design Surrogates
Authors: Patricia Pauli, Matthieu Barreau
Organizations: Mechanical Engineering Department, Eindhoven University of Technology, Eindhoven, The Netherlands · Division of Decision and Control Systems, KTH Royal Institute of Technology, Stockholm, Sweden
Training and verifying neural network-based control Lyapunov functions (CLFs) can be computationally demanding. To address this challenge, we propose a framework that jointly learns a CLF and a stabilizing controller using structured neural networks and a projection-based parameterization. Specifically, we employ Polyak-Lojasiewicz networks (PLNets), which parameterize nonconvex, positive-definite CLF candidates, and construct a stable-by-design surrogate of the closed-loop dynamics that satisfies the Lyapunov decrease condition by construction. The controller combines linear state feedback with a higher-order correction term resembling Sontag's universal formula and is trained to align the resulting closed-loop dynamics with the stable-by-design surrogate. Unlike classical approaches based on Sontag's formula, our framework does not require a known CLF; instead, it learns the CLF and controller simultaneously. We establish local and global stability guarantees for the learned controller and demonstrate the effectiveness of the proposed approach through numerical examples.
Figures & tables
in Ω
beyond
ts [s]
∫∥u∥2
peak ∥u∥
Academic example
LQR
4
4
5.0
3.5
4
Sontag (analytic CLF)
12
12
6.4
19.3
19.6
Sontag (learned CLF)
12
12
6.2
43.3
45.2
CLF-QP (analytic CLF)
12
12
7.1
18.8
9.87
CLF-QP (learned CLF)
12
12
6.6
16.7
12
Table 1: Stabilization performance (five seeds). “In Ω ” and “beyond” give the number of converged runs out of 12 (mean over seeds, with ± the standard deviation if nonzero). Settling time ts=inf{t∈[0,T]:∥x(τ)∥<10−2∀τ∈[t,T]} , energy and peak input use converged runs from the starts inside Ω , pooled over seeds (medians for ts and energy, maximum for the peak). On the Van der Pol oscillator, Sontag’s formula and the CLF-QP with a learned CLF use the Vθ of the learned ωϕ/(1+∥x∥) model with α=0.5 .
Figure 1: Closed-loop responses (seed 0). (a) Academic example: LQR diverges from 8 of 12 initial conditions, whereas the learned controller converges from all. (b) Van der Pol oscillator: states and control input from x(0)=(9.8,0) . The input uses the signed logarithmic scale sign(u)log10(1+∣u∣) ; dense bands indicate rapid input oscillations.
This work investigates the joint learning of neural controllers and control Lyapunov functions (CLFs) for nonlinear discrete-time systems. We exploit the structural properties of Input Convex Neural Networks and Polyak-Łojasiewicz Networks to guarantee positive definiteness of CLF candidates by construction, reducing the conditions requiring formal verification. We develop a mixed-integer programming framework to formally verify the remaining stability conditions and introduce an iterative region of attraction (ROA) expansion procedure to enlarge the certified ROA without retraining. Experiments on an inverted pendulum and a path-tracking system demonstrate the advantages of structured neural CLFs, particularly PLNets, and illustrate how their structural bounds influence the geometry of the certified regions. Our code is available at: https://github.com/JLedesMart/Structured-Neural-Control-Lyapunov-Functions.
Jose Luis Ledesma Martin, Patricia Pauli
Department of Mechanical Engineering, Delft University of Technology, Delft, 2628 CD Delft, The Netherlands · Department of Mechanical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands
We propose a framework for training neural network controllers with certified robust forward invariant polytopes. First, we parameterize a family of lifted control systems in a higher dimensional space, where the original neural controlled system evolves on an invariant subspace of each lifted system. We use interval analysis and neural network verifiers to further construct a family of lifted embedding systems, carefully capturing the knowledge of this invariant subspace. If the vector field of any lifted embedding system satisfies a sign constraint at a single point, then a certain convex polytope of the original system is robustly forward invariant. Treating the neural network controller and the lifted system parameters as variables, we propose an algorithm to train controllers with certified forward invariant polytopes in the closed-loop control system. Through two examples, we demonstrate how the simplicity of the sign constraint allows our approach to scale with system dimension to over 50 states, and outperform state-of-the-art Lyapunov-based sampling approaches in runtime.
Akash Harapanahalli, Samuel Coogan
School of Electrical and Computer Engineering, Georgia Institute of Technology
We augment the Slotine--Li adaptive controller for Euler--Lagrange systems with three learned components: a structured-quadratic Lyapunov function Vψ whose positive-definiteness follows from a Cholesky parameterization, a residual Soft Actor--Critic policy that adds bounded torque corrections to the analytic baseline, and a physics-informed neural network that estimates unmodeled dynamics. A closed-form safety filter, derived from the single affine constraint V˙ψ+αVψ≤0, projects every policy output onto the safe set without requiring an online QP solver. We prove: global feasibility of the filter under a drift-decay condition on the control-degeneracy set; exponential stability under exact shielding, with a robust extension whose margin depends on the PINN approximation error; almost-sure convergence of the three-timescale policy--certificate--multiplier updates to a KKT point; and a PAC generalization bound for the certificate over compacts. On a 2-DOF manipulator with nonlinear friction and variable payload, the learned certificate accounts for most of the empirical gain: tracking error drops by 41% on nominal friction and 24% on aggressive friction at the centroid of the training distribution. A 7-DOF scalability study on a Franka Emika Panda confirms clean convergence of the full pipeline at industrial scale, identifies the conditions under which gains over exact model-based baselines should and should not be expected, and documents a warm-start pathology of the learned certificate that has practical implications for deployment.
Giansalvo Cirrincione, Adriano Fagiolini
Laboratoire LTI, Université de Picardie Jules Verne, Amiens, France. · MIRPALab, Department of Engineering, University of Palermo, 90128 Palermo, Italy.