Toward AI Trustworthiness: Finding Analytically Proven Forward-Invariant Sets for AI-Controlled Systems

Haoyang Song; Xikun Yang; Qixin Wang

cs.AIOct 5, 2026

Toward AI Trustworthiness: Finding Analytically Proven Forward-Invariant Sets for AI-Controlled Systems

Authors: Haoyang Song, Xikun Yang, Qixin Wang

Organizations: Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China.

Abstract

Neural-network (NN) controllers are increasingly used in nonlinear control systems, but their highly nonlinear behavior makes them difficult to explain and verify, raising trustworthiness concerns in safety- and mission-critical applications. A key step toward certifiable trustworthiness is to find a Forward-Invariant Set (FIS): a state-space region such that any trajectory starting inside remains inside. If the FIS excludes unsafe states, safety can be guaranteed for initial states within it. Finding an analytically proven FIS for a given AI-controlled system with a fixed controller is difficult. We propose a framework that uses an Invertible Neural Network (INN) to transform the original state space into a latent space where a regular-shaped FIS is more likely to exist. We train the INN so that a preferred hyper-rectangular candidate becomes invariant in the latent space, then formally verify it. We prove that, whenever verification succeeds, both the latent-space candidate and its inverse-transformed counterpart in the original state space are analytically proven FISs. We evaluate the approach on 45 AI-controlled systems across three representative control testbeds. Our method finds certified FISs for all 45 systems, whereas an adapted state-of-the-art baseline finds none. It is also faster on 40 of the 45 systems, and the centers of the resulting FISs roughly match domain-expert preferences.

Figures & tables

Fig. 1 : Solution heuristics: a hyper-rectangular FIS constructed and proven in the latent state space maps to a nonlinear FIS in the original state space.

Fig. 2 : Training subsets: $\mathcal{Z}_{\mathsf{inr}}$ , $\mathcal{Z}_{\mathsf{b}}$ , $\mathcal{Z}_{\mathsf{adv}}$ , and $\mathcal{Z}_{\mathsf{rep}}$ .

	Input : controller $\pi_{0}$ , $f_{\mathsf{cld},0}$ , xxxxxxxxxxx /* see ( 13 ) */
		$\vec{\mu}$ , $\vec{\sigma}$ , xxx /* see ( 1 ) and Overall Procedure Step V */
		$R_{\mathsf{fis}}$ , $K$ , $\rho_{\mathsf{shr}}$ , $\rho_{\mathsf{hrd}}$ , xxl /* see ( 21 ), ( 22 ), ( 23 ), ( 31 ) */
		$I_{\mathsf{inr}}$ , $I_{\mathsf{b}}$ , $I_{\mathsf{adv}}$ , $I_{\mathsf{rep}}$ , xxx /* see ( 26 ), ( 27 ), ( 28 ), ( 29 ) */
		total number of training epochs $E$ ,
		early-stop test triggering threshold $\theta_{\mathsf{stp}}$ ,
		early-stop test set size $I_{\mathsf{stp}}$ ;

Algorithm 1 NINN Training for Overall Procedure Step V

Application	$n$	$m$	$\eta$	activation function	Number of AI-Controllers Trained
Pendulum	2	1	128	SB3 default	20
Cart pole	4	1	128		20
PVTOL	6	2	24		5

TABLE I : Summary of AI-controllers

Application	$\vec{\mu}$	$\vec{\sigma}$
Pendulum	(0, 0) T	(0.3, 1.5) T
Cart pole	(0, 0, 0, 0) T	(0.35, 1.0, 0.1, 1.6) T
PVTOL	(0, 0, 0, 0, 0, 0) T	(1.0, 1.0, 0.8, 1.0, 1.0, 1.0) T

TABLE II : Configuration parameter settings

Application	Proposed Solution	Baseline Solution
Pendulum	100% (20/20)	0% (0/20)
Cart pole	100% (20/20)	0% (0/20)
PVTOL	100% (5/5)	0% (0/5)

TABLE III : Success rates of finding FIS

Fig. 3 : Comparisons: time cost to run the FIS finding solution.

Application	min	median	mean	max	preferred
Pendulum	0.929	1.307	1.288	1.308	1.800
Cart Pole	0.033	0.346	0.473	1.348	0.896
PVTOL	0.008	0.012	0.011	0.013	51.200

TABLE IV :

\mathbb{X}_{0}

FIS size statistics on our proposed solution

Application	Dimension Index $i$	min	median	mean	max	preferred range size
Pendulum	1	0.822	0.822	0.823	0.840	0.600
	2	2.110	3.108	3.058	3.108	3.000
Cart Pole	1	1.000	1.913	2.018	3.014	0.700
	2	1.729	3.360	3.304	4.234	2.000
	3	0.381	0.632	0.626	0.847	0.200
	4	1.056	2.674	2.656	3.416	3.200

TABLE V : Dimension-wise range size statistics on

\mathbb{X}_{0}

FISs found by our proposed solution

Application	Dimension Index $i$	min	median	mean	max
Pendulum	1	-6.557	-0.370	-0.679	-0.370
	2	-38.995	-2.024	-3.872	-2.024
Cart Pole	1	-29.255	1.053	-0.240	45.220
	2	-51.584	-4.826	-3.262	24.570
	3	-11.171	-0.255	-1.298	4.192
	4	-12.437	1.516	1.062	11.026

TABLE VI : Dimension-wise center drift (unit:

10^{-3}

) statistics on

\mathbb{X}_{0}

FISs found by our proposed solution

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

	Input : $\mathcal{Q}_{\mathsf{rep}}$ , $I^{\mathsf{max}}_{\mathsf{rep}}$ , $e$ , $\mathcal{Z}_{\mathsf{b}}$ , $J_{\mathsf{add}}$ , $E_{\mathsf{rep}}$ ;
	Output : the updated $\mathcal{Q}_{\mathsf{rep}}$ ;
	Sort all elements of $\mathcal{Z}_{\mathsf{b}}$ into $\vec{z}_{\mathsf{b},(1)},\ldots,\vec{z}_{\mathsf{b},(I_{\mathsf{b}})}$ so that
	$\nu(\vec{z}_{\mathsf{b},(1)})\geqslant\dots\geqslant\nu(\vec{z}_{\mathsf{b},(I_{\mathsf{b}})})$ ; x // $\nu(\vec{\bullet})$ is defined by ( 33 )
	$J^{\prime}_{\mathsf{add}}$ := $\mathsf{min}\{J_{\mathsf{add}},I_{\mathsf{b}}\}$ ;
	$\mathcal{A}_{\mathsf{rep}}$ := $\{\vec{z}_{\mathsf{b},(j)}\mid 1\leqslant j\leqslant J^{\prime}_{\mathsf{add}}\}$ ;
	$\mathcal{Q}^{\prime}_{\mathsf{rep}}$ := $\mathcal{Q}_{\mathsf{rep}}\cup\{(\vec{z},\nu(\vec{z}))\mid\vec{z}\in\mathcal{A}_{\mathsf{rep}}\}$ ;

Appendix

Algorithm 2 Replay Buffer Update

Explore similar work

Certified Robust Invariant Polytope Training in Neural Controlled ODEs
Aug 2, 2024Akash Harapanahalli, Samuel CooganNeural Controlled Differential Equations Robust Control
School of Electrical and Computer Engineering, Georgia Institute of Technology
Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks
Aug 5, 2026Berk Altiner, Rajasree Sarkar, Arunava Banerjee +2Robot Trajectory Tracking
Department of Mechanical Engineering, University of Minnesota–Twin Cities, Minneapolis, MN 55455, USA · DEVCOM Army Research Laboratory, Aberdeen Proving Ground, Aberdeen, MD 21005, USA
Implicit Neural Networks as Static Controllers: Certificates and Performance Separation
Jul 13, 2026Giuseppe C. Calafiore, Laurent El GhaouiImplicit Neural Representations Feedback Control

Aug 2, 2024, cs.LGJ/K move · Enter open · S save

Certified Robust Invariant Polytope Training in Neural Controlled ODEs

Akash Harapanahalli, Samuel Coogan

School of Electrical and Computer Engineering, Georgia Institute of Technology

Neural Controlled Differential Equations Robust Control Neural ODEs

Open paper AskPDF arXiv

We propose a framework for training neural network controllers with certified robust forward invariant polytopes. First, we parameterize a family of lifted control systems in a higher dimensional space, where the original neural controlled system evolves on an invariant subspace of each lifted system. We use interval analysis and neural network verifiers to further construct a family of lifted embedding systems, carefully capturing the knowledge of this invariant subspace. If the vector field of any lifted embedding system satisfies a sign constraint at a single point, then a certain convex polytope of the original system is robustly forward invariant. Treating the neural network controller and the lifted system parameters as variables, we propose an algorithm to train controllers with certified forward invariant polytopes in the closed-loop control system. Through two examples, we demonstrate how the simplicity of the sign constraint allows our approach to scale with system dimension to over

50

states, and outperform state-of-the-art Lyapunov-based sampling approaches in runtime.

Abstract

Explore similar work

Certified Robust Invariant Polytope Training in Neural Controlled ODEs

Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks

Implicit Neural Networks as Static Controllers: Certificates and Performance Separation