Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.
Figures & tables
Figure 1: Closed-loop system architecture including a learning system influencing the controller. In this generalized architecture, the “Plant” represents the target physical system or dynamical environment (e.g., the 1-DOF nonlinear Duffing oscillator in Section 6 ), which outputs physical state variables xk+1 (such as position and velocity). The z−1 block denotes the standard discrete-time unit delay operator mapping the next state to the current state xk , and xref represents the desired reference trajectory or setpoint.
Figure 2: Experimental Evaluation Phases: The three-phase deployment strategy utilized in Section 6 to explicitly isolate and quantify the proposed error taxonomy. While practical deployments may initialize directly into online adaptation, this staged experimental approach is necessary to independently evaluate the Closed-Loop Identification Error (Phase 1), the Action Error induced by static deployment (Phase 2), and the continuous Tracking Error mitigated by time-scale separated learning (Phase 3).
Architecture
Id. Error (MSE)
Action Error
Tracking Error
Baseline PD
-
-
0.2037 ( σ2 : 0.0010)
Static Offline NN
0.0001 ( σ2 : 0.0000)
0.0463 ( σ2 : 0.0014)
0.1218 ( σ2 : 0.0103)
Optimal Online NN
-
-
0.0044 ( σ2 : 0.0000)
Ablation: Sluggish LR
-
-
0.7096 ( σ2 : 13.9977)
Ablation: Unstable LR
-
-
19.3110 ( σ2 : 1.3640)
Table 1: Monte Carlo Summary of Experimental Error Metrics ( N=100 ). Results report the mean and variance across the 36 stable runs.
Figure 3: Simulation of the Action Error and Online Adaptation in a noisy nonlinear tracking task. (a, c) Standard Architectures: The Baseline PD (dashed black) exhibits phase lag due to uncompensated cubic stiffness. The Static Offline NN (red) pushes the system out-of-distribution, forcing an Action Error. The Optimal Online NN (blue) utilizes continuous gradient descent with a bounded learning rate ( ∥∇θL∥≤0.5 ) to mitigate noise and track the limit cycle. (b, d) Ablation Studies: A sluggish learning rate ( α=0.01 , green) fails to adapt to the moving target, causing persistent phase lag. Conversely, applying the aggressive rate ( α=0.50 , purple) without gradient clipping violates time-scale separation; the network overreacts to transient errors, inducing mechanical chatter.
Architecture
Tracking Error (MSE)
Amplitude Distortion (%)
MRAC (Linear-in-the-Parameters)
0.0062
2.49
Dual IOP (Frozen Phase 2)
0.2122
48.04
Table 2: Empirical Tracking Error of Structural and Adaptive Baselines
Figure 4: Evaluation of proposed baselines on the 1-DOF nonlinear Duffing oscillator. While the MRAC bounds the trajectory via its linear-in-the-parameters Lyapunov formulation, it suffers from a persistent tracking lag due to its inability to capture the unmodeled cubic stiffness. Conversely, the Dual IOP model, despite achieving unbiased closed-loop identification in Phase 1, distorts the amplitude upon static deployment, demonstrating the persistence of the Action Error independent of calibration bias.
The growing complexity of modern control tasks calls for controllers that can react online as objectives and disturbances change, while preserving closed-loop stability. Recent approaches for improving the performance of nonlinear systems while preserving closed-loop stability rely on time-invariant recurrent neural-network controllers, but offer no principled way to update the controller during operation. Most importantly, switching from one stabilizing policy to another can itself destabilize the closed-loop. We address this problem by introducing a stability-preserving update mechanism for nonlinear, neural-network-based controllers. Each controller is modeled as a causal operator with bounded ℓp-gain, and we derive gain-based conditions under which the controller may be updated online. These conditions yield two practical update schemes, time-scheduled and state-triggered, that guarantee the closed-loop remains ℓp-stable after any number of updates. Our analysis further shows that stability is decoupled from controller optimality, allowing approximate or early-stopped controller synthesis. We demonstrate the approach on nonlinear systems with time-varying objectives and disturbances, and show consistent performance improvements over static and naive online baselines while guaranteeing stability.
Institute of Mechanical Engineering, Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland. · Department of Engineering Science, University of Oxford, United Kingdom
Neural ODEs learn dynamics from trajectory losses, but their adjoint gradients lack the regressor-times-parameter-error structure on which Lyapunov analyses of online adaptation rest, so training on streaming data comes without stability guarantees. We show that this structure is in fact present: the adjoint gradient decomposes exactly into a positive semi-definite trajectory operator acting on the parameter error plus a nonlinear perturbation with explicit, horizon-dependent bounds. A quadratic Lyapunov function then certifies online Neural ODE training over sliding windows under computable gain and horizon conditions, and the same certificate extends to stored data: its drift branch recovers concurrent learning, and its trajectory branch yields NODE-CL, a stored-segment Gauss-Newton method built on batched forward sensitivities that needs no state-derivative estimates. On four DeepMind Control Suite domains, NODE-CL attains the lowest median prediction error on three under velocity measurement noise, where observer-based concurrent learning degrades by up to 8x; with clean measurements it is best on the pendulum and within a factor of 1.6 of the best stored-data baseline on the cartpole and reacher.
Omkar Sudhir Patil
Division of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803, USA
A learning-enabled disturbance-rejection framework based on a Neural Extended State Observer (Neural-ESO) is presented in this letter. Unlike existing learning-based control methods that largely rely on the learned model once deployed, Neural-ESO adopts a dual-pathway architecture: a predictive pathway uses a neural network to provide a feedforward disturbance estimate that accelerates convergence, while a corrective pathway employs a conventional ESO to compensate prediction errors and prevent over-reliance on the neural component. Using Lyapunov theory and a small-gain analysis, we show that enforcing a Lipschitz bound on the learning component guarantees uniform ultimate boundedness of the closed-loop error dynamics. The proposed framework is validated on a quadrotor landing task subject to strong ground-effect disturbances across normal and out-of-distribution scenarios, demonstrating accuracy-robustness trade-off and greater operational reliability during training, deployment, and transfer compared with state-of-the-art baselines.
Fan Zhang, Richie Suganda, Jinfeng Chen +4
Department of Engineering Technology, University of Houston, USA · Department of Electrical and Computer Engineering, University of Houston, USA · Department of Electrical Engineering, Rice University, USA