End-to-End Stability of Online Active Learning in Large-Scale Neural Systems
Authors: Shuiwen Shen, Jinhua Zhang, Chenlei Hou, Peng Shi, Yong Han, Bo Gao, Xiaoxiao Na, Zhihui Lin, Pengtao Du
Organizations: School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen, 361024, China. · State Innovative Centre for Automotive Electric Driving at Xiamen, Xiamen, Fujian, 361010, China. · China Innovation Center for Intelligent and Connected Vehicles, Beijing, 100176, China. · King Long United Automotive Industry Co. Ltd., Xiamen, Fujian, 361023, China. · School of Electrical and Mechanical Engineering, Adelaide University, Adelaide, SA 5005, Australia. · Syselek UK Ltd, Warwick, CV34 6RG, United Kingdom. · Department of Engineering, Cambridge University, Cambridge, CB2 1TN, United Kingdom.
Online Active Learning (OAL) aims to extend deep learning from static inference after offline training to continuous adaptation during real-time interaction with a changing physical environment. This transition is essential for safety-critical applications, but it raises a central theoretical question: whether the online parameter update process remains stable when scaled to multilayer neural architectures. This paper develops a stability-focused OAL framework. The framework uses prediction error as a feedback signal to update a structured deep model, thereby linking online learning with Lyapunov-based energy minimization. We show that prediction error convergence alone is insufficient for global parameter convergence because multilayer networks may admit high-dimensional manifolds of equivalent output solutions. The analysis therefore separates output error regulation, internal zero dynamics, interlayer coupling, and parameter convergence. Persistency of excitation and active damping are then used to remove the residual null-space modes that remain after output error convergence. A cascaded Lyapunov analysis establishes asymptotic stability of the overall OAL learning process, while monotonic activation functions and diffeomorphic layer mappings provide the uniqueness condition for the target parameter trajectory. The resulting proof provides a theoretical basis for applying online adaptive learning to large-scale neural systems in nonstationary and safety-critical settings.
Figures & tables
Figure 1: From the principal Online Active Learning framework to the primary framework. (a) The external stimulus x drives both the true physical process and the structured deep model, while the sensor measurement y is the comparison target. The Physical World block denotes the true process, whereas Eqn ( 4 ) describes its parameterized internal model. The gradient arrows correspond to J\uptheta[i]e[i] in Eqn ( 11 ), up to the gradient-sign convention. (b) The primary framework FOALPRY sets x=ys and decomposes the model into prediction P and reconstruction R , with the resulting error ey distributed across the layers for online parameter correction.
Figure 2: Structural Isomorphism of Large-Scale System Dynamics in Engineering
Figure 3: Topological Isomorphism of Feedback Learning
Figure 4: Topological Isomorphism of Intra-Layer Feedback Learning
The transition of information-centric artificial intelligence to safety-critical applications is impeded by the limitations of conventional offline-training paradigms in open-world environments. This paper presents Control AI, an architecture for Online Active Learning (OAL) inspired by the continuous Prediction-Evaluation-Correction process of biological intelligence. The framework combines the hierarchical representation capacity of deep neural networks with the real-time error-feedback adaptation of adaptive control. A prediction model maps sensory input into a latent manifold, an optional latent model represents transitions in that space, and a reconstruction model maps the latent state back to observable signals. Residuals between predictions and measurements provide the internal learning signal. Spatial and temporal cross-reconstruction architectures prevent OAL from collapsing to a trivial identity mapping, while coupled, dual, and localized variants shorten error-transmission paths. Localized predictive coding style OAL is implemented through guidance-backpropagation and target-backpropagation architectures, and hybrid and scaled variants support high-dimensional deployment. The paper focuses on the architectural rationale and the progression from a conceptual feedback loop to practical large-scale configurations.
Shuiwen Shen, Jinhua Zhang, Yan Wang +5
School of Mechanical and Automotive Engineering, Xiamen University of Technology, Xiamen, 361024, China. · State Innovative Centre for Automotive Electric Driving at Xiamen, Xiamen, Fujian, 361010, China. · China Innovation Center for Intelligent and Connected Vehicles, Beijing, 100176, China. +2
Neural ODEs learn dynamics from trajectory losses, but their adjoint gradients lack the regressor-times-parameter-error structure on which Lyapunov analyses of online adaptation rest, so training on streaming data comes without stability guarantees. We show that this structure is in fact present: the adjoint gradient decomposes exactly into a positive semi-definite trajectory operator acting on the parameter error plus a nonlinear perturbation with explicit, horizon-dependent bounds. A quadratic Lyapunov function then certifies online Neural ODE training over sliding windows under computable gain and horizon conditions, and the same certificate extends to stored data: its drift branch recovers concurrent learning, and its trajectory branch yields NODE-CL, a stored-segment Gauss-Newton method built on batched forward sensitivities that needs no state-derivative estimates. On four DeepMind Control Suite domains, NODE-CL attains the lowest median prediction error on three under velocity measurement noise, where observer-based concurrent learning degrades by up to 8x; with clean measurements it is best on the pendulum and within a factor of 1.6 of the best stored-data baseline on the cartpole and reacher.
Omkar Sudhir Patil
Division of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803, USA
The growing complexity of modern control tasks calls for controllers that can react online as objectives and disturbances change, while preserving closed-loop stability. Recent approaches for improving the performance of nonlinear systems while preserving closed-loop stability rely on time-invariant recurrent neural-network controllers, but offer no principled way to update the controller during operation. Most importantly, switching from one stabilizing policy to another can itself destabilize the closed-loop. We address this problem by introducing a stability-preserving update mechanism for nonlinear, neural-network-based controllers. Each controller is modeled as a causal operator with bounded ℓp-gain, and we derive gain-based conditions under which the controller may be updated online. These conditions yield two practical update schemes, time-scheduled and state-triggered, that guarantee the closed-loop remains ℓp-stable after any number of updates. Our analysis further shows that stability is decoupled from controller optimality, allowing approximate or early-stopped controller synthesis. We demonstrate the approach on nonlinear systems with time-varying objectives and disturbances, and show consistent performance improvements over static and naive online baselines while guaranteeing stability.
Institute of Mechanical Engineering, Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland. · Department of Engineering Science, University of Oxford, United Kingdom