stat.MLOct 15, 2026

Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop

Authors: Vittorio Lippi

Organizations: Calejo Hybrid Intelligence AB, Mellansjö 109, 92592 Tärnaby, Sweden

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.

Figures & tables

Figure 1: Closed-loop system architecture including a learning system influencing the controller. In this generalized architecture, the “Plant” represents the target physical system or dynamical environment (e.g., the 1-DOF nonlinear Duffing oscillator in Section 6 ), which outputs physical state variables xk+1x_{k+1} (such as position and velocity). The z−1z^{-1} block denotes the standard discrete-time unit delay operator mapping the next state to the current state xkx_{k} , and xrefx_{ref} represents the desired reference trajectory or setpoint.
Figure 2: Experimental Evaluation Phases: The three-phase deployment strategy utilized in Section 6 to explicitly isolate and quantify the proposed error taxonomy. While practical deployments may initialize directly into online adaptation, this staged experimental approach is necessary to independently evaluate the Closed-Loop Identification Error (Phase 1), the Action Error induced by static deployment (Phase 2), and the continuous Tracking Error mitigated by time-scale separated learning (Phase 3).
ArchitectureId. Error (MSE)Action ErrorTracking Error
Baseline PD--0.2037 ( σ2\sigma^{2} : 0.0010)
Static Offline NN0.0001 ( σ2\sigma^{2} : 0.0000)0.0463 ( σ2\sigma^{2} : 0.0014)0.1218 ( σ2\sigma^{2} : 0.0103)
Optimal Online NN--0.0044 ( σ2\sigma^{2} : 0.0000)
Ablation: Sluggish LR--0.7096 ( σ2\sigma^{2} : 13.9977)
Ablation: Unstable LR--19.3110 ( σ2\sigma^{2} : 1.3640)
Table 1: Monte Carlo Summary of Experimental Error Metrics ( N=100N=100 ). Results report the mean and variance across the 36 stable runs.
Figure 3: Simulation of the Action Error and Online Adaptation in a noisy nonlinear tracking task. (a, c) Standard Architectures: The Baseline PD (dashed black) exhibits phase lag due to uncompensated cubic stiffness. The Static Offline NN (red) pushes the system out-of-distribution, forcing an Action Error. The Optimal Online NN (blue) utilizes continuous gradient descent with a bounded learning rate ( ∥∇θL∥≤0.5\|\nabla_{\theta}L\|\leq 0.5 ) to mitigate noise and track the limit cycle. (b, d) Ablation Studies: A sluggish learning rate ( α=0.01\alpha=0.01 , green) fails to adapt to the moving target, causing persistent phase lag. Conversely, applying the aggressive rate ( α=0.50\alpha=0.50 , purple) without gradient clipping violates time-scale separation; the network overreacts to transient errors, inducing mechanical chatter.
ArchitectureTracking Error (MSE)Amplitude Distortion (%)
MRAC (Linear-in-the-Parameters)0.00622.49
Dual IOP (Frozen Phase 2)0.212248.04
Table 2: Empirical Tracking Error of Structural and Adaptive Baselines
Figure 4: Evaluation of proposed baselines on the 1-DOF nonlinear Duffing oscillator. While the MRAC bounds the trajectory via its linear-in-the-parameters Lyapunov formulation, it suffers from a persistent tracking lag due to its inability to capture the unmodeled cubic stiffness. Conversely, the Dual IOP model, despite achieving unbiased closed-loop identification in Phase 1, distorts the amplitude upon static deployment, demonstrating the persistence of the Action Error independent of calibration bias.
More in this topic

Distribution Shift

Distribution shift studies how changes between training and deployment data affect model reliability, and how to detect, measure, and address them.

See all 168 papers

Explore similar work

CardsList

  1. Stability-Preserving Online Adaptation of Neural Closed-loop Maps

    Mar 23, 2026Danilo Saccani, Luca Furieri, Giancarlo Ferrari-TrecateFeedback ControlNeural Network Stability

    Institute of Mechanical Engineering, Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland. · Department of Engineering Science, University of Oxford, United Kingdom

  2. Neural ODEs Meet Concurrent Learning: Stable Online Learning with Lyapunov Guarantees

    Sep 26, 2026Omkar Sudhir PatilNeural ODEsNeural Network Training Dynamics

    Division of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803, USA

  3. Neural-ESO: A Dual-Pathway Architecture for Provably Robust Learning-Based Control

    Jul 7, 2026Fan Zhang, Richie Suganda, Jinfeng Chen +4Robust ControlDisturbance Rejection

    Department of Engineering Technology, University of Houston, USA · Department of Electrical and Computer Engineering, University of Houston, USA · Department of Electrical Engineering, Rice University, USA