eess.SYJul 6, 2025

Action-Increment-Constrained Policy Optimization for Smooth Online Adaptive Critic Flight Control with Actuator Rate Constraints and Low Computational Complexity (extended version with Ultimate Uniform Boundedness analysis)

Authors: Yifei Li, Erik-Jan van Kampen

Organizations: Section Control and Simulation, Faculty of Aerospace Engineering, Delft University of Technology, Kluyverweg 1, 2629 HS, Delft, the Netherlands

Abstract

Approximate Dynamic Programming (ADP) enables online control-law adaptation through adaptive critic methods. A safety challenge in applying ADP to angle-of-attack (AoA) tracking control is that an overly sensitive neural network (NN) controller may generate oscillatory or saturated control actions under nonlinear dynamics, potentially compromising flight stability and increasing actuator stress. To address this challenge, this paper proposes two innovations: (1) the actuator rate constraints are enforced within a an Action-Increment-Constrained Policy Optimization (AI-CPO) framework inspired by safe reinforcement learning (RL), in which a primal-dual approach adaptively adjusts the penalty weight to address actuator rate constraints. This optimization is solved online using adaptive Temporally Smoothed Incremental Model-Based Heuristic Dynamic Programming (TS-IHDP) for smooth and computationally efficient online adaptive control; and (2) a low-pass filter integrated into the cascaded NN architecture to attenuate high-frequency pitch-rate commands while preserving the Markov property during online learning by exploiting the hierarchical actor structure. Compared with the IHDP baseline, simulations using a longitudinal air vehicle dynamics model demonstrate that TS-IHDP improves control smoothness and tracking performance, reduces system oscillations, and enhances the trained policy's robustness to model uncertainties.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 7, 2026

Learned Lyapunov Shielding for Adaptive Control

We augment the Slotine--Li adaptive controller for Euler--Lagrange systems with three learned components: a structured-quadratic Lyapunov function VψV_ψ whose positive-definiteness follows from a Cholesky parameterization, a residual Soft Actor--Critic policy that adds bounded torque corrections to the analytic baseline, and a physics-informed neural network that estimates unmodeled dynamics. A closed-form safety filter, derived from the single affine constraint V˙ψ+αVψ≤0\dot V_ψ+ αV_ψ\le 0, projects every policy output onto the safe set without requiring an online QP solver. We prove: global feasibility of the filter under a drift-decay condition on the control-degeneracy set; exponential stability under exact shielding, with a robust extension whose margin depends on the PINN approximation error; almost-sure convergence of the three-timescale policy--certificate--multiplier updates to a KKT point; and a PAC generalization bound for the certificate over compacts. On a 2-DOF manipulator with nonlinear friction and variable payload, the learned certificate accounts for most of the empirical gain: tracking error drops by 41% on nominal friction and 24% on aggressive friction at the centroid of the training distribution. A 7-DOF scalability study on a Franka Emika Panda confirms clean convergence of the full pipeline at industrial scale, identifies the conditions under which gains over exact model-based baselines should and should not be expected, and documents a warm-start pathology of the learned certificate that has practical implications for deployment.
Jun 11, 2026

An integrated interpretable control effectiveness learning and nonlinear control allocation methodology for overactuated aircraft

Nonlinear dynamics and the strong couplings that arise between multiple effectors undermine the assumptions behind conventional, linear control allocation techniques. When flight enters regimes where nonlinear effects dominate, linear allocators exhibit reduced accuracy due to increased model mismatch, which subsequently degrades performance and robustness of the flight control system. High fidelity onboard models and black box data driven approaches can recover accuracy across the flight envelope, but respectively impose computational burdens prohibitive for real time allocation and sacrifice the interpretability required for verification and fault diagnosis. This paper addresses these limitations by learning an explicit, physics constrained analytical model of the control effectiveness mapping from representative flight data using Sparse Identification of Nonlinear Dynamics. The resulting mapping is compact, interpretable, and admits analytical derivatives, enabling efficient computation within nonlinear solvers that additionally incorporate actuator dynamics, without requiring an onboard model. An online adaptation mechanism monitors prediction residuals and refreshes the model when significant plant changes are detected, providing graceful reconfiguration under actuator failures and varying operating conditions. The methodology is evaluated on a high fidelity nonlinear benchmark aircraft across a range of aggressive maneuvers, achieving accuracy comparable to a full nonlinear onboard model while substantially reducing computational cost relative to established baselines.
Aug 15, 2026

Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints

This paper develops Admissibility-Preserving Control (APC), a realization-centered safety-critical control framework for strict-feedback systems subject to asymmetric actuator limits, time-varying output constraints, and actuator-rate limitations. APC denotes the overall control architecture, whereas an Admissibility-Preserving Input Realization (APIR) denotes its constraint-realization module. Therein, the APIR dynamically generates the physical plant input while rendering its prescribed asymmetric actuator set forward invariant. In contrast to algebraic clipping and post-design saturation compensation, the actuator limits are embedded directly in a continuously differentiable dynamic realization with user-selectable regularity and interpretable tuning parameters. The APIR is integrated with recursive backstepping by treating the realized plant input as an additional state. The resulting design does not require an input-to-state stability assumption on the uncontrolled plant. Instead, the nonlinear drift terms are compensated recursively, subject to an explicit compatibility condition between the desired motion, the available control authority, and the APIR interior gain. The framework is further extended to time-varying output-safe tracking through a smooth asymmetric logarithmic barrier coordinate and its associated Lyapunov function and to simultaneous actuator-magnitude and rate constraints through a cascaded APIR. Rigorous Lyapunov and invariance analyses establish regional asymptotic tracking, forward invariance of the compatible admissible sets, and boundedness of all closed-loop signals. Numerical studies illustrate asymmetric actuator utilization, output-safety preservation, and magnitude-rate constraint enforcement.