eess.SYJul 6, 2025

Action-Increment-Constrained Policy Optimization for Smooth Online Adaptive Critic Flight Control with Actuator Rate Constraints and Low Computational Complexity (extended version with Ultimate Uniform Boundedness analysis)

Authors: Yifei Li, Erik-Jan van Kampen

Organizations: Section Control and Simulation, Faculty of Aerospace Engineering, Delft University of Technology, Kluyverweg 1, 2629 HS, Delft, the Netherlands

Abstract

Approximate Dynamic Programming (ADP) enables online control-law adaptation through adaptive critic methods. A safety challenge in applying ADP to angle-of-attack (AoA) tracking control is that an overly sensitive neural network (NN) controller may generate oscillatory or saturated control actions under nonlinear dynamics, potentially compromising flight stability and increasing actuator stress. To address this challenge, this paper proposes two innovations: (1) the actuator rate constraints are enforced within a an Action-Increment-Constrained Policy Optimization (AI-CPO) framework inspired by safe reinforcement learning (RL), in which a primal-dual approach adaptively adjusts the penalty weight to address actuator rate constraints. This optimization is solved online using adaptive Temporally Smoothed Incremental Model-Based Heuristic Dynamic Programming (TS-IHDP) for smooth and computationally efficient online adaptive control; and (2) a low-pass filter integrated into the cascaded NN architecture to attenuate high-frequency pitch-rate commands while preserving the Markov property during online learning by exploiting the hierarchical actor structure. Compared with the IHDP baseline, simulations using a longitudinal air vehicle dynamics model demonstrate that TS-IHDP improves control smoothness and tracking performance, reduces system oscillations, and enhances the trained policy's robustness to model uncertainties.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learned Lyapunov Shielding for Adaptive Control

    May 7, 2026Giansalvo Cirrincione, Adriano FagioliniLyapunov-Based ControlSafe RL

  2. An integrated interpretable control effectiveness learning and nonlinear control allocation methodology for overactuated aircraft

    Jun 11, 2026Umut Demir, Aamir Ahmad, Torbjørn Cunis +1Online System IdentificationFeedback Control

  3. Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints

    Aug 15, 2026Saurabh Kumar, Shashi Ranjan Kumar, Abhinav SinhaBackstepping ControlSafety-Critical Control