math.OCNov 13, 2026

Averaged-Cost Control with Shared Feedback under Model Uncertainty

Authors: Piernicola Bettiol, Daisuke Inoue, Matías Gómez-Aedo, Dante Kalise, Richard B. Vinter

Organizations: Université de Bretagne Occidentale · Imperial College London

Abstract

This paper studies the shared-feedback design problem for finite-horizon averaged-cost control under model uncertainty represented by a joint law over the model parameter and initial state. One feedback map is shared across all realizations and uses only time and the current state as online inputs. We develop two methods: the nonlinear adjoint method combines forward closed-loop solves, backward adjoints, and joint-law policy gradients, while the linear moment method gives exact cost and gradient formulas for a homogeneous linear-quadratic design model by propagating conditional second moments. Numerical studies compare averaged and nominal designs for time-dependent neural feedback and static linear feedback in nonlinear Van der Pol and Stuart-Landau systems; the static gains are designed from oscillator linearizations and evaluated on the original nonlinear systems. Averaged design has lower cost in most comparisons.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jun 8, 2026

Families of Control-Cost-Parametrized Inverse-Optimal Universal Stabilizers

A classical universal stabilization formula offers the practitioner no design freedom: it is a single, parameter-free object. We introduce a cost-parametrized family of stabilizing feedback laws, where (1) the user chooses a function that serves as the running cost on control in an inverse-optimal cost functional, and (2) obtains, through a formula, a nonlinear "expander" of a pre-existing universal controller, which solves an infinite-horizon optimal control problem with a meaningful cost on the state. The cost-to-expander formula is a three-step construction, involving, inter alia, cost differentiation and function inversion-overall, a nonlinear infinite-dimensional operator. The cost-to-expander operator is proven Lipschitz, which enables uniform neural operator approximation of the entire family and supports both offline performance exploration and online adaptation. Semiglobal practical asymptotic stability and second-order suboptimality bounds are established under the approximation. The operator learning and its use in semiglobal stabilization are illustrated numerically. We call the result 'half-direct-optimal' because the paper's design is less than a general 'direct optimal' (HJB-inducing) control, but more than the fully inverse optimal, since the user performs minimization for an arbitrary given cost on control. The dual to the half-direct problem we solve is the problem in which the cost on the state is arbitrary and given. This dual problem is easier and outside of the scope of the paper.
Oct 26, 2026

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.
Nov 9, 2026

Generalizable Output Feedback Optimal Control via Transformers Without Explicit State Estimation

Classical optimal control is used to design a separate controller for each plant. This paper asks a different question. Can one learned policy serve a whole family of partially observed, noisy linear systems? We show that one transformer can. The policy is trained to imitate optimal LQG control on a heterogeneous family of Linear Time-Invariant (LTI) systems with Multiple Inputs and Multiple Outputs (MIMO), whose dimensions, cost matrices, and noise levels all vary. A shared representation with system-wise standardization, zero-padding, masking, and an encoding of the cost and noise covariances lets one set of weights control every member of the family. At run time, a short window of noisy measurements is all the policy needs to produce a control action, with no plant matrices and no state estimator. We characterize the relation between the optimal LQG policy (represented as a dynamic state-space model) and a static transformer acting on a finite window of measurements. We then express the closed-loop excess cost of any causal policy as the energy of its deviation from the optimal action. Finally, we evaluate the approach on 28 simulated systems. The trained policy reaches the optimal cost to within a median gap of 0.4%0.4\% for seen systems. For unseen systems, a light fine-tuning step brings the median gap to 1.0%1.0\%.