A Decade of Learning-Based Control: From Theoretic Foundations to Practical Deployment

Angela P. Schoellig; Matthias A. Müller; Sebastian Trimpe; Melanie N. Zeilinger; Mahathi Anand; Andrea Carron; Oliver Hausdörfer; Victor G. Lopez

eess.SYOct 21, 2026

A Decade of Learning-Based Control: From Theoretic Foundations to Practical Deployment

Authors: Angela P. Schoellig, Matthias A. Müller, Sebastian Trimpe, Melanie N. Zeilinger, Mahathi Anand, Andrea Carron, Oliver Hausdörfer, Victor G. Lopez,

Organizations: Chair of Safety, Performance, and Reliability for Learning Systems, Technical University of Munich · Institute of Automatic Control, Leibniz University Hannover · Institute for Data Science in Mechanical Engineering, RWTH Aachen University · Institute for Dynamical Systems and Control, ETH Zurich ·

Abstract

Motivated by ten years of Learning-Based Control Invited Sessions at the IEEE Conference on Decision and Control (CDC), this tutorial paper reviews key advances at the intersection of machine learning and control. We focus on methods that combine the flexibility of machine learning techniques with the structure and theoretical foundations of control to design autonomous systems that are capable, reliable, and safe. We first introduce core notions for establishing stability and safety guarantees, then review approaches that learn dynamics models to improve control under uncertainty, and finally discuss data-driven methods that directly synthesize or learn policies, including reinforcement learning, imitation learning, and Bayesian optimization. We further discuss representative applications and open challenges. The goal is to provide an accessible entry point to the field and to support continued progress toward reliable learning-based control in real-world systems. To support learning and practical exploration, we share implementations and illustrative examples along with our discussion, together with additional resources on our website: https://learning-based-control.github.io/tutorial/.

Figures & tables

Fig. 1 : Scope of the tutorial paper. We review learning-based control methods that combine machine learning with control-theoretic structure to improve system capabilities while preserving desired closed-loop properties. The paper first introduces foundations for stability and safety guarantees (green), then covers methodological paradigms ranging from structured model-based learning to increasingly data-driven policy learning (orange), and finally discusses representative application domains (grey).

Fig. 2 : Evolution of recurring research threads in the CDC Learning-Based Control Invited Sessions from 2016 through 2025. The heatmap shows the annual frequency of representative concept phrases extracted from paper titles and abstracts. Details of the keyword extraction and analysis procedure are provided in the appendix.

Fig. 3 : Citation statistics for papers presented in the CDC Learning-Based Control Invited Sessions. The bars show the total number of citations received per year, while the line indicates cumulative citations based on Google Scholar data collected in May 2026.

Fig. 4 : The percentage of papers with the term “learning” in their titles across three control conferences (CDC, ACC, and ECC) from 2010 to 2025. Pertinent data was obtained from the DBLP database in May 2026.

Method Family	What is Learned?	Prior Knowledge Required	Learning from Data	Estimation Errors	Compatible Controller Synthesis Methods	Resulting Closed-Loop Guarantees
Dynamics learning for control ( Section IV ) . All methods can additionally include a known nominal model.
Linear regression ( Section IV-B )	point estimates of coefficients in a linear / fixed-basis nonlinear dynamics model	(locally) linear structure / (nonlinear) basis functions	closed-form least-squares solution based on (offline) input-state data	norm bounds or ellipsoidal uncertainty sets for model coefficients	certainty-equivalent and robust LQR; MPC	robust stability; robust constraint satisfaction; probabilistic constraint satisfaction
Neural networks ( Section IV-C )	point estimates of parameters in a nonlinear neural-network dynamics model	network architecture and activation functions; optional physical or dynamical structure (e.g., Lipschitz regularities)	gradient-based training based on (offline) input-state data	deterministic error or output bounds based on regularities ; possibly, predictive uncertainty estimates via approximate (Bayesian) inference	nonlinear MPC ; Lyapunov and small-gain design	stability via Lipschitz or local sector bounds; ISS / dISS of recurrent models
Bayesian linear regression ( Section IV-D , Section IV-D )	posterior distribution over coefficients in a fixed-basis nonlinear dynamics model	(nonlinear) basis functions, prior parameter distribution, and noise model	closed-form posterior inference based on (offline) input-state data under Gaussian prior and iid noise assumptions	probabilistic predictive uncertainty bounds derived from the posterior parameter distribution	chance-constrained MPC with constraint tightening; robust MPC with conservative error bounds	probabilistic constraint satisfaction or worst-case bounds; overconfident if the basis is misspecified
Gaussian processes ( Section IV-D , Section IV-D , Section IV-D )	posterior distribution over the dynamics function represented by a nonparametric model	prior mean function, covariance kernel [ 15 ] , and noise model	closed-form posterior inference based on (offline) input-state data under Gaussian prior and iid noise assumptions	probabilistic predictive uncertainty bounds derived from the posterior distribution over the dynamics function	chance-constrained MPC; robust MPC using confidence regions; CLF -based synthesis	probabilistic constraint satisfaction; stability via robust invariant terminal sets and worst-case safety from confidence regions
Direct data-driven control ( Section V )

TABLE I : An overview of the learning-based control method families reviewed in Section IV - Section VI . For each method family, we detail the specific learning objective, requisite prior knowledge, and the data-driven learning mechanism. Additionally, we summarize how each method quantifies estimation errors, and how those uncertainty models pair with compatible controller synthesis techniques to yield closed-loop guarantees discussed in Section III . Note that the control methods and closed-loop guarantees presented in the table serve as representative examples, and are as such non-exhaustive.

Panel of Figure 6 — Fig. 5 : Illustration of the stability and safety notions of this section. (a) Nominal stability: a Lyapunov function $V$ decreases along closed-loop trajectories (level sets in grey) and the state converges to the equilibrium $\bar{x}$ . (b) Robust and stochastic stability: under disturbances $w$ , the state converges to a neighbourhood of $\bar{x}$ (grey), whose size depends on the disturbance bound or the noise covariance. (c) Safety: constraint satisfaction at one time instant does not imply safety (red dashed arrow); safety is guaranteed by an invariant set $\mathcal{S}$ (blue) contained in the constraint set (white). (d)–(f) Safety filters: the desired input $u_{\mathrm{des},k}$ would violate the constraint and is modified to $u_{k}^{\star}$ to stay inside of the constraint set (white). HJ reachability computes the maximal safe set $\mathcal{S}$ offline, the largest invariant set within the constraint set. A CBF certifies invariance of $\{b(x)\geq 0\}$ by the one-step condition on $b(x_{k+1})$ . A PSF does not compute the safe set but certifies safety by a backup trajectory (dotted) into a smaller terminal set $\mathcal{S}_{f}$ .

Figure 8

Figure 9

Figure 12

Fig. 13 : Number of papers that released open-source code throughout the years for the CDC Learning-Based Control Invited Sessions. In total, 38 of 182 papers, or 20.9% of the papers, released open-source code. Details of the data source and analysis procedure are included in the appendix.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 272 papers

Apr 2, 2026

Model-Based Reinforcement Learning for Control under Time-Varying Dynamics

Learning-based control methods typically assume stationary system dynamics, an assumption often violated in real-world systems due to drift, wear, or changing operating conditions. We study reinforcement learning for control under time-varying dynamics. We consider a continual model-based reinforcement learning setting in which an agent repeatedly learns and controls a dynamical system whose transition dynamics evolve across episodes. We analyze the problem using Gaussian process dynamics models under frequentist variation-budget assumptions. Our analysis shows that persistent non-stationarity requires explicitly limiting the influence of outdated data to maintain calibrated uncertainty and meaningful dynamic regret guarantees. Motivated by these insights, we propose a practical optimistic model-based reinforcement learning algorithm with adaptive data buffer mechanisms and demonstrate improved performance on continuous control benchmarks with non-stationary dynamics.

Klemens Iten, Bruce Lee, Chenhao Li +3

ETH Zürich, Rämistrasse 101, 8092 Zürich, Switzerland · ETH AI Center at ETH Zürich

RL Control Non-Stationary RL Model-Based RL Continual RL

Ask PDF

Apr 1, 2026

Safe learning-based control via function-based uncertainty quantification

Uncertainty quantification is essential when deploying learning-based control methods in safety-critical systems. This is commonly realized by constructing uncertainty tubes that enclose the unknown function of interest, e.g., the reward and constraint functions or the underlying dynamics model, with high probability. However, existing approaches for uncertainty quantification typically rely on restrictive assumptions that encode smoothness properties of the unknown function, such as a known norm in a function space. Moreover, these methods usually struggle with discontinuities. In this paper, we model the unknown function as a random function from which independent and identically distributed realizations can be generated. We then construct uncertainty tubes via the scenario approach that hold with high probability. Our uncertainty tubes rely solely on sampled realizations and can therefore accommodate discontinuities represented by the sampling model. We integrate these uncertainty tubes into a safe Bayesian optimization algorithm with which we safely tune control parameters on a real Furuta pendulum.

Abdullah Tokmak, Toni Karvonen, Thomas B. Schön +1

Cyber-physical Systems Group, Aalto University, Espoo, Finland · School of Engineering Sciences, Lappeenranta–Lahti University of Technology LUT, Lappeenranta, Finland · Department of Information Technology, Uppsala University, Uppsala, Sweden

Uncertainty Quantification Safety-Critical Control Bayesian Optimization Safe Bayesian Optimization

Ask PDF

Oct 4, 2026

Optimal Control with Learned Critics under Unmodeled State Dependencies

Model Predictive Control (MPC) provides a structured and constraint-aware mechanism for decision-making, but its reliance on optimization-friendly analytical dynamics models limits its use in tasks with contacts and other hard-to-model state dependencies. Model-free reinforcement learning avoids explicit modeling assumptions but typically requires large amounts of interaction data. We present a learning-based MPC framework that combines the data efficiency and structure of local model-based planning with learned components that compensate for incomplete dynamics and finite-horizon myopia. The method augments a nominal analytical model with a residual dynamics network that learns missing state-dependent effects from data and combines the resulting planner with a learned action-value critic that injects long-horizon MDP structure into the local iLQR optimization. To make this practical at reinforcement-learning scale, we develop a GPU-accelerated batched iLQR solver that evaluates learned dynamics and critic networks inside the optimal-control loop and solves thousands of trajectory-optimization problems in parallel. The complete system is integrated into a robotics simulator, enabling scalable model-based reinforcement learning under incomplete dynamics. Experiments on biased and incompletely modeled control tasks show that the approach improves closed-loop control performance while preserving the model-based structure needed for efficient constrained trajectory optimization.

Philipp Schoch, Markus Ryll

Autonomous Aerial Systems Lab Technical University of Munich

Residual Dynamics Learning Robot Dynamics Learning Model Predictive Control Model-Based RL

Ask PDF

Abstract

RL Control

Explore similar work

Model-Based Reinforcement Learning for Control under Time-Varying Dynamics

Safe learning-based control via function-based uncertainty quantification

Optimal Control with Learned Critics under Unmodeled State Dependencies