cs.LGNov 6, 2026

From PAC bounds on the generalization error to PAC bounds on the parameter estimation error

Authors: Mihaly Petreczky, John Leth, Emma Lukacs, Balint Daroczy

Organizations: Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France · Department of Electronic Systems, Aalborg University, Denmark · Institute for Computer Science and Control, HUN-REN SZTAKI, Budapest, Hungary

Abstract

Probably Approximately Correct (PAC) bounds on the generalization error (the difference between the empirical prediction error computed on the training data and its expectation) are an established tool in statistical machine learning, and there have been several extensions of these bounds to dynamical systems. In parallel, in recent years, there has been growing interest in finite-sample bounds for parameter estimation error in system identification. The latter bounds tend to be algorithm-specific, relying on reformulating the identification algorithm as a least-squares regression problem. This is in contrast to classical PAC bounds, which are algorithm-agnostic and can be applied to any identification algorithm based on minimizing a cost function involving the empirical loss. In this paper, we show how oracle bounds derived from PAC bounds can be used to derive finite-sample bounds for dynamical systems. As an illustration of this approach, we present finite-sample parameter-estimation bounds for linear and bilinear systems with bounded inputs and noise. These bounds apply to identification algorithms based on minimizing a regularized empirical loss function. The key ingredient is to establish an oracle inequality, i.e., an inequality between the true prediction error of the learned model and the minimum achievable true prediction error, and then use either the convexity of the true loss or system-theoretic properties of the loss function to derive bounds on the parameter estimation error. We also analyze the rates of the obtained bounds and compare them with those of classical asymptotic results.

Explore similar work

CardsList
  1. PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise

    Sep 8, 2026Mihaly Petreczky, Mohamad Al Ahdab, John LethPAC-Bayesian Generalization BoundsStochastic Linear Dynamical Systems

  2. CLT-Optimal Parameter Error Bounds for Linear System Identification

    Apr 23, 2026Yichen Zhou, Stephen TuDynamical SystemsParameter Estimation

  3. Uniform error bounds for quantized dynamical models

    Feb 17, 2026Abdelkader Metakalard, Fabien Lauer, Kevin Colin +1System IdentificationGeneralization Bounds