This paper provides statistical guarantees on the accuracy of dynamical models learned from dependent data sequences. Specifically, we develop uniform error bounds that apply to quantized models and imperfect optimization algorithms commonly used in practical contexts for system identification, and in particular hybrid system identification. Two families of bounds are obtained: slow-rate bounds via a block decomposition and fast-rate, variance-adaptive, bounds via a novel spaced-point strategy. The bounds scale with the number of bits required to encode the model and thus translate hardware constraints into interpretable statistical complexities.
Figures & tables
Figure 1: Two intertwined block sequences: light gray for S1 , black for S2 .
Figure 2: Spaced point selection in ( 3 ), showing the first four and the last points, with a points between each pair.
Figure 3: Slow and fast rate error bounds as functions of n when modeling ( 5 ).
Slow (Theorem 4 )
Fast (Theorem 5 )
a
μ
L^n(f)
Confidence interval
Total
μ′
L^nspaced(f)
Confidence interval
Total
17
5882
0.9823
3.312
4.355
11764
0.9869
0.630
1.691
21
4761
0.9823
3.701
4.683
9523
0.9650
0.729
1.766
25
3999
0.9823
4.036
5.018
7999
0.9862
0.834
1.902
30
3333
0.9823
4.421
5.403
6666
0.9820
0.954
2.031
40
2499
0.9823
5.105
6.088
4999
0.9679
1.183
2.270
Table 1: Comparison between the slow and fast rate bounds on the linear system identification example of Sect. 6.1 for δ=0.05 and various block sizes a .
Bound
Full confidence
Complexity term
interval
(mixing-free)
(with mixing)
Massucci et al. (2022) , ( 9 )
23.48
27.77
Theorem 4 , ( 22 )
19.06
19.23
Theorem 5 , ( 23 )
10.23
10.41
Table 2: Comparison of error bounds for switched system identification.
In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.
Mihaly Petreczky, Mohamad Al Ahdab, John Leth
CRIStAL, Centrale Lille, Universit´e de Lille, France · Department of Electronic Systems, Aalborg University, Denmark
There has been remarkable progress over the past decade in establishing finite-sample, non-asymptotic bounds on recovering unknown system parameters from observed system behavior. Surprisingly, however, we show that the current state-of-the-art bounds do not accurately capture the statistical complexity of system identification, even in the most fundamental setting of estimating a discrete-time linear dynamical system (LDS) via ordinary least-squares regression (OLS). Specifically, we utilize asymptotic normality to identify classes of problem instances for which current bounds overstate the squared parameter error, in both spectral and Frobenius norm, by a factor of the state-dimension of the system. Informed by this discrepancy, we then sharpen the OLS parameter error bounds via a novel second-order decomposition of the parameter error, where crucially the lower-order term is a matrix-valued martingale that we show correctly captures the CLT scaling. From our analysis we obtain finite-sample bounds for both (i) stable systems and (ii) the many-trajectories setting that match the instance-specific optimal rates up to constant factors in Frobenius norm, and polylogarithmic state-dimension factors in spectral norm.
Yichen Zhou, Stephen Tu
Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, California, USA.
We introduce the Trajectory-based Quantization Sensitivity Score (TQS), a metric that reframes post-training quantization (PTQ) through the lens of dynamical-systems stability. By modeling the network's rollout as a discrete-time dynamical system, TQS characterizes how quantization-induced errors propagate and amplify over the rollout horizon. Unlike conventional PTQ methods, where sensitivity analysis is often coupled to the quantization procedure, TQS enables a priori sensitivity estimation decoupled from quantizer selection and bit-width assignment. This separation allows for quantization budget planning even for black-box or compiled networks with fused operators. Building on this, we present TQS-PTQ, a flexible mixed-precision framework that requires no calibration data or costly second-order approximations. Our experiments show that a dynamical-systems perspective provides a robust, high-performing pathway for low-precision deployment in resource-constrained settings.
Mariya Pavlova, Harrison Bo Hua Zhu, Lidia Vitanova +2
Department of Computing, Imperial College London, London, United Kingdom · Department of Public Health, University of Copenhagen, Copenhagen, Denmark · GATE Institute, Sofia University St. Kliment Ohridski, Sofia, Bulgaria +1