Ordinary differential equations (ODEs) provide a powerful framework for modeling dynamic systems arising in a wide range of scientific domains. However, most existing ODE methods focus on a single system, and do not adequately address the problem of learning shared patterns from multiple heterogeneous dynamic systems. In this article, we propose a novel distributionally robust learning approach for modeling heterogeneous ODE systems. Specifically, we construct a robust dynamic system by maximizing a worst-case reward over an uncertainty class formed by convex combinations of the derivatives of trajectories. We show the resulting estimator admits an explicit weighted average representation, where the weights are obtained from a quadratic optimization that balances information across multiple data sources. We further develop a bi-level stabilization procedure to address potential instability in estimation. We establish rigorous theoretical guarantees for the proposed method, including consistency of the stabilized weights, error bound for robust trajectory estimation, and asymptotical validity of pointwise confidence interval. We demonstrate that the proposed method considerably improves the generalization performance compared to the alternative solutions through both extensive simulations and the analysis of an intracranial electroencephalogram data.
Figures & tables
Figure 1: Stability of estimating the weight ω∗ .
K=5
K=10
Case
Level
Loss
Proposed
ERM
IVS
Proposed
ERM
IVS
Stable
I
Max.
0.068 (0.002)
1.067 (0.002)
0.424 (0.003)
0.066 (0.002)
1.072 (0.001)
0.418 (0.004)
Avg.
0.068 (0.002)
1.053 (0.002)
0.411 (0.003)
0.066 (0.002)
1.055 (0.001)
0.399 (0.004)
Gen.
0.068 (0.002)
1.049 (0.001)
0.406 (0.003)
0.066 (0.002)
1.050 (0.001)
0.393 (0.004)
II
Max.
0.089 (0.003)
1.246 (0.002)
0.454 (0.002)
0.134 (0.003)
1.433 (0.002)
0.527 (0.009)
Avg.
0.076 (0.002)
1.197 (0.002)
0.439 (0.002)
0.095 (0.002)
1.328 (0.001)
0.500 (0.008)
Table 1: Estimation performance for Example 1 . Reported are the average max loss, average loss, and generalization loss, with the standard deviation in the parenthesis, based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} . Three solutions are compared, the proposed method, the empirical risk minimization method (ERM), and the invariance score method (IVS).
K=5
K=10
Case
Level
Loss
Proposed
ERM
IVS
Proposed
ERM
IVS
Stable
I
Max.
0.044 (0.000)
4.843 (0.001)
4.798 (0.006)
0.043 (0.000)
4.851 (0.001)
4.775 (0.003)
Avg.
0.044 (0.000)
4.838 (0.001)
4.790 (0.006)
0.043 (0.000)
4.845 (0.001)
4.766 (0.003)
Gen.
0.044 (0.000)
4.758 (0.002)
4.708 (0.008)
0.043 (0.000)
4.770 (0.001)
4.675 (0.004)
II
Max.
0.162 (0.000)
6.108 (0.001)
6.038 (0.006)
0.238 (0.000)
7.522 (0.001)
7.417 (0.004)
Avg.
0.139 (0.000)
5.258 (0.001)
5.191 (0.006)
0.209 (0.000)
5.534 (0.001)
5.427 (0.003)
Table 2: Estimation performance for Example 2 . Reported are the average max loss, average loss, and generalization loss ( ×104 ), with the standard deviation in the parenthesis, based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} . Three solutions are compared, the proposed method, the empirical risk minimization method (ERM), and the invariance score method (IVS).
Stable
Unstable
K=5
K=10
K=5
K=10
Example
Level
ECP
CIL
ECP
CIL
ECP
CIL
ECP
CIL
Example 1
I
97.0%
0.469
95.8%
0.453
96.2%
0.476
96.1%
0.431
II
96.9%
0.499
95.3%
0.439
96.7%
0.472
95.5%
0.444
III
96.7%
0.453
96.1%
0.439
96.4%
0.450
95.8%
0.459
Example 2
I
94.5%
2.786
94.9%
2.490
94.8%
2.872
94.9%
2.582
Table 3: Inference performance for Examples 1 and 2 . Reported are the empirical coverage probability (ECP) and confidence interval length (CIL) based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} .
Subject ID
Proposed vs ERM
Proposed vs IVS
BJH016
124/190 (65.3%)
113/190 (59.5%)
BJH021
92/164 (56.1%)
92/164 (56.1%)
BJH025
85/132 (64.4%)
85/132 (64.4%)
LL12
76/151 (50.3%)
59/151 (39.1%)
SLCH002
64/130 (49.2%)
64/130 (49.2%)
Overall
441/767 (57.5%)
413/767 (53.8%)
Table 4: Leave-one-subject-out cross-validation. Reported are the trial-level pairwise comparison frequencies, defined as the percentage of held-out trials for which the proposed method achieves a normalized loss ( 14 ) no greater than the competing method. A higher frequency indicates a more reliable generalization of the proposed method.
Figure 2: Reconstruction of brain connectivity network during AAC by ERM (bottom left), IVS (bottom right), and the proposed method (top right), compared to the reference network (top left). Green solid edges denote the correctly recovered reference edges (true positive), red dashed edges denote the incorrectly recovered reference edges (false positive), and gray dotted edges denote the missed reference edges (false negative).
Dynamical systems studies how states evolve over time, including stability, feedback, chaos, and transitions in physical, biological, and engineered systems.
By focusing on algorithmic stability as a means of establishing out-of-sample bounds, we provide a system-theoretic interpretation of generalization in learning-enabled dynamical systems arising in data-driven optimization and feedback control approximation. Given two neighboring datasets, we specifically model sample replacement as an exogenous disturbance acting on a sensitivity system, while the incremental behavior of the data-dependent operator is encoded through an integral quadratic constraint. By relying on dissipativity arguments, we establish a matrix inequality-based certificate and a uniform stability bound that separates the one-sample sensitivity of the learned operator, and an algorithm-dependent dynamical gain. The latter can then be optimized, offering a tractable tool for certifying and comparing generalization capabilities of learning dynamics. We show that our results recover classical ones for gradient descent, apply naturally to momentum-based methods such as heavy-ball and Nesterov acceleration, and extend to data-driven control.
Filippo Fabiani
IMT School for Advanced Studies Lucca, Piazza San Francesco 19, 55100, Lucca, Italy
We consider the problem of learning linear dynamical systems under adversarial contamination from a single trajectory of length T. While identification of linear dynamical systems itself is well-studied, the problem of robust system identification under adversarial contamination is relatively less explored. In this work, we study the setting where a fraction of the T observations are contaminated by adversarial outliers. We propose different estimators based on relaxations of least-trimmed squares along with an alternating minimization algorithm. Furthermore, we also propose two estimators which exploit the group-sparsity (through penalization/hard-constraints) of the outliers. For the estimator with group-sparse penalty, we derive non-asymptotic error bounds which establish its robustness to outliers. We also show empirically that the proposed estimators work well in practice.
Aravinda Kanchana Ruwanpathirana, Hemant Tyagi
Division of Mathematical Sciences, SPMS, NTU Singapore 637371
Neural ODEs learn dynamics from trajectory losses, but their adjoint gradients lack the regressor-times-parameter-error structure on which Lyapunov analyses of online adaptation rest, so training on streaming data comes without stability guarantees. We show that this structure is in fact present: the adjoint gradient decomposes exactly into a positive semi-definite trajectory operator acting on the parameter error plus a nonlinear perturbation with explicit, horizon-dependent bounds. A quadratic Lyapunov function then certifies online Neural ODE training over sliding windows under computable gain and horizon conditions, and the same certificate extends to stored data: its drift branch recovers concurrent learning, and its trajectory branch yields NODE-CL, a stored-segment Gauss-Newton method built on batched forward sensitivities that needs no state-derivative estimates. On four DeepMind Control Suite domains, NODE-CL attains the lowest median prediction error on three under velocity measurement noise, where observer-based concurrent learning degrades by up to 8x; with clean measurements it is best on the pendulum and within a factor of 1.6 of the best stored-data baseline on the cartpole and reacher.
Omkar Sudhir Patil
Division of Electrical and Computer Engineering Louisiana State University Baton Rouge, LA 70803, USA