Ordinary differential equations (ODEs) provide a powerful framework for modeling dynamic systems arising in a wide range of scientific domains. However, most existing ODE methods focus on a single system, and do not adequately address the problem of learning shared patterns from multiple heterogeneous dynamic systems. In this article, we propose a novel distributionally robust learning approach for modeling heterogeneous ODE systems. Specifically, we construct a robust dynamic system by maximizing a worst-case reward over an uncertainty class formed by convex combinations of the derivatives of trajectories. We show the resulting estimator admits an explicit weighted average representation, where the weights are obtained from a quadratic optimization that balances information across multiple data sources. We further develop a bi-level stabilization procedure to address potential instability in estimation. We establish rigorous theoretical guarantees for the proposed method, including consistency of the stabilized weights, error bound for robust trajectory estimation, and asymptotical validity of pointwise confidence interval. We demonstrate that the proposed method considerably improves the generalization performance compared to the alternative solutions through both extensive simulations and the analysis of an intracranial electroencephalogram data.
Figures & tables
Figure 1: Stability of estimating the weight ω∗ .
K=5
K=10
Case
Level
Loss
Proposed
ERM
IVS
Proposed
ERM
IVS
Stable
I
Max.
0.068 (0.002)
1.067 (0.002)
0.424 (0.003)
0.066 (0.002)
1.072 (0.001)
0.418 (0.004)
Avg.
0.068 (0.002)
1.053 (0.002)
0.411 (0.003)
0.066 (0.002)
1.055 (0.001)
0.399 (0.004)
Gen.
0.068 (0.002)
1.049 (0.001)
0.406 (0.003)
0.066 (0.002)
1.050 (0.001)
0.393 (0.004)
II
Max.
0.089 (0.003)
1.246 (0.002)
0.454 (0.002)
0.134 (0.003)
1.433 (0.002)
0.527 (0.009)
Avg.
0.076 (0.002)
1.197 (0.002)
0.439 (0.002)
0.095 (0.002)
1.328 (0.001)
0.500 (0.008)
Table 1: Estimation performance for Example 1 . Reported are the average max loss, average loss, and generalization loss, with the standard deviation in the parenthesis, based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} . Three solutions are compared, the proposed method, the empirical risk minimization method (ERM), and the invariance score method (IVS).
K=5
K=10
Case
Level
Loss
Proposed
ERM
IVS
Proposed
ERM
IVS
Stable
I
Max.
0.044 (0.000)
4.843 (0.001)
4.798 (0.006)
0.043 (0.000)
4.851 (0.001)
4.775 (0.003)
Avg.
0.044 (0.000)
4.838 (0.001)
4.790 (0.006)
0.043 (0.000)
4.845 (0.001)
4.766 (0.003)
Gen.
0.044 (0.000)
4.758 (0.002)
4.708 (0.008)
0.043 (0.000)
4.770 (0.001)
4.675 (0.004)
II
Max.
0.162 (0.000)
6.108 (0.001)
6.038 (0.006)
0.238 (0.000)
7.522 (0.001)
7.417 (0.004)
Avg.
0.139 (0.000)
5.258 (0.001)
5.191 (0.006)
0.209 (0.000)
5.534 (0.001)
5.427 (0.003)
Table 2: Estimation performance for Example 2 . Reported are the average max loss, average loss, and generalization loss ( ×104 ), with the standard deviation in the parenthesis, based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} . Three solutions are compared, the proposed method, the empirical risk minimization method (ERM), and the invariance score method (IVS).
Stable
Unstable
K=5
K=10
K=5
K=10
Example
Level
ECP
CIL
ECP
CIL
ECP
CIL
ECP
CIL
Example 1
I
97.0%
0.469
95.8%
0.453
96.2%
0.476
96.1%
0.431
II
96.9%
0.499
95.3%
0.439
96.7%
0.472
95.5%
0.444
III
96.7%
0.453
96.1%
0.439
96.4%
0.450
95.8%
0.459
Example 2
I
94.5%
2.786
94.9%
2.490
94.8%
2.872
94.9%
2.582
Table 3: Inference performance for Examples 1 and 2 . Reported are the empirical coverage probability (ECP) and confidence interval length (CIL) based on 100 data replications. Considered models include the stable and unstable case, each with three levels of heterogeneity, and two values of number of subjects K={5,10} .
Subject ID
Proposed vs ERM
Proposed vs IVS
BJH016
124/190 (65.3%)
113/190 (59.5%)
BJH021
92/164 (56.1%)
92/164 (56.1%)
BJH025
85/132 (64.4%)
85/132 (64.4%)
LL12
76/151 (50.3%)
59/151 (39.1%)
SLCH002
64/130 (49.2%)
64/130 (49.2%)
Overall
441/767 (57.5%)
413/767 (53.8%)
Table 4: Leave-one-subject-out cross-validation. Reported are the trial-level pairwise comparison frequencies, defined as the percentage of held-out trials for which the proposed method achieves a normalized loss ( 14 ) no greater than the competing method. A higher frequency indicates a more reliable generalization of the proposed method.
Figure 2: Reconstruction of brain connectivity network during AAC by ERM (bottom left), IVS (bottom right), and the proposed method (top right), compared to the reference network (top left). Green solid edges denote the correctly recovered reference edges (true positive), red dashed edges denote the incorrectly recovered reference edges (false positive), and gray dotted edges denote the missed reference edges (false negative).
Dynamical systems studies how states evolve over time, including stability, feedback, chaos, and transitions in physical, biological, and engineered systems.