High-Dimensional Markov-Switching Ordinary Differential Processes
Organizations: Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign · Marshall School of Business, University of Southern California · Department of Computer Science, Stanford University
Abstract
Complex systems often evolve in continuous time but are observed only at discrete time points with measurement noise. When the interactions can change across unobserved states, recovering both the state transitions and the differential equations is important for mechanistic insight and for identifying time-varying directed graphs. However, this recovery is challenging in high dimensions. In this paper, we study estimation and graph recovery for high-dimensional Markov-switching ordinary differential equation models from noisy discrete observations. We model the switching between states by a latent continuous-time Markov chain, and within each state the drift is a sparse additive function with unknown nonlinear components. We approximate each component by a finite basis expansion and estimate the parameters under group sparsity. Our procedure estimates the latent trajectories by wavelet smoothing and then runs a penalized surrogate expectation-maximization algorithm. Under stationarity and geometric -mixing, given a sufficiently accurate estimated trajectory and other local regularity conditions, we show that the algorithm converges geometrically to a neighborhood of the true parameters. Finally, in an application to resting-state fMRI data, the procedure identifies multiple connectivity states, whose occupation times we compare descriptively between the children with combined-type ADHD and the typically developing children used in the fit.
Figures & tables
| Data-Generating Process 1 | Data-Generating Process 2 | |||||||
| Proposed Method | Oracle Method | Proposed Method | Oracle Method | |||||
| State | 1 | 2 | 1 | 2 | 1 | 2 | 1 | 2 |
| N | ||||||||
| 40 | — | — | 0.89(0.025) | 0.88(0.020) | 0.53(0.025) | 0.60(0.016) | 0.77(0.021) | 0.50(0.037) |
| 80 | 0.53(0.048) | 0.52(0.073) | 0.90(0.025) | 0.90(0.039) | 0.62(0.068) | 0.83(0.027) | 0.79(0.026) | 0.92(0.012) |
| 120 | 0.63(0.173) | 0.64(0.177) | 0.91(0.017) | 0.90(0.023) | 0.81(0.128) | 0.95(0.005) | 0.93(0.018) | 1.00(0.002) |
| group | state | state | state | |
|---|---|---|---|---|
| total occupation time | TDC | |||
| ADHD-C | ||||
| average occupation time ( ) | TDC | |||
| ADHD-C |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Data-Generating Process 1 | Data-Generating Process 2 | |||||||
| Proposed Method | Oracle Method | Proposed Method | Oracle Method | |||||
| State | 1 | 2 | 1 | 2 | 1 | 2 | 1 | 2 |
| N | ||||||||
| 40 | — | — | 0.90(0.016) | 0.80(0.027) | 0.53(0.049) | 0.84(0.010) | 0.86(0.006) | 0.90(0.005) |
| 80 | 0.93(0.056) | 0.83(0.029) | 0.94(0.026) | 0.92(0.012) | 0.81(0.059) | 0.98(0.002) | 1.00(0.001) | 1.00(0.000) |
| 120 | 0.92(0.046) | 0.85(0.040) | 0.95(0.017) | 0.93(0.014) | 0.90(0.135) | 0.98(0.000) | 1.00(0.000) | 1.00(0.000) |
| Data-Generating Process 1 | Data-Generating Process 2 | |||||||
|---|---|---|---|---|---|---|---|---|
| MS-VAR Baseline | Oracle MS-VAR Baseline | MS-VAR Baseline | Oracle MS-VAR Baseline | |||||
| State | 1 | 2 | 1 | 2 | 1 | 2 | 1 | 2 |
| N | ||||||||
| 80 | 0.65(0.070) | 0.82(0.020) | 0.83(0.007) | 0.64(0.030) | 0.78(0.048) | 0.86(0.014) | 0.82(0.009) | 0.94(0.002) |
| 120 | 0.58(0.030) | 0.79(0.025) | 0.84(0.003) | 0.67(0.012) | 0.85(0.041) | 0.91(0.003) | 0.93(0.006) | 0.98(0.001) |
| 160 | 0.62(0.047) | 0.81(0.025) | 0.84(0.004) | 0.72(0.014) | 0.72(0.077) | 0.91(0.002) | 0.94(0.003) | 0.96(0.001) |
| State | State | State | |
|---|---|---|---|
| Nonzero estimates (density) | |||
| Positive / negative nonzero estimates | |||
| Total absolute magnitude | |||
| Mean absolute magnitude | |||
| Edges displayed |
Latent Dynamics Modeling
Latent dynamics modeling learns compact hidden-state representations and transition rules to predict, explain, and control evolving systems.
See all 274 papers