DS3M: Deep structured state space models for nonlinear system identification on modern hardware
Organizations: University of Wuppertal, Wuppertal, Germany
Abstract
Neural state-space identification learns dynamical models from measured inputs and outputs. SUBNET combines this task with an encoder that estimates initial states from short measurement histories. Its continuous-time version requires many successive neural-network evaluations during numerical integration, resulting in slow training speed which is a major limitation for practical use cases. We propose deep structured state space models (DS3M), which combine compact nonlinear dynamical components in parallel and in series. Custom GPU kernels implemented with Triton reduce the launch and memory overhead of their sequential evaluation. We explain this construction, describe a port-Hamiltonian variant, and present various benchmark comparisons of accuracy and training time.
Figures & tables
| Baseline | Base [ms] | Triton [ms] | Speedup | |
|---|---|---|---|---|
| 100 | CPU eager | 154.651 | 1.850 | |
| 100 | CPU compiled | 52.854 | 1.850 | |
| 100 | CUDA eager | 280.963 | 1.850 | |
| 100 | CUDA compiled | 79.145 | 1.850 | |
| 200 | CPU eager | 370.954 | 3.779 | |
| 200 | CPU compiled | 117.881 | 3.779 |
| Dataset | N | n | m | h | training iterations | batch size | validation frequency | history length | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Silverbox | RK4 | |||||||||||
| EMPS | Heun | |||||||||||
| WH | RK4 | |||||||||||
| WH (port-Hamiltonian) | RK4 | |||||||||||
| ParWH | RK4 |
| Method | Training time | RMSE [mV] |
|---|---|---|
| SUBNET | ||
| PNLSS | n.a. | |
| DS3M | ||
| DS3M (port-Hamiltonian) |
| Method | Training time | RMSE [mm] |
|---|---|---|
| CT-SUBNET | – | |
| dynoNet (original) | n.a. | |
| PNARX | – | |
| DS3M |
| Method | Training time | RMSE [mV] |
|---|---|---|
| NL-LFR | – | |
| SUBNET | – | |
| DS3M |
State Space Models
State space models represent sequences through evolving latent states, enabling efficient temporal modeling, system identification, and forecasting.
See all 324 papers