SINDy-KANs: Sparse identification of non-linear dynamics through Kolmogorov-Arnold networks
Organizations: Pacific Northwest National Laboratory, Richland, WA 99354 · University of Washington, Seattle, WA 98195
Abstract
Kolmogorov-Arnold networks (KANs) have arisen as a potential way to enhance the interpretability of machine learning. However, solutions learned by KANs are not necessarily interpretable, in the sense of being sparse or parsimonious. Sparse identification of nonlinear dynamics (SINDy) is a complementary approach that allows for learning sparse equations for dynamical systems from data; however, learned equations are limited by the library. In this work, we present SINDy-KANs, which simultaneously train a KAN and a SINDy-like representation to increase interpretability of KAN representations with SINDy applied at the level of each activation function, while maintaining the function compositions possible through deep KANs. We apply our method to a number of symbolic regression tasks, including dynamical systems, to show accurate equation discovery across a range of systems.
Figures & tables
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| (From Sec. 4.1 ) | ||
|---|---|---|
| Exact eq. | ||
| Training time (s) | 145 | 215 |
| (From Sec. 4.2.1 ) | ||
|---|---|---|
| Data relative noise | error | error | Learned equation |
|---|---|---|---|
| 0% | 5.688e-08 | – | |
| 6.27% | 1.255e-05 | 0.00014 | |
| 31.37% | 0.00043 | 0.0101 |
| Parameter | Sec. 4.1 |
|---|---|
| KAN architecture | [2, 2, 1] |
| [3, 6] | |
| Learning rate scales | [1, 0.6] |
| Boundaries | [0, 40k] |
| 5 | |
| ADAM learning rate | 1e-3 |
| Parameter | Sec. 4.2.1 | Sec. 4.2.2 | Sec. 4.2.3 | Sec. 4.2.4 | Sec. 4.2.5 |
|---|---|---|---|---|---|
| KAN architecture | [3, 3] | [2, 2] | [3, 6, 3] | [3, 5, 3] | [3, 6, 3] |
| [3] | [5, 10] | [5, 7] | [5, 7, 10] | [5, 7, 10, 15] | |
| Learning rate scales | [1] | [1, .6] | [1, .6] | [1, .6, .6] | [1, .6, .6, .1] |
| Boundaries | [0] | [0, 50k] | [0, 30k] | [0, 30k, 60k] | [0, 30k, 60k, 300k] |
| 5 | 5 | 5 | 5 | 5 | |
| ADAM learning rate | 1e-3 | 1e-3 | 5e-3 | 5e-3 | 1e-2 |