LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models
Organizations: Department of Computing Imperial College London · Department of Bioengineering Imperial College London
Abstract
Time-series data are often sampled irregularly at high frequencies and exhibit long-range dependencies, which makes long-horizon modelling difficult. Continuous-time models such as neural controlled differential equations (NCDEs) and neural rough differential equations (NRDEs) can handle irregular sampling, but they scale poorly to long sequences. Selective state-space models (SSMs) such as Mamba scale linearly with sequence length, but they provide limited recurrent mixing across hidden dimensions within a single block. We propose LogSig-SSM (Log-Signature Compression for State-Space Models), which first compresses long multivariate time series into a shorter sequence of tokens using multi-scale windowed log-signatures, and then processes these tokens with a selective SSM backbone. LogSig-SSM is scalable and robust to irregular sampling, combining log-signature tokens that capture higher-order cross-channel interactions with a selective SSM that models long-range dependencies. The model also admits a continuous-time interpretation as an NCDE/NRDE-style system driven by a log-signature-based input, in which selectivity induces an input-dependent rescaling of the latent dynamics. Across four benchmarks, namely long-sequence classification on UEA, high-frequency physiological regression on PPG-DaLiA, multivariate weather forecasting, and irregularly sampled clinical prediction on PhysioNet Sepsis, LogSig-SSM outperforms or matches strong SSM and continuous-time baselines while training up to faster and using up to less GPU memory than Mamba on the longest sequences.
Figures & tables
| Model | EigenWorms | SCP1 | SCP2 | Ethanol | Heartbeat | Motor | Avg | Rank |
|---|---|---|---|---|---|---|---|---|
| Seq. length | 17,984 | 896 | 1,152 | 1,751 | 405 | 3,000 | ||
| # classes | 5 | 2 | 2 | 4 | 2 | 2 | ||
| Continuous-time models | ||||||||
| NRDE [ 25 ] | ||||||||
| NCDE [ 19 ] | ||||||||
| Log-NCDE [ 38 ] | ||||||||
| NCDE | NRDE | Log-NCDE | Mamba | S5 | LinOSS-IM | D-LinOSS | LogSig-SSM | |
| Mem. (MB) | 2484 | 2506 | 2510 | 13486 | 6646 | 3488 | 3488 | 362 |
| Time (s/1k steps) | 24595 | 5386 | 1956 | 122 | 31 | 14 | 14 | 4 |
| Accuracy (%) | 75.0 | 83.9 | 85.6 | 70.9 | 81.1 | 95.0 | 93.9 | 95.3 |
| Regression (PPG-DaLiA) | Forecasting (Weather) | Irregular Sampling (PhysioNet Sepsis) | ||||
|---|---|---|---|---|---|---|
| Model | MSE | Model | MAE | Method | OI | No OI |
| NRDE | Informer | GRU- | ||||
| NCDE | Informer † | GRU-D | ||||
| Log-NCDE | LogTrans | GRU-ODE | ||||
| LRU | Reformer | ODE-RNN | ||||
| S5 | LSTMa | Neural CDE | ||||
| Configuration | Avg. acc. (%) | (pp) | What it adds |
| Window descriptor (all rows compress the sequence), Mamba backbone fixed | |||
| Mamba (raw input) | 58.6 | – | no compression, no channel mixing |
| Catch24 statistics [ 22 ] | 58.9 | path discarded, summary statistics only | |
| Depthwise convolution | 60.1 | channels kept separate | |
| Standard convolution | 62.9 | first-order (linear) channel mixing | |
| Wavelet scattering | 67.6 | multi-scale spectral content, no cross-channel geometry | |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| NRDE | NCDE | Log-NCDE | LRU | S5 | Mamba | S6 | LinOSS-IMEX | LinOSS-IM | D-LinOSS | LogSig-SSM | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EigenWorms | 105110 | 166789 | 37977 | 101129 | 22007 | 27381 | 15045 | 26119 | 134279 | 134279 | 52933 | |
| mem. | 2506 | 2484 | 2510 | 10716 | 6646 | 13486 | 7922 | 6556 | 3488 | 3488 | 362 | |
| time | 5386 | 24595 | 1956 | 94 | 31 | 122 | 68 | 37 | 14 | 14 | 4 | |
| SCP1 | 117187 | 166274 | 91557 | 25892 | 226328 | 184194 | 24898 | 447944 | 991240 | 992776 | 310786 | |
| mem. | 716 | 694 | 724 | 960 | 1798 | 1110 | 904 | 4768 | 4772 | 4790 | 202 | |
| time | 1014 | 973 | 635 | 9 | 17 | 7 | 3 | 42 | 38 | 38 | 3 |
| Dataset | Tokenisation (s) | % of one training run | ||
|---|---|---|---|---|
| EigenWorms | 6 | 17,984 | 1.2 | |
| SCP1 | 6 | 896 | 0.8 | |
| SCP2 | 7 | 1,152 | 0.1 | |
| Ethanol | 3 | 1,751 | 0.1 | |
| Heartbeat | 61 | 405 | 0.9 | |
| Motor | 64 | 3,000 | 1.5 |
| Model | EigenWorms | SCP1 | SCP2 | Ethanol | Heartbeat | Motor | Avg |
|---|---|---|---|---|---|---|---|
| Tokeniser variants (Mamba backbone) | |||||||
| Mamba (raw input) | |||||||
| ConvDepthwise-Mamba | |||||||
| ConvMixing-Mamba | |||||||
| Catch24-Mamba | |||||||
| Wst-Mamba | |||||||
| Accuracy (%) | Tokenisation (s) | |||
|---|---|---|---|---|
| Dataset | Log-signature | Signature | Log-signature | Signature |
| EigenWorms | ||||
| SCP1 | ||||
| SCP2 | ||||
| Average / total | ||||
| Dataset | ||||
|---|---|---|---|---|
| EigenWorms | 6 | |||
| SCP1 | 6 | |||
| SCP2 | 7 | |||
| Ethanol | 3 | |||
| Heartbeat | 61 | |||
| Motor | 64 |
| Model | EigenWorms | SCP1 | SCP2 | Ethanol | Heartbeat | Motor | Avg |
|---|---|---|---|---|---|---|---|
| LogSig-SSM (searched) | |||||||
| LogSig-SSM (rule, no search) | |||||||
| S5 | |||||||
| LRU | |||||||
| Mamba |
| Dataset | LR | Hidden | State | Blocks | Time | Conv | Exp | Norm | Lengths / Depths | Stride | Global |
|---|---|---|---|---|---|---|---|---|---|---|---|
| EigenWorms | 1e-3 | 64 | 64 | 2 | F | 2 | 1 | T | [281] / [2] | 21 | – |
| Ethanol | 1e-3 | 16 | 256 | 6 | T | 2 | 2 | T | [18, 132] / [2, 2] | 66 | – |
| Heartbeat | 1e-3 | 128 | 64 | 2 | F | 4 | 1 | T | [143] / [2] | 26 | – |
| Motor | 1e-4 | 16 | 16 | 2 | T | 3 | 1 | F | [51, 16] / [2, 2] | 50 | – |
| SCP1 | 1e-3 | 128 | 64 | 2 | F | 3 | 2 | T | [56] / [2] | 5 | 1 |
| SCP2 | 1e-4 | 128 | 16 | 2 | T | 2 | 3 | T | [36] / [2] | 46 | – |
| Dataset | LR | Hidden | State | Blocks | Time | Norm | Lengths | Depths | Stride | Global |
|---|---|---|---|---|---|---|---|---|---|---|
| EigenWorms | 1e-3 | 64 | 16 | 4 4 | F | F | [562, 281] | [2, 2] | 57 | F |
| Ethanol | 1e-3 | 16 | 64 | 2 2 | F | T | [109, 43] | [2, 1] | 70 | F |
| Heartbeat | 1e-5 | 64 | 16 | 6 2 | T | T | [14] | [1] | 3 | F |
| Motor | 1e-4 | 64 | 16 | 4 4 | T | F | [93, 250, 125] | [2,1,2] | 35 | T |
| SCP1 | 1e-3 | 16 | 256 | 6 4 | F | F | [112, 18] | [1, 1] | 30 | F |
| SCP2 | 1e-4 | 16 | 64 | 4 2 | F | T | [18, 144, 192] | [4,2,3] | 13 | T |
| Dataset | LR | Hidden | Heads | FFN | State | Blocks | Time | Norm | Lengths | Depths | Stride | Global |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EigenWorms | 1e-3 | 128 | 4 | 256 | 64 | 2 | T | T | [374, 899] | [2, 1] | 55 | – |
| Ethanol | 1e-3 | 16 | 8 | 128 | 64 | 6 | T | F | [22, 48, 114] | [1, 1, 1] | 74 | – |
| Heartbeat | 1e-5 | 16 | 8 | 256 | 64 | 4 | T | T | [16, 25] | [1, 1] | 14 | – |
| Motor | 1e-4 | 16 | 8 | 512 | 64 | 4 | T | T | [250, 46] | [1, 1] | 46 | – |
| SCP1 | 1e-4 | 16 | 8 | 512 | 64 | 4 | T | T | [74] | [1] | 14 | 1 |
| SCP2 | 1e-3 | 128 | 8 | 512 | 64 | 2 | T | T | [68] | [2] | 76 | – |
| Dataset | LR | Hidden | State | Blocks | Time | Conv | Exp | Norm | Lengths | Stride |
|---|---|---|---|---|---|---|---|---|---|---|
| EigenWorms | 1e-3 | 64 | 256 | 6 | T | 2 | 4 | T | [70, 749, 499, 70, 35] | 135 |
| Ethanol | 1e-3 | 128 | 256 | 4 | T | 3 | 3 | T | [109, 350, 13, 109, 350] | 79 |
| Heartbeat | 1e-5 | 128 | 64 | 2 | T | 3 | 2 | F | [11, 16, 50, 8, 3] | 38 |
| Motor | 1e-3 | 64 | 16 | 6 | F | 3 | 3 | T | [187, 11] | 163 |
| SCP1 | 1e-4 | 64 | 16 | 2 | T | 4 | 2 | T | [179, 9, 179, 32, 224] | 98 |
| SCP2 | 1e-3 | 64 | 256 | 6 | T | 4 | 3 | T | [192, 48, 115, 192] | 67 |
| Dataset | LR | Hidden | State | Blocks | Conv | Exp | WST Lengths | Stride |
|---|---|---|---|---|---|---|---|---|
| EigenWorms | 1e-3 | 128 | 64 | 6 | 2 | 1 | [749, 2997] | 48 |
| Ethanol | 1e-4 | 128 | 64 | 2 | 3 | 4 | [18, 291, 62, 48] | 66 |
| Heartbeat | 1e-4 | 64 | 64 | 2 | 3 | 4 | [67, 67, 40] | 75 |
| Motor | 1e-5 | 16 | 16 | 2 | 2 | 1 | [375, 125, 150, 187] | 125 |
| SCP1 | 1e-5 | 128 | 256 | 2 | 4 | 2 | [179, 28] | 122 |
| SCP2 | 1e-3 | 16 | 16 | 2 | 3 | 1 | [48, 4, 96, 48, 192] | 141 |