Backpropagation-free Deep Learning for Time-Series Classification
Organizations: ∗Department of Communication Systems, Jožef Stefan Institute, Slovenia
Abstract
Deep neural networks for time series classification (TSC) are trained almost exclusively by error backpropagation, which imposes update locking and the weight-transport problem. Backpropagation-free learning rules such as Direct Feedback Alignment (DFA), Direct Random Target Projection (DRTP), and the diffusion-inspired NoProp relax these constraints but have been evaluated almost exclusively on 2D image benchmarks. We adapt these three learners to univariate TSC with 1D convolutional architectures and present the first systematic study of them across 112 datasets from the UCR time series archive, with one hyperparameter-optimized configuration per method. The optimized learners reach mean F1 scores of 0.76 to 0.79, below ROCKET (0.83) and InceptionTime (0.84). We propose an analysis framework that relates the per-dataset gap to InceptionTime to dataset properties and tests convolutional feature learning by replacing the flatten readout with global average pooling, measured as a difference-in-differences (DD) against a backpropagation control on the identical architecture. Intra-class temporal variability is the dataset property most consistently associated with the gap for all three learners (partial Spearman from to ). Pooling harms DFA and DRTP more than backpropagation (DD and ), consistent with near-random convolutional features, whereas NoProp tolerates pooling as backpropagation does (DD ). Relative to backpropagation on the same architectures, DRTP reduces training floating-point operations (FLOPs) by 33% and trains faster only on series longer than about 700 time steps, DFA yields no computational reduction, and NoProp reduces peak memory by 38% at the cost of 48% longer training.
Figures & tables
| Statistic | Train | Test | Length | Class |
| Min | 16 | 20 | 15 | 2 |
| Max | 8926 | 16 800 | 2844 | 60 |
| Mean | 507.02 | 1108.62 | 551.36 | 8.28 |
| Median | 190.5 | 362 | 315 | 3 |
| StdDev | 1175.71 | 2115.24 | 591.69 | 12.47 |
| Parameter | Model | Search space |
| Learning rate | All | |
| Dropout | All | |
| Conv. modules | DFA/DRTP | |
| Base channels | DFA/DRTP | |
| Kernel size | DFA/DRTP | |
| Freeze conv. layers | DFA/DRTP |
| Hyperparameters | Values |
| Learning rate | |
| Dropout | |
| Number of NoProp blocks | |
| Number of convolutional modules | |
| Number of filters | |
| Kernel size |
| Model | Accuracy | Precision | Recall | F1 score | Training time (s) | |||||
| Mean | StdDev | Mean | StdDev | Mean | StdDev | Mean | StdDev | Mean | StdDev | |
| DFA | 0.748 | 0.231 | 0.731 | 0.249 | 0.722 | 0.248 | 0.719 | 0.252 | 12.19 | 25.50 |
| DRTP | 0.744 | 0.217 | 0.725 | 0.236 | 0.715 | 0.236 | 0.712 | 0.239 | 11.81 | 24.99 |
| NoProp | 0.766 | 0.196 | 0.769 | 0.181 | 0.741 | 0.209 | 0.738 | 0.210 | 423.78 | 908.71 |
| Model | Configuration |
| DFA | |
| DRTP | |
| NoProp |
| Model | Accuracy | Precision | Recall | F1 score | Training time (s) | |||||
| Mean | StdDev | Mean | StdDev | Mean | StdDev | Mean | StdDev | Mean | StdDev | |
| DFA | 0.784 | 0.190 | 0.772 | 0.198 | 0.763 | 0.202 | 0.762 | 0.204 | 14.97 | 27.74 |
| DRTP | 0.804 | 0.183 | 0.792 | 0.194 | 0.783 | 0.198 | 0.782 | 0.199 | 21.23 | 41.28 |
| NoProp | 0.810 | 0.178 | 0.804 | 0.178 | 0.788 | 0.195 | 0.787 | 0.197 | 500.01 | 1085.31 |
| ROCKET | 0.851 | 0.135 | 0.842 | 0.150 | 0.829 | 0.160 | 0.829 | 0.160 | 3.74 | 16.49 |
| InceptionTime | 0.851 | 0.148 | 0.844 | 0.160 | 0.835 | 0.168 | 0.839 | 0.164 | 4067.79 | 5139.57 |
| Native | Backprop. | DD | |||
| Method | mean | deg. | mean | deg. | |
| DFA | 101/112 | 99/112 | |||
| DRTP | 93/112 | 65/112 | |||
| NoProp | 45/112 | 58/112 | |||
| DFA | DRTP | NoProp | ||||
| Property | zero | partial | zero | partial | zero | partial |
| Model configuration | Training method | Training time [s] | Total FLOPs | Total energy consumed [kWh] | Total CO 2 emissions [kg] | Peak memory consumption [MB] |
| DFA | Native | 14.97 | 0.151 | 0.035 | 530.97 | |
| BP | 13.65 | 0.148 | 0.034 | 521.40 | ||
| DRTP | Native | 21.23 | 0.193 | 0.045 | 532.36 | |
| BP | 19.33 | 0.204 | 0.047 | 574.09 | ||
| NoProp | Native | 500.01 | 3.963 | 0.854 | 132.79 | |
| BP | 337.80 | 4.957 | 1.146 | 215.07 |
| Name | Train / Test | Length / Class | DFA | DRTP | NoProp |
|---|---|---|---|---|---|
| ACSF1 | 100/100 | 1460/10 | 0.741 ± 0.036 | 0.788 ± 0.025 | 0.672 ± 0.052 |
| Adiac | 390/391 | 176/37 | 0.754 ± 0.017 | 0.749 ± 0.018 | 0.708 ± 0.018 |
| ArrowHead | 36/175 | 251/3 | 0.810 ± 0.039 | 0.840 ± 0.050 | 0.828 ± 0.048 |
| Beef | 30/30 | 470/5 | 0.760 ± 0.055 | 0.655 ± 0.085 | 0.649 ± 0.083 |
| BeetleFly | 20/20 | 512/2 | 0.790 ± 0.107 | 0.714 ± 0.129 | 0.802 ± 0.065 |
| BirdChicken | 20/20 | 512/2 | 0.668 ± 0.160 | 0.714 ± 0.131 | 0.789 ± 0.117 |
Time Series Classification
Time series classification assigns labels to sequential data, using temporal patterns to support decisions in health, sensing, science, and industry.
See all 192 papers