Wavelet-Energy-Driven Dynamic Patching for Time Series Forecasting
Organizations: Department of Engineering Science, University of Oxford
Abstract
Patch-based transformers tokenise a time series into fixed-size, uniform windows, which commits the model to a single temporal resolution even though real signals are multi-scale. The discrete wavelet transform (DWT) is a natural alternative because it decomposes a signal across scales. A growing body of work makes the wavelet filter learnable, on the assumption that a more flexible wavelet forecasts better. That assumption has rarely been tested in isolation, because a single reported improvement usually credits three things at once. These are the use of any multi-resolution decomposition, the choice of wavelet and training the filter. We introduce a forecaster that turns a learnable-DWT per-scale energy map into variable-length patches by merging adjacent windows that share a dominant scale. Three findings follow. First, most of the improvement is there before any filter is learned. A frozen wavelet beats a fixed-patch baseline on eight of the nine datasets by 1.5% to 7.8%, and on four of them it recovers all of the gain. Traffic is the one exception, where it is 1.2% worse. Second, learning the filter gives a further improvement on only some datasets. It helps on three of the nine datasets and makes Traffic 2.7% worse. Third, having variable-length patches helps, but exactly where their boundaries fall does not. A fixed-size uniform grid, even with its patch size tuned per task, is worse than our variable-length patches on 27 of 36 settings. Recomputing the boundaries from the learned filter at every step, rather than caching them once, never helps. The improvement in wavelet-based forecasting comes from the multi-resolution structure and the variable-length patches it induces, not from making the wavelet learnable. In practice, start from any valid wavelet, cache the patch boundaries once and treat training the filter as a per-dataset decision rather than a default.
Figures & tables
| Dataset | Ours | Autoformer | PatchTST | TimesNet | Transformer | TimeMixer | TimeXer | iTransformer | DLinear | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTh1 | 0.421 | 0.438 | 0.515 | 0.495 | 0.449 | 0.449 | 0.460 | 0.455 | 0.946 | 0.774 | 0.467 | 0.448 | 0.444 | 0.445 | 0.457 | 0.449 | 0.460 | 0.457 |
| ETTh2 | 0.371 | 0.400 | 0.450 | 0.461 | 0.382 | 0.410 | 0.407 | 0.421 | 4.522 | 1.696 | 0.393 | 0.411 | 0.378 | 0.401 | 0.383 | 0.407 | 0.564 | 0.519 |
| ETTm1 | 0.374 | 0.395 | 0.578 | 0.515 | 0.388 | 0.402 | 0.411 | 0.418 | 0.929 | 0.730 | 0.381 | 0.397 | 0.382 | 0.397 | 0.407 | 0.411 | 0.404 | 0.408 |
| ETTm2 | 0.280 | 0.328 | 0.333 | 0.370 | 0.291 | 0.335 | 0.295 | 0.332 | 1.553 | 0.889 | 0.276 | 0.323 | 0.274 | 0.321 | 0.291 | 0.334 | 0.355 | 0.402 |
| Weather | 0.245 | 0.274 | 0.358 | 0.392 | 0.259 | 0.281 | 0.259 | 0.286 | 0.625 | 0.557 | 0.244 | 0.274 | 0.241 | 0.271 | 0.260 | 0.281 | 0.265 | 0.317 |
| Dataset | Ours | EntroPE | Pathformer | TimeMosaic | HDMixer | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTh1 | 0.421 | 0.438 | 0.438 | 0.437 | 0.445 | 0.426 | 0.450 | 0.435 | 0.441 | 0.433 |
| ETTh2 | 0.371 | 0.400 | 0.381 | 0.409 | 0.346 | 0.379 | 0.373 | 0.394 | 0.353 | 0.387 |
| ETTm1 | 0.374 | 0.395 | 0.386 | 0.397 | 0.381 | 0.382 | 0.382 | 0.380 | – | – |
| ETTm2 | 0.280 | 0.328 | 0.294 | 0.338 | 0.273 | 0.316 | 0.278 | 0.317 | – | – |
| Weather | 0.245 | 0.274 | 0.249 | 0.279 | – | – | 0.252 | 0.267 | – | – |
| Dataset | PatchTST | Fixed | Learned | Dynamic |
|---|---|---|---|---|
| ETTh1 | 0.449 | 0.421 | 0.421 | 0.421 |
| ETTh2 | 0.382 | 0.371 | 0.371 | 0.373 |
| ETTm1 | 0.388 | 0.379 | 0.374 | 0.373 |
| ETTm2 | 0.291 | 0.283 | 0.280 | 0.279 |
| Weather | 0.259 | 0.255 | 0.245 | 0.245 |
| Exchange | 0.396 | 0.366 | 0.369 | 0.379 |
| Dataset | PatchTST | none | Fixed | Fixed | Learned | Learned | Learned | Dynamic |
|---|---|---|---|---|---|---|---|---|
| uniform | uniform | merged | uni-16 | uni- | merged | |||
| ETTh1 | 0.449 | 0.421 | 0.418 | 0.418 | 0.424 | 0.419 | 0.418 | 0.418 |
| ETTh2 | 0.382 | 0.375 | 0.371 | 0.369 | 0.380 | 0.368 | 0.368 | 0.370 |
| ETTm1 | 0.388 | 0.383 | 0.379 | 0.379 | 0.375 | 0.374 | 0.373 | 0.372 |
| ETTm2 | 0.291 | 0.283 | 0.280 | 0.281 | 0.289 | 0.277 | 0.277 | 0.277 |
| Weather | 0.259 | 0.258 | 0.255 | 0.255 | 0.246 | 0.246 | 0.245 | 0.245 |
| Dataset | Wavelet | Xavier | Random |
|---|---|---|---|
| ETTh1 | 0.365 | 0.382 | 0.383 |
| ETTh2 | 0.281 | 0.305 | 0.297 |
| ETTm1 | 0.313 | 0.323 | 0.318 |
| ETTm2 | 0.173 | 0.178 | 0.181 |
| Weather | 0.163 | 0.164 | 0.163 |
| Exchange | 0.082 | 0.088 | 0.086 |
| Dataset | Ours-Learned MSE | Ours-Uniform MSE | MSE | Horizons won |
|---|---|---|---|---|
| ETTh1 | 0.418 | 0.432 | 4/4 | |
| ETTh2 | 0.368 | 0.382 | 4/4 | |
| ETTm1 | 0.373 | 0.373 | 1/4 | |
| ETTm2 | 0.277 | 0.280 | 4/4 | |
| Weather | 0.245 | 0.246 | 3/4 | |
| Exchange | 0.354 | 0.397 | 3/4 |
| MSE | Patches | Mean length | |||||
|---|---|---|---|---|---|---|---|
| Dataset | Ours | Uniform 16 | Random | Ours | Uniform 16 | Ours | Uniform 16 |
| ETTh1 | 0.418 | 0.424 | 0.418 | 4.3 | 6 | 24.2 | 16 |
| ETTh2 | 0.368 | 0.380 | 0.372 | 14.0 | 6 | 8.4 | 16 |
| ETTm1 | 0.373 | 0.375 | 0.374 | 13.8 | 6 | 13.3 | 16 |
| ETTm2 | 0.277 | 0.289 | 0.279 | 19.8 | 6 | 4.9 | 16 |
| Weather | 0.245 | 0.246 | 0.246 | 6.0 | 6 | 16.7 | 16 |
| Test A — shrink rows, sampling fixed (isolates volume) | ||||
|---|---|---|---|---|
| Arm | Train rows | Drift | Drift | Test MSE |
| full | 34,560 | 0.082 | 0.116 | 0.313 |
| 17,280 | 0.057 | 0.057 | 0.392 | |
| (ETTh size) | 8,640 | 0.050 | 0.036 | 0.447 |
| 4,320 | 0.038 | 0.015 | 0.578 | |
| 2,160 | 0.020 | 0.013 | 0.826 | |
| Dataset | Drift | Grad share | Scale energy shift (%) | Learnable helps? | |
|---|---|---|---|---|---|
| ETTm2 | 0.070 | 1.43 | 0.78 | 15.9 | yes, |
| Weather | 0.069 | 1.33 | 1.08 | 11.5 | yes, |
| ETTh2 | 0.044 | 1.20 | 0.87 | 9.0 | no, |
| ETTm1 | 0.078 | 1.65 | 0.43 | 8.9 | yes, |
| ETTh1 | 0.049 | 0.92 | 0.26 | 2.7 | no, |
| ILI | 0.007 | 0.89 | 0.46 | 2.5 | no, |
| Dataset | (main) | (%) | |
|---|---|---|---|
| ETTh1 | 0.4178 | 0.4211 | |
| ETTh2 | 0.3704 | 0.3763 | |
| ETTm1 | 0.3724 | 0.3726 | |
| ETTm2 | 0.2770 | 0.2773 | |
| Weather | 0.2453 | 0.2452 | |
| Exchange | 0.3761 | 0.3765 |
| Dataset | Power below 2 h | Nyquist (cyc/h) | Learnable helps? |
|---|---|---|---|
| Weather | 10.0% | 3.0 | yes |
| ETTm1 | 1.9% | 2.0 | yes |
| ETTm2 | 0.9% | 2.0 | yes |
| ETTh1 | 0.0% | 0.5 | no |
| ETTh2 | 0.0% | 0.5 | no |
| ECL | 0.0% | 0.5 | yes |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Channels | Frequency | Timesteps | Split (train/val/test) | Seed (MSE) |
|---|---|---|---|---|---|
| ETTh1 | 7 | 1 hour | 17,420 | 12/4/4 months | 0.0034 |
| ETTh2 | 7 | 1 hour | 17,420 | 12/4/4 months | 0.0037 |
| ETTm1 | 7 | 15 min | 69,680 | 12/4/4 months | 0.0006 |
| ETTm2 | 7 | 15 min | 69,680 | 12/4/4 months | 0.0028 |
| Weather | 21 | 10 min | 52,696 | 70/10/20% | 0.0005 |
| Exchange | 8 | 1 day | 7,587 | 70/10/20% | 0.0129 |
| Dataset | Ours | EntroPE | Pathformer | TimeMosaic | HDMixer |
|---|---|---|---|---|---|
| ETTh1 | 0.0034 | 0.0015 | 0.0008 | 0.0056 | 0.0030 |
| ETTh2 | 0.0037 | 0.0019 | 0.0039 | 0.0019 | 0.0026 |
| ETTm1 | 0.0006 | 0.0024 | 0.0027 | 0.0010 | – |
| ETTm2 | 0.0028 | 0.0014 | 0.0007 | 0.0011 | – |
| Weather | 0.0005 | 0.0006 | – | 0.0002 | – |
| ECL | 0.0001 | 0.0007 | 0.0011 | 0.0010 | – |
| Data | drop | fc | head | wavelet | level | FW | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ETTh1 | 96 | 512 | 2048 | 4 | 16 | 0.2 | 0.1 | 0.1 | db8 | 3 | 8 | 16 |
| ETTh1 | 192 | 512 | 2048 | 4 | 2 | 0.3 | 0.3 | 0.2 | db8 | 2 | 16 | 16 |
| ETTh1 | 336 | 512 | 2048 | 4 | 16 | 0.3 | 0.1 | 0.2 | db8 | 2 | 16 | 16 |
| ETTh1 | 720 | 256 | 1024 | 4 | 8 | 0.0 | 0.3 | 0.2 | db8 | 4 | 8 | 16 |
| ETTh2 | 96 | 512 | 512 | 1 | 16 | 0.0 | 0.2 | 0.2 | db8 | 6 | 2 | 16 |
| ETTh2 | 192 | 512 | 512 | 1 | 16 | 0.3 | 0.2 | 0.2 | db8 | 6 | 4 | 16 |
| Ours | Autoformer | PatchTST | TimesNet | Transformer | TimeMixer | TimeXer | iTransformer | DLinear | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Data | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTh1 | 96 | 0.368 | 0.397 | 0.479 | 0.476 | 0.379 | 0.399 | 0.389 | 0.412 | 0.862 | 0.726 | 0.381 | 0.398 | 0.382 | 0.403 | 0.386 | 0.405 | 0.396 | 0.411 |
| 192 | 0.408 | 0.427 | 0.525 | 0.492 | 0.425 | 0.431 | 0.439 | 0.442 | 0.945 | 0.771 | 0.442 | 0.430 | 0.429 | 0.435 | 0.441 | 0.436 | 0.445 | 0.440 | |
| 336 | 0.443 | 0.448 | 0.509 | 0.492 | 0.469 | 0.457 | 0.495 | 0.471 | 1.017 | 0.826 | 0.501 | 0.460 | 0.467 | 0.449 | 0.491 | 0.461 | 0.487 | 0.465 | |
| 720 | 0.465 | 0.479 | 0.545 | 0.521 | 0.523 | 0.507 | 0.518 | 0.494 | 0.958 | 0.771 | 0.544 | 0.505 | 0.499 | 0.493 | 0.509 | 0.494 | 0.513 | 0.510 | |
| ETTh2 | 96 | 0.287 | 0.342 | 0.368 | 0.406 | 0.294 | 0.347 | 0.337 | 0.371 | 2.583 | 1.289 | 0.287 | 0.340 | 0.285 | 0.338 | 0.300 | 0.350 | 0.341 | 0.395 |
| Ours | EntroPE | Pathformer | TimeMosaic | HDMixer | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Data | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE | |
| ETTh1 | 96 | 0.368 | 0.397 | 0.390 | 0.406 | 0.385 | 0.390 | 0.369 | 0.385 | 0.387 | 0.401 |
| 192 | 0.408 | 0.427 | 0.432 | 0.430 | 0.444 | 0.421 | 0.428 | 0.418 | 0.441 | 0.428 | |
| 336 | 0.443 | 0.448 | 0.469 | 0.447 | 0.462 | 0.429 | 0.512 | 0.469 | 0.449 | 0.432 | |
| 720 | 0.465 | 0.479 | 0.463 | 0.466 | 0.490 | 0.463 | 0.492 | 0.470 | 0.486 | 0.470 | |
| ETTh2 | 96 | 0.287 | 0.342 | 0.305 | 0.353 | 0.283 | 0.334 | 0.302 | 0.343 | 0.280 | 0.336 |
| Dataset | Patches | Tokens saved (%) | Mean length (steps) | |||
|---|---|---|---|---|---|---|
| ETTh1 | 96 | 8, 16 | 9.0 | 4.3 | 53 | 24.2 |
| ETTh2 | 96 | 2, 4, 8 | 27.0 | 14.0 | 48 | 8.4 |
| ETTm1 | 96 | 2, 8, 16 | 28.5 | 13.8 | 52 | 13.3 |
| ETTm2 | 96 | 2 | 48.0 | 19.8 | 59 | 4.9 |
| Weather | 96 | 4, 8 | 15.0 | 6.0 | 60 | 16.7 |
| Exchange | 96 | 2, 8, 16 | 28.5 | 13.2 | 54 | 13.5 |
| Model | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Weather | Exchange | ILI | ECL | Traffic |
|---|---|---|---|---|---|---|---|---|---|
| Standard baselines | |||||||||
| PatchTST | O | O | O | O | O | O | O | O | O |
| TimesNet | O | O | O | O | O | O | O | O | O |
| Autoformer | O | O | O | O | O | O | O | O | O |
| Transformer | O | O | O | O | O | O | O | O | O |
| TimeXer | O | O | O | O | O | T | T | O | O |
| Comparison | wins/losses | median MSE | Wilcoxon |
|---|---|---|---|
| vs. Autoformer | 9/0 | ||
| vs. PatchTST | 8/1 | ||
| vs. TimesNet | 9/0 | ||
| vs. Transformer | 9/0 | ||
| vs. TimeMixer | 6/3 | ||
| vs. TimeXer | 5/4 |
| Setting | Value |
|---|---|
| Straight-through strength | 1.0 |
| Optimiser / schedule | Adam / type1 |
| Batch size | 32 (16 ECL, 4 Traffic) |
| Seeds (main table) | {2021, 42, 123} |
| Filter learning rate | same as the model |
| Soft-weight temperature | 1.0 |
| Dataset | Wavelet family | Level | Base-patch |
|---|---|---|---|
| ETTh1 | 0.12% | 0.23% | 0.50% |
| ETTh2 | 1.73% | 2.63% | 1.20% |
| ETTm1 | 1.31% | 0.65% | 0.74% |
| ETTm2 | 0.39% | 0.67% | 2.00% |
| Weather | 0.38% | 0.71% | 0.10% |
| Exchange | 1.18% | 4.94% | 2.38% |
| Dataset | 2 | 4 | 6 | 8 |
|---|---|---|---|---|
| ETTh1 | 0.3681 | 0.3689 | 0.3691 | 0.3669 |
| ETTh2 | 0.2934 | 0.2880 | 0.2837 | 0.3043 |
| ETTm1 | 0.3204 | 0.3151 | 0.3198 | 0.3187 |
| ETTm2 | 0.1746 | 0.1745 | 0.1724 | 0.1721 |
| Weather | 0.1658 | 0.1633 | 0.1630 | 0.1631 |
| Exchange | 0.0819 | 0.0912 | 0.0896 | 0.0938 |
| Dataset | 4 | 8 | 16 | 24 |
|---|---|---|---|---|
| ETTh1 | 0.3714 | 0.3662 | 0.3684 | 0.3682 |
| ETTh2 | 0.2907 | 0.2877 | 0.2971 | 0.2939 |
| ETTm1 | 0.3154 | 0.3149 | 0.3200 | 0.3139 |
| ETTm2 | 0.1744 | 0.1756 | 0.1760 | 0.1834 |
| Weather | 0.1630 | 0.1630 | 0.1626 | 0.1628 |
| Exchange | 0.0865 | 0.0871 | 0.0819 | 0.0843 |
| Learned | Dynamic | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | Drift | Drift | Grad share | Scale energy (%) | Drift | Drift | Grad share | Scale energy (%) | ||
| ETTm2 | 0.070 | 0.099 | 1.43 | 0.78 | 15.9 | 0.072 | 0.090 | 1.25 | 0.90 | 18.1 |
| Weather | 0.069 | 0.092 | 1.33 | 1.08 | 11.5 | 0.071 | 0.086 | 1.21 | 0.99 | 11.9 |
| ETTh2 | 0.044 | 0.053 | 1.20 | 0.87 | 9.0 | 0.046 | 0.043 | 0.94 | 1.06 | 6.8 |
| ETTm1 | 0.078 | 0.128 | 1.65 | 0.43 | 8.9 | 0.079 | 0.113 | 1.43 | 0.47 | 8.6 |
| ETTh1 | 0.049 | 0.045 | 0.92 | 0.26 | 2.7 | 0.048 | 0.049 | 1.01 | 0.28 | 2.4 |
| Dataset | QMF | Non-QMF | QMF Non-QMF %Imp | (Non-QMF) |
|---|---|---|---|---|
| ETTh1 | 0.3671 | 0.3691 | 1.27 | |
| ETTh2 | 0.2943 | 0.2879 | 1.03 | |
| ETTm1 | 0.3238 | 0.3197 | 1.25 | |
| ETTm2 | 0.1818 | 0.1741 | 1.91 |
| Dataset | CV% | |||||
|---|---|---|---|---|---|---|
| ETTh1 | 0.3702 | 0.3702 | 0.3702 | 0.3702 | 0.3704 | 0.02 |
| ETTh2 | 0.2880 | 0.2880 | 0.2881 | 0.2881 | 0.2881 | 0.01 |
| ETTm1 | 0.3251 | 0.3251 | 0.3249 | 0.3244 | 0.3245 | 0.09 |
| ETTm2 | 0.1745 | 0.1745 | 0.1745 | 0.1744 | 0.1744 | 0.03 |
| Weather | 0.1632 | 0.1632 | 0.1633 | 0.1634 | 0.1638 | 0.14 |
| Exchange | 0.0825 | 0.0825 | 0.0825 | 0.0825 | 0.0825 | 0.00 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 939 papers