Patch-based transformers tokenise a time series into fixed-size, uniform windows, which commits the model to a single temporal resolution even though real signals are multi-scale. The discrete wavelet transform (DWT) is a natural alternative because it decomposes a signal across scales. A growing body of work makes the wavelet filter learnable, on the assumption that a more flexible wavelet forecasts better. That assumption has rarely been tested in isolation, because a single reported improvement usually credits three things at once. These are the use of any multi-resolution decomposition, the choice of wavelet and training the filter. We introduce a forecaster that turns a learnable-DWT per-scale energy map into variable-length patches by merging adjacent windows that share a dominant scale. Three findings follow. First, most of the improvement is there before any filter is learned. A frozen wavelet beats a fixed-patch baseline on eight of the nine datasets by 1.5% to 7.8%, and on four of them it recovers all of the gain. Traffic is the one exception, where it is 1.2% worse. Second, learning the filter gives a further improvement on only some datasets. It helps on three of the nine datasets and makes Traffic 2.7% worse. Third, having variable-length patches helps, but exactly where their boundaries fall does not. A fixed-size uniform grid, even with its patch size tuned per task, is worse than our variable-length patches on 27 of 36 settings. Recomputing the boundaries from the learned filter at every step, rather than caching them once, never helps. The improvement in wavelet-based forecasting comes from the multi-resolution structure and the variable-length patches it induces, not from making the wavelet learnable. In practice, start from any valid wavelet, cache the patch boundaries once and treat training the filter as a per-dataset decision rather than a default.
Figures & tables
Figure 1: End-to-end architecture. The learnable DWT decomposes each channel, per-scale energy sets variable-length patch boundaries, and the merged patches feed a transformer backbone with a scale embedding.
Figure 2: Scale-energy-driven boundary patching (hard assignment). A real input window (ETTm1, first channel, L=96 , base-window width b=8 , J=4 detail bands). Top: the input x , shaded by the patch each region is assigned to. Middle: the dominant detail band mw of each base window (the argmax of per-band energy Ew[s] ). Bottom: adjacent windows sharing a dominant band merge into variable-length patches. Patch i has band tag σi and length Li (its width). A band that stays dominant over several windows forms a longer patch. Rapidly changing dominance yields shorter patches.
Figure 3: Patch-token construction. The variable-length patches from §3.4 (lengths Li differ) are each turned into a fixed-size token. We embed every time step to width d , then pool the patch’s steps with the element-wise max, mean, and min over time (collapsing the variable length Li ), project to d with Wp , and add the learned band embedding ei=ScaleEmb[σi] for the patch’s scale σi . Every patch, of any length Li becomes a token pi∈Rd of the same size.
Dataset
Ours
Autoformer
PatchTST
TimesNet
Transformer
TimeMixer
TimeXer
iTransformer
DLinear
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.421
0.438
0.515
0.495
0.449
0.449
0.460
0.455
0.946
0.774
0.467
0.448
0.444
0.445
0.457
0.449
0.460
0.457
ETTh2
0.371
0.400
0.450
0.461
0.382
0.410
0.407
0.421
4.522
1.696
0.393
0.411
0.378
0.401
0.383
0.407
0.564
0.519
ETTm1
0.374
0.395
0.578
0.515
0.388
0.402
0.411
0.418
0.929
0.730
0.381
0.397
0.382
0.397
0.407
0.411
0.404
0.408
ETTm2
0.280
0.328
0.333
0.370
0.291
0.335
0.295
0.332
1.553
0.889
0.276
0.323
0.274
0.321
0.291
0.334
0.355
0.402
Weather
0.245
0.274
0.358
0.392
0.259
0.281
0.259
0.286
0.625
0.557
0.244
0.274
0.241
0.271
0.260
0.281
0.265
0.317
Table 1: Main results: MSE / MAE averaged over horizons, lookback L=96 (ILI L=36 ). Best in bold , second best underlined . Ours is the Learned variant (§3.2), the per-task mean over seeds {2021, 42, 123}. Per-dataset standard deviations are in Appendix A .
Dataset
Ours
EntroPE
Pathformer
TimeMosaic
HDMixer
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
0.421
0.438
0.438
0.437
0.445
0.426
0.450
0.435
0.441
0.433
ETTh2
0.371
0.400
0.381
0.409
0.346
0.379
0.373
0.394
0.353
0.387
ETTm1
0.374
0.395
0.386
0.397
0.381
0.382
0.382
0.380
–
–
ETTm2
0.280
0.328
0.294
0.338
0.273
0.316
0.278
0.317
–
–
Weather
0.245
0.274
0.249
0.279
–
–
0.252
0.267
–
–
Table 2: Dynamic-patching methods: MSE / MAE averaged over horizons, lookback L=96 . Best in bold , second best underlined . Every model is run in our harness from its released code and published configuration and is the per-task mean over seeds {2021, 42, 123}. Per-dataset standard deviations are in Table 13 . TimeMosaic and HDMixer run their released scripts with the lookback set to 96. HDMixer releases configurations only for ETTh1 and ETTh2. Pathformer diverged on Weather and its Traffic runs did not complete.
Dataset
PatchTST
Fixed
Learned
Dynamic
ETTh1
0.449
0.421
0.421
0.421
ETTh2
0.382
0.371
0.371
0.373
ETTm1
0.388
0.379
0.374
0.373
ETTm2
0.291
0.283
0.280
0.279
Weather
0.259
0.255
0.245
0.245
Exchange
0.396
0.366
0.369
0.379
Table 3: Mode ladder: test MSE averaged over the four horizons. PatchTST (no decomposition), then the three DWT modes of our model: Fixed (frozen filter), Learned (trainable filter, cached boundaries, the main model) and Dynamic (boundaries recomputed each step, straight-through band embedding of §3.5). Best per row in bold .
Dataset
PatchTST
none
Fixed
Fixed
Learned
Learned
Learned
Dynamic
uniform
uniform
merged
uni-16
uni- b
merged
ETTh1
0.449
0.421
0.418
0.418
0.424
0.419
0.418
0.418
ETTh2
0.382
0.375
0.371
0.369
0.380
0.368
0.368
0.370
ETTm1
0.388
0.383
0.379
0.379
0.375
0.374
0.373
0.372
ETTm2
0.291
0.283
0.280
0.281
0.289
0.277
0.277
0.277
Weather
0.259
0.258
0.255
0.255
0.246
0.246
0.245
0.245
Table 4: Within-architecture ladder. Four-horizon mean test MSE, seed 2021. none/uniform keeps the whole backbone, the pooled patch tokens, the cross-attention and the head, with the raw series as its single scale token, no decomposition and one patch per base window, Fixed/uniform adds the frozen DWT decomposition with the patching still uniform, Fixed/merged merges same-band windows. Learned/uni-16 and Learned/uni- b keep the trained filter and replace merging with a fixed grid, of 16 steps and of the task’s tuned base width b . Learned/merged trains the filter with cached boundaries (the main model) and Dynamic recomputes the boundaries every step. Best per row in bold .
Dataset
Wavelet
Xavier
Random
ETTh1
0.365
0.382
0.383
ETTh2
0.281
0.305
0.297
ETTm1
0.313
0.323
0.318
ETTm2
0.173
0.178
0.181
Weather
0.163
0.164
0.163
Exchange
0.082
0.088
0.086
Table 5: Initialisation sensitivity: test MSE for the learnable filter (Learned mode, the main model) started from three initialisations. Wavelet is each dataset’s tuned wavelet (db8; haar for Exchange); Xavier and Random are non-wavelet filters of the same width. Single horizon (p96; ILI p24). Best per row in bold .
Dataset
Ours-Learned MSE
Ours-Uniform MSE
Δ MSE
Horizons won
ETTh1
0.418
0.432
−3.2%
4/4
ETTh2
0.368
0.382
−3.5%
4/4
ETTm1
0.373
0.373
−0.0%
1/4
ETTm2
0.277
0.280
−1.0%
4/4
Weather
0.245
0.246
−0.4%
3/4
Exchange
0.354
0.397
−10.8%
3/4
Table 6: Within-model uniform control (Learned mode, seed 2021). Only the patcher differs. MSE averaged over four horizons; Δ MSE <0 favours dynamic.
MSE
Patches
Mean length
Dataset
Ours
Uniform 16
Random
Ours
Uniform 16
Ours
Uniform 16
ETTh1
0.418
0.424
0.418
4.3
6
24.2
16
ETTh2
0.368
0.380
0.372
14.0
6
8.4
16
ETTm1
0.373
0.375
0.374
13.8
6
13.3
16
ETTm2
0.277
0.289
0.279
19.8
6
4.9
16
Weather
0.245
0.246
0.246
6.0
6
16.7
16
Table 7: Cut positions and token count (Learned mode, seed 2021). Four-horizon mean test MSE for our patcher, a non-overlapping uniform grid of 16 steps and random merging, which keeps our patch count per window and channel and so our mean patch length. Patches are per window and channel on the training split, length in time steps. Best per row in bold .
Test A — shrink rows, sampling fixed (isolates volume)
Arm
Train rows
Drift h
Drift g
Test MSE
full
34,560
0.082
0.116
0.313
21
17,280
0.057
0.057
0.392
41 (ETTh size)
8,640
0.050
0.036
0.447
81
4,320
0.038
0.015
0.578
161
2,160
0.020
0.013
0.826
Table 8: Truncation control (Learned Mode, the main model, 10 epochs / patience 3 at H=96 ). Only the training-row count changes, with dmodel , wavelet, filter width, channels and spectrum frozen. Test A shrinks ETTm1’s rows at fixed 15-min sampling (isolates volume). Test B holds 8,640 rows and switches 15-min → hourly (isolates resolution). Drift is filter movement from init for the low-pass h and high-pass g
Dataset
Drift h
g/h
Grad share
Scale energy shift (%)
Learnable helps?
ETTm2
0.070
1.43
0.78
15.9
yes, −1.1%
Weather
0.069
1.33
1.08
11.5
yes, −3.9%
ETTh2
0.044
1.20
0.87
9.0
no, 0.0%
ETTm1
0.078
1.65
0.43
8.9
yes, −1.3%
ETTh1
0.049
0.92
0.26
2.7
no, 0.0%
ILI
0.007
0.89
0.46
2.5
no, +0.2%
Table 9: Final-epoch mechanism statistics (Learned), averaged over the four horizons, ordered by scale energy shift. Scale energy shift is measured on a set of training windows tracked through training. High-pass drift is omitted since it equals drift h times the ratio g/h . The full set for both modes is in Appendix I . The last column gives the change in MSE from freezing the filter to learning it (§5.1), with negative meaning learning helps.
Figure 4: Adaptation settles by epoch three. Filter drift (left) and scale energy shift (right), Learned mode, 96-step horizon (ILI 24). Table 9 reports the same statistics averaged over the four horizons, so its values differ from the right edge here. The grouping of datasets is the same in both.
Dataset
α=1 (main)
α=0
Δ (%)
ETTh1
0.4178
0.4211
−0.78
ETTh2
0.3704
0.3763
−1.56
ETTm1
0.3724
0.3726
−0.05
ETTm2
0.2770
0.2773
−0.13
Weather
0.2453
0.2452
+0.02
Exchange
0.3761
0.3765
−0.10
Table 10: Boundary-gradient ablation (Dynamic mode). Four-horizon mean test MSE, straight-through strength α on (main) vs off. Δ is α=1 relative to α=0 , so Δ<0 favours the gradient.
Dataset
Power below 2 h
Nyquist (cyc/h)
Learnable helps?
Weather
10.0%
3.0
yes
ETTm1
1.9%
2.0
yes
ETTm2
0.9%
2.0
yes
ETTh1
0.0%
0.5
no
ETTh2
0.0%
0.5
no
ECL
0.0%
0.5
yes
Table 11: Frequency content by sampling rate.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Channels
Frequency
Timesteps
Split (train/val/test)
Seed σ (MSE)
ETTh1
7
1 hour
17,420
12/4/4 months
0.0034
ETTh2
7
1 hour
17,420
12/4/4 months
0.0037
ETTm1
7
15 min
69,680
12/4/4 months
0.0006
ETTm2
7
15 min
69,680
12/4/4 months
0.0028
Weather
21
10 min
52,696
70/10/20%
0.0005
Exchange
8
1 day
7,587
70/10/20%
0.0129
Appendix
Table 12: Benchmark statistics. Lookback L=96 (ILI 36 ); horizons {96,192,336,720} (ILI {24,36,48,60} ). Splits follow the Time-Series-Library convention.
Dataset
Ours
EntroPE
Pathformer
TimeMosaic
HDMixer
ETTh1
0.0034
0.0015
0.0008
0.0056
0.0030
ETTh2
0.0037
0.0019
0.0039
0.0019
0.0026
ETTm1
0.0006
0.0024
0.0027
0.0010
–
ETTm2
0.0028
0.0014
0.0007
0.0011
–
Weather
0.0005
0.0006
–
0.0002
–
ECL
0.0001
0.0007
0.0011
0.0010
–
Appendix
Table 13: Seed standard deviation of the four-horizon mean MSE over seeds {2021,42,123} for the models in Table 2 . A dash marks a cell with no complete three-seed result.
Data
H
d
dff
Le
nh
drop
fc
head
wavelet
level
b
FW
ETTh1
96
512
2048
4
16
0.2
0.1
0.1
db8
3
8
16
ETTh1
192
512
2048
4
2
0.3
0.3
0.2
db8
2
16
16
ETTh1
336
512
2048
4
16
0.3
0.1
0.2
db8
2
16
16
ETTh1
720
256
1024
4
8
0.0
0.3
0.2
db8
4
8
16
ETTh2
96
512
512
1
16
0.0
0.2
0.2
db8
6
2
16
ETTh2
192
512
512
1
16
0.3
0.2
0.2
db8
6
4
16
Appendix
Table 14: Optuna-selected per-task hyperparameters (lookback L=96 , ILI 36 ). H is the forecast horizon, d the model width, dff the FFN width, Le the encoder layers and nh the attention heads. The three dropout rates are the main, fully-connected and head rates. b is the base-patch width and FW the filter width in taps, the number of coefficients in h .
Ours
Autoformer
PatchTST
TimesNet
Transformer
TimeMixer
TimeXer
iTransformer
DLinear
Data
H
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.368
0.397
0.479
0.476
0.379
0.399
0.389
0.412
0.862
0.726
0.381
0.398
0.382
0.403
0.386
0.405
0.396
0.411
192
0.408
0.427
0.525
0.492
0.425
0.431
0.439
0.442
0.945
0.771
0.442
0.430
0.429
0.435
0.441
0.436
0.445
0.440
336
0.443
0.448
0.509
0.492
0.469
0.457
0.495
0.471
1.017
0.826
0.501
0.460
0.467
0.449
0.491
0.461
0.487
0.465
720
0.465
0.479
0.545
0.521
0.523
0.507
0.518
0.494
0.958
0.771
0.544
0.505
0.499
0.493
0.509
0.494
0.513
0.510
ETTh2
96
0.287
0.342
0.368
0.406
0.294
0.347
0.337
0.371
2.583
1.289
0.287
0.340
0.285
0.338
0.300
0.350
0.341
0.395
Appendix
Table 15: Per-horizon test MSE/MAE at lookback L=96 (ILI L=36 ). Best per row per metric in bold . Ours is the 3-seed mean {2021,42,123} .
Ours
EntroPE
Pathformer
TimeMosaic
HDMixer
Data
H
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
0.368
0.397
0.390
0.406
0.385
0.390
0.369
0.385
0.387
0.401
192
0.408
0.427
0.432
0.430
0.444
0.421
0.428
0.418
0.441
0.428
336
0.443
0.448
0.469
0.447
0.462
0.429
0.512
0.469
0.449
0.432
720
0.465
0.479
0.463
0.466
0.490
0.463
0.492
0.470
0.486
0.470
ETTh2
96
0.287
0.342
0.305
0.353
0.283
0.334
0.302
0.343
0.280
0.336
Appendix
Table 16: Per-horizon test MSE/MAE of the dynamic-patching methods at lookback L=96 . Best per row per metric in bold , second best underlined . Every model is the mean over seeds {2021,42,123} . HDMixer releases configurations only for ETTh1 and ETTh2. Pathformer diverged on Weather and its Traffic runs did not complete. All other runs converged.
Dataset
L
b
L/b
Patches
Tokens saved (%)
Mean length (steps)
ETTh1
96
8, 16
9.0
4.3
53
24.2
ETTh2
96
2, 4, 8
27.0
14.0
48
8.4
ETTm1
96
2, 8, 16
28.5
13.8
52
13.3
ETTm2
96
2
48.0
19.8
59
4.9
Weather
96
4, 8
15.0
6.0
60
16.7
Exchange
96
2, 8, 16
28.5
13.2
54
13.5
Appendix
Table 17: Merged patch counts on the training split, four-horizon mean. b lists the tuned base window widths across the four horizons, L/b is the uniform token count at those widths, tokens saved is 1−patches/(L/b) and the last column is the mean patch length in time steps.
Figure 5: Patches per window and channel on the training split at the 96-step horizon (ILI 24), one panel per dataset at its tuned base window width. The dotted line is the mean. A uniform grid at the same width would use L/b patches, the right end of each axis.
Model
ETTh1
ETTh2
ETTm1
ETTm2
Weather
Exchange
ILI
ECL
Traffic
Standard baselines
PatchTST
O
O
O
O
O
O
O
O
O
TimesNet
O
O
O
O
O
O
O
O
O
Autoformer
O
O
O
O
O
O
O
O
O
Transformer
O
O
O
O
O
O
O
O
O
TimeXer
O
O
O
O
O
T
T
O
O
Appendix
Table 18: Configuration source per model and dataset. O is the authors’ own configuration for that dataset at our lookback, run unchanged, from the benchmark library or the model’s own repository. T means we found no configuration at our lookback in the library, the model’s repository or its paper, so we transfer the model’s nearest published configuration. F is the authors’ released configuration with only the lookback set to 96. A dash means not reported.
Comparison
wins/losses
median Δ MSE
Wilcoxon p
vs. Autoformer
9/0
−0.113
0.004
vs. PatchTST
8/1
−0.014
0.055
vs. TimesNet
9/0
−0.037
0.004
vs. Transformer
9/0
−0.555
0.004
vs. TimeMixer
6/3
−0.011
0.055
vs. TimeXer
5/4
−0.007
0.496
Appendix
Table 19: Paired Wilcoxon signed-rank, our model vs. each baseline over the nine datasets (four-horizon mean MSE). Δ MSE is ours minus baseline, so negative favours ours.
Setting
Value
Straight-through strength α
1.0
Optimiser / schedule
Adam / type1
Batch size
32 (16 ECL, 4 Traffic)
Seeds (main table)
{2021, 42, 123}
Filter learning rate
same as the model
Soft-weight temperature τ
1.0
Appendix
Table 20: Deployed model settings held fixed across tasks. Per-task architecture ( d_model , d_ff , layers, heads, dropouts, wavelet, level, base-patch, filter width) is selected by Optuna and released with the code.
Dataset
Wavelet family
Level
Base-patch
ETTh1
0.12%
0.23%
0.50%
ETTh2
1.73%
2.63%
1.20%
ETTm1
1.31%
0.65%
0.74%
ETTm2
0.39%
0.67%
2.00%
Weather
0.38%
0.71%
0.10%
Exchange
1.18%
4.94%
2.38%
Appendix
Table 21: Structural sensitivity (std/mean of MSE across settings, %). Wavelet over the ten wavelets of Figure 6 , level over {2,4,6,8} , base-patch over {4,8,16,24} .
Figure 6: Daubechies filter width, normalised per dataset. Mean bowl (bold) with ±1 std band; shallow, floor flat between db8 and db16. ILI omitted (db20 diverges).
Dataset
2
4
6
8
ETTh1
0.3681
0.3689
0.3691
0.3669
ETTh2
0.2934
0.2880
0.2837
0.3043
ETTm1
0.3204
0.3151
0.3198
0.3187
ETTm2
0.1746
0.1745
0.1724
0.1721
Weather
0.1658
0.1633
0.1630
0.1631
Exchange
0.0819
0.0912
0.0896
0.0938
Appendix
Table 22: Test MSE by decomposition level (p96; ILI p24). Best per row in bold .
Dataset
4
8
16
24
ETTh1
0.3714
0.3662
0.3684
0.3682
ETTh2
0.2907
0.2877
0.2971
0.2939
ETTm1
0.3154
0.3149
0.3200
0.3139
ETTm2
0.1744
0.1756
0.1760
0.1834
Weather
0.1630
0.1630
0.1626
0.1628
Exchange
0.0865
0.0871
0.0819
0.0843
Appendix
Table 23: Test MSE by base-patch size in time steps (p96; ILI p24). Best per row in bold .
Learned
Dynamic
Dataset
Drift h
Drift g
g/h
Grad share
Scale energy (%)
Drift h
Drift g
g/h
Grad share
Scale energy (%)
ETTm2
0.070
0.099
1.43
0.78
15.9
0.072
0.090
1.25
0.90
18.1
Weather
0.069
0.092
1.33
1.08
11.5
0.071
0.086
1.21
0.99
11.9
ETTh2
0.044
0.053
1.20
0.87
9.0
0.046
0.043
0.94
1.06
6.8
ETTm1
0.078
0.128
1.65
0.43
8.9
0.079
0.113
1.43
0.47
8.6
ETTh1
0.049
0.045
0.92
0.26
2.7
0.048
0.049
1.01
0.28
2.4
Appendix
Table 24: Final-epoch mechanism statistics for both trained modes, all nine datasets. Both modes were run with the straight-through path off ( α=0 ). In Learned mode, α has no effect, so that column is unaffected.
Dataset
QMF
Non-QMF
QMF → Non-QMF %Imp
g/h (Non-QMF)
ETTh1
0.3671
0.3691
−0.54
1.27
ETTh2
0.2943
0.2879
+2.19
1.03
ETTm1
0.3238
0.3197
+1.27
1.25
ETTm2
0.1818
0.1741
+4.24
1.91
Appendix
Table 25: Relaxing the QMF constraint (Non-QMF), per-feature, p96, best filter width. %Imp is QMF → Non-QMF (positive = Non-QMF better); g/h is the high-/low-pass drift ratio under Non-QMF (QMF fixes it at 1). Best per row bold .
Dataset
λ=0
10−3
10−2
0.1
1.0
CV%
ETTh1
0.3702
0.3702
0.3702
0.3702
0.3704
0.02
ETTh2
0.2880
0.2880
0.2881
0.2881
0.2881
0.01
ETTm1
0.3251
0.3251
0.3249
0.3244
0.3245
0.09
ETTm2
0.1745
0.1745
0.1745
0.1744
0.1744
0.03
Weather
0.1632
0.1632
0.1633
0.1634
0.1638
0.14
Exchange
0.0825
0.0825
0.0825
0.0825
0.0825
0.00
Appendix
Table 26: Test MSE as the vanishing-moment penalty λ is swept (per-feature, p96).
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
Adaptive patching has been proposed for time-series forecasting: allocate finer patches where the sequence looks locally informative. We compare three adaptive patching methods with tuned uniform counterparts that use one fixed patch size throughout the input, while preserving each method's backbone family and shared training protocol. Across the evaluated long-horizon benchmarks, we observe mostly small accuracy differences and no consistent advantage for adaptive patching; uniform variants generally train faster and avoid adaptive routing at inference. Motivated by these observations, we develop a theoretical framework for budgeted patch allocation through a common resolution-density representation of uniform, nonuniform, and overlapping layouts, independently of the patching rule. Within a separable shared-loss model, we derive an exact criterion for when nonuniform allocation improves on uniform allocation at the same budget. We characterize modeled gains locally through a quadratic surrogate and bound them globally under strong convexity and a bounded loss derivative. The analysis distinguishes where resolution is placed from how much it is redistributed: even an allocation that correctly ranks regions by their weights can redistribute too much resolution and increase modeled loss. Without coupling across time, positive regional weights preserve the minimizers of the shared loss. When the shared loss has a common minimizing rate, the remaining absolute headroom for improvement tends to zero as the uniform rate approaches it. These results characterize the roles of regional weighting, placement, and redistribution strength under the stated model. The empirical findings support evaluating adaptive patching against a tuned uniform baseline and weighing any accuracy gain against the additional training and inference cost of adaptive routing.
Federico Zucchi, Yi Xie, Chao Zhang +2
ICube, University of Strasbourg, Illkirch-Graffenstaden, France · Cephalgo, Strasbourg, France · Technical University of Munich +2
Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies how predictive structure, head count, training sample size, and long-run dependence enter the selection criterion.
Junyu Chen, Likai Chen, Weining Wang
Department of Economics, Econometrics and Finance University of Groningen · Department of Statistics and Data Science Washington University in St. Louis · School of Economics University of Bristol
Transformer-based models have shown strong performance in time-series forecasting by leveraging self-attention to model long-range temporal dependencies. However, their effectiveness depends critically on the quality and structure of input representations derived from raw multivariate time-series data, particularly as sequence length and data scale increase. This paper proposes a two-stage forecasting framework that explicitly separates local temporal representation learning from global dependency modelling. In the proposed approach, a convolutional neural network operates on fixed-length temporal patches to extract short-range temporal dynamics and non-linear feature interactions, producing compact patch-level token embeddings. Token-level self-attention is applied during representation learning to refine these embeddings, after which a Transformer encoder models inter-patch temporal dependencies to generate forecasts. The method is evaluated on a synthetic multivariate time-series dataset with controlled static and dynamic factors, using an extended sequence length and a larger number of samples. Experimental results demonstrate that the proposed framework consistently outperforms a convolutional baseline under increased temporal context and remains competitive with a strong patch-based Transformer model. These findings indicate that structured patch-level tokenization provides a scalable and effective representation for multivariate time-series forecasting, particularly when longer input sequences are considered.