Transformer architectures have established strong baselines in time series forecasting, yet they typically rely on positional encodings that assume uniform, index-based temporal progression. However, real-world systems, from shifting financial cycles to elastic biological rhythms, frequently exhibit "time-warped" dynamics where the effective flow of time decouples from the sampling index. In this work, we first formalize this misalignment and prove that rotary position embedding (RoPE) is mathematically incapable of representing non-affine temporal warping. To address this, we propose Symplectic Positional Embeddings (SyPE), a learnable encoding framework derived from Hamiltonian mechanics. SyPE strictly generalizes RoPE by extending the rotation group SO(2) to the symplectic group Sp(2,R), modulated by a novel input-dependent adaptive warp module. By allowing the attention mechanism to adaptively dilate or contract temporal coordinates end-to-end, our approach captures locally varying periodicities without requiring pre-defined warping functions. We implement this mechanism in StretchTime, a multivariate forecasting architecture that achieves state-of-the-art performance on standard benchmarks, demonstrating superior robustness on datasets exhibiting non-stationary temporal dynamics.
Figures & tables
Figure 1: Visualization of Temporal Stretching
Figure 2: Overview of the SyPE-Augmented Transformer architecture.
Horizon
StretchTime
RoPE
w/o MLP
MSE
MAE
MSE
MAE
MSE
MAE
96
0.053
0.180
0.058
0.186
0.078
0.218
192
0.073
0.207
0.084
0.219
0.105
0.250
336
0.120
0.255
0.184
0.311
0.161
0.298
720
0.331
0.416
0.411
0.480
0.358
0.439
Avg
0.144
0.265
0.184
0.299
0.176
0.301
Table 1: Results on the synthetic dataset with temporally warped seasonal dynamics (see Section 3.2 ). We report MSE and MAE for varying prediction lengths ( H ). Best results are in bold and second best are underlined .
Figure 3: Forecast visualization on warped seasonal dynamics. StretchTime (left) corrects the phase alignment errors observed in the static RoPE baseline (right).
Model
StretchTime
RoPE
OLinear
TimeMixer++
TimeMixer
iTransformer
PatchTST
TimesNet
DLinear
Weather
0.237
0.244
0.237
0.226
0.240
0.258
0.265
0.259
0.265
Solar
0.195
0.214
0.215
0.203
0.216
0.233
0.287
0.403
0.330
ECL
0.174
0.179
0.159
0.165
0.182
0.178
0.216
0.192
0.225
ETTh1
0.424
0.449
0.424
0.419
0.447
0.454
0.516
0.458
0.461
ETTh2
0.384
0.401
0.367
0.339
0.364
0.383
0.391
0.414
0.563
ETTm1
0.367
0.379
0.374
0.369
0.381
0.407
0.406
0.400
0.404
Table 2: Summary of Multivariate TSF Results. Averaged test set MSE are reported. Best results are in bold and second best are underlined .
Horizon
StretchTime
TimeMixer++
TimeMixer
iTransformer
PatchTST
TimesNet
DLinear
FLOPs
Params
FLOPs
Params
FLOPs
Params
FLOPs
Params
FLOPs
Params
FLOPs
Params
FLOPs
Params
96
1.24G
471K
6.24G
1.19M
2.77G
1.13M
210M
9.56M
2.51G
10.1M
2.26G
1.19M
259K
18.6K
192
1.86G
480K
6.25G
1.2M
2.81G
1.14M
211M
9.61M
2.52G
10.6M
3.38G
1.2M
517K
37.2K
336
2.79G
494K
6.25G
1.22M
2.85G
1.17M
213M
9.68M
2.54G
11.5M
5.06G
1.22M
905K
65.2K
720
5.27G
530K
6.26G
1.27M
2.98G
1.24M
217M
9.88M
2.59G
13.9M
9.57G
1.25M
1.94M
140K
Avg Cost
2.79G
493.75K
6.25G
1.22M
2.85G
1.17M
212.75M
9.68M
2.54G
11.53M
5.07G
1.22M
905.25K
65.25K
Table 3: Computational efficiency comparison (FLOPs and Parameters) with Average Rank. Models are ordered to match the main results. Best results are in bold and second best are underlined .
V1
V2
V3
V4
V5
Dataset (Len)
Warp Only
Fixed RoPE
Learnable Symplectic
Fixed RoPE + Warp
Full StretchTime
PEMS04 (12)
0.075
0.072
0.072
0.073
0.071
PEMS04 (24)
0.082
0.080
0.078
0.079
0.081
PEMS04 (48)
0.097
0.093
0.094
0.095
0.090
PEMS04 (96)
0.119
0.118
0.118
0.115
0.112
PEMS04 Avg
0.093
0.091
0.091
0.090
0.089
Table 4: Factorized ablation study on stable traffic forecasting benchmarks (MSE). We decouple the adaptive warp module from the underlying positional geometry. Best results are in bold and second best are underlined .
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
StretchTime
RoPE
OLinear
TimeMixer++
TimeMixer
iTransformer
PatchTST
TimesNet
DLinear
Dataset
Len
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Weather
96
0.143
0.203
0.146
0.206
0.153
0.190
0.155
0.205
0.163
0.209
0.174
0.214
0.186
0.227
0.172
0.220
0.195
0.252
192
0.196
0.258
0.201
0.262
0.200
0.235
0.201
0.245
0.208
0.250
0.221
0.254
0.234
0.265
0.219
0.261
0.237
0.295
336
0.253
0.304
0.270
0.317
0.258
0.280
0.237
0.265
0.251
0.287
0.278
0.296
0.284
0.301
0.280
0.306
0.282
0.331
720
0.355
0.373
0.360
0.374
0.337
0.333
0.312
0.334
0.339
0.341
0.358
0.347
0.356
0.349
0.365
0.359
0.345
0.382
Solar
96
0.158
0.241
0.190
0.268
0.179
0.191
0.171
0.231
0.189
0.259
0.203
0.237
0.265
0.323
0.373
0.358
0.290
0.378
Appendix
Table 5: Full Multivariate TSF Results. We report MSE and MAE. Best results are in bold and second best are underlined . AvgRank and #Top1 are calculated based on MSE across all horizons.
Model
StretchTime
RoPE
Pure Softmax
LinAttn+RoPE
Dataset
Len
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Weather
96
0.143
0.203
0.146
0.206
0.146
0.209
0.152
0.211
192
0.196
0.258
0.201
0.262
0.198
0.260
0.210
0.270
336
0.253
0.304
0.270
0.317
0.269
0.317
0.280
0.329
720
0.355
0.373
0.360
0.374
0.359
0.378
0.369
0.378
Solar
96
0.158
0.241
0.190
0.268
0.183
0.249
0.187
0.245
Appendix
Table 6: Ablation Study Results. Comparison of StretchTime with component variants. Best results are in bold and second best are underlined .
Dataset
Horizon
Seed 2024
Seed 2025
Seed 2026
Seed 2027
Seed 2028
Mean
Std. Dev
PEMS04
12
0.069
0.069
0.071
0.070
0.070
0.0698
0.0008
24
0.079
0.081
0.081
0.081
0.080
0.0804
0.0009
48
0.096
0.097
0.090
0.095
0.097
0.0950
0.0029
96
0.115
0.110
0.112
0.111
0.112
0.1120
0.0019
Appendix
Table 7: Robustness analysis of StretchTime on the PEMS04 dataset (MSE) across five random seeds. The low standard deviation confirms the stability of the performance improvements.
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.