Echoes of Error: Residual-Directional Local Rollout Consistency for Timeseries Forecasting
Organizations: Bosch Center for Artificial Intelligence (BCAI) Bosch (China) Investment Co., Ltd.
Abstract
In time-series forecasting, rollout errors are not merely one-step fitting discrepancies: once fed back into the evolving state, they can perturb future predictions. We formalize this effect through local rollout consistency, which compares the next prediction produced under teacher forcing with that produced under free running. In principle, this consistency can be enforced directly by tracking the residual-induced perturbation of the next state. In practice, however, the residual-to-state alignment operator is often difficult to design when the input state and output block have different dimensions, heterogeneous components, or architecture-dependent layouts. We therefore use a first-order local surrogate that penalizes the Jacobian-vector product of the predictor along the actual feedback perturbation, suppressing precisely the directions through which self-generated errors remain visible to future predictions. We further give the same loss a geometric interpretation near a predictive manifold: by controlling residual-induced directions that probe the normal bundle, the objective yields theoretical conditions under which free-running trajectories remain inside a stability tube around the manifold. Experiments on time series forecasting tasks support this view, showing that the proposed regularization improves predictive accuracy. More broadly, our results suggest a training principle for time-series forecasters and sequential predictors: a model should not only make small one-step errors, but also learn not to let its own errors echo into the future. Code is available at https://github.com/Wangzzzzzzzz/Echoes-Of-Error.
Figures & tables
| Dataset | PatchTST | FilterNet | SparseTSF | TSLANet | AMD | DUET | TimeDistill | RootPurge | MLP + CL | ||
|---|---|---|---|---|---|---|---|---|---|---|---|
| ETTh1 | 96 | 0.385 | 0.375 | 0.362 | 0.387 | 0.364 | 0.362 | 0.369 | 0.373 | 0.362 | 0.358 |
| 192 | 0.413 | 0.436 | 0.403 | 0.425 | 0.399 | 0.401 | 0.401 | 0.411 | 0.398 | 0.395 | |
| 336 | 0.440 | 0.472 | 0.435 | 0.443 | 0.436 | 0.418 | 0.417 | 0.439 | 0.434 | 0.434 | |
| 720 | 0.456 | 0.474 | 0.426 | 0.463 | 0.487 | 0.439 | 0.445 | 0.495 | 0.423 | 0.458 | |
| ETTh2 | 96 | 0.274 | 0.301 | 0.294 | 0.297 | 0.281 | 0.274 | 0.276 | 0.273 | 0.272 | 0.269 |
| 192 | 0.338 | 0.363 | 0.339 | 0.348 | 0.351 | 0.351 | 0.334 | 0.334 | 0.332 | 0.331 |
| Model | Metric | Method | ETTh2 | ETTm2 | ECL | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 96 | 192 | 336 | 720 | 96 | 192 | 336 | 720 | 96 | 192 | 336 | 720 | |||
| FilterNet | MSE | Original | 0.312 | 0.363 | 0.402 | 0.438 | 0.178 | 0.243 | 0.293 | 0.370 | 0.137 | 0.155 | 0.170 | 0.206 |
| +CL | 0.278 | 0.341 | 0.382 | 0.428 | 0.164 | 0.224 | 0.273 | 0.352 | 0.137 | 0.153 | 0.169 | 0.206 | ||
| MAE | Original | 0.365 | 0.398 | 0.431 | 0.468 | 0.265 | 0.309 | 0.345 | 0.396 | 0.236 | 0.253 | 0.269 | 0.302 | |
| +CL | 0.340 | 0.383 | 0.420 | 0.464 | 0.255 | 0.298 | 0.332 | 0.383 | 0.236 | 0.252 | 0.269 | 0.301 | ||
| TSLANet | MSE | Original | 0.297 | 0.348 | 0.376 | 0.431 | 0.197 | 0.251 | 0.302 | 0.369 | 0.129 | 0.147 | 0.163 | 0.196 |
| Forecasting | Method | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Weather | ECL | Traffic | |
|---|---|---|---|---|---|---|---|---|---|
| Rollout | 336 | Original | 0.461 | 0.403 | 0.383 | 0.299 | 0.263 | 0.162 | 0.392 |
| +CL | 0.411 | 0.361 | 0.353 | 0.265 | 0.235 | 0.161 | 0.391 | ||
| 720 | Original | 0.508 | 0.428 | 0.452 | 0.381 | 0.345 | 0.200 | 0.428 | |
| +CL | 0.427 | 0.399 | 0.412 | 0.346 | 0.308 | 0.200 | 0.428 | ||
| One-shot | 336 | Original | 0.458 | 0.395 | 0.377 | 0.296 | 0.258 | 0.163 | 0.388 |
| +CL | 0.434 | 0.362 | 0.352 | 0.268 | 0.235 | 0.160 | 0.388 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Datasets | ETTh1 | ETTh2 | ETTm1 | ETTm2 | Weather | Traffic | Electricity |
|---|---|---|---|---|---|---|---|
| No. of Channels | 7 | 7 | 7 | 7 | 21 | 862 | 321 |
| Timesteps | 17420 | 17420 | 69680 | 69680 | 52696 | 17544 | 26304 |
| Frequency | Hourly | Hourly | 15min | 15min | 10min | Hourly | Hourly |
| Application Domain | Electricity | Electricity | Electricity | Electricity | Weather | Traffic | Electricity |
| Dataset | Forecasting horizon | |||
|---|---|---|---|---|
| 96 | 192 | 336 | 720 | |
| ETTh1 | ||||
| ETTh2 | ||||
| ETTm1 | ||||
| ETTm2 | ||||
| Weather | ||||
| Dataset | Forecasting horizon | |||
|---|---|---|---|---|
| 96 | 192 | 336 | 720 | |
| ETTh1 | ||||
| ETTh2 | ||||
| ETTm1 | ||||
| ETTm2 | ||||
| Weather | ||||
| Dataset | Avg Padding | Repeat Padding | ||||||
|---|---|---|---|---|---|---|---|---|
| 96 | 192 | 336 | 720 | 96 | 192 | 336 | 720 | |
| ETTh1 | ||||||||
| ETTh2 | ||||||||
| ETTm1 | ||||||||
| ETTm2 | ||||||||
| Weather | ||||||||
| Dataset | Forecasting horizon | |||
|---|---|---|---|---|
| 96 | 192 | 336 | 720 | |
| ETTh1 | ||||
| ETTh2 | ||||
| ETTm1 | ||||
| ETTm2 | ||||
| Weather | ||||
| Dataset | Forecasting horizon | |||
|---|---|---|---|---|
| 96 | 192 | 336 | 720 | |
| ETTh1 | ||||
| ETTh2 | ||||
| ETTm1 | ||||
| ETTm2 | ||||
| Weather | ||||
| Dataset | Method | + Consistency Loss | Baseline (without CL) | ||
|---|---|---|---|---|---|
| 336 | 720 | 336 | 720 | ||
| ETTh1 | Rollout | ||||
| One-shot | |||||
| ETTh2 | Rollout | ||||
| One-shot | |||||
| ETTm1 | Rollout | ||||
| Backbone | Metric | MSE only | Full JVP | JVP | |
|---|---|---|---|---|---|
| ResMLP | 128 | Time (s) | 17.564 ( ) | 38.397 ( ) | 19.009 ( ) |
| Memory (GiB) | 1.918 ( ) | 3.931 ( ) | 1.837 ( ) | ||
| FilterNet | 64 | Time (s) | 8.847 ( ) | 9.146 ( ) | 9.154 ( ) |
| Memory (GiB) | 0.610 ( ) | 1.055 ( ) | 0.638 ( ) | ||
| iTransformer | 64 | Time (s) | 9.437 ( ) | 29.380 ( ) | 11.925 ( ) |
| Memory (GiB) | 3.258 ( ) | 11.205 ( ) | 3.756 ( ) |
| Dataset | Training Setting | Forecasting horizon | |||
|---|---|---|---|---|---|
| 96 | 192 | 336 | 720 | ||
| ETTh1 | Full JVP | 0.358 | 0.395 | 0.434 | 0.458 |
| JVP ( ) | 0.359 | 0.396 | 0.435 | 0.449 | |
| JVP ( ) | 0.361 | 0.398 | 0.437 | 0.456 | |
| ETTh2 | Full JVP | 0.269 | 0.331 | 0.362 | 0.397 |
| JVP ( ) | 0.269 | 0.330 | 0.362 | 0.399 | |
| Training Setting | Overall | ||||
|---|---|---|---|---|---|
| No CL on MLP branch ( ) | 0.2621 | 0.3019 | 0.3347 | 0.3936 | 0.3231 |
| Full JVP | 0.2420 | 0.2803 | 0.3141 | 0.3644 | 0.3002 |
| JVP ( ) | 0.2424 | 0.2801 | 0.3141 | 0.3634 | 0.3000 |
| JVP ( ) | 0.2434 | 0.2814 | 0.3150 | 0.3653 | 0.3013 |
| Backbone | MSE only | Full JVP | JVP | (%) |
|---|---|---|---|---|
| FilterNet | 0.2723 | 0.2590 | 0.2583 | |
| TSLANet | 0.2672 | 0.2561 | 0.2548 | |
| iTransformer | 0.2730 | 0.2602 | 0.2626 | |
| 0.2582 | 0.2503 | 0.2504 |
| Model | Training Setting | ETTh2 | ETTm2 | ECL | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 96 | 192 | 336 | 720 | 96 | 192 | 336 | 720 | 96 | 192 | 336 | 720 | ||
| FilterNet | Full JVP | 0.278 | 0.341 | 0.382 | 0.428 | 0.164 | 0.224 | 0.273 | 0.352 | 0.137 | 0.153 | 0.169 | 0.206 |
| JVP | 0.280 | 0.341 | 0.375 | 0.420 | 0.164 | 0.225 | 0.277 | 0.352 | 0.137 | 0.153 | 0.169 | 0.206 | |
| TSLANet | Full JVP | 0.270 | 0.332 | 0.364 | 0.411 | 0.190 | 0.239 | 0.282 | 0.358 | 0.128 | 0.144 | 0.160 | 0.194 |
| JVP | 0.269 | 0.331 | 0.362 | 0.399 | 0.189 | 0.240 | 0.284 | 0.359 | 0.127 | 0.144 | 0.159 | 0.193 | |
| iTransformer | Full JVP | 0.284 | 0.351 | 0.391 | 0.432 | 0.164 | 0.224 | 0.278 | 0.355 | 0.132 | 0.153 | 0.167 | 0.191 |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 930 papers