A Verifier from the Past: Test-Time Compute for Time Series Foundation Models
Organizations: Georgia Institute of Technology · Synthefy · The University of Texas at Austin
Abstract
Test-time compute (TTC) improves language models by spending additional inference compute to generate, verify, and select among candidate outputs, but this recipe has not transferred directly to time series forecasting. A language model can verify a proof or vote across samples, but a forecaster must be causal; it cannot use future information to score a candidate. We propose Backtest-Gated Augmentation Averaging (BGAA), a TTC algorithm for time series forecasting that builds a causal verifier from the series's own history, using rolling-origin backtests to score structured views of the context, discard those that underperform the untransformed context, and average the survivors' predictive distributions. BGAA improves every tested TSFM-benchmark combination, reducing MASE by 0.5-1.0% on GIFT-Eval, 1.3-5.0% on fev-bench, and 1.7-5.9% on BOOM, where a 313M model with TTC outperforms the 1B and 2.5B models in its family. We further show that useful views have structured, model-dependent effects consistent with a prior-transport interpretation. On the 53 cross-fittable GIFT-Eval configs, the deployed policy gains only 0.4-1.3% CRPS, versus 1.4-4.1% when choosing the best view for each dataset configuration and 17-19% when choosing the best view separately for each test window. This gap suggests that a major remaining challenge is window-level view selection.
Figures & tables
| benchmark | model | base (MASE / CRPS † ) | Bgaa (MASE / CRPS † ) | (MASE / CRPS † ) | % (MASE / CRPS † ) | win/loss |
| GIFT-Eval | Chronos-2 | 0.7043 / 0.4830 | 0.6975 / 0.4778 | 0.004 / 0.004 | 55/38 | |
| TiRex-2 | 0.6972 / 0.4791 | 0.6940 / 0.4776 | 0.021 / 0.028 | 54/38 | ||
| PatchTST-FM-R1 | 0.7052 / 0.4820 | 0.7018 / 0.4811 | 0.001 / 0.001 | 64/28 | ||
| Toto-2-313M | 0.7032 / 0.4838 | 0.6984 / 0.4806 | 0.023 / 0.042 | 60/32 | ||
| BOOM | Chronos-2 | 0.6547 / 0.3935 | 0.6259 / 0.3803 | 0.001 / 0.001 | 4165/1691 | |
| TiRex-2 | 0.6418 / 0.3855 | 0.6191 / 0.3763 | 0.001 / 0.001 | 3917/1941 |
| system | MASE | CRPS | note |
| Toto-2-313M + Bgaa | 0.575 | 0.350 | ours |
| Toto-2-2.5B | 0.581 | 0.349 | published |
| Toto-2-1B | 0.582 | 0.349 | published |
| Toto-2-313M | 0.585 | 0.351 | published |
| Toto-2-313M base | 0.585 | 0.353 | our reproduction |
| Toto (v1) | 0.617 | 0.375 | published |
| final policy | conservative ( routemix05-crps ) | ||||||
| benchmark | configs | equal or better | no sig. regression | worst | equal or better | no sig. regression | worst |
| GIFT-Eval | 97 | 61% | 95% | 67% | 93% | ||
| BOOM | 6750 | 69% | 92% | 85% | 95% | ||
| fev-bench | 100 | 70% | 94% | 77% | 97% | ||
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
| tier | family | # | passes | degrees, and how the fan is inverted |
| crop | identity | 1 | 1 | – (cached base forecast) |
| crop | lookback | 18 | 18 | keep the last samples, 0.98, 0.95, 0.925, 0.9, 0.875, 0.85, 0.8, 0.75, 0.7, 0.625, 0.55, 0.5, 0.4375, 0.375, 0.3125, 0.25, 0.1875, 0.125 ; restriction map, no fan inversion |
| crop | offset | 14 | 14 | shift the patch grid by 1, 2, 3, 4, 6, 8, 10, 12, 14, 16, 20, 24, 32, 48 samples ( ); restriction map |
| micro | jitter | 1 | 3 | additive noise, seeded draws averaged in quantile space; median-neutral |
| micro | jitternorm | 1 | 3 | as above, rescaled to preserve the context’s scale |
| micro | winsor | 1 | 1 | clip at the tails |
| Chronos-2 | TiRex-2 | PatchTST-FM-R1 | Toto-2-313M | |||||
| mode | cand | cost | MASE% | CRPS% | worst% | cost / MASE% | ||
| base | — | — | — | |||||
| low | / | / | / | |||||
| medium | / | / | / | |||||
| high | / | / | / | |||||
| xhigh | / | / | / | |||||
| GIFT-Eval | Time for forecasts (s) | |||||||||
| System | Mode | MASE | WQL | PFLOPs | ms/fc | GPU (GiB) | ||||
| toto_2.0_2.5b | — | 0.6956 | 121.0 | 0.133 | 0.319 | 2.86 | 28.5 | 2.85 | 10.30 | |
| TiRex-2 | base | 0.6972 | 0.4791 | 15.6 | 0.0562 | 0.212 | 1.77 | 17.7 | 1.77 | 5.96 |
| TiRex-2 + Bgaa | xhigh | 0.4776 | 148.8 | 40.4 | 41.1 | 47.7 | 114 | 7.40 | 5.96 | |
| Chronos-2 | base | 0.7043 | 0.4830 | 22.1 | 0.0143 | 0.0203 | 0.173 | 1.71 | 0.17 | 0.48 |
| Chronos-2 + Bgaa | low | 0.7012 | 0.4805 | 53.0 | 0.125 | 0.160 | 0.780 | 7.02 | 0.69 | 0.48 |
| Time (s) | Relative to frozen base ( ) | GPU (GiB) | |||||||
| Model | Base | BGAA cold | Cold batched | Cold unbatched | Passes | Warm batched | Base | BGAA | |
| Chronos-2 | 1 | 0.0143 | 4.49 | 313 | 22,594 | 22,276 | 1.12 | 0.26 | 0.46 |
| 10 | 0.0158 | 4.49 | 285 | 2,260 | 2,231 | 1.30 | 0.27 | 0.46 | |
| 100 | 0.0203 | 4.58 | 226 | 230 | 227 | 5.55 | 0.42 | 0.47 | |
| 500 | 0.0864 | 4.97 | 57.5 | 49.2 | 48.5 | 5.77 | 0.47 | 0.47 | |
| 1,000 | 0.173 | 5.47 | 31.7 | 26.6 | 26.3 | 5.78 | 0.47 | 0.48 | |
| pool tier | MASE | CRPS | policy | MASE | CRPS |
| base (no TTC) | 0.7043 | 0.4830 | margin (conservative) | 0.7019 | 0.4823 |
| crop core | 0.7005 | 0.4811 | top-2 Vincent | 0.6990 | 0.4792 |
| + micro views | 0.7007 | 0.4809 | top-3 Vincent | 0.6978 | 0.4780 |
| + 9 ideas families | 0.7000 | 0.4809 | CDF mixture top-3 | 0.6975 | 0.4778 |
| + mixture aggregation | 0.7001 | 0.4805 | spread-routed mixture | 0.6996 | 0.4796 |
| + decomposition family | 0.6975 | 0.4778 | tempvinc-mase | 0.6991 | 0.4798 |
| control alone | full pool vs control | ||||||
| pool | MASE% | w/l | share | % | w/l | ||
| full pool | 9/5 | 0.011 | 100% | — | — | — | |
| lookback-only | 8/2 | 0.037 | 30% | 7/6 | 0.065 | ||
| offset-only | 3/4 | 0.100 | 1% | 10/4 | 0.001 | ||
| cleaning-only | 3/5 | 0.484 | % | 10/4 | 0.030 | ||
| vs base (MASE) | vs base (CRPS) | BGAA vs arm (MASE) | |||||||
| arm | % | w/l | % | w/l | % | w/l | |||
| BGAA top-2 | 63/29 | 64/29 | — | — | — | ||||
| random- | 35/60 | 31/63 | 77/20 | ||||||
| noise- | 47/50 | 0.079 | 45/52 | 0.86 | 65/32 | ||||
| lag ( ) | MASE% | CRPS% | (MASE) | Jaccard vs lag 0 |
| 0 | 0.011 | 1.00 | ||
| 1 | 0.043 | 0.59 | ||
| 2 | 0.015 | 0.48 | ||
| 4 | 0.050 | 0.44 |
| Bgaa | Top-1 | Base + top-1 | L2-convex | L2-anchored | ||||||
| MASE | WQL | MASE | WQL | MASE | WQL | MASE | WQL | MASE | WQL | |
| Chronos-2 , GIFT-Eval | ||||||||||
| Chronos-2 , fev | ||||||||||
| Toto-2, GIFT-Eval | ||||||||||
| Toto-2, fev | ||||||||||
| TiRex-2 , GIFT-Eval | ||||||||||
| (a) Chronos-2/GIFT-Eval pool transferred across models and benchmarks | |||||||||
| Full pool (51) | Restricted pool (22) | Retained gain, top- (%) | |||||||
| Model / benchmark | MASE | WQL | MASE | WQL | Kept (%) | Gap [95% CI] | |||
| Chronos-2 , GIFT-Eval ∗ | |||||||||
| Chronos-2 , fev | [0.47, 8.42] | ||||||||
| Toto-2, GIFT-Eval | [ 0.13, 2.15] | ||||||||
| Toto-2, fev | [ 0.04, 0.68] | ||||||||
| Model / benchmark | Selected views (frequency) | Distinct | Outside |
| Chronos-2 , GIFT-Eval ∗ | decomp m (7), decomp 2m (6), rankgauss (5), multideseason m (4) | 22 | |
| Chronos-2 , fev | decomp 2m (5), decomp m (5), diff1 † (4), lookback (3), rankgauss (3), multideseason m (3) | 24 | |
| Toto-2, GIFT-Eval | recursive 0.25 (5), decomp m (3) | 27 | |
| Toto-2, fev | winsor 0.005 (4), recursive (4), multideseason m (3), diff1 † (3), rankgauss (3), decomp m (3) | 22 | |
| TiRex-2 , GIFT-Eval | supportfloor † (3), nonneg † (3), decomp m (3) | 27 | |
| TiRex-2 , fev | rankgauss (5), recursive (4), decomp m (4), winsor 0.005 (3), decomp m/2 (3), multideseason m (3) | 25 |
| model | policy | MASE | CRPS | MASE% | CRPS% | w/l |
| Chronos-2 | base | 0.7043 | 0.4830 | — | — | — |
| top2-mase | 0.6990 | 0.4792 | 63/29 | |||
| top3-mase | 0.6978 | 0.4780 | 65/27 | |||
| routemix05-crps | 0.6996 | 0.4796 | 63/29 | |||
| mixtop3-mase | 0.6975 | 0.4778 | 55/38 | |||
| TiRex-2 | base | 0.6972 | 0.4791 | — | — | — |
| model | policy | SQL | WQL | MASE | SQL skill | MASE skill | SQL w/l |
| Chronos-2 | base | 1.3416 | 0.2193 | 1.6332 | 0.438 | 0.312 | — |
| routemix05-crps | 1.2882 | 0.2179 | 1.5593 | 0.447 | 0.324 | 71/23 | |
| mixtop3-mase | 1.2836 | 0.2163 | 1.5513 | 0.447 | 0.325 | 56/38 | |
| TiRex-2 | base | 1.3653 | 0.2275 | 1.6192 | 0.426 | 0.305 | — |
| routemix05-crps | 1.3460 | 0.2266 | 1.6098 | 0.431 | 0.309 | 64/27 | |
| mixtop3-mase | 1.3436 | 0.2263 | 1.5989 | 0.430 | 0.309 | 54/39 |
| system | MASE | CRPS | system | MASE | CRPS |
| Toto-2-313M + Bgaa | 0.575 | 0.350 | moirai_2_small | 0.672 | 0.427 |
| toto_2.0_2.5b | 0.581 | 0.349 | moirai_1.1_base | 0.710 | 0.428 |
| toto_2.0_1b | 0.582 | 0.349 | moirai_1.1_large | 0.720 | 0.436 |
| toto_2.0_313m | 0.585 | 0.351 | moirai_1.1_small | 0.729 | 0.442 |
| Toto-2-313M base | 0.585 | 0.353 | timesfm_2_0_500m | 0.725 | 0.447 |
| toto_2.0_22m | 0.601 | 0.363 | chronos_bolt_base | 0.726 | 0.451 |
| policy | MASE | % | w/l | worst% | CRPS | % | w/l | ||
| base | 0.7043 | — | — | — | — | 0.4830 | — | — | — |
| margin-mase | 0.7019 | 0.080 | 17/9 | 0.4823 | 0.980 | 15/11 | |||
| top2-mase | 0.6990 | 0.001 | 63/29 | 0.4792 | 0.021 | 64/29 | |||
| top3-mase | 0.6978 | 0.001 | 65/27 | 0.4780 | 0.002 | 66/27 | |||
| mixtop2-mase | 0.6991 | 0.020 | 58/35 | 0.4791 | 0.026 | 57/36 | |||
| mixtop3-mase | 0.6975 | 0.004 | 55/38 | 0.4778 | 0.004 | 60/33 |
| model | Econ/Fin | Energy | Health | Nature | Sales | Transport | Web/CloudOps |
| Chronos-2 | |||||||
| (4/2) | (20/12) | (4/1) | (8/7) | (3/0) | (11/4) | (10/7) | |
| TiRex-2 | |||||||
| (1/5) | (19/12) | (5/0) | (6/9) | (3/0) | (9/6) | (10/7) | |
| PatchTST-FM-R1 | |||||||
| (3/3) | (20/12) | (4/0) | (10/5) | (3/0) | (12/3) | (11/5) |
| Normalized MSE | Normalized MAE | ||||||
| Model | Base | Bgaa | Base | Bgaa | |||
| Chronos-2 | 96 | 0.2405 | 0.2723 | ||||
| 192 | 0.2920 | 0.3105 | |||||
| 336 | 0.3328 | 0.3402 | |||||
| 720 | 0.3901 | 0.3827 | |||||
| Toto-2-313M | 96 | 0.2392 | 0.2677 | ||||
| all configs | configs long enough to cross-fit | |||||
| model | realized ( Table 1 ) | realized | label-ranked gate | realized / label-ranked | best-view gain (x-fit) | per-window hindsight |
| % | % | % | ratio | % | % | |
| Chronos-2 | 1.07 | 1.32 | 2.06 | 0.64 | 4.10 | 19.4 |
| TiRex-2 | 0.32 | 0.50 | 1.21 | 0.41 | 1.48 | 16.6 |
| PatchTST-FM-R1 | 0.19 | 0.41 | 1.20 | 0.34 | 1.42 | 17.5 |
| Toto-2-313M | 0.66 | 0.61 | 1.16 | 0.52 | 1.37 | 16.6 |
| CRPS | MASE | |||||||||
| best-view gain | label-ranked gate | realized | label-ranked | realized/ | best-view gain | per-window | ||||
| model | in | x-fit | in | x-fit | gate (x-fit) | label-ranked | in | x-fit | hindsight | |
| Chronos-2 | 5.36 | 4.10 | 2.01 | 2.06 | 0.92 | 1.39 | 0.67 | 2.44 | 0.76 | 20.7 |
| TiRex-2 | 3.43 | 1.48 | 1.06 | 1.21 | 0.23 | 0.75 | 0.31 | 1.79 | 1.09 | 19.0 |
| PatchTST-FM-R1 | 3.54 | 1.42 | 1.09 | 1.20 | 0.35 | 1.23 | 0.28 | 3.46 | 2.36 | 21.3 |
| Toto-2-313M | 2.95 | 1.37 | 1.26 | 1.16 | 1.08 | 1.44 | 0.75 | 1.74 | 0.72 | 19.4 |
| Model | Series/config | Configs | Series | Backtests/series | Per-config MASE | Per-series MASE | Gap [95% CI] |
| Chronos-2 | Low ( – ) | 17 | 114 | 47 | [ , ] | ||
| Medium ( ) | 7 | 147 | 18 | [ , ] | |||
| High ( – ) | 6 | 204 | 12 | [ , ] | |||
| Very high ( – ) | 11 | 1,507 | 3 | [0.50, 2.70] | |||
| All (41) | 41 | 1,972 | 3–47 | [ , ] | |||
| TiRex-2 | Low ( – ) | 17 | 114 | 47 | [ , ] |
| family | Chronos-2 | TiRex-2 | PatchTST-FM-R1 | Toto-2-313M |
| changepoint | ||||
| decomp | ||||
| diff | ||||
| fft | ||||
| jitter | ||||
| jitternorm |
| model A | model B | perm. | |
| Chronos-2 | TiRex-2 | ||
| Chronos-2 | PatchTST-FM-R1 | ||
| Chronos-2 | Toto-2-313M | ||
| TiRex-2 | PatchTST-FM-R1 | ||
| TiRex-2 | Toto-2-313M | ||
| PatchTST-FM-R1 | Toto-2-313M |
| Characteristic | Tercile | Chronos-2 | Toto-2-313M | TiRex-2 |
| Seasonality | Low | diff BGAA | winsor † BGAA | offset † BGAA |
| High | decomp BGAA | decomp BGAA | multideseason BGAA | |
| Trend strength | Low | decomp BGAA | decomp BGAA | multideseason BGAA |
| High | decomp BGAA | winsor † BGAA | multideseason † BGAA | |
| Lag-1 autocorr. | Low | decomp BGAA | winsor † BGAA | multideseason BGAA |
| High | diff BGAA | decomp BGAA | multideseason † BGAA |
| Selection method | Chronos-2 | Toto-2-313M | TiRex-2 |
| Historical top-1 + identity | |||
| Global view router | |||
| Four-characteristic router | |||
| Best of 12 routers (post hoc) | |||
| Bgaa ( ; reference) |
| group | features | description |
| Autocorrelation and memory | acf1 , acf2 , acf5 , acf10 , acf_first_zero , pacf2 | Autocorrelation at lags ; first non-positive ACF lag up to (scaled by ); and lag- partial autocorrelation. |
| Trend and oscillation | trend_r2 , turning_rate , crossing_rate | Linear-trend , frequency of sign changes in first differences, and frequency of crossings of the series mean. |
| Spectrum | spectral_entropy , seasonal_peak_power , seasonal_peak_freq | Normalized spectral entropy, power share of the strongest non-DC Fourier component, and its normalized frequency. The spectrum uses a Hann window. |
| Distribution shape | skew , kurtosis , iqr_over_std , mad_over_std , tail_frac_2sig | Skewness, excess kurtosis, normalized IQR and MAD, and the fraction of observations with . |
| Smoothness, jumps, and persistence | diff_std_ratio , step_max , runlen_mean | Relative first-difference variability, largest one-step change, and mean run length above or below the median. |
| Chronos-2 | TiRex-2 | |||||
| view | magnitude | consistency | CI | magnitude | consistency | CI |
| diff | ||||||
| smooth_ma7 | ||||||
| jitter_0.002 | ||||||
| sdiff | ||||||
| rankgauss | ||||||
| feature space | embedding space | |||
| model | mean off-diag | max | mean off-diag | max |
| TiRex-2 | ||||
| Chronos-2 | ||||
| PatchTST-FM-R1 | ||||
| Toto-2-313M | ||||
| model | top mode for diff | cosine | mass |
| TiRex-2 | high kurtosis / high skew | ||
| Chronos-2 | low iqr_over_std / high step_max | ||
| PatchTST-FM-R1 | high tail_frac_2sig / high seasonal_peak_freq | ||
| Toto-2-313M | low iqr_over_std / low mad_over_std |
| Chronos-2 | TiRex-2 | |||||||
| family | width | sign | shift | cov | width | sign | shift | cov |
| diff | ||||||||
| sdiff | ||||||||
| recursive | ||||||||
| fft | ||||||||
| rankgauss | ||||||||
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 947 papers