Organizations: Institute of Computing Technology, CAS · Zhejiang Lab · Beijing University of Posts and Telecommunications · Beijing Jiaotong University · Beijing Language and Culture University · Chengdu University of Technology
Time series foundation models (TSFMs) based on autoregressive (AR) next-token prediction entangle two factors: variable-context conditioning and prediction-unit factorization with autoregressive rollout. Strong AR performance therefore does not reveal whether generalization stems from the flexible training distribution, the local prediction target, the left-to-right inference process, or their combination. We propose FlexCast, a flexible context-target learning framework that disentangles these factors. FlexCast reformulates forecasting as variable-length masked prediction over time series patches: during pretraining, it dynamically samples visible context and masked target patches, exposing the model to diverse context-target configurations without sequential decoding. At inference time, the same pretrained model supports one-step forecasting, confidence-based iterative refinement, and next-token-style generation through a unified mask-and-predict interface, enabling controlled comparison of inference schedules under identical architecture, data, and objective. Evaluated on BLAST and multiple zero-shot benchmarks, FlexCast achieves accuracy competitive with or better than strong TSFM baselines while substantially reducing inference latency. Ablations confirm that context diversity and contiguous target masking drive performance, while left-to-right decoding does not yield consistent gains once flexible context-target relationships have been learned, suggesting that the key benefit of AR-style TSFMs lies in flexible training rather than sequential inference. The code is available at https://github.com/BenchCouncil/FlexCast.
Figures & tables
Figure 1. Comparison of forecasting patterns and inference modes. (a) Conventional Non-Autoregressive forecasting is rigid: a task-specific model usually learns one fixed observation-length–prediction-length pattern, so new context or horizon settings require another model, head, or task configuration, although inference is one-step. Recent Non-AR TSFMs can relax this restriction through variable-length or masked training. (b) Autoregressive forecasting is flexible on the observation side: next-token training lets one model learn multiple variable-observation-to-unit-target distributions, but the full horizon must be generated by multi-step inference with predictions fed back as input. (c) FlexCast is flexible on both sides: masked forecasting lets one model learn multiple observation-length–prediction-length patterns while preserving one-step inference. A three-panel schematic comparing conventional Non-Autoregressive, Autoregressive, and FlexCast forecasting. The conventional Non-Autoregressive panel shows rigid observation and prediction windows, single-pattern training, and one-step inference. The Autoregressive panel shows flexible observation histories, prediction-as-input tokens, multi-pattern training, and multi-step inference. The FlexCast panel shows flexible observations and predictions with multi-pattern training and one-step inference.
Figure 2. Overview of the FlexCast framework. (a) Model Backbone: the input series is normalized with RevIN and an asinh transform, embedded together with context/prediction mask indicators, processed by a shared Transformer encoder whose additive attention bias sets padding keys to −∞ , and mapped to quantile forecasts before de-normalization. (b) Training Strategy: FlexCast uses random contiguous masking to sample visible context, masked target, and padding positions, feeds them into the shared backbone, and computes the quantile pinball loss only at prediction positions. (c) Inference Strategy: the same quantile predictor supports one-step forecasting and multi-step generation, where predicted quantiles provide confidence scores and selected predictions can be fed back as input in later rounds. Three-panel diagram of FlexCast. Panel a shows normalization, embedding with context and prediction mask indicators, a Transformer encoder with negative-infinity attention bias for padding keys, a quantile head, and de-normalization. Panel b shows random contiguous masking, a shared backbone, and quantile pinball loss at prediction positions. Panel c shows quantile prediction for one-step and multi-step inference, with confidence scores and predictions reused as input.
Models
FlexCast(Ours) ♣
PatchTST-FM ⧫
Sundial l ♠
TimeMoE l ♣
TimeMoE b ♣
MOIRAI l ♣
MOIRAI b ♣
Chronos b ♣
Chronos s ♣
#Params
131.4M
260M
444M
453M
113M
311M
91M
200M
46M
Metrics
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTh1
96
.362
.381
.358
.378
.346
.383
.348
.375
.352
.376
.359
.383
.362
.384
.357
.375
.359
.376
192
.395
.403
.400
.407
.386
.410
.381
.399
.389
.401
.395
.404
.400
.405
.397
.401
.403
.402
336
.409
.414
.438
.439
.410
.426
.409
.424
.408
.419
.411
.416
.416
.420
.422
.417
.431
.416
720
.461
.458
.500
.464
.438
.459
.447
.451
.450
.455
.420
.430
.430
.439
.460
.443
.449
.439
Table 1. Zero-shot forecasting performance compared with strong forecasting baselines. The best results are marked in red, and the second-best results are marked in light blue. The symbols s , b , and l represent the small, base, and large versions, respectively. Superscripts indicate different pretraining corpora: ♣ for BLAST, ⧫ for the PatchTST-FM corpus, and ♠ for the Sundial corpus.
Figure 3. Inference-latency scaling across four prediction lengths with a fixed 512-step context. The three-step FlexCast curve and shorter-horizon baseline points are estimated from the measured endpoints. Lower is better. Line chart comparing measured endpoint and estimated scaling trends for one-step and three-step FlexCast and baseline time series foundation models at prediction lengths 96, 192, 336, and 720.
Table 2. Settings for the variable context-target ablation.
Figure 4. Average MSE of different context-target pattern designs across downstream datasets. Bar chart comparing average MSE for rigid and flexible context-target pattern designs. The full flexible context and flexible target setting performs best.
Strategy
Mask rate
Position
Shape
♣ Tail
r∼U(0.1,0.3)
Tail
Contig.
♠ CPM
rmax=0.4,cmax=6
Random
Contig. blocks
Fixed
r∈{0.1,0.2,0.3}
Random
Contig.
▲ NTP
r∼U(0.1,0.3)
Tail
Next-patch
[0pt][0pt] \varheartsuit Random (Ours)
r∼U(0.1,0.3)
Random
Contig.
Table 3. Settings for the pretraining-strategy ablation.
Figure 5. Average MSE of different pretraining strategies across downstream datasets. Bar chart comparing Tail, Contiguous Patch Masking, Fixed-rate, NTP, and Random-rate pretraining strategies by average MSE. NTP has the highest error and Random-rate contiguous masking has the lowest error across the evaluated datasets.
Figure 6. Relative MSE change compared with one-step inference across datasets, horizons, and inference schedules. Positive values indicate higher MSE than one-step inference, while negative values indicate lower MSE. Relative MSE comparison of one-step, confidence-based multi-step, and blockwise autoregressive inference schedules across datasets and horizons.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
Model Architecture
Maximum context window
4096
Patch length
16
Quantile levels
99
Encoder layers
10
Model dimension
1024
Attention heads
16
Appendix
Table 4. Common experimental settings used in the ablation studies.
Model
#Params
Layers
dmodel
dff
Avg MSE
FlexCast s
15M
6
256
1024
.360
FlexCast b
40M
8
512
2048
.332
FlexCast l
131.4M
10
1024
4096
.308
Appendix
Table 5. Model capacity analysis. Average MSE across six benchmarks (four horizons each) for three FlexCast model sizes. All variants use the same BLAST pretraining data, training steps, and default mask-rate range.
Figure 7. Context-length sensitivity across datasets with prediction length 336. Line chart showing how forecasting MSE changes as the input context length varies across datasets.
Figure 8. Parameter sensitivity on ETTh2 (context 3376, horizon 336). Each subplot varies one hyperparameter and marks the default. Three sensitivity plots showing MSE changes under different attention head counts, encoder depths, and model dimensions.
Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.
Longer histories can improve time-series foundation models (TSFMs), but require substantially higher inference cost. We therefore ask whether contextual information can be provided more efficiently through a compact set of learned token embeddings. We introduce PaCTS, which generates a small set of instance-adaptive latent prompts in the form of continuous embedding tokens conditioned on the visible context. These prompts serve as compact context surrogates for frozen TSFMs. PaCTS constructs them from instance-specific global statistics and further refines them with segment-level temporal information, capturing both global characteristics and local temporal variations. The prompt module is jointly trained and deployed across heterogeneous time series with the frozen backbone. Extensive experiments demonstrate the effectiveness of prompts as context, consistently improving forecasting across context lengths and model architectures. With a shorter input context, PaCTS can outperform the same frozen backbone using double context while requiring substantially less inference computation. Compared with weight-space adaptation methods, PaCTS achieves stronger improvements and better out-of-distribution generalization.
Zehao Xiao, Shifeng Xie, Lei Zan +5
Huawei Noah’s Ark Lab, Paris, France · LIPADE, Universit´e Paris Cit´e, Paris, France · Huawei Noah’s Ark Lab, Shenzhen, China
Time Series Foundation Models (TSFMs) have borrowed the long context paradigm from natural language processing under the premise that feeding more history into the model improves forecast quality. But in stochastic domains, distant history is often just high-frequency noise, not signal. Hence, the proposed work tests whether this premise actually holds by running continuous context architectures (PatchTST included) through the ETTh1 benchmark. The obtained results contradict the premise: an inverse scaling law shows up clearly, with forecasting error rising as context gets longer. A 3,000-step window causes performance to drop by over 68%, evidence that attention mechanisms are poor at ignoring irrelevant historical volatility. Retrieval-Augmented Forecasting (RAFT) is evaluated as an alternative. RAFT achieves a mean squared error (MSE) of 0.379 with a fixed 720-step window and selective retrieval, outperforming both long-context configurations and zero-shot foundation models (Chronos, Moirai) despite requiring far less computation. In addition, the retrieval step injects only the most relevant historical segments as dynamic exogenous variables, which gives the model a context-informed inductive bias it cannot build on its own from raw sequences. Therefore, foundation models going forward need to shift architecturally toward selective retrieval.
Rishi Ahuja, Kumar Prateek, Simranjit Singh +1
Department of Information Technology, Dr. B.R. Ambedkar National Institute of Technology Jalandhar, Punjab, 144008, India.
Time Series Foundation Models (TSFMs) currently provide state-of-the-art results in forecasting tasks. They are available out-of-the-box and rely on in-context learning to make their predictions, which makes the quality of their performance highly sensitive to the user-selected lookback, covariates, horizon and training data distributions. In practise, the quality of the forecasts are variable but complementary, which highlights the need for a principled ensembling approach, rather than selecting the best context. This paper introduces Latent Inference-Time Guidance for TSFMs, which adaptively combines a pool of TSFM forecasts through a time-dependent latent space with independent components. The framework comes equipped with identifiability and reconstruction guarantees, whilst maintaining the off-the-shelf aspect of foundation models. We provide experiments on datasets at various frequencies and from multiple domains: these show that the approach is competitive with traditional ensembling approaches.