Investigating the effects of the patch size in Time Series Foundation Models

Jad Yehya; Andrei-Tiberiu Pantea; Aymeric Dieuleveut; Matthieu Kowalski; Thomas Moreau

cs.LGNov 2, 2026

Investigating the effects of the patch size in Time Series Foundation Models

Authors: Jad Yehya, Andrei-Tiberiu Pantea, Aymeric Dieuleveut, Matthieu Kowalski, Thomas Moreau

Organizations: Université Paris-Saclay, Inria, CEA Palaiseau, France · CMAP, École polytechnique, IP Paris Électricité de France R&D Palaiseau, France · CMAP, École polytechnique, IP Paris Palaiseau, France · Université Paris-Saclay, Inria, CNRS, LISN Gif-sur-Yvette, France

Abstract

Patch size is a central design choice in patch-based time-series foundation models, controlling temporal resolution, sequence length, and computational cost. We study how this choice interacts with transformer architecture through a controlled comparison of encoder-only and decoder-only forecasting models trained with the same data mixture and optimization budget. Across 97 GIFT-Eval tasks, encoder-only models consistently benefit from smaller patches, whereas decoder-only performance follows a U-shaped trend, with an intermediate patch size performing best in aggregate. Dataset-level results further show that the preferred decoder patch size increases with the forecasting horizon. For encoder-only models, increasing context length improves performance across patch sizes, although the gains diminish beyond a context length of 2048. Finally, training-compute and CPU-latency Pareto analyses reveal complementary regimes: decoder-only models provide efficient low-cost alternatives, while encoder-only models attain the highest overall accuracy. These results show that the choice of patch size and architecture should be made depending on the forecasting horizon and deployment constraints.

Figures & tables

Figure 1: Normalized MASE (left) and CRPS (right) against CPU latency for encoder-only and decoder-only models for various values of the patch size

P

. The latency is computed over evaluations on the GIFT-Eval horizon lengths and averaged according to GIFT-Eval’s proportions.

Model family	Patch size $P$	Context length $C$
Encoder-only	4	1024
Encoder-only	8	1024, 2048
Encoder-only	16, 32, 64	1024, 2048, 4096
Decoder-only	16, 32, 64, 128, 256	2048

Table 1: Configurations considered in our experiments. All the configurations are trained for 100k optimization steps.

Figure 2: Effect of patch size on the encoder-only model for

C=2048

. Left: normalized MASE and CRPS on GIFT-Eval. Right: dataset win rates by forecasting term. The encoder-only model with patch size

P=8

is the best in both aggregate performance and dataset win rates within each term.

Figure 3: Encoder-only context and patch-size sweep after 100k training steps. Within each context length, smaller patches perform better. Increasing the context length substantially improves performance, although the gains from

2048

4096

are smaller than those from

1024

2048

Figure 4: Effect of patch size on the decoder-only model. Left: aggregate MASE and CRPS on GIFT-Eval. Right: dataset win rates separated by forecasting term. Aggregate performance is best at

P=64

, while the patch sizes favored by individual tasks shift as the forecasting term increases.

Figure 5: Normalized MASE (left) and CRPS (right) against training compute for encoder-only and decoder-only models with various patch sizes

P

. Training compute denotes the number of matrix-multiplication FLOPs required to train each model for 100k steps with an effective batch size of 512 and is reported in EFLOP. Each point shows the mean performance over pretraining seeds and the dashed line indicates the Pareto front.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	$d_{\mathrm{model}}$	Layers	Heads	Quantiles	Parameters
Encoder-only	1024	20	16	99	$255.0$ - $267.5$ M
Decoder-only	1024	20	16	9	$253.4$ – $255.9$ M

Appendix

Table 2: Architecture and parameter counts of the evaluated model families. Parameter counts vary slightly with patch size and context length; ranges are reported over all evaluated configurations.

Model family	Context length	$P=4$	$8$	$16$	$32$	$64$	$128$	$256$
Encoder-only	1024	5	5	5	5	5	–	–
Encoder-only	2048	–	5	5	5	1	–	–
Encoder-only	4096	–	–	5	5	5	–	–
Decoder-only	2048	–	–	1	2	3	3	3

Appendix

Table 3: Number of pretraining seeds per configuration.

Time-Series Foundation Models

Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.

See all 320 papers

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting
Mar 11, 2026Sravan Kumar Ankireddy, Nikita Seleznev, Nam H. Nguyen +4Time Series Forecasting Time Series Transformers
University of Texas at Austin, Austin, TX, USA · Capital One, USA · University of Maryland, College Park, MD, USA
Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
Feb 19, 2026Xinghong Fu, Yanhong Li, Georgios Papaioannou +1Time-Series Foundation Models Zero-Shot Time Series Forecasting
Massachusetts Institute of Technology · Allen Institute for AI · Qube Research & Technologies
Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting
Jun 2, 2026Federico Zucchi, Yi Xie, Chao Zhang +2Long-Term Time Series Forecasting Time Series Forecasting
ICube, University of Strasbourg, Illkirch-Graffenstaden, France · Cephalgo, Strasbourg, France · Technical University of Munich +2

Mar 11, 2026J/K move · Enter open · S save

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting

Sravan Kumar Ankireddy, Nikita Seleznev, Nam H. Nguyen, Yulun Wu+3

University of Texas at Austin, Austin, TX, USA · Capital One, USA · University of Maryland, College Park, MD, USA

Time Series Forecasting Time Series Transformers Time-Series Foundation Models

Open paper AskPDF arXiv

Transformer-based time series foundation models face a fundamental trade-off in choice of tokenization: point-wise embeddings preserve temporal fidelity but scale poorly with sequence length, whereas fixed-length patching improves efficiency by imposing uniform boundaries that may disrupt natural transitions and blur informative local dynamics. In order to address these limitations, we introduce TimeSqueeze, a dynamic patching mechanism that adaptively selects patch boundaries within each sequence based on local signal complexity. TimeSqueeze first applies a lightweight state-space encoder to extract full-resolution point-wise features, then performs content-aware segmentation by allocating short patches to information-dense regions and long patches to smooth or redundant segments. This variable-resolution compression preserves critical temporal structure while substantially reducing the token sequence presented to the Transformer backbone. Specifically for large-scale pretraining, TimeSqueeze attains up to 20x faster convergence and 8x higher data efficiency compared to equivalent point-token baselines. Experiments across long-horizon forecasting benchmarks show that TimeSqueeze consistently outperforms comparable architectures that use either point-wise tokenization or fixed-size patching.

Abstract

Time-Series Foundation Models

Explore similar work

TimeSqueeze: Dynamic Patching for Efficient Time Series Forecasting

Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting