Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting

Junyu Chen; Likai Chen; Weining Wang

stat.MLNov 5, 2026

Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting

Authors: Junyu Chen, Likai Chen, Weining Wang

Organizations: Department of Economics, Econometrics and Finance University of Groningen · Department of Statistics and Data Science Washington University in St. Louis · School of Economics University of Bristol

Abstract

Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies how predictive structure, head count, training sample size, and long-run dependence enter the selection criterion.

Figures & tables

Figure 1: Excess MSE across forecast horizons for fixed one-head attention, with $n=200$ independent training series. Curves show means over $50$ data repetitions. Shaded bands are pointwise $95\%$ bootstrap intervals. Dashed lines mark $p=\sqrt{L}$ . Patch candidates are displayed at equally spaced categorical positions, and panel scales differ.

Figure 2: Total test MSE at $H=1$ for fixed one-head attention as the number of independent training series grows. Means and pointwise $95\%$ bootstrap intervals use $50$ repetitions. Other plotting conventions match Figure 1 .

$L$	$n$	$p=1$	$p=\sqrt{L}$	$p=L$
64	5	$238.6\pm 2.2$	$184.4\pm 1.3$	$238.9\pm 2.1$
64	25	$182.8\pm 0.8$	$172.0\pm 0.6$	$182.8\pm 0.8$
64	200	$171.6\pm 0.6$	$169.8\pm 0.6$	$172.0\pm 0.6$
144	5	$320.7\pm 3.0$	$194.1\pm 1.7$	$328.4\pm 3.7$
144	25	$200.5\pm 1.0$	$173.5\pm 0.7$	$201.5\pm 1.0$
144	200	$174.7\pm 0.6$	$170.2\pm 0.6$	$175.4\pm 0.6$

Table 1: One-step test MSE in units of

10^{-3}

for the balanced patch and both extremes. Entries are means

\pm

standard errors over

50

data repetitions, and

n

counts independent training series.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Force	Quantity represented in the theory	Typical pressure on $p$
Predictive approximation	$A_{H,L,h}(p)$ or its multi-horizon aggregate	Depends on the coefficient reshape under a rank constraint. Patching alone is lossless. No monotone direction is proved.
Statistical complexity	$d_{p}/n_{\mathrm{eff}}(p)$ , with $d_{p}=h(L/p+p-h)$ when $h\leq\min(N,p)$	Toward a balanced factorization near $\sqrt{L}$ only when approximation and the dependence proxy vary little.
Serial dependence	Patch-dependent score concentration and restricted curvature	Smaller $n_{\mathrm{eff}}$ magnifies complexity control. The direction is not universally larger because $n_{\mathrm{eff}}(p)$ can change with tokenization.
Token mixing	$(L/p)^{2}$ for dense all-token mixing, and $L/p$ for fixed-width linear attention	Larger patches reduce mixing. The $L^{2/3}$ scale belongs only to the dense quadratic proxy.
Multiple horizons	Weighted-average or worst-case approximation over $H\in\mathcal{H}$	One shared patch only if horizon-specific curves share a near-minimizer. Otherwise multiresolution or horizon-specific tokens.

Appendix

Table 2: Approximation, estimation, and computational considerations in patch selection.

Panel of Figure 3 — Figure 3: Near-aligned observation design, fixed one-head attention, $50$ data repetitions. Top: excess MSE across forecast horizons with $n=200$ training series. Bottom: total test MSE at $H=1$ as the number of training series grows. Conventions match Figures 1 and 2 .

$L$	$n$	$p=1$	$p=\sqrt{L}$	$p=L$
64	5	$195.6\pm 1.4$	$155.1\pm 0.8$	$196.4\pm 1.5$
64	25	$155.3\pm 0.5$	$145.9\pm 0.4$	$155.5\pm 0.6$
64	200	$146.0\pm 0.4$	$144.3\pm 0.4$	$146.1\pm 0.5$
144	5	$265.0\pm 3.0$	$165.6\pm 1.2$	$270.1\pm 3.3$
144	25	$169.5\pm 0.7$	$147.4\pm 0.5$	$170.9\pm 0.7$
144	200	$148.3\pm 0.5$	$144.6\pm 0.4$	$148.8\pm 0.5$

Appendix

Table 3: Near-aligned observation design: one-step test MSE in units of

10^{-3}

for the balanced patch and both extremes (means

\pm

standard errors over

50

data repetitions). The one-step Bayes MSE is

144

in these units. Compare Table 1 .

Design	$L$	$H$	Signal	Approx. bound	Max over $p$	Trained, $n=5$	Trained, $n=200$
Dense	64	1	0.830	$2.3\cdot 10^{-5}$	$4.6\cdot 10^{-5}$	$1.50\cdot 10^{-2}$	$9.2\cdot 10^{-4}$
Dense	64	12	0.206	$2.6\cdot 10^{-4}$	$4.7\cdot 10^{-4}$	$5.80\cdot 10^{-2}$	$3.8\cdot 10^{-3}$
Dense	144	1	0.830	$9.3\cdot 10^{-6}$	$3.5\cdot 10^{-5}$	$2.52\cdot 10^{-2}$	$1.3\cdot 10^{-3}$
Dense	144	12	0.206	$1.1\cdot 10^{-4}$	$3.8\cdot 10^{-4}$	$1.08\cdot 10^{-1}$	$5.9\cdot 10^{-3}$
Near-aligned	64	1	0.856	$8.2\cdot 10^{-10}$	$1.6\cdot 10^{-9}$	$1.12\cdot 10^{-2}$	$6.9\cdot 10^{-4}$
Near-aligned	64	12	0.277	$1.0\cdot 10^{-8}$	$2.0\cdot 10^{-8}$	$5.40\cdot 10^{-2}$	$3.3\cdot 10^{-3}$

Appendix

Table 4: One-head approximation error versus trained excess risk. “Approx. bound” is an upper bound on

\mathcal{E}_{1}^{\Gamma}(B^{\circ}_{H,L,p})

p=\sqrt{L}

, and “max over

p

” its largest value over the patch grid. “Trained” is the mean population excess risk of the fitted model at

p=\sqrt{L}

(

50

repetitions).

Figure 5: Population ratio $\sigma_{2}/\sigma_{1}$ of the truncated infinite-past coefficient matrix $B_{H,p}$ against the horizon, $L=144$ , logarithmic axes. Left: near-aligned design. Right: dense design. Dashed lines: the analytic long-horizon limits $\kappa_{\infty,1}(p)$ of Theorem 4 .

Figure 6: Nearly rank-one target (dense design): population excess risk against the number of heads, with the patch selected by validation in each repetition. Panels show horizons, colors show window lengths, and shading shows pointwise

95\%

bootstrap intervals over

20

repetitions.

Weights	Heads	Lower mean	Favors more heads	Favors fewer heads
$(0.30,0.30,0.30)$	1 to 2	12/12	12/12	0/12
	2 to 3	12/12	12/12	0/12
	3 to 4	0/12	0/12	10/12
$(0.45,0.30,0.15)$	1 to 2	12/12	12/12	0/12
	2 to 3	9/12	9/12	1/12
	3 to 4	0/12	0/12	10/12

Appendix

Table 5: Rank-three targets: effect of adding one head at

p=\sqrt{L}

, over

12

settings (

4

windows,

H\in\{1,2,4\}

) and

20

repetitions. “Lower mean” counts settings where the larger model has lower mean population excess risk. The interval columns count paired

95\%

intervals entirely favoring the larger or the smaller model.

Figure 7: Rank-three target with equal weights,

H=1

p=\sqrt{L}

: population excess risk of the trained model (blue) and the rank lower bound

\lambda_{\min}(\Gamma_{L,p})\,0.09\,(3-m)_{+}

(orange) against the number of heads

m

, for four window lengths.

$H$	Baseline	Changed $A$	Changed $C$	Changed $A$ and $C$
1	1.04 (0.94)	0.96 (0.83)	1.06 (0.94)	1.33 (0.64)
2	1.08 (0.93)	1.00 (0.99)	1.15 (0.94)	1.06 (0.87)
4	1.13 (0.90)	1.14 (0.94)	0.98 (0.96)	1.36 (0.75)
8	1.43 (0.89)	1.41 (0.76)	1.32 (0.85)	0.13 ( $-$ 0.74)
12	1.33 (0.84)	1.66 (0.94)	1.55 (0.78)	0.13 ( $-$ 0.73)
16	1.58 (0.85)	1.60 (0.92)	1.41 (0.34)	0.14 ( $-$ 0.66)

Appendix

Table 6: Fitted constants

\widehat{C}

p=C\sqrt{L}

over nine window lengths, with centered

R^{2}

in parentheses (one head,

100

training series,

20

repetitions). Negative

R^{2}

means that the fit is worse than a constant.

Figure 8: Baseline (dense) design: the patch length $\widehat{p}^{\star}$ with the lowest mean population excess risk (blue) against $\sqrt{L}$ for nine window lengths, with the through-origin fit $p=\widehat{C}\sqrt{L}$ and its $95\%$ bootstrap band (orange). Panels: horizons, with centered $R^{2}$ .

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 905 papers

Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting
Jun 2, 2026Federico Zucchi, Yi Xie, Chao Zhang +2Long-Term Time Series Forecasting Time Series Forecasting
ICube, University of Strasbourg, Illkirch-Graffenstaden, France · Cephalgo, Strasbourg, France · Technical University of Munich +2
Investigating the effects of the patch size in Time Series Foundation Models
Nov 2, 2026Jad Yehya, Andrei-Tiberiu Pantea, Aymeric Dieuleveut +2Time-Series Foundation Models Time Series Forecasting
Université Paris-Saclay, Inria, CEA Palaiseau, France · CMAP, École polytechnique, IP Paris Électricité de France R&D Palaiseau, France · CMAP, École polytechnique, IP Paris Palaiseau, France +1
From HL to H+L-1 Parameters: A Hankel-Toeplitz Forecaster for Long-Term Time Series Forecasting
Sep 27, 2026Chaoqi Zhang, Yu Wang, Haixu TangLong-Term Time Series Forecasting Time Series Forecasting
Indiana University Bloomington, IN, USA

Jun 2, 2026J/K move · Enter open · S save

Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting

Federico Zucchi, Yi Xie, Chao Zhang, Thomas Lampert+1

ICube, University of Strasbourg, Illkirch-Graffenstaden, France · Cephalgo, Strasbourg, France · Technical University of Munich+2

Long-Term Time Series Forecasting Time Series Forecasting Time Series Transformers

Open paper AskPDF arXiv

Adaptive patching has been proposed for time-series forecasting: allocate finer patches where the sequence looks locally informative. We compare three adaptive patching methods with tuned uniform counterparts that use one fixed patch size throughout the input, while preserving each method's backbone family and shared training protocol. Across the evaluated long-horizon benchmarks, we observe mostly small accuracy differences and no consistent advantage for adaptive patching; uniform variants generally train faster and avoid adaptive routing at inference. Motivated by these observations, we develop a theoretical framework for budgeted patch allocation through a common resolution-density representation of uniform, nonuniform, and overlapping layouts, independently of the patching rule. Within a separable shared-loss model, we derive an exact criterion for when nonuniform allocation improves on uniform allocation at the same budget. We characterize modeled gains locally through a quadratic surrogate and bound them globally under strong convexity and a bounded loss derivative. The analysis distinguishes where resolution is placed from how much it is redistributed: even an allocation that correctly ranks regions by their weights can redistribute too much resolution and increase modeled loss. Without coupling across time, positive regional weights preserve the minimizers of the shared loss. When the shared loss has a common minimizing rate, the remaining absolute headroom for improvement tends to zero as the uniform rate approaches it. These results characterize the roles of regional weighting, placement, and redistribution strength under the stated model. The empirical findings support evaluating adaptive patching against a tuned uniform baseline and weighing any accuracy gain against the additional training and inference cost of adaptive routing.

Abstract

Time Series Forecasting

Explore similar work

Adaptive Patching Is Harder Than It Looks For Time-Series Forecasting

Investigating the effects of the patch size in Time Series Foundation Models

From HL to H+L-1 Parameters: A Hankel-Toeplitz Forecaster for Long-Term Time Series Forecasting