Learning What to Share: Expert Specialization for Time-Series Forecasting

Bashar Khoury; Shadi Omari; Assaf Schuster

cs.LGOct 24, 2026

Learning What to Share: Expert Specialization for Time-Series Forecasting

Authors: Bashar Khoury, Shadi Omari, Assaf Schuster

Organizations: Department of Computer Science, Technion – Israel Institute of Technology

Abstract

A central challenge in multivariate forecasting is deciding what to share across channels and where to specialize. Mixture-of-experts models offer flexible sharing, yet it remains unclear how much of their forecasting gains comes from added capacity or expert allocation. Through controlled comparisons across eleven datasets and three backbones, we show that informative channel assignments improve forecasting beyond matched shared capacity and random allocation. On frozen transformer backbones, persistent channel-specific mixtures outperform input-dependent routing on average. To examine when and how much specialization helps, we introduce GalaxyTS, a controlled synthetic dataset with exact shared and channel-specific forecasting references. It reveals how persistent channel mixtures exploit predictive differences concealed by short input windows and measures gains over matched shared corrections. These insights motivate Persistent Closed-Form Routing (PCR), a plug-and-play method that alternates exact conditional estimation of channel-specific mixture weights with gradient-based training of low-rank residual experts around a frozen pretrained forecaster. Averaged over four forecasting horizons, PCR-K4 achieves dataset-balanced mean squared error reductions of 3.86-14.25% relative to the three frozen backbones. In matched comparisons, PCR outperforms competing specialization methods on average while requiring substantially fewer trainable parameters.

Figures & tables

Figure 1: Persistent Closed-Form Routing (PCR). Channel-specific mixtures combine low-rank experts to correct a frozen forecast. Training alternates constrained ridge allocation with expert updates, preserving the backbone throughout.

Variant	DLinear	PatchTST	iTransformer
Global correction	$0.96\pm 0.30$	$5.79\pm 0.05$	$0.60\pm 0.05$
Random routing	$0.85\pm 0.11$	$6.27\pm 0.15$	$1.18\pm 0.11$
Ridge-based	$2.32\pm 0.18$	$7.95\pm 0.06$	$2.24\pm 0.04$
Persistent ID	$3.57\pm 0.20$	$\mathbf{12.41}\pm 0.29$	$\mathbf{7.62}\pm 0.10$
Input-dependent	$\underline{10.46}\pm 0.45$	$6.46\pm 0.10$	$0.49\pm 0.03$
Identity + input	$\mathbf{11.49}\pm 0.22$	$\underline{10.36}\pm 0.13$	$\underline{7.20}\pm 0.19$

Table 1: Meaningful allocation beyond shared capacity. MSE reduction over the pretrained backbone (%), averaged over 22 settings. Global uses rank 64; routed variants use four rank-16 experts, two active. Bold/underline indicate best/second-best means.

Figure 2: Channel layout of GalaxyTS. Dots sample the layout, colored by family; direction sets the echo mix. Larger

s

unwinds the arms around a shared core. Boxes mark the channels of Figure 3 .

Figure 3: Predictive grouping and echo dependence. (a) Generating families and Ridge groups at

s=0

; (b) boxed channels at

s=1

with echo links; (c) their exact autocorrelation near the echo delays (same scale).

Figure 4: Reference-scaled gains over Global.

H=96

. Curves: approximate mean

G

values; shaded bands: range across backbones. See Equation 16 .

$H$	$s$	Random	Ridge	Input- dependent	Persistent ID	PCR- $K2$
		Allocation study				PCR evaluation
$96$	$0$	$-1.0$	$55.1$	$-1.1$	$70.8$	$75.7$
$96$	$1$	$-0.1$	$47.9$	$-0.3$	$79.2$	$82.4$
$720$	$0$	$-1.4$	$50.5$	$-0.9$	$58.0$	$61.8$
$720$	$1$	$-0.4$	$42.5$	$-0.3$	$64.0$	$65.9$

Table 2: Reference-scaled gains over Global (

K=2

). Mean

G

(%) over backbones, realizations, and training seeds.

Backbone	PCR-K4 (ours)	PCR-K2 (ours)	CCM original	CCM matched	MoLE original	MoLE matched	VE original	VE matched
DLinear	$\underline{3.86}\,/\,\underline{1.87}$	$3.63\,/\,1.76$	$2.14\,/\,1.59$	$1.00\,/\,0.41$	$\mathbf{10.37}\,/\,\mathbf{9.17}$	$1.68\,/\,0.72$	$-2.49\,/\,-1.11$	$-1.65\,/\,-1.46$
PatchTST	$\mathbf{14.25}\,/\,\mathbf{11.17}$	$\underline{13.31}\,/\,\underline{10.44}$	$8.05\,/\,7.78$	$6.27\,/\,5.67$	—	—	$8.97\,/\,6.75$	$11.29\,/\,7.14$
iTransformer	$\mathbf{10.22}\,/\,\mathbf{5.87}$	$\underline{8.56}\,/\,\underline{4.75}$	$4.70\,/\,3.36$	$-1.70\,/\,-1.91$	—	—	$7.32\,/\,4.10$	$3.57\,/\,0.93$

Table 3: Forecasting gains across backbones. Dataset-balanced MSE/MAE reductions (%) over the frozen base. Bold/underline mark best/second best values. Detailed MSE and MAE results across datasets and backbones are reported in Appendices E.3.4 and E.4 .

Figure 5: Accuracy-latency trade-offs. PCR-K4 and original-capacity CCM, MoLE, and VE. Relative MSE to frozen backbone versus median single-window inference latency; lower-left is better. Colours identify backbones, and circle area denotes gradient-trained parameters. Matched variants appear in Figure 9 .

Appendix figures & tables57 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Channels	Rows	Sampling interval
PEMS03	358	26,208	5 minutes
PEMS04	307	16,992	5 minutes
PEMS07	883	28,224	5 minutes
PEMS08	170	17,856	5 minutes
ETTh1 / ETTh2	7	17,420	1 hour
ETTm1 / ETTm2	7	69,680	15 minutes

Appendix

Table 4: Dataset dimensions before splitting. Each dataset in an ETT pair has the listed channel count, length, and sampling interval.

Asset	Creator and source	License or reuse status
ETTh1/2, ETTm1/2	Zhou et al.; ETDataset ( Zhou et al.,, 2021 ) .	CC BY-ND 4.0 .
Electricity	Trindade; UCI archive ( Trindade,, 2015 ) ; hourly benchmark distributed through Autoformer/LTSF ( Wu et al.,, 2021 ; Zeng et al.,, 2023 ) .	Original archive: CC BY 4.0 . Benchmark preprocessing is described separately from this source license.
Weather	MPI-BGC Jena station ( Max Planck Institute for Biogeochemistry, nd, ) ; Autoformer benchmark ( Wu et al.,, 2021 ) .	CC BY 4.0 , as stated by the station archive.
Traffic	Caltrans PeMS ( Caltrans, nd, ) ; hourly benchmark distributed through Autoformer/LTSF ( Wu et al.,, 2021 ; Zeng et al.,, 2023 ) .	Caltrans Conditions of Use ; no separate benchmark-specific data license identified.
PEMS03/04/07/08	Caltrans PeMS ( Caltrans, nd, ) ; five-minute benchmark releases from ASTGNN ( Guo et al.,, 2022 ) .	Caltrans source terms; no explicit separate benchmark-data license identified in the inspected ASTGNN release.
TSLib	THUML, Tsinghua University ( Wang et al.,, 2026 ) .	MIT .

Appendix

Table 5: *

Backbone	Datasets / horizon	$d$	$F$	$N$	$A$	$d_{\rm ff}$	Dropout
DLinear	All	-	192	-	-	-	-
PatchTST	ETTh1, ETTh2	16	192	3	4	128	0.3
	All others	128	1536	3	16	256	0.2
iTransformer	ETTh1 / 96	256	256	2	8	256	0.1
	ETTh1 / 720	512	512	2	8	512	0.1
	Other ETT	128	128	2	8	128	0.1

Appendix

Table 5: Backbone architecture recipes for the frozen benchmark.

d

is model width,

F

correction-feature width,

N

encoder layers,

A

attention heads, and

d_{\rm ff}

feed-forward width. All iTransformer recipes use four time-mark tokens. PatchTST uses twelve length-16 patches at stride eight. DLinear uses decomposition width 25.

Dataset	$C$	DLinear	PatchTST	iTransformer
ETTh1	7	0.01	0.001	0.0001
ETTh2	7	0.01	0.001	0.0001
ETTm1	7	0.01/0.001	0.0001/0.001	0.0001
ETTm2	7	0.01	0.0001	0.0001
Electricity	321	0.01	0.001	0.0005
PEMS03	358	0.001/0.01	0.001	0.0005

Appendix

Table 6: Base learning rates. A single value applies to both horizons;

a/b

gives

H=96/720

. DLinear and PatchTST select rates on seed-zero validation; iTransformer rates are fixed by dataset.

Study	Evaluated settings and comparisons
Frozen allocation	Three backbones, eleven datasets, $H\in\{96,720\}$ : 66 settings. Input-dependent and Ridge-based routing cover this grid.
Backbone adaptation	The same 66 settings: three jointly adapted correction variants, backbone-only fine-tuning, and Input-dependent adaptation.
Lookback	iTransformer on Electricity, PEMS04, and Traffic, $L,H\in\{96,720\}$ : twelve settings, including Input-dependent routing.
GalaxyTS	Three backbones, four separation levels, four horizons: 48 settings. Raw-window checks use three realizations; backbone-feature allocation and PCR use three realizations and three training seeds.
PCR benchmark	Three backbones, eleven datasets, four horizons: 132 settings, four PCR/learned-allocation arms and the frozen base. MoLE is evaluated on DLinear; CCM on DLinear, PatchTST, and our iTransformer adaptation.
Resource measurements	Allocation inference: 64 of 66 settings; frozen/joint fitting: 24 settings; iTransformer lookback inference: twelve settings. PCR and external deployment: 660 and 396 configurations, respectively, including 44 MoLE expert-budget controls.

Appendix

Table 7: Experimental coverage. A setting is a backbone-dataset-horizon combination, extended by lookback or separation when these vary. Real-data accuracy uses three reporting seeds.

Alternative	Allocation	Expert weights	Router weights
Base	Frozen forecaster, without correction	$0$	$0$
Global correction	One shared rank-64 map	$64(F+H)$	$0$
Random routing	Balanced, equally weighted expert pairs	$64(F+H)$	$0$
Ridge-based routing	Predictive cluster and nearest alternative, equally weighted	$64(F+H)$	$0$
Persistent ID routing	Channel-specific top-2 logits	$64(F+H)$	$4C$
Input-dependent routing	Affine feature projection, top-2	$64(F+H)$	$4(F+1)$

Appendix

Table 8: Allocation rules and trainable parameters.

C

F

, and

H

are channel count, feature width, and horizon. Dense ID uses

E\in\{2,4\}

and

r=16

; per-channel correction is a larger-capacity reference.

Dataset	$H=96$	$H=720$
ETTh1	0.1	0.1
ETTh2	0.001	10
ETTm1	0.01	0.01
ETTm2	0.001	1
Electricity	0.001	0.01
PEMS03	0.001	0.001

Appendix

Table 9: Validation-selected relative ridge penalties. Each value multiplies the number of inner fitting windows; the full-training refit divides that penalty by

0.8

. These settings are shared across backbones.

Alternative	DLinear	PatchTST	iTransformer
Global correction	$0.96$	$5.79$	$0.60$
Random routing	$0.85$	$6.27$	$1.18$
Ridge-based routing	$2.32$	$7.95$	$2.24$
Persistent ID routing	$3.57$	$12.41$	$7.62$
Input-dependent routing	$\underline{10.46}$	$6.46$	$0.49$
Identity + input routing	$\mathbf{11.49}$	$10.36$	$7.20$

Appendix

Table 10: Frozen-backbone MSE reductions (%). Results average all eleven datasets. Bold/underline mark best/second means, including the larger per-channel reference. All backbone comparisons use matched checkpoints within each setting and seed.

Dataset	Persistent ID routing vs. Global correction	Persistent ID routing vs. Random routing	Identity + input routing vs. Persistent ID routing
DLinear
ETTh1	$2.89\pm 1.00$	$0.47\pm 0.67$	$-2.38\pm 0.70$
ETTh2	$-6.45\pm 6.02$	$-1.50\pm 0.72$	$-0.39\pm 2.57$
ETTm1	$3.34\pm 0.11$	$0.32\pm 0.31$	$-2.05\pm 0.13$
ETTm2	$-5.58\pm 5.27$	$-0.56\pm 0.71$	$2.84\pm 1.45$
Weather	$10.89\pm 0.63$	$9.41\pm 0.33$	$-1.80\pm 1.62$

Appendix

Table 11: Dataset-level relative MSE reductions (%), mean

\pm

sample SD over three seed-level horizon averages. Positive favors the first method. All backbone comparisons use matched checkpoints.

Method	Reference	DLinear	PatchTST	iTransformer
Persistent ID routing	Global correction	$2.48\pm 0.57$	$7.41\pm 0.30$	$7.07\pm 0.07$
Persistent ID routing	Random routing	$2.74\pm 0.13$ [9/11]	$6.96\pm 0.34$ [10/11]	$6.53\pm 0.15$ [10/11]
Ridge-based routing	Random routing	$1.47\pm 0.28$	$1.91\pm 0.12$	$1.08\pm 0.15$
Identity + input routing	Persistent ID routing	$8.29\pm 0.30$	$-2.73\pm 0.51$	$-0.39\pm 0.20$
Input-dependent routing	Persistent ID routing	$7.10\pm 0.63$	$-8.12\pm 0.42$	$-8.53\pm 0.08$
Identity + input routing	Input-dependent routing	$1.20\pm 0.55$	$4.46\pm 0.11$	$6.77\pm 0.20$

Appendix

Table 12: Pairwise MSE reductions (%), mean

\pm

sample SD over three dataset-balanced seed averages. Brackets count favorable dataset averages. Every comparison covers 22 settings per backbone and uses matched checkpoints.

Dataset	$H$	MSE	MAE	Gain (%)	$R_{\mathrm{Ridge}\mid\mathrm{Random}}$
DLinear
ETTh1	96	$0.3825\pm 0.0029$	$0.3956\pm 0.0039$	$0.91\pm 0.08$	$-0.31\pm 0.56$
ETTh1	720	$0.5061\pm 0.0157$	$0.4998\pm 0.0138$	$5.56\pm 2.61$	$1.57\pm 0.50$
ETTh2	96	$0.3184\pm 0.0393$	$0.3701\pm 0.0319$	$0.75\pm 0.78$	$0.16\pm 1.61$
ETTh2	720	$0.7485\pm 0.0564$	$0.6225\pm 0.0237$	$0.00\pm 0.00$	$2.04\pm 2.57$
ETTm1	96	$0.3311\pm 0.0015$	$0.3638\pm 0.0018$	$3.68\pm 0.67$	$-0.67\pm 0.71$

Appendix

Table 13: Ridge-based routing,

L=96

: mean

\pm

sample SD over three seeds. MSE/MAE use train-standardized coordinates; Gain is own-base MSE reduction (%).

R_{\mathrm{Ridge}\mid\mathrm{Random}}

uses matched checkpoints in all 66 settings.

Dataset	$H$	Base MSE	Input-dependent MSE	Input-dependent MAE	Gain (%)	$R_{I\mid ID}$	$R_{ID+I\mid I}$
DLinear
ETTh1	96	$\underline{0.3860}\pm 0.0031$	$\mathbf{0.3859}\pm 0.0017$	$0.3981\pm 0.0025$	$0.01\pm 0.40$	$-1.60\pm 0.25$	$-0.06\pm 0.07$
ETTh1	720	$\underline{0.5363}\pm 0.0294$	$\mathbf{0.5346}\pm 0.0049$	$0.5205\pm 0.0043$	$0.15\pm 4.67$	$-4.61\pm 1.00$	$1.44\pm 1.58$
ETTh2	96	$\underline{0.3208}\pm 0.0397$	$\mathbf{0.3101}\pm 0.0230$	$0.3650\pm 0.0216$	$2.99\pm 4.57$	$2.10\pm 5.41$	$-0.91\pm 1.02$
ETTh2	720	$\mathbf{0.7485}\pm 0.0564$	$\underline{0.7599}\pm 0.0827$	$0.6213\pm 0.0271$	$-1.48\pm 6.78$	$3.63\pm 9.01$	$-6.61\pm 11.71$
ETTm1	96	$\underline{0.3438}\pm 0.0014$	$\mathbf{0.3360}\pm 0.0021$	$0.3661\pm 0.0017$	$2.27\pm 1.02$	$-2.58\pm 0.65$	$-0.54\pm 1.06$

Appendix

Table 14: Input-dependent routing on DLinear and PatchTST,

L=96

: mean

\pm

sample SD over three seeds. Gain is same-backbone MSE reduction (%);

R_{I\mid ID}

and

R_{ID+I\mid I}

compare the indicated routers on matched checkpoints before horizon averaging. Bold/underline mark best/second-best Base versus Input-dependent MSE; ties share rank. iTransformer appears in Tables 16 and 17 .

Router	MSE All	MAE All	MSE $\alpha=1$
Input-dependent	$0.49\pm 0.03$	$0.28\pm 0.02$	$0.29\pm 0.06$
Persistent ID	$7.62\pm 0.10$	$4.14\pm 0.05$	$7.26\pm 0.09$
Identity + input	$7.20\pm 0.19$	$3.79\pm 0.12$	$7.02\pm 0.23$

Appendix

Table 15: iTransformer routing: backbone-relative reductions (%), mean

\pm

sample SD over three seed-level averages. All results average the 22 settings across eleven datasets. The uncalibrated readout uses each router’s selected learning rate, with the checkpoint selected at

\alpha=1

Dataset	$H$	Input	ID	ID+input
ETTh1	96	$0.3928\pm 0.0016$ $0.4087\pm 0.0013$	$0.3886\pm 0.0019$ $0.4062\pm 0.0015$	$0.3939\pm 0.0002$ $0.4096\pm 0.0008$
ETTh1	720	$0.5020\pm 0.0069$ $0.4887\pm 0.0038$	$0.4990\pm 0.0061$ $0.4861\pm 0.0039$	$0.5018\pm 0.0076$ $0.4887\pm 0.0042$
ETTh2	96	$0.3006\pm 0.0023$ $0.3504\pm 0.0008$	$0.2958\pm 0.0040$ $0.3465\pm 0.0021$	$0.3007\pm 0.0021$ $0.3506\pm 0.0007$
ETTh2	720	$0.4296\pm 0.0032$ $0.4482\pm 0.0023$	$0.4272\pm 0.0025$ $0.4455\pm 0.0017$	$0.4286\pm 0.0026$ $0.4473\pm 0.0021$
ETTm1	96	$0.3440\pm 0.0020$ $0.3775\pm 0.0014$	$0.3353\pm 0.0017$ $0.3704\pm 0.0011$	$0.3380\pm 0.0033$ $0.3733\pm 0.0022$
ETTm1	720	$0.4929\pm 0.0020$ $0.4582\pm 0.0007$	$0.4866\pm 0.0019$ $0.4538\pm 0.0006$	$0.4923\pm 0.0014$ $0.4559\pm 0.0014$

Appendix

Table 16: iTransformer routing errors. Each entry gives MSE (top) and MAE (bottom), mean

\pm

sample SD over three seeds in train-standardized coordinates. Input, ID and ID+input denote the three routing rules. All 22 settings are retained.

Dataset	$H$	$R_{ID\mid I}$	$R_{ID+I\mid I}$	$R_{ID\mid ID+I}$
ETTh1	96	$1.07\pm 0.87$	$-0.28\pm 0.35$	$1.35\pm 0.54$
ETTh1	720	$0.61\pm 1.35$	$0.04\pm 0.45$	$0.56\pm 1.76$
ETTh2	96	$1.57\pm 0.60$	$-0.05\pm 0.17$	$1.62\pm 0.71$
ETTh2	720	$0.56\pm 0.18$	$0.23\pm 0.65$	$0.33\pm 0.48$
ETTm1	96	$2.52\pm 0.44$	$1.76\pm 0.62$	$0.77\pm 0.47$
ETTm1	720	$1.27\pm 0.17$	$0.12\pm 0.16$	$1.15\pm 0.26$

Appendix

Table 17: iTransformer relative MSE reductions (%), mean

\pm

sample SD over three seeds. Positive favors the first router; the reference router supplies the denominator. All averages the 22 settings within each seed.

Correction	Frozen	Joint	Residual off	Joint vs. frozen (%)	Joint vs. fine-tuned (%)
DLinear
Global correction	$0.4335(26)$ $0.96(30)$	$0.4368(32)$ $0.23(11)$	$0.4375(25)$ $0.10(9)$	$-0.85(46)$	$0.23(11)$
Persistent ID routing	$0.4245(38)$ $3.57(20)$	$0.4193(68)$ $4.31(98)$	$0.4421(65)$ $-1.22(40)$	$0.72(92)$	$4.31(98)$
Identity + input routing	$\mathbf{0.3860}(35)$ $\mathbf{11.49}(22)$	$\mathbf{0.3822}(32)$ $\mathbf{11.94}(17)$	$0.4839(30)$ $-10.58(67)$	$0.66(39)$	$11.94(18)$
Input-dependent routing	$\underline{0.3882}(10)$ $\underline{10.46}(45)$	$\underline{0.3884}(48)$ $\underline{10.33}(48)$	$0.4940(34)$ $-12.37(53)$	$0.09(102)$	$10.33(49)$
PatchTST

Appendix

Table 18: Adaptation summary over three seeds. Frozen, joint, and residual-off columns give MSE above and improvement over the pretrained backbone (%) below; parentheses denote SD in final decimal places. The final columns compare joint fitting with the corresponding frozen correction and independently fine-tuned backbone. Bold/underline mark the best/second frozen or joint result within each backbone.

Dataset	$H$	Backbone fine-tuning without correction	Global correction	Persistent ID routing	Identity + input routing
DLinear
ETTh1	96	$-0.09\pm 0.15$	$\underline{0.00}\pm 0.00$	$\mathbf{1.54}\pm 0.50$	$-0.22\pm 0.31$
ETTh1	720	$\underline{0.00}\pm 0.00$	$-0.06\pm 0.10$	$\mathbf{3.88}\pm 4.03$	$-0.87\pm 4.74$
ETTh2	96	$0.18\pm 0.32$	$\underline{2.18}\pm 3.25$	$\mathbf{4.34}\pm 7.51$	$0.25\pm 0.23$
ETTh2	720	$0.00\pm 0.00$	$\underline{3.18}\pm 5.52$	$\mathbf{8.47}\pm 14.67$	$0.00\pm 0.00$
ETTm1	96	$0.00\pm 0.00$	$0.00\pm 0.00$	$\mathbf{5.12}\pm 0.74$	$\underline{3.28}\pm 0.39$

Appendix

Table 19: Joint fitting: backbone-relative MSE reduction (%), mean

\pm

SD over three seeds. Bold/underline mark the best/second result per row. Input-dependent results appear in Table 20 .

		DLinear		PatchTST		iTransformer
Dataset	$H$	Joint	Residual off	Joint	Residual off	Joint	Residual off
ETTh1	96	$0.3868(29)$	$0.3849(22)$	$0.3923(32)$	$0.6531(35)$	$0.3922(7)$	$0.3882(8)$
ETTh1	720	$0.5586(358)$	$0.5403(295)$	$0.4768(87)$	$0.6538(101)$	$0.5004(43)$	$0.4984(44)$
ETTh2	96	$0.3193(407)$	$0.3224(389)$	$0.2929(32)$	$0.3084(34)$	$0.3001(16)$	$0.2976(14)$
ETTh2	720	$0.7678(784)$	$0.8111(1123)$	$0.4294(69)$	$0.4308(43)$	$0.4280(13)$	$0.4282(14)$
ETTm1	96	$0.3438(14)$	$0.3438(14)$	$0.3267(13)$	$0.6019(59)$	$0.3359(32)$	$0.3377(40)$

Appendix

Table 20: Input-dependent adaptation: joint and residual-off test MSE, mean(SD in final decimal places) over three seeds. Residual-off disables the correction at the selected joint checkpoint without refitting.

Backbone	Dataset	$H$	Fit frozen / joint	Time ratio	Peak frozen / joint	Memory ratio	$\Delta$ MSE (%)
DLinear	ETTm1	96	$107\pm 30$ / $84\pm 31$	$0.86\pm 0.44$	$0.020$ / $0.021$	$1.02$	$0.41\pm 0.20$
DLinear	ETTm1	720	$116\pm 33$ / $53\pm 7$	$0.47\pm 0.08$	$0.024$ / $0.030$	$1.24$	$-2.58\pm 0.45$
DLinear	Electricity	96	$69\pm 38$ / $59\pm 3$	$1.08\pm 0.68$	$0.081$ / $0.111$	$1.37$	$0.30\pm 0.15$
DLinear	Electricity	720	$93\pm 5$ / $44\pm 10$	$0.47\pm 0.12$	$0.199$ / $0.324$	$1.62$	$-0.04\pm 0.91$
DLinear	PEMS04	96	$50\pm 14$ / $35\pm 6$	$0.74\pm 0.33$	$0.067$ / $0.094$	$1.40$	$0.12\pm 0.28$
DLinear	PEMS04	720	$55\pm 5$ / $34\pm 4$	$0.63\pm 0.12$	$0.181$ / $0.299$	$1.65$	$-0.10\pm 0.20$

Appendix

Table 21: Frozen/joint Persistent ID fitting on 24 settings. Times (seconds), joint/frozen time ratios, and joint-over-frozen MSE reduction (%) are mean

\pm

SD over three seeds. Memory gives frozen/joint mean peaks in GiB (SD rounds to zero); ratios are joint/frozen. Arithmetic mean ratios here differ from the geometric aggregation in the text.

Dataset	$H$	$L$	Global correction	Random routing	Ridge-based routing	Persistent ID routing	Identity + input routing
PEMS04	96	96	$0.3014(61)$ $0.90(1)$	$0.2985(58)$ $1.85(16)$	$0.2887(56)$ $5.05(12)$	$\underline{0.2334}(42)$ $\underline{23.24}(24)$	$\mathbf{0.2308}(71)$ $\mathbf{24.12}(87)$
PEMS04	96	720	$0.1083(13)$ $0.78(22)$	$0.1082(13)$ $0.82(20)$	$0.1075(14)$ $1.51(17)$	$\mathbf{0.1038}(15)$ $\mathbf{4.82}(8)$	$\underline{0.1047}(15)$ $\underline{4.06}(22)$
PEMS04	720	96	$0.3367(60)$ $0.75(27)$	$0.3352(58)$ $1.18(29)$	$0.3232(57)$ $4.72(16)$	$\mathbf{0.2757}(63)$ $\mathbf{18.74}(54)$	$\underline{0.2785}(96)$ $\underline{17.93}(129)$
PEMS04	720	720	$0.1488(33)$ $0.39(30)$	$0.1491(33)$ $0.24(33)$	$0.1473(32)$ $1.43(29)$	$\mathbf{0.1416}(32)$ $\mathbf{5.26}(37)$	$\underline{0.1428}(31)$ $\underline{4.46}(45)$
Electricity	96	96	$0.1491(2)$ $0.50(8)$	$0.1489(2)$ $0.65(9)$	$0.1471(1)$ $1.89(2)$	$\underline{0.1450}(3)$ $\underline{3.29}(28)$	$\mathbf{0.1449}(6)$ $\mathbf{3.30}(36)$
Electricity	96	720	$\underline{0.1340}(17)$ $\underline{1.56}(18)$	$0.1341(17)$ $1.43(19)$	$\mathbf{0.1339}(17)$ $\mathbf{1.57}(20)$	$0.1341(17)$ $1.43(20)$	$\underline{0.1340}(18)$ $1.54(10)$

Appendix

Table 22: Frozen iTransformer at two lookbacks: MSE above and same-lookback backbone-relative reduction (%) below, mean(SD in final decimal places) over three seeds. All bank-based routers use

E=4,K=2

; Global correction matches factor count. Bold/underline mark the best/second result per row and metric.

Dataset	$H$	$L$	Pretrained backbone MSE	Input-dependent routing MSE	Improvement (%)
PEMS04	96	96	${0.3041}\pm 0.0062$	$\mathbf{0.3013}\pm 0.0060$	$0.92\pm 0.05$
PEMS04	96	720	${0.1091}\pm 0.0015$	$\mathbf{0.1084}\pm 0.0013$	$0.68\pm 0.18$
PEMS04	720	96	${0.3392}\pm 0.0065$	$\mathbf{0.3359}\pm 0.0062$	$0.99\pm 0.30$
PEMS04	720	720	${0.1494}\pm 0.0029$	$\mathbf{0.1489}\pm 0.0031$	$0.35\pm 0.12$
Electricity	96	96	${0.1499}\pm 0.0001$	$\mathbf{0.1492}\pm 0.0002$	$0.45\pm 0.07$
Electricity	96	720	${0.1361}\pm 0.0020$	$\mathbf{0.1341}\pm 0.0017$	$1.45\pm 0.20$

Appendix

Table 23: Input-dependent lookback results for iTransformer, mean

\pm

SD over three seeds. Improvements use each row’s pretrained backbone; the checkpoint is matched to the other routing methods in Table 22 at the same setting and seed. Bold marks the lower MSE in each row.

Figure 6: Autocorrelation of the illustrated channels. Exact profiles for the five channels highlighted in Figure 3 at

s=1

. The shaded region marks lags within the input window, where the channels share the same autocorrelation; channel-specific echo peaks occur at longer lags.

Gap closed by one exact map per group (%)
	$s=0$	$0.33$	$0.66$	$1$
Optimal shared MSE	0.9257	0.9000	0.8683	0.8297
Optimal per-channel MSE	0.8997	0.8561	0.8030	0.7127
Specialization gap (%)	2.81	4.87	7.52	14.10
Family labels	9.4	23.3	52.6	77.4
Random groups	0.5	0.4	0.4	0.5

Appendix

Table 24: Exact population references at

L=H=96

. The lower block gives the specialization gap closed by one exact map per group (%; Ridge averages realizations). Bold/underline mark the best/second distinct grouping values at each separation; ties share rank. Population errors in the upper block are unranked.

Evaluation	Correction input	Rank / Global	Seeds	Fitting rule
Backbone-feature allocation	Prediction-head features	16 / 64	0,1,2	Appendix B.3
Raw-window check	Channel history	32 / 128	0	Fixed-rate Adam

Appendix

Table 25: GalaxyTS allocation protocols. Rank is per expert; Global gives the shared correction’s factor width. Seeds are per realization.

$s$	$H$	Base	Global	Random	Family	Ridge	ID	Input	ID+input	Dense
DLinear
0	96	$0.9342(0)[34]$	$0.9288(0)[34]$	$0.9295(1)[34]$	$0.9278(0)[36]$	$0.9153(4)[35]$	$0.9113(8)[38]$	$0.9293(0)[34]$	$\underline{0.9101}(2)[36]$	$\mathbf{0.9068}(3)[39]$
0	192	$0.9537(0)[33]$	$0.9509(0)[33]$	$0.9513(0)[33]$	$0.9500(0)[36]$	$0.9406(1)[34]$	$\underline{0.9381}(3)[35]$	$0.9511(0)[33]$	$0.9382(9)[36]$	$\mathbf{0.9343}(3)[36]$
0	336	$0.9709(0)[29]$	$0.9693(0)[29]$	$0.9695(0)[29]$	$0.9687(0)[30]$	$0.9624(0)[29]$	$\underline{0.9608}(2)[30]$	$0.9694(0)[29]$	$0.9610(6)[35]$	$\mathbf{0.9585}(2)[29]$
0	720	$0.9863(0)[24]$	$0.9855(0)[24]$	$0.9857(0)[24]$	$0.9853(0)[24]$	$0.9822(0)[25]$	$0.9815(2)[26]$	$0.9856(0)[24]$	$\underline{0.9814}(1)[26]$	$\mathbf{0.9805}(1)[23]$
0.33	96	$0.9147(0)[27]$	$0.9091(0)[27]$	$0.9099(1)[27]$	$0.9012(1)[24]$	$0.8842(2)[29]$	$0.8779(3)[32]$	$0.9097(0)[27]$	$\underline{0.8770}(8)[32]$	$\mathbf{0.8700}(2)[33]$

Appendix

Table 26: Backbone-feature test MSE for all 48 settings, with uncertainty notation defined in Appendix D.3 . Random, Family, and Ridge use one active expert; ID, Input, ID+input, and Dense denote Persistent ID, Input-dependent, Identity + input, and Dense ID routing. Bold/underline mark the best/second distinct displayed means per row.

Figure 7: One-expert Ridge-based routing. Other curves match Figure 4 .

$s$	$H$	Random MSE	Ridge MSE	$G_{\rm Random}$	$G_{\rm Ridge}$	Ridge vs Random
DLinear
0	96	$\underline{0.9292}(0)[33]$	$\mathbf{0.9148}(0)[35]$	$\underline{-1.7}$	$\mathbf{54.1}$	1.55
0	720	$\underline{0.9856}(0)[24]$	$\mathbf{0.9819}(0)[23]$	$\underline{-1.5}$	$\mathbf{50.5}$	0.38
1	96	$\underline{0.8447}(2)[91]$	$\mathbf{0.7879}(2)[73]$	$\underline{-0.4}$	$\mathbf{47.4}$	6.73
1	720	$\underline{0.9056}(2)[86]$	$\mathbf{0.8691}(1)[78]$	$\underline{-0.2}$	$\mathbf{41.9}$	4.03
PatchTST

Appendix

Table 27: Matched two-expert Random and Ridge-based controls on all twelve endpoint settings. MSE uses

\mu(a)[b]

notation;

G

is the reference-scaled gain over the same study’s Global correction (%). The last column gives Ridge’s MSE reduction over Random (%). Bold/underline compare the two controls.

Figure 8: Raw-window allocation patterns. The 512 channels at

s=0

and

s=1

, colored by family, Random, Ridge-based, Persistent ID, and Input-dependent allocation (left to right). Random and Ridge show one-expert partitions, not the matched two-expert controls. Persistent ID shows the highest-weight expert; Input-dependent shows the most frequent highest-weight expert across test windows. These summaries omit mixture weights. Percentages give agreement with the dominant generating echo under the displayed color alignment.

	$H=96$				$H=192$
$s$	$0$	$0.33$	$0.66$	$1$	$0$	$0.33$	$0.66$	$1$
Optimal shared	0.9274(34)	0.9076(27)	0.8727(48)	0.8410(87)	0.9500(34)	0.9316(24)	0.8953(45)	0.8572(85)
Optimal per-family	0.9250(36)	0.8972(25)	0.8385(43)	0.7493(68)	0.9481(36)	0.9224(22)	0.8637(43)	0.7672(70)
Optimal per-channel	0.9015(37)	0.8631(32)	0.8070(37)	0.7220(60)	0.9283(35)	0.8924(30)	0.8344(38)	0.7402(64)
DLinear
Frozen backbone	0.9342(34)	0.9147(27)	0.8803(49)	0.8505(91)	0.9537(33)	0.9356(24)	0.8998(46)	0.8635(87)

Appendix

Table 28: Raw-window test MSE at

L=96

H=96,192

, with rank-32 corrections: mean (sample SD) over three realizations. SD is in units of

10^{-4}

, e.g.

0.9342(34)=0.9342\pm 0.0034

. Bold/underline mark the lowest/second distinct displayed means within each backbone, horizon, and separation, excluding exact references. Ties share rank. Backbone-feature results appear in Table 26 .

	$H=336$				$H=720$
$s$	$0$	$0.33$	$0.66$	$1$	$0$	$0.33$	$0.66$	$1$
Optimal shared	0.9687(29)	0.9540(22)	0.9177(46)	0.8705(82)	0.9851(24)	0.9808(24)	0.9531(50)	0.9029(85)
Optimal per-family	0.9674(30)	0.9472(21)	0.8916(44)	0.7851(67)	0.9845(25)	0.9773(24)	0.9374(46)	0.8363(69)
Optimal per-channel	0.9541(29)	0.9248(26)	0.8672(40)	0.7594(62)	0.9780(24)	0.9656(26)	0.9227(43)	0.8163(69)
DLinear
Frozen backbone	0.9709(29)	0.9565(23)	0.9207(47)	0.8754(83)	0.9863(24)	0.9823(23)	0.9551(51)	0.9069(86)

Appendix

Table 29: Raw-window test MSE at

L=96

H=336,720

. Protocol, uncertainty notation, and ranking follow Table 28 ; backbones are reported separately.

$s$	$H$	Exact shared	Exact channel	Base	Global	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
DLinear
0	96	$0.92738$	$0.90152$	$0.9342(0)[34]$	$0.9281(0)[34]$	$\mathbf{0.9047}(1)[39]$	$0.9082(3)[38]$	$\underline{0.9050}(2)[38]$	$0.9087(1)[37]$
0	192	$0.95004$	$0.92827$	$0.9537(0)[33]$	$0.9505(0)[33]$	$\mathbf{0.9335}(5)[31]$	$\underline{0.9359}(2)[36]$	$\mathbf{0.9335}(4)[34]$	$0.9366(1)[36]$
0	336	$0.96869$	$0.95405$	$0.9709(0)[29]$	$0.9691(0)[29]$	$\underline{0.9587}(6)[25]$	$0.9595(0)[30]$	$\mathbf{0.9581}(3)[29]$	$0.9599(1)[30]$
0	720	$0.98513$	$0.97799$	$0.9863(0)[24]$	$0.9854(0)[24]$	$\underline{0.9807}(4)[23]$	$0.9809(1)[25]$	$\mathbf{0.9802}(1)[24]$	$0.9811(1)[25]$
0.33	96	$0.90762$	$0.86314$	$0.9147(0)[27]$	$0.9084(0)[27]$	$\mathbf{0.8675}(2)[31]$	$0.8737(8)[28]$	$\underline{0.8676}(2)[32]$	$0.8741(2)[34]$

Appendix

Table 30: PCR and matched learned-allocation test MSE on all 48 GalaxyTS settings, alongside exact shared and channel-specific reference errors. The exact reference predictors use only the target channel’s input history and are evaluated on the same test windows; their errors are averaged across realizations. Fitted-model uncertainty uses

\mu(a)[b]

as defined in Appendix D.3 . Global is fitted under this study’s budget, separately from the allocation-study control. Bold/underline rank fitted-model means only; the reference columns are unranked.

$s$	$H$	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
DLinear
0	96	5.24 (19.7)	4.50 (18.1)	5.16 (19.5)	4.37 (17.2)
0	720	1.00 (4.6)	0.96 (4.3)	1.09 (4.6)	0.92 (4.1)
1	96	30.63 (60.0)	26.74 (54.0)	30.91 (60.3)	26.01 (55.1)
1	720	14.66 (28.7)	13.84 (29.5)	15.26 (30.4)	13.46 (29.3)
PatchTST

Appendix

Table 31: Effect of permuting fitted routing vectors on all endpoint settings: relative MSE increase (%), followed in parentheses by mean within-checkpoint MSE SD across permutations in units of

10^{-4}

. Each entry averages nine checkpoints and twenty permutations per checkpoint; every permutation increases MSE.

Method	Run epochs (range)	Cap hits
DLinear
Global correction	68-90	0
PCR- $K4$	78-152	0
PCR- $K2$	72-120	1
Learned- $K4$	70-122	0
Learned- $K2$	76-94	0

Appendix

Table 32: GalaxyTS stopping summaries over four endpoint settings per backbone. Epoch ranges span setting summaries, each the median of three within-realization medians. Cap hits count individual fits across realizations and seeds; two of the full grid’s eight PCR cap hits occur here. Seed variability appears in Table 30 .

Backbone	$H$	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$	Peak
DLinear	96	237.9 / 385.0	214.0 / 349.1	230.7 / 329.4	306.2 / 410.3	0.134
DLinear	720	157.7 / 338.6	101.3 / 239.2	215.8 / 335.8	221.0 / 318.9	0.539
PatchTST	96	149.9 / 1501.6	141.9 / 1531.5	156.5 / 637.0	165.9 / 1197.3	1.340
PatchTST	720	114.8 / 1405.7	98.7 / 696.1	169.7 / 1345.9	160.1 / 696.9	1.663-1.664
iTransformer	96	404.6 / 599.0	237.3 / 358.7	239.4 / 318.8	380.7 / 478.1	12.498
iTransformer	720	157.7 / 347.9	158.6 / 308.6	208.0 / 308.6	139.5 / 216.3	10.995

Appendix

Table 33: GalaxyTS fitting costs at

s=1

, realization 1, training seed 0, on exclusive RTX 3090 GPUs. Entries give method / complete elapsed time (s); peak memory is the range across four methods (GiB). Each entry is one fit without a variance estimate; all stop by the plateau rule.

Backbone	Comparison	Mean (%)	95% interval	Positive in 3/3
DLinear	PCR versus learned, $K4$	$-0.07$	$[-0.50,+0.32]$	12/44
	PCR versus learned, $K2$	$+0.02$	$[-0.51,+0.40]$	18/44
	$K2$ versus $K4$ , PCR	$-0.25$	$[-0.43,-0.03]$	3/44
	$K2$ versus $K4$ , learned	$-0.33$	$[-0.61,-0.04]$	3/44
PatchTST	PCR versus learned, $K4$	$+0.38$	$[+0.05,+0.75]$	19/44
	PCR versus learned, $K2$	$+0.20$	$[-0.36,+0.88]$	13/44

Appendix

Table 34: Paired relative MSE reductions (%); positive values favor the first method. Means weight datasets equally; the upper panel’s 95% percentile intervals use 100,000 bootstrap resamples of the eleven dataset means. Its last column counts improvements in all three seeds, out of 44 settings. The lower panel reports final-protocol Global-control comparisons on the same 44 settings and three seeds per backbone.

Backbone	Arm	Selected epoch	Epochs run	Caps	Gain, $\leq 30$	Final gain
DLinear	PCR- $K4$	61	82	0	3.65 %	3.86 %
	PCR- $K2$	47	80	0	3.49%	3.63%
	Learned- $K4$	48	80	0	3.53 %	3.92 %
	Learned- $K2$	65	86	0	3.14%	3.62%
PatchTST	PCR- $K4$	86	102	2	13.45 %	14.25 %
	PCR- $K2$	76	99	2	12.54 %	13.31%

Appendix

Table 35: Training diagnostics over 132 runs per backbone and arm: median epochs, cap counts, and dataset-balanced MSE reductions (%). Bold/underline mark the best/second distinct gain within each backbone and column; ties share rank.

Dataset	$H$	Frozen base	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
ETTh1	96	$0.3860\pm 0.0031$	$\mathbf{0.3793}\pm 0.0004$	$0.3810\pm 0.0027$	$0.3828\pm 0.0036$	$\underline{0.3808}\pm 0.0010$
ETTh1	192	$0.4512\pm 0.0301$	$\underline{0.4404}\pm 0.0114$	$\mathbf{0.4395}\pm 0.0108$	$0.4465\pm 0.0019$	$0.4444\pm 0.0035$
ETTh1	336	$0.4994\pm 0.0249$	$\mathbf{0.4814}\pm 0.0031$	$0.4832\pm 0.0041$	$0.4843\pm 0.0032$	$\underline{0.4824}\pm 0.0025$
ETTh1	720	$0.5363\pm 0.0294$	$\mathbf{0.5081}\pm 0.0047$	$\underline{0.5115}\pm 0.0050$	$0.5174\pm 0.0031$	$0.5213\pm 0.0056$
ETTh2	96	$0.3208\pm 0.0397$	$0.3223\pm 0.0422$	$0.3212\pm 0.0404$	$\underline{0.3190}\pm 0.0365$	$\mathbf{0.3175}\pm 0.0340$
ETTh2	192	$\mathbf{0.4372}\pm 0.0065$	$\mathbf{0.4372}\pm 0.0065$	$\mathbf{0.4372}\pm 0.0065$	$0.4381\pm 0.0063$	$\underline{0.4380}\pm 0.0063$

Appendix

Table 36: DLinear: test MSE, mean

\pm

sample SD over three seeds.

Dataset	$H$	Frozen base	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
ETTh1	96	$0.4050\pm 0.0004$	$\mathbf{0.3869}\pm 0.0021$	$0.3897\pm 0.0005$	$0.3887\pm 0.0013$	$\underline{0.3874}\pm 0.0027$
ETTh1	192	$0.4507\pm 0.0008$	$0.4426\pm 0.0020$	$0.4418\pm 0.0017$	$\underline{0.4417}\pm 0.0012$	$\mathbf{0.4413}\pm 0.0009$
ETTh1	336	$0.4860\pm 0.0024$	$0.4843\pm 0.0042$	$0.4838\pm 0.0031$	$\underline{0.4797}\pm 0.0055$	$\mathbf{0.4773}\pm 0.0042$
ETTh1	720	$0.4883\pm 0.0092$	$0.4894\pm 0.0103$	$0.4886\pm 0.0119$	$\underline{0.4865}\pm 0.0114$	$\mathbf{0.4844}\pm 0.0114$
ETTh2	96	$0.2931\pm 0.0015$	$0.2908\pm 0.0021$	$0.2903\pm 0.0023$	$\underline{0.2873}\pm 0.0019$	$\mathbf{0.2871}\pm 0.0026$
ETTh2	192	$0.3751\pm 0.0048$	$\mathbf{0.3677}\pm 0.0032$	$\underline{0.3695}\pm 0.0032$	$0.3705\pm 0.0059$	$0.3698\pm 0.0038$

Appendix

Table 37: PatchTST: test MSE, mean

\pm

sample SD over three seeds.

Dataset	$H$	Frozen base	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
ETTh1	96	$0.3898\pm 0.0007$	$\mathbf{0.3824}\pm 0.0014$	$0.3838\pm 0.0027$	$\underline{0.3835}\pm 0.0003$	$0.3857\pm 0.0016$
ETTh1	192	$0.4445\pm 0.0006$	$\mathbf{0.4363}\pm 0.0010$	$\underline{0.4379}\pm 0.0032$	$0.4415\pm 0.0008$	$0.4394\pm 0.0042$
ETTh1	336	$0.4856\pm 0.0017$	$\underline{0.4805}\pm 0.0039$	$\mathbf{0.4789}\pm 0.0055$	$0.4821\pm 0.0025$	$0.4811\pm 0.0050$
ETTh1	720	$0.5137\pm 0.0089$	$\underline{0.5082}\pm 0.0055$	$0.5083\pm 0.0049$	$\mathbf{0.5080}\pm 0.0037$	$\underline{0.5082}\pm 0.0049$
ETTh2	96	$0.2988\pm 0.0009$	$\mathbf{0.2918}\pm 0.0014$	$0.2930\pm 0.0022$	$\underline{0.2927}\pm 0.0006$	$0.2931\pm 0.0010$
ETTh2	192	$0.3800\pm 0.0009$	$\underline{0.3738}\pm 0.0009$	$\mathbf{0.3735}\pm 0.0010$	$0.3756\pm 0.0015$	$0.3745\pm 0.0019$

Appendix

Table 38: iTransformer: test MSE, mean

\pm

sample SD over three seeds.

Backbone	PCR- $K4$	PCR- $K2$	Learned- $K4$	Learned- $K2$
DLinear	$1.87$ $[0.96,2.86]$	$1.76$ $[0.83,2.76]$	$1.87$ $[1.13,2.69]$	$1.85$ $[1.08,2.75]$
PatchTST	$11.17$ $[5.41,17.01]$	$10.44$ $[5.07,15.81]$	$10.99$ $[5.39,16.78]$	$10.24$ $[5.03,15.48]$
iTransformer	$5.87$ $[2.59,9.50]$	$4.75$ $[2.19,7.57]$	$5.71$ $[2.39,9.37]$	$4.29$ $[2.06,6.79]$

Appendix

Table 39: Dataset-balanced MAE reductions (%) from the frozen base. Brackets give 95% dataset-bootstrap intervals; higher is better.

	Original-expert benefit		Calendar benefit
Dataset	MSE	MAE	MSE	MAE
ETTh1	$+0.70$	$+1.08$	$-2.02$	$-1.86$
ETTh2	$-42.63$	$-15.33$	$-2.19$	$-0.93$
ETTm1	$-1.59$	$-0.96$	$-5.93$	$-4.99$
ETTm2	$-4.62$	$-1.85$	$-1.33$	$+0.06$
Electricity	$+5.81$	$+3.25$	$+2.46$	$+0.11$

Appendix

Table 40: MoLE capacity and calendar contrasts (%). Each dataset averages all four horizons and each error uses three seeds. Positive values favor original experts or calendar access. The final row weights datasets equally; these contrasts are not method rankings.

Backbone	Dataset	$H$	CCM original	CCM matched	MoLE original	MoLE expert- budget control	MoLE matched
DLinear	ETTh1	96	$0.385\!\pm\!0.002$ $0.396\!\pm\!0.001$	$0.383\!\pm\!0.001$ $0.399\!\pm\!0.001$	$0.391\!\pm\!0.003$ $0.401\!\pm\!0.005$	$0.400\!\pm\!0.009$ $0.411\!\pm\!0.012$	$0.381\!\pm\!0.003$ $0.395\!\pm\!0.004$
DLinear	ETTh1	192	$0.436\!\pm\!0.003$ $0.431\!\pm\!0.002$	$0.438\!\pm\!0.003$ $0.436\!\pm\!0.004$	$0.438\!\pm\!0.003$ $0.429\!\pm\!0.006$	$0.441\!\pm\!0.007$ $0.436\!\pm\!0.010$	$0.436\!\pm\!0.003$ $0.433\!\pm\!0.003$
DLinear	ETTh1	336	$0.483\!\pm\!0.002$ $0.459\!\pm\!0.004$	$0.500\!\pm\!0.011$ $0.478\!\pm\!0.012$	$0.489\!\pm\!0.016$ $0.459\!\pm\!0.015$	$0.488\!\pm\!0.008$ $0.457\!\pm\!0.009$	$0.477\!\pm\!0.003$ $0.446\!\pm\!0.001$
DLinear	ETTh1	720	$0.524\!\pm\!0.010$ $0.515\!\pm\!0.008$	$0.515\!\pm\!0.011$ $0.509\!\pm\!0.006$	$0.507\!\pm\!0.005$ $0.495\!\pm\!0.005$	$0.508\!\pm\!0.024$ $0.498\!\pm\!0.021$	$0.509\!\pm\!0.024$ $0.497\!\pm\!0.023$
DLinear	ETTh2	96	$0.301\!\pm\!0.009$ $0.357\!\pm\!0.008$	$0.311\!\pm\!0.012$ $0.367\!\pm\!0.010$	$0.433\!\pm\!0.052$ $0.426\!\pm\!0.024$	$0.341\!\pm\!0.005$ $0.391\!\pm\!0.004$	$0.328\!\pm\!0.012$ $0.383\!\pm\!0.010$
DLinear	ETTh2	192	$0.397\!\pm\!0.019$ $0.418\!\pm\!0.015$	$0.423\!\pm\!0.019$ $0.439\!\pm\!0.017$	$0.492\!\pm\!0.049$ $0.464\!\pm\!0.020$	$0.430\!\pm\!0.018$ $0.443\!\pm\!0.011$	$0.462\!\pm\!0.051$ $0.456\!\pm\!0.026$

Appendix

Table 41: Competitor and calendar-ablation results for all 44 dataset-horizon settings per backbone. Entries stack MSE above MAE, with mean

\pm

sample SD over three seeds. The corresponding frozen-base, PCR, and learned-mixture results appear in Appendix E.3.4 . iTransformer evaluates our CCM adaptation; dashes indicate methods not evaluated on that backbone.

Backbone	Datasets	$H=96$	$H\in\{192,336,720\}$
DLinear	ETTh1, ETTm1	$(16,1)$	$(2,1)$
	Weather	$(8,4)$	$(8,4)$
	Remaining	$(2,4)$	$(2,4)$
PatchTST	ETTh1	$(4,4)$	$(16,1)$
	ETTh2	$(16,1)$	$(128,4)$
	Remaining	$(4,1)$	$(2,1)$

Appendix

Table 42: Original VE capacity recipes. Entries are (number of experts

k

, capacity ratio

p

); “remaining” covers the other datasets within each backbone. Here

d=L

for each DLinear bank and

d=F

for the transformers.

Backbone	Method	MSE gain (%)	MAE gain (%)	MSE $R$ (wins)	MAE $R$ (wins)
DLinear	PCR-K4	$3.86\pm 0.21$	$1.87\pm 0.21$	—	—
	VE original	$-2.49\pm 5.47$	$-1.11\pm 2.38$	$+5.12\ (43)$	$+2.67\ (40)$
	VE matched	$-1.65\pm 1.97$	$-1.46\pm 1.08$	$+4.33\ (39)$	$+2.87\ (38)$
	$\delta$ -Adapter original	$11.01\pm 0.10$	$7.85\pm 0.09$	$-9.97\ (15)$	$-7.57\ (16)$
	$\delta$ -Adapter reduced-width	$4.40\pm 0.56$	$2.84\pm 0.47$	$-0.92\ (24)$	$-1.22\ (22)$
PatchTST	PCR-K4	$14.25\pm 0.10$	$11.17\pm 0.09$	—	—

Appendix

Table 43: VE and

\delta

-Adapter across 44 settings per backbone. Backbone-relative gains (%) are means

\pm

sample SD over three seed aggregates, averaging horizons within datasets and weighting datasets equally.

R=100(m_{\mathrm{method}}-m_{\mathrm{PCR}})/m_{\mathrm{method}}

uses each setting’s three-seed mean error before averaging; positive

R

favors PCR-K4. Parentheses count PCR-K4 wins out of 44. Comparisons are descriptive.

		Original		Expert-budget matched
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.385\pm 0.003$	$0.397\pm 0.003$	$0.384\pm 0.005$	$0.396\pm 0.005$
	192	$0.442\pm 0.004$	$0.431\pm 0.004$	$0.440\pm 0.004$	$0.429\pm 0.004$
	336	$0.482\pm 0.001$	$0.449\pm 0.001$	$0.482\pm 0.000$	$0.451\pm 0.002$
	720	$0.520\pm 0.009$	$0.502\pm 0.008$	$0.512\pm 0.008$	$0.499\pm 0.006$
ETTh2	96	$0.384\pm 0.099$	$0.406\pm 0.058$	$0.404\pm 0.062$	$0.426\pm 0.032$

Appendix

Table 44: VE on DLinear: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

		Original		Reduced-width
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.388\pm 0.003$	$0.402\pm 0.004$	$0.386\pm 0.003$	$0.400\pm 0.004$
	192	$0.453\pm 0.029$	$0.445\pm 0.028$	$0.452\pm 0.030$	$0.444\pm 0.028$
	336	$0.525\pm 0.023$	$0.496\pm 0.021$	$0.533\pm 0.020$	$0.502\pm 0.016$
	720	$0.586\pm 0.022$	$0.563\pm 0.014$	$0.555\pm 0.039$	$0.539\pm 0.027$
ETTh2	96	$0.323\pm 0.042$	$0.376\pm 0.037$	$0.321\pm 0.040$	$0.374\pm 0.036$

Appendix

Table 45:

\delta

-Adapter on DLinear: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

. The reduced-width variant uses the closest feasible adapter budget.

		Original		Expert-budget matched
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.397\pm 0.002$	$0.411\pm 0.001$	$0.395\pm 0.002$	$0.412\pm 0.003$
	192	$0.442\pm 0.004$	$0.439\pm 0.002$	$0.445\pm 0.003$	$0.441\pm 0.000$
	336	$0.485\pm 0.004$	$0.464\pm 0.002$	$0.488\pm 0.004$	$0.466\pm 0.001$
	720	$0.478\pm 0.014$	$0.481\pm 0.011$	$0.475\pm 0.004$	$0.481\pm 0.002$
ETTh2	96	$0.296\pm 0.006$	$0.345\pm 0.004$	$0.294\pm 0.003$	$0.344\pm 0.002$

Appendix

Table 46: VE on PatchTST: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

		Original		Reduced-width
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.395\pm 0.001$	$0.411\pm 0.001$	$0.403\pm 0.002$	$0.416\pm 0.003$
	192	$0.439\pm 0.002$	$0.437\pm 0.001$	$0.451\pm 0.002$	$0.444\pm 0.003$
	336	$0.480\pm 0.004$	$0.462\pm 0.003$	$0.488\pm 0.003$	$0.467\pm 0.001$
	720	$0.488\pm 0.015$	$0.492\pm 0.012$	$0.496\pm 0.024$	$0.495\pm 0.017$
ETTh2	96	$0.293\pm 0.002$	$0.342\pm 0.002$	$0.293\pm 0.002$	$0.342\pm 0.002$

Appendix

Table 47:

\delta

-Adapter on PatchTST: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

. The reduced-width variant uses the closest feasible adapter budget.

		Original		Expert-budget matched
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.390\pm 0.002$	$0.408\pm 0.002$	$0.396\pm 0.004$	$0.415\pm 0.003$
	192	$0.443\pm 0.004$	$0.440\pm 0.002$	$0.454\pm 0.006$	$0.449\pm 0.002$
	336	$0.487\pm 0.003$	$0.461\pm 0.001$	$0.491\pm 0.004$	$0.466\pm 0.001$
	720	$0.505\pm 0.004$	$0.491\pm 0.004$	$0.504\pm 0.006$	$0.490\pm 0.003$
ETTh2	96	$0.299\pm 0.002$	$0.348\pm 0.001$	$0.302\pm 0.004$	$0.353\pm 0.003$

Appendix

Table 48: VE on iTransformer: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

		Original		Reduced-width
Dataset	$H$	MSE	MAE	MSE	MAE
ETTh1	96	$0.394\pm 0.001$	$0.409\pm 0.001$	$0.394\pm 0.002$	$0.410\pm 0.002$
	192	$0.445\pm 0.003$	$0.445\pm 0.002$	$0.457\pm 0.006$	$0.450\pm 0.005$
	336	$0.494\pm 0.005$	$0.471\pm 0.005$	$0.498\pm 0.003$	$0.470\pm 0.003$
	720	$0.528\pm 0.005$	$0.515\pm 0.003$	$0.526\pm 0.011$	$0.508\pm 0.004$
ETTh2	96	$0.298\pm 0.001$	$0.349\pm 0.000$	$0.299\pm 0.001$	$0.349\pm 0.000$

Appendix

Table 49:

\delta

-Adapter on iTransformer: test MSE and MAE, mean

\pm

sample SD over three seeds, with

L=96

. The reduced-width variant uses the closest feasible adapter budget.

Backbone	Method	Stored params.	Trainable params.	FLOPs (M)
DLinear	PCR-K4/K2	80,400	29,184	21.8
	Learned-K4/K2	82,180	30,964	21.8
	CCM original	212,236	212,236	50.8
	CCM matched	36,992	36,992	12.1
	MoLE original	671,344	671,344	49.7
	MoLE matched / expert-budget control	495,664	495,664	10.9

Appendix

Table 50: Median parameters and FLOPs per window over 44 settings. Trainable counts include retained router weights with zero data gradient. MoLE matched and its expert-budget control have identical counts. PCR stores fitted mixtures/supports in addition to model parameters. iTransformer FLOPs use a PyTorch counter; other rows are analytical estimates, both omitting uncounted overhead. Bold/underline mark the lowest/second distinct reported values per backbone and metric; ties share rank.

Figure 9: Original and expert-budget-matched specialization. PCR-K4 is repeated in both panels. Relative MSE is dataset-balanced across eleven datasets and four horizons, averaged over three seeds; latency and gradient-trained parameter counts are medians over the 44 settings per backbone. Colours identify backbones and bubble area is proportional to parameter count, with a common scale across panels. The dashed line denotes the frozen reference. Expert-budget matching does not match total trainable parameters; MoLE matched also removes calendar inputs. Lower MSE and latency are better.

Backbone	Method	Fit (s)	Excl. cache setup (s)	Tuning (s)	Peak memory
DLinear	PCR-K4	562.9	560.0	-	0.54 / 5.72
	PCR-K2	546.3	538.0	-	0.54 / 5.72
	Learned-K4	527.6	527.6	-	0.08 / 0.97
	Learned-K2	639.6	639.6	-	0.08 / 0.97
PatchTST	PCR-K4	1,101.9	1,068.6	-	1.95 / 17.65
	PCR-K2	968.2	929.5	-	1.95 / 17.65

Appendix

Table 51: Median fitting costs. PCR/learned rows summarize 132 fits per method; external rows give selected joint-fit time and non-selected tuning time per setting. Memory is median / maximum, retaining recorded units (PCR/learned: GB; external: GiB). Dashes mark unreported quantities or inherited tuning for MoLE matched. CCM-iTransformer fitting medians use all 132 selected trajectories per variant; tuning sums non-selected trials per setting. Training hardware and concurrency differ.

Method	Adaptation / joint fit	Including base	Peak GiB
PCR-K4	976.7	1,978.8	1.948
PCR-K2	927.5	1,906.0	1.948
Learned-K4	4,570.2	5,792.8	0.424
Learned-K2	4,014.1	4,998.9	0.424
CCM original	2,586.9	2,586.9	2.452
CCM matched	2,361.3	2,361.3	2.430

Appendix

Table 52: PatchTST costs with and without selected pretraining: medians of three-seed means over 44 settings. Time is in seconds and peak memory in GiB. Pretraining is included once; external joint fits already train the full model. Search is reported separately.

Backbone	Configuration	Latency	Throughput	Peak / incremental	FLOPs (M)	State (MiB)
DLinear	Backbone	0.325	65,559	0.108 / 0.017	11.6	0.195
DLinear	PCR-K4	0.514	59,052	0.111 / 0.021	21.8	0.313
DLinear	PCR-K2	0.510	58,791	0.121 / 0.021	21.8	0.313
DLinear	Learned-K4	0.511	58,536	0.121 / 0.021	21.8	0.313
DLinear	Learned-K2	0.508	59,597	0.121 / 0.021	21.8	0.313
PatchTST	Backbone	2.138	1,086	0.419 / 0.316	1,788.1	5.093

Appendix

Table 53: Deployment medians over 44 settings per backbone. Latency: batch-one ms; throughput: batch-32 windows/s; peak/incremental memory: batch-32 GiB. State includes deployed parameters, buffers, gates, and calibration; FLOPs are per window. All profiles use seed-zero selected checkpoints. External rows show the available latency and throughput measurements; dashes denote quantities not reported here.

DLinear
Correction / deployment	Added (k)	Latency (ms)	Overhead (%)	Activation (MB)
Global (unmerged)	38.40	0.56	25.94	17.65
Random (unmerged)	38.40	0.58	29.40	17.65
Persistent ID (unmerged)	40.18	0.84	83.91	17.74
Identity + input (unmerged)	40.95	0.92	98.43	17.74
Dense ID (unmerged)	40.18	0.58	30.41	17.65

Appendix

Table 54: Median allocation-study deployment costs. Added parameters are in thousands; latency is batch-32 ms; overhead is the setting-wise increase over the backbone (%); activation memory is incremental MB. Static counts exclude discarded fitting factors and routers. Dashes denote unreported memory.

Dataset	$L$	$H$	Base	Input	Global	ID	ID + input	Dense ID
Electricity	96	96	13.275	13.277	13.012	13.121	13.131	12.865
Electricity	96	720	13.428	13.729	13.474	13.691	13.700	13.443
Electricity	720	96	13.812	13.836	13.540	13.699	13.747	13.503
Electricity	720	720	14.460	14.637	14.271	14.450	14.530	14.264
PEMS04	96	96	12.981	13.194	12.895	12.900	12.868	12.659
PEMS04	96	720	12.591	13.198	12.885	13.108	13.170	12.921

Appendix

Table 55: iTransformer lookback profiles: median batch-32 latency (ms) over 50 synchronized forwards. Base and Input use trained checkpoints; the four remaining columns are untrained architecture probes. ID denotes persistent routing.

Multivariate Time Series Forecasting

Multivariate time series forecasting predicts future values across related variables by learning their temporal and cross-variable dependencies.

See all 302 papers

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting
May 24, 2026Rui Wang, Renhao Xue, Ray Razi +2Sparse Mixture-of-Experts Time Series Forecasting
Amazon Web Services
Routing Channel-Patch Dependencies in Time Series Forecasting with Graph Spectral Decomposition
Mar 14, 2026Dongyuan Li, Shun Zheng, Chang Xu +2Multivariate Time Series Forecasting Multiscale Time-Series Modeling
The University of Tokyo · Microsoft Research Asia
Multi-Modal Time Series Prediction via Mixture of Modulated Experts
Jan 29, 2026Lige Zhang, Ali Maatouk, Jialin Chen +3Multivariate Time Series Forecasting Time Series Forecasting
Emory University · Yale University · Amazon

May 24, 2026J/K move · Enter open · S save

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting

Rui Wang, Renhao Xue, Ray Razi, Huan Song+1

Amazon Web Services

Sparse Mixture-of-Experts Time Series Forecasting Expert Routing Time-Series Foundation Models Time Series Modeling

Open paper AskPDF arXiv

Time series forecasting models are increasingly scaled through large Transformer backbones, yet most existing approaches process all series through a shared dense computation path despite substantial heterogeneity in temporal structure. Mixture-of-Experts (MoE) offers a natural alternative by enabling conditional computation, but standard MoE routing leaves expert specialization weakly identified and often unstable during downstream adaptation. We propose AME-TS, a structure-guided sparse time series foundation model that aligns expert routing with interpretable temporal structure. AME-TS first uses a lightweight regime predictor to estimate series-level descriptors, including forecastability, seasonality, trend, and sparsity, and maps them to a soft structural prior over experts. This series-level prior guides token-level routing during training, encouraging structure-aligned specialization. On the GIFT-Eval benchmark, AME-TS delivers a strong accuracy-efficiency tradeoff across model scales: it substantially outperforms existing time series foundation models at small model scales and remains competitive with the strongest models at larger scales, while activating substantially fewer parameters through sparse routing. We further show that AME-TS learns more interpretable routing geometry and substantially more stable expert specialization than standard MoE during fine-tuning on the M5 dataset. These results suggest that structure-aware routing is an effective and reliable way to realize the benefits of sparse expert models for time series forecasting.

Abstract

Multivariate Time Series Forecasting

Explore similar work

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting

Routing Channel-Patch Dependencies in Time Series Forecasting with Graph Spectral Decomposition

Multi-Modal Time Series Prediction via Mixture of Modulated Experts