Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Models

Hung Phan; Thuy T. Nguyen; Minh Ngoc Dinh; Nhat-Quang Tran

cs.LGSep 30, 2026

Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Models

Authors: Hung Phan, Thuy T. Nguyen, Minh Ngoc Dinh, Nhat-Quang Tran

Abstract

Time series foundation models (TSFMs) commonly adapt to new data by attaching a single trainable head to a frozen backbone, a one-size-fits-all setup that underfits heterogeneous regimes. Replacing the head with a mixture of experts is the standard upgrade, but on instance-normalized backbones (the dominant TSFM design class) it fails: routing entropy collapses to zero and one expert absorbs every input, a failure we call normalization-induced routing collapse. Standard MoE rescue mechanisms do not repair it, because the cause is in the router's input, not its optimization. Pre-encoder normalization strips the statistics a router would need to tell regimes apart. A mutual-information decomposition makes this precise and yields a signal-ratio that, computed before training, predicts dataset vulnerability (Spearman $ρ= -0.88$ ). Eight causal controls, including a vision-modality replication, isolate instance normalization as the cause. The prescription is a minimal causal intervention: Raw-Routed Mixture of Adapters (RR-MoA), which routes on the raw, pre-normalization input. Under a strictly frozen backbone, RR-MoA wins 54/54 comparisons against the strongest fixed adapter and significantly outperforms LoRA, TRACE, AdaMix, and full fine-tuning. The effect generalizes across six backbones and an imputation task. Frozen RR-MoA also beats full fine-tuning by 12-79% (the Frozen Paradox); two architecturally distinct variants confirm the principle generalizes beyond this specific router.

Figures & tables

Panel of Figure 1 — Figure 1: Problem overview. (a) TSFM adapters use the same head for every window. (b) The standard mixture-of-experts extension fails on instance-normalized TSFMs under unfreezing: entropy collapses ( $0.000{\pm}0.000$ ) and the remaining experts receive no gradient. Same backbone, different heads.

Panel of Figure 2 — Figure 2: RR-MoA architecture. Path A (blue): frozen TSFM produces $\mathbf{H}$ for all experts. Path B (orange): Conv1d gate ( $853$ params) reads $\mathbf{X}_{\mathrm{raw}}$ and selects Top-2 experts; inactive experts (gray) receive no gradient. Standard hidden-state MoE collapses (Table 2 ); RR-MoA wins 54/54 (Table 1 ).

Dataset	Freeze level	RR-MoA	Best single-head	MSE $\Delta\%$ vs. single-head
ETTh1	Frozen (0/8)	$\mathbf{0.690\pm 0.021}$	$1.241\pm 0.015$	$\mathbf{-44.4\%}$
	Last-2 (2/8)	$\mathbf{0.727\pm 0.074}$	$1.030\pm 0.139$	$-29.4\%$
	Last-4 (4/8)	$\mathbf{0.749\pm 0.036}$	$1.101\pm 0.120$	$-31.9\%$
ETTm1	Frozen (0/8)	$\mathbf{0.571\pm 0.073}$	$1.115\pm 0.075$	$\mathbf{-48.7\%}$
	Last-2 (2/8)	$\mathbf{0.623\pm 0.032}$	$0.891\pm 0.049$	$-30.1\%$
	Last-4 (4/8)	$\mathbf{0.571\pm 0.034}$	$0.866\pm 0.016$	$-34.1\%$

Table 1: Freeze-level ablation (test MSE, mean

\pm

std, 3 seeds

\{42,43,44\}

, H=96). RR-MoA (sparse Top-2 routing over 5 experts) vs. the best single-head adapter from

\{

linear, attention, conv

\}

trained with the same protocol. “MSE

\Delta\%

vs. single-head” is the relative MSE change of RR-MoA over the single-head baseline; negative

=

lower MSE

=

better. RR-MoA wins 27/27 paired (cell, seed) comparisons in this table; combined with Appendix G the score is 54/54 across 6 datasets

\times

3 freeze levels

\times

3 seeds. The gain is largest when frozen (“Frozen Paradox”).

Dataset	Freeze level	MSE (mean $\pm$ std)	Routing entropy (mean $\pm$ std)
ETTh1	Frozen	$1.105\pm 0.026$	$0.629\pm 0.436$
	Last-2	$1.153\pm 0.000$	$\mathbf{0.000\pm 0.000}$
	Last-4	$1.154\pm 0.001$	$\mathbf{0.000\pm 0.000}$
	Last-4 (no RevIN)	$0.575\pm 0.075$	$\mathbf{1.315\pm 0.148}$
ETTm1	Frozen	$1.008\pm 0.012$	$0.487\pm 0.371$
	Last-2	$1.061\pm 0.088$	$0.218\pm 0.309$

Table 2: AdaMix routing collapse (entropy max

{\approx}1.609

). Under unfreezing, entropy

{\to}0.000

. Green : disabling RevIN inside MOMENT recovers entropy (

0.66

–

1.32

), proving RevIN is the causal mechanism. Extended grid in Appendix G .

Figure 3: Causal contrast: MOMENT vs Timer-XL (ETTh1, last-4 unfreezing). Solid: unfrozen; MOMENT entropy collapses

0.51{\to}0.000

{\sim}40

steps; Timer-XL holds

{\approx}1.60

. Dotted:

\log K{=}1.609

ceiling. Per-step trajectories in Figure G.1 (App. G.2 ).

Panel of Figure 4 — Figure 4: Left: Frozen Paradox , frozen RR-MoA beats full FT by $12$ – $79\%$ across the six LTSF datasets at $H{=}96$ . Right: Observation 1 validated : $R(\mathcal{D})$ predicts raw-routing benefit at Spearman $\rho{=}{-}0.88$ ( $p{<}0.002$ , $n{=}9$ ). Traffic and Solar are boundary cases.

#	Control	Hypothesis (rejected)	Observed result $\Rightarrow$ verdict
C1	RevIN ablation	Collapse is intrinsic to MoE itself, not caused by normalization	Entropy $0.000\to 0.66$ – $1.32$ when RevIN disabled (Table 2 ) $\Rightarrow$ RevIN is necessary for collapse
C2	MOMENT vs Timer-XL	Architecture, not normalization, causes collapse	Opposite trajectories with identical code, only norm differs (Fig. 3 ) $\Rightarrow$ normalization is sufficient
C3	BatchNorm1d / GroupNorm	Collapse is unique to RevIN; other per-window normalizers are immune	All three per-window normalizers collapse identically (App. H ) $\Rightarrow$ any norm that strips $(\mu,\sigma)$ triggers collapse
C4	5-family rescue sweep	Optimization problem (load-balance, z-loss, ReLU, expert-choice)	0/12 configs recover; best is $2.7{\times}$ worse than RR-MoA (Table G.3 ) $\Rightarrow$ not in the optimizer
C5	AdaMix with raw router	Router architecture is the bottleneck	Same router, swap input only: entropy $\to 1.41$ – $1.57$ ; MSE $-41$ – $88\%$ (App. G.2 ) $\Rightarrow$ the input matters, not the router
C6	SR-MoA raw vs hidden routing	The fix only works for RR-MoA’s specific Conv1d router design	Different router (per-expert sigmoid); hidden-routed $+87$ – $768\%$ worse (Table F.2 ) $\Rightarrow$ raw-input principle generalizes

Table 3: Eight causal controls. Each rules out a competing explanation; the only mechanism consistent with all 8 is that instance normalization strips the per-window

(\mu,\sigma)

signal the router needs .

Method	ETTh1	ETTm1	Weather	$p$ -value
Best fixed adapter	$1.254\pm 0.026$	$1.148\pm 0.035$	$0.528\pm 0.018$	${<}0.001^{***}$
Best LoRA (108-run sweep)	$1.154\pm 0.000$	$0.956\pm 0.072$	$0.600\pm 0.052$	${<}0.001^{***}$
TRACE (frozen)	$1.195\pm 0.026$	$1.014\pm 0.073$	$0.457\pm 0.015$	${<}0.001^{***}$
Ind. Ensemble (5 experts)	$1.081\pm 0.021$	$1.049\pm 0.032$	$0.509\pm 0.024$	${<}0.001^{***}$
AdaMix (frozen)	$1.109\pm 0.031$	$1.008\pm 0.012$	$0.459\pm 0.017$	${<}0.001^{***}$
Full fine-tuning (all unfrozen)	$1.060\pm 0.059$	$0.871\pm 0.029$	$0.468\pm 0.009$	${<}0.001^{***}$

Table 4: Baseline comparison (MOMENT-small, Top-2, 5 seeds, H=96). Above double rule: frozen; full FT unfreezes all 8 blocks. Wilcoxon, Bonferroni-corrected. Grey rows: from-scratch calibration anchors.

Appendix figures & tables43 assets

Supplementary material from the paper’s appendix.

Appendix

Data and task
Symbol	Meaning
$\mathbf{x}\in\mathbb{R}^{T\times c}$	Input window: $T{=}512$ time steps, $c$ channels
$\mathbf{y}\in\mathbb{R}^{h\times c}$	Forecast target at horizon $h\in\{96,192,336,720\}$
$\hat{\mathbf{y}}$	Model prediction
$M_{i},\Sigma_{i},\mathbf{S}_{i}$	Per-window mean, scale, RevIN-normalized shape ( $\mathbf{S}_{i}{=}(\mathbf{x}_{i}{-}M_{i})/\Sigma_{i}$ )
Backbone and adapter

Appendix

Table A.1: Notation. Symbols used throughout the paper.

Method	Latency (ms)	$\Delta$ vs. backbone	Peak GPU (MB)	Adapter params
Backbone only	$52.89\pm 0.36$	—	$359$	—
Single adapter (Conv1dPool)	$53.57\pm 0.37$	$+1.3\%$	$360$	$268$ K
RR-MoA (Top-2, $K{=}5$ )	$\mathbf{56.02\pm 0.44}$	$\mathbf{+5.9\%}$	$\mathbf{362}$	$\mathbf{426}$ K
SR-MoA (dense $K{=}5$ )	$\mathbf{54.20\pm 0.79}$	$\mathbf{+2.5\%}$	$\mathbf{364}$	$\mathbf{466}$ K
Residual-IA +	$\mathbf{54.53\pm 0.43}$	$\mathbf{+3.1\%}$	$\mathbf{365}$	$\mathbf{354}$ K
DLinear (no backbone)	$0.04\pm 0.00$	—	$160$	$49$ K

Appendix

Table B.1: Inference latency and memory across all three architectural variants (MOMENT-small, A10G GPU, batch size

128

200

repeats with

20

-step CUDA-event warmup; synthetic random inputs at the standard LTSF batch shape, since latency for non-autoregressive deterministic forward passes is data-oblivious). All three architectural fixes (RR-MoA, SR-MoA, Residual-IA + ) add only single-digit milliseconds of adapter overhead on top of the backbone forward pass; the frozen-backbone fast-inference property is preserved by each.

	ETTh1			ETTm1			Weather
$H$	RR-MoA	Best fix	DLin	RR-MoA	Best fix	DLin	RR-MoA	Best fix	DLin
96	$0.680{\pm}.030$	$1.233$	$\mathit{0.420}$	$0.564{\pm}.066$	$1.169$	$\mathit{0.326}$	$0.276{\pm}.019$	$0.522$	$\mathit{0.207}$
192	$0.773{\pm}.071$	$1.326$	$\mathit{0.476}$	$0.632{\pm}.039$	$1.141$	$\mathit{0.365}$	$0.292{\pm}.009$	$0.498$	$\mathit{0.246}$
336	$0.776{\pm}.069$	$1.316$	$\mathit{0.502}$	$0.658{\pm}.060$	$1.125$	$\mathit{0.415}$	$0.329{\pm}.016$	$0.516$	$\mathit{0.286}$
720	$0.816{\pm}.049$	$1.379$	$\mathit{0.553}$	$0.713{\pm}.048$	$1.121$	$\mathit{0.454}$	$0.401{\pm}.017$	$0.548$	$\mathit{0.350}$

Appendix

Table C.1: Multi-horizon RR-MoA vs. DLinear (MOMENT-small, strictly frozen, Top-2, 5 seeds, test MSE mean

\pm

std). RR-MoA wins all 12 configurations vs. best fixed baseline across horizons

96

–

720

. DLinear column provides supervised calibration: the gap narrows from

+63\%

+48\%

on ETTh1 and from

+33\%

+15\%

on Weather as

H

grows, consistent with the frozen backbone’s horizon-invariant representation becoming relatively more informative at longer horizons.

Panel of Figure 12 — Figure D.1: Normalization dose-response (frozen, 5 seeds $\times$ 6 datasets; std smaller than markers). MSE vs $\alpha$ ( $0{=}$ raw, $1{=}$ RevIN). Phase transition at $\alpha{=}1.0$ ( $+283\%$ ), mild for $\alpha{\leq}0.75$ ( $+5.4\%$ ); routing entropy stays in $[1.43,1.57]$ across all $\alpha$ (Appendix F ).

Dataset	Rank	Targets	Head	MSE (mean $\pm$ std)
ETTh1 (RR-MoA frozen: $\mathbf{0.690\pm 0.021}$ )
	$r{=}8$	q,v	linear	$1.559\pm 0.022$
	$r{=}8$	q,v	mlp	$1.217\pm 0.010$
	$r{=}8$	qkvo	linear	$1.286\pm 0.052$
	$r{=}8$	qkvo	mlp	$1.154\pm 0.000$
	$r{=}16$	q,v	linear	$1.347\pm 0.056$

Appendix

Table E.1: Full LoRA sweep (strictly frozen backbone, 3 seeds, test MSE mean

\pm

std). Bold rows are the best-per-dataset configuration used in Table 4 . No LoRA variant beats the RR-MoA (Top-2) result on any dataset.

Dataset	Raw router (main)	RevIN router	Degradation ( $\Delta\%$ , $\uparrow$ worse)
ETTh1	$\mathbf{0.690\pm 0.021}$	$1.101\pm 0.008$	$\mathbf{+59.6\%}$
ETTm1	$\mathbf{0.571\pm 0.073}$	$1.077\pm 0.020$	$\mathbf{+88.6\%}$
Weather	$\mathbf{0.289\pm 0.008}$	$0.542\pm 0.026$	$\mathbf{+87.5\%}$

Appendix

Table F.1: Rawness vs. bypass (frozen, Top-2, 3 seeds). RevIN router reads the same input after per-window normalization. Stripping statistics degrades MSE by 60–88%.

Panel of Figure 15 — Figure F.1: SR-MoA architecture (Self-Routed Mixture of Adapters). Each of $K{=}5$ experts has its own sigmoid gate $\sigma_{k}$ reading $\mathbf{X}_{\mathrm{raw}}$ directly, with no external router. The frozen TSFM produces $\mathbf{H}$ , and each expert’s contribution is scaled by its self-gate. SR-MoA shares only the raw-input principle with RR-MoA (Figure 2 ): the routing architecture is otherwise entirely different. SR-MoA outperforms RR-MoA by $13$ – $42\%$ across all 6 datasets, providing architecture-agnostic validation that the principle, not the specific router, drives the gains.

Dataset	Config	Backbone	MSE ( $\downarrow$ )	Entropy
Routing input control (frozen, MOMENT-small, 5 seeds):
ETTh1	Raw	MOMENT-sm	$\mathbf{0.464\pm 0.009}$	$1.53$
ETTh1	Hidden	MOMENT-sm	$1.075\pm 0.102$ ( $+131\%$ )	$1.60$
ETTh2	Raw	MOMENT-sm	$\mathbf{0.577\pm 0.105}$	$1.38$
ETTh2	Hidden	MOMENT-sm	$2.840\pm 0.164$ ( $+393\%$ )	$1.58$
ETTm1	Raw	MOMENT-sm	$\mathbf{0.388\pm 0.023}$	$1.53$

Appendix

Table F.2: Self-Routed MoA (SR-MoA): routing input control (dense, 5 seeds unless noted). Each expert has its own sigmoid gate, with no external router. Top : gating on raw input vs hidden states (frozen, MOMENT-small). Second : Timer-XL negative control with no RevIN, so raw

\approx

hidden. Third : freeze-level ablation confirms Frozen Paradox (frozen best 4/6). Bottom : cross-backbone generalization (H=96, frozen). 252 runs total.

Figure F.2: Per-sample expert assignment vs. input statistics (ETTh1, seed 42, $N{=}5{,}000$ test windows). Left: amplitude vs. volatility; right: mean level vs. amplitude. Colors indicate the top-1 expert per window. Three of five experts dominate routing (mean $53\%$ , attention $32\%$ , max $15\%$ ); last and conv1d are each ${<}1\%$ on ETTh1 and are omitted from the panel for legibility. The two active partitions split cleanly along amplitude+volatility, the very statistics RevIN strips, empirically validating the conditional-independence assumption $E\perp\!\!\!\perp S\mid(M,\Sigma)$ from Observation 1 , Part (iii) (Part iii condition (a)): expert assignment is determined by location-scale, with negligible residual dependence on shape.

Dataset	Reinjected MSE	Raw RR-MoA MSE	$\Delta\%$	Entropy
ETTh1	$1.357\pm 0.128$	$0.690$	$+97\%$	$0.000$
ETTh2	$2.669\pm 0.104$	$0.788$	$+239\%$	$0.153$
ETTm1	$0.899\pm 0.168$	$0.571$	$+57\%$	$0.376$
ETTm2	$1.895\pm 0.866$	$0.588$	$+222\%$	$0.378$
Weather	$0.490\pm 0.039$	$0.289$	$+70\%$	$0.185$
Electricity	$0.548\pm 0.064$	$0.386$	$+42\%$	$0.307$

Appendix

Table F.3: Statistics re-injection fails (MOMENT-small, frozen, Top-2, 3 seeds). Re-injecting

(\mu,\sigma)

into mean-pooled hidden states degrades MSE by

+42

–

239\%

vs raw routing, with

11/18

runs collapsed (entropy

<0.3

). Two scalars grafted onto a

512

-dim homogenized representation are insufficient to recover routing signal.

Dataset	Freeze level	RR-MoA	Best single-head	MSE $\Delta\%$ vs. single-head
ETTh2	Frozen (0/8)	$\mathbf{0.788\pm 0.096}$	$2.748\pm 0.201$	$\mathbf{-71.3\%}$
	Last-2 (2/8)	$\mathbf{1.302\pm 0.134}$	$2.599\pm 0.085$	$-49.9\%$
	Last-4 (4/8)	$\mathbf{1.745\pm 0.187}$	$2.540\pm 0.110$	$-31.3\%$
ETTm2	Frozen (0/8)	$\mathbf{0.588\pm 0.021}$	$2.744\pm 0.266$	$\mathbf{-78.6\%}$
	Last-2 (2/8)	$\mathbf{0.867\pm 0.083}$	$2.915\pm 0.202$	$-70.3\%$
	Last-4 (4/8)	$\mathbf{0.731\pm 0.157}$	$2.740\pm 0.092$	$-73.3\%$

Appendix

Table G.1: Extended freeze-level ablation (ETTh2, ETTm2, Electricity; test MSE, mean

\pm

std, 3 seeds, H=96). Continues Table 1 from the main text.

Dataset	Freeze level	MSE (mean $\pm$ std)	Routing entropy (mean $\pm$ std)
ETTh2	Frozen	$2.625\pm 0.219$	$0.636\pm 0.385$
	Last-2	$2.985\pm 0.024$	$0.353\pm 0.499$
	Last-4	$2.885\pm 0.150$	$0.541\pm 0.648$
ETTm2	Frozen	$2.946\pm 0.076$	$0.316\pm 0.342$
	Last-2	$3.114\pm 0.004$	$\mathbf{0.000\pm 0.000}$
	Last-4	$2.997\pm 0.162$	$0.505\pm 0.714$

Appendix

Table G.2: Extended AdaMix routing collapse (ETTh2, ETTm2, Electricity). Continues Table 2 from the main text.

Rescue mechanism	MSE (pooled)	Routing $H$	Collapsed	$n/60$
Baseline (Switch LB $\alpha{=}0.01$ )	$1.654$	$0.21$	46/60 (77%)	60/60
+ Entropy reg $\lambda{=}0.01$	$1.666$ ( $+0.7\%$ )	$0.26$	44/60 (73%)	60/60
+ Entropy reg $\lambda{=}0.1$	$1.599$ ( $-3.3\%$ )	$1.03$	19/60 (32%)	60/60
+ Entropy reg $\lambda{=}1.0$	$1.546$ ( $-6.5\%$ )	$1.41$	7/60 (12%)	60/60
+ Z-loss $c_{z}{=}0.001$ (ST-MoE)	$1.561$ ( $-5.6\%$ )	$1.01$	12/60 (20%)	60/60
+ Z-loss $c_{z}{=}0.01$	$\mathbf{1.473}$ ( $\mathbf{-10.9\%}$ )	$1.24$	3/60 (5%)	60/60

Appendix

Table G.3: Rescue-baseline sweep (MOMENT-small

+

RevIN, 720 runs). “Collapsed” = fraction of cells with entropy

<0.3

(max

\log K{=}1.609

). No rescue mechanism recovers routing to within

2.7\times

of RR-MoA.

Figure G.1: Per-step routing / gradient trajectory (AdaMix, ETTh1). (a) Unfrozen+RevIN (red) collapses within

{\sim}40

steps; strictly frozen control (blue) stays stable, isolating gradient co-adaptation from optimizer or data effects. (b) The dominant expert (Attn) retains a gradient of

{\sim}10^{-2}

throughout training, while the other four experts crash to the numerical noise floor (

{<}10^{-3}

) by step 30 and never recover. Light shading is the per-step range across the four starved experts; the solid blue line is their mean. Removing RevIN (separate experiment, Figure 3 ) inverts the trajectory.

Dataset	Freeze	AdaMix MSE	AdaMix $H$	AdaMix-Raw MSE	AdaMix-Raw $H$
ETTh1	frozen	$0.912$	$0.49$	$\mathbf{0.456}$ ( $-50\%$ )	$1.55$
ETTh1	last-4	$0.935$	$0.39$	$\mathbf{0.547}$ ( $-42\%$ )	$1.54$
ETTm1	frozen	$1.008$	$0.49$	$\mathbf{0.397}$ ( $-61\%$ )	$1.56$
ETTm1	last-4	$1.123$	$0.00$	$\mathbf{0.431}$ ( $-62\%$ )	$1.56$
ETTm2	frozen	$2.946$	$0.32$	$\mathbf{0.348}$ ( $-88\%$ )	$1.48$
ETTm2	last-4	$2.997$	$0.51$	$\mathbf{0.375}$ ( $-88\%$ )	$1.44$

Appendix

Table G.4: AdaMix-Raw: architecture-agnostic validation (MOMENT-small, 3 seeds). Replacing AdaMix’s hidden-state router with a raw-input router recovers routing entropy and MSE on all 12 cells, confirming the causal intervention generalizes beyond RR-MoA.

Normalizer	Entropy (step 0)	Entropy (step 400)	Collapses?
RevIN (default)	$0.507$	$0.000$	Yes
BatchNorm1d	$0.619$	$0.004$	Yes
GroupNorm	$0.507$	$0.000$	Yes
None (identity)	$0.623$	$0.817$	No

Appendix

Table H.1: Routing collapse generalizes across normalizer types. All three instance-level normalizers (RevIN, BatchNorm1d, GroupNorm) cause entropy collapse under unfreezing; removing normalization prevents collapse. Per-step trajectories logged over 400 optimizer steps on ETTh1, last-4 unfreezing, seed 42.

Backbone	Cond	Configuration	Entropy	Collapses?	Acc
ResNet-18	A	Unfrozen + InstanceNorm2d	$1.475\pm 0.021$	No	$0.919$
	B	Frozen + InstanceNorm2d	$0.992\pm 0.041$	No	$0.577$
	C	Unfrozen + No norm	$\mathbf{0.184\pm 0.260}$	Yes	$0.921$
	D	Unfrozen + BatchNorm2d	$1.476\pm 0.031$	No	$0.919$
ViT-B/16	E	Frozen + InstanceNorm1d	$\mathbf{0.000\pm 0.000}$	Yes	$0.953$
ViT-B/16	F	Frozen + No norm	$0.494\pm 0.130$	No	$0.958$

Appendix

Table H.2: Vision MoE routing collapse control (CIFAR-10,

K{=}5

diverse experts, mean

\pm

std over 3 seeds). ResNet-18 (top): InstanceNorm2d prevents collapse because spatial patterns (the routing signal) survive normalization. ViT-B/16 (bottom): InstanceNorm1d on patch embeddings causes collapse (

0.000\pm 0.000

) because it strips per-patch statistics that carry routing information, the same mechanism as RevIN in time series.

Sparsity ( $k$ )	Active experts	Active expert FLOPs	Test MSE	$\Delta$ vs Dense
$k{=}1$	1/5	20%	$1.268\pm 0.072$	$+130\%$
$k{=}2$ (default)	2/5	40%	$\mathbf{0.727\pm 0.074}$	$+32\%$
$k{=}3$	3/5	60%	$0.679\pm 0.042$	$+23\%$
Dense $k{=}K{=}5$	5/5	100%	$0.550\pm 0.029$	—

Appendix

Table I.1: Top-

k

sparse routing ablation on ETTh1 (last-2 freeze, 3 seeds). Top-2 is the operating point used throughout the paper.

Expert pool	ETTh1	ETTm1	Weather
Canonical (diverse)	$\mathbf{0.690\pm 0.021}$	$\mathbf{0.571\pm 0.073}$	$0.289\pm 0.008$
$5{\times}$ Mean-pool	$0.827\pm 0.077$ ( $+20\%$ )	$0.739\pm 0.023$ ( $+29\%$ )	$\mathbf{0.245\pm 0.007}$ ( $-15\%$ )
$5{\times}$ Conv1d-pool	$0.687\pm 0.039$ ( $-0\%$ )	$0.628\pm 0.040$ ( $+10\%$ )	$0.243\pm 0.005$ ( $-16\%$ )
$5{\times}$ Attention-pool	$0.779\pm 0.026$ ( $+13\%$ )	$0.685\pm 0.054$ ( $+20\%$ )	$0.264\pm 0.006$ ( $-9\%$ )

Appendix

Table I.2: Expert pool diversity ablation (MOMENT-small frozen, Top-2, H=96, 3 seeds). Canonical diverse pool wins on ETTh1 and ETTm1; identical pools are competitive on Weather.

$K$	Routing	ETTh1	ETTm1	Weather
1	dense	$1.306\pm 0.089$	$1.183\pm 0.051$	$0.576\pm 0.027$
2	top-2	$\mathbf{0.554\pm 0.051}$	$\mathbf{0.457\pm 0.018}$	$\mathbf{0.227\pm 0.008}$
3	top-2	$0.703\pm 0.048$	$0.573\pm 0.065$	$0.276\pm 0.023$
5	top-2	$0.690\pm 0.021$	$0.571\pm 0.073$	$0.289\pm 0.008$
7	top-2	$0.724\pm 0.043$	$0.600\pm 0.029$	$0.253\pm 0.003$
10	top-2	$0.669\pm 0.052$	$0.582\pm 0.052$	$0.258\pm 0.024$

Appendix

Table I.3: Expert count scaling (MOMENT-small frozen, Top-2, H=96, 3 seeds). Performance is stable for

K{\geq}2

;

K{=}1

(no routing) is substantially worse (

57

–

107\%

Dataset	Best fixed adapter	RR-MoA	$\Delta$ %
ETTh1	$0.259\pm 0.018$	$\mathbf{0.144\pm 0.006}$	$-44\%$
ETTh2	$0.451\pm 0.014$	$\mathbf{0.161\pm 0.028}$	$-64\%$
ETTm1	$0.194\pm 0.017$	$\mathbf{0.100\pm 0.008}$	$-49\%$
ETTm2	$0.498\pm 0.020$	$\mathbf{0.104\pm 0.006}$	$\mathbf{-79\%}$
Weather	$0.207\pm 0.008$	$\mathbf{0.067\pm 0.005}$	$-67\%$
Electricity	$0.166\pm 0.008$	$\mathbf{0.131\pm 0.007}$	$-21\%$

Appendix

Table J.1: Imputation task (20% masked reconstruction, MOMENT-small frozen, test MSE mean

\pm

std over 5 seeds). RR-MoA wins 7/8 datasets (

-21\%

-79\%

); the sole loss is Solar where dense_mlp dominates.

Dataset	Learned $\alpha$ (mean $\pm$ std)	MSE
ETTh1	$0.468\pm 0.006$	$0.463\pm 0.015$
ETTh2	$0.482\pm 0.007$	$0.553\pm 0.020$
ETTm1	$0.473\pm 0.005$	$0.397\pm 0.012$
ETTm2	$0.466\pm 0.003$	$0.355\pm 0.055$
Electricity	$0.462\pm 0.004$	$0.269\pm 0.008$
Weather	$0.459\pm 0.009$	$0.245\pm 0.010$

Appendix

Table K.1: Learned normalization coefficient (

\alpha

, 5 seeds). All 30 runs converge to

\alpha<0.5

, confirming gradient descent prefers raw input for routing.

Dataset	$H(E)$	$H(E\mid S)$	Ret%	LB	Gap	$\Delta\%$
ETTh1	0.73	0.61	16.3	0.02	0.06	$-43.2$
ETTh2	0.76	0.57	26.3	0.00	0.10	$-71.0$
ETTm1	0.75	0.57	24.2	0.18	0.03	$-51.1$
ETTm2	0.82	0.53	35.1	0.18	0.07	$-77.2$
Weather	1.37	0.98	29.1	0.61	0.08	$-44.6$
Electricity	0.68	0.46	32.4	0.13	0.02	$-26.8$

Appendix

Table L.1: Exact MI loss measurement and bound tightness (3 seeds).

H(E\mid S)

is the exact routing information lost to normalization. Retention

=I(S;E)/H(E)

. LB = lower bound from Observation 1 , Part (ii). Gap

=I(M,\Sigma;S\mid E)

via within-expert CCA.

	k-NN (MSE)		Ridge (MSE)		Backbone
Dataset	Raw input	Backbone	Raw input	Backbone	penalty
ETTh1	$0.636$	$0.737$	$0.551$	$1.230$	$+16\%$
ETTm1	$0.498$	$0.614$	$0.363$	$0.708$	$+23\%$
Weather	$0.453$	$0.601$	$0.300$	$0.661$	$+33\%$

Appendix

Table M.1: Backbone features vs. raw input (H=96, 80/20 train/test split). k-NN: k-nearest neighbor regression (

k{=}20

, distance-weighted) on mean-pooled backbone features vs. raw 512-dim input. Ridge: regularized linear regression (cross-validated

\alpha

). Backbone features are

16

–

33\%

worse than raw input even with a universal approximator (k-NN), confirming genuine information destruction.

	ETTh1		Weather		Electricity
$N$	DLinear	RR-MoA	DLinear	RR-MoA	DLinear	RR-MoA
10	$\mathit{0.859}$	$1.530$	$\mathit{0.361}$	$1.265$	$\mathit{0.601}$	$1.384$
100	$\mathit{0.630}$	$1.172$	$\mathit{0.286}$	$0.773$	$\mathit{0.373}$	$1.103$
1000	$\mathit{0.453}$	$0.816$	$\mathit{0.213}$	$0.378$	$\mathit{0.188}$	$0.732$
5000	$\mathit{0.421}$	$0.692$	$\mathit{0.205}$	$0.288$	$\mathit{0.158}$	$0.396$

Appendix

Table M.2: Few-shot learning curve (H=96, mean

\pm

std over seeds

\{42,43,44\}

). DLinear (49K params, trained from scratch) dominates frozen RR-MoA-full (426K params) at every tested

N\in\{10,\ldots,5000\}

, confirming the gap reflects encoder information loss rather than data efficiency.

	MOMENT-small		MOMENT-large		Moirai-small
Dataset	RR-MoA	Best fixed	RR-MoA	Best fixed	RR-MoA	Best fixed
ETTh1	$0.690{\pm}0.021$	$1.241$ ( $-44\%$ )	$0.803{\pm}0.032$	$1.173$ ( $-32\%$ )	$\mathbf{0.471{\pm}0.002}$	$0.664$ ( $-29\%$ )
ETTh2			$1.224{\pm}0.311$	$3.075$ ( $-60\%$ )
ETTm1	$0.571{\pm}0.073$	$1.115$ ( $-49\%$ )	$0.732{\pm}0.038$	$1.126$ ( $-35\%$ )	$\mathbf{0.396{\pm}0.041}$	$0.471$ ( $-16\%$ )
ETTm2			$0.790{\pm}0.117$	$3.136$ ( $-75\%$ )
Weather	$0.289{\pm}0.008$	$0.530$ ( $-45\%$ )	$0.276{\pm}0.017$	$0.606$ ( $-54\%$ )	$\mathbf{0.209{\pm}0.004}$	$0.238$ ( $-12\%$ )

Appendix

Table N.1: Cross-backbone RR-MoA (strictly frozen, Top-2 sparse, H=96). MOMENT-small/Moirai-small RR-MoA at 3 seeds (matching Tables 1 , G.1 ); MOMENT-large RR-MoA at 5 seeds. Best fixed: strongest single-adapter baseline. RR-MoA wins all 15 configurations. MOMENT-large expanded to 6 datasets confirms generalization (

-32\%

-75\%

Dataset	RR-MoA (ours)	AdaMix MSE	$\Delta$ %	AdaMix $H$
ETTh1	$\mathbf{0.723\pm 0.008}$	$1.154\pm 0.001$	$\mathbf{-37.3\%}$	$1.6094\pm 0.0000$
ETTh2	$\mathbf{0.487\pm 0.041}$	$3.056\pm 0.009$	$\mathbf{-84.1\%}$	$1.6094\pm 0.0000$
ETTm1	$\mathbf{0.694\pm 0.008}$	$1.123\pm 0.001$	$\mathbf{-38.2\%}$	$1.6094\pm 0.0000$
ETTm2	$\mathbf{0.298\pm 0.016}$	$3.110\pm 0.004$	$\mathbf{-90.4\%}$	$1.6094\pm 0.0000$
Weather	$\mathbf{0.246\pm 0.007}$	$0.607\pm 0.003$	$\mathbf{-59.5\%}$	$1.6094\pm 0.0000$
Electricity	$\mathbf{0.530\pm 0.031}$	$1.055\pm 0.006$	$\mathbf{-49.8\%}$	$1.6094\pm 0.0000$

Appendix

Table N.2: Extension to a native-MoE backbone: Moirai-MoE (strictly frozen, Top-2 sparse, H=96, 5 seeds, 60 runs total). Moirai-MoE is a TSFM whose internal FFN layers are themselves sparsely-gated experts [ 8 ] , using RMSNorm internally with only non-learnable per-instance I/O scaling (no learnable-affine RevIN inside the encoder pipeline). RR-MoA wins all 6 datasets by

-37\%

-90\%

, suggesting adapter-level routing stacks on top of backbone-internal routing. AdaMix’s hidden-state routing does not collapse on Moirai-MoE: under a strictly frozen backbone the router-input gradient is dominated by the load-balance term and the softmax saturates at uniform mixing, so routing entropy hovers at

\log K=1.6094

to four decimal places across all 30 cells (

\sigma{=}0.0000

, indicating mathematically uniform rather than specialized routing); yet AdaMix still only matches the best fixed baseline MSE, consistent with raw-signal routing being information-richer than hidden-state routing even when the hidden states retain enough signal to avoid collapse.

Dataset	Freeze	AdaMix MSE	AdaMix Ent	RR-MoA MSE	Winner
ETTh1	frozen	$\mathbf{0.477\pm 0.012}$	$1.53$	$0.497\pm 0.020$	AdaMix
	last-2	$\mathbf{0.533\pm 0.010}$	$1.60$	$0.547\pm 0.027$	AdaMix
	last-4	$\mathbf{0.582\pm 0.118}$	$1.60$	$0.646\pm 0.088$	AdaMix
ETTm1	frozen	$\mathbf{0.398\pm 0.007}$	$1.52$	$0.408\pm 0.006$	AdaMix
	last-2	$\mathbf{0.479\pm 0.023}$	$1.59$	$0.491\pm 0.029$	AdaMix
	last-4	$0.715\pm 0.299$	$1.59$	$\mathbf{0.704\pm 0.031}$	RR-MoA

Appendix

Table N.3: Negative control: Timer-XL [ 6 ] (decoder-only, 84M params, LayerNorm only, no RevIN; H=96, 5 seeds). Key causal result : on MOMENT (RevIN), unfreezing drives entropy DOWN toward

0.000

(

71\%

collapsed). On Timer-XL, unfreezing drives entropy UP toward the maximum

1.609

(

0/90

collapsed). The trajectories are opposite , isolating RevIN as the causal mechanism. AdaMix wins

5/6

when frozen, confirming RR-MoA is unnecessary without instance normalization.

Dataset	Method	MSE (mean $\pm$ std)	Entropy	Best fixed	$\Delta$ %
Exchange	RR-MoA (frozen)	$\mathbf{1.531\pm 0.185}$	$0.97$	$4.568$	$\mathbf{-66.5\%}$
	AdaMix (frozen)	$5.571\pm 0.511$	$0.61$		$+22.0\%$
	AdaMix (last-4)	$3.851\pm 0.012$	$\mathbf{0.000}$		$-15.7\%$
	DLinear (from scratch)	$0.149\pm 0.017$	—		—
Solar	RR-MoA (frozen)	$\mathbf{0.252\pm 0.021}$	$1.53$	$0.376$	$\mathbf{-32.9\%}$
	AdaMix (frozen)	$0.264\pm 0.017$	$0.75$		$-29.8\%$

Appendix

Table N.4: Extension to Exchange and Solar datasets (MOMENT-small, strictly frozen, Top-2 sparse, H=96, 5 seeds for RR-MoA/AdaMix, 3 seeds for DLinear/AdaMix-last4). Exchange (

R{=}2.17

, highest signal ratio) confirms the theory prediction:

-66.5\%

improvement. Solar (

R{=}0.06

, lowest) is an outlier: RR-MoA still achieves

-32.9\%

improvement, indicating the Conv1d router exploits temporal shape patterns beyond location-scale (consistent with the SSR ablation in Appendix F ). AdaMix collapses to

0.000

entropy under last-4 unfreezing on both datasets.

Configuration	ETTh1	ETTm1	Weather
Full-FT 15ep Adam (lr=1e-4)	$1.086$	$0.937$	$0.485$
Full-FT 50ep Adam (lr=1e-4)	$1.054$	$0.884$	$0.505$
Full-FT 50ep cosine+warmup	$1.067$	$1.079$	$0.607$
Full-FT 50ep cosine+layerwise	$\mathbf{1.017}$	$\mathbf{0.881}$	$\mathbf{0.470}$
Full-FT 50ep Adam (lr=1e-5)	$1.093$	$1.059$	$0.586$
Full-FT best (any config)	$1.017$	$0.881$	$0.470$

Appendix

Table O.1: Extended full fine-tuning does not close the Frozen Paradox gap. Best of 5 training configurations (15ep Adam, 50ep Adam, 50ep cosine, 50ep cosine+layerwise, 50ep lr=1e-5). Frozen RR-MoA wins by

33

–

41\%

even against the strongest optimization schedule.

Setting	ETTh1	ETTm1	Weather
Full FT (RevIN on)	$1.063$	$0.910$	$0.481$
Full FT (RevIN off)	$\mathbf{0.490}$	$\mathbf{0.364}$	$\mathbf{0.187}$
$\Delta$ %	$-54\%$	$-60\%$	$-61\%$

Appendix

Table O.2: Full fine-tuning, RevIN on vs off (MOMENT-small, all 8 blocks unfrozen, best of 5 heads

\times

2 LRs, 50 epochs cosine; mean test MSE over the same seeds as Table 4 ). Disabling RevIN inside MOMENT recovers full-FT MSE by

54

–

61\%

, isolating the normalization layer rather than the unfreezing protocol as the source of the Frozen Paradox.

Epochs	ETTh1	ETTm1	Weather
15	$0.690$	$0.572$	$0.289$
50	$\mathbf{0.633}$	$\mathbf{0.494}$	$\mathbf{0.250}$
$\Delta$ %	$-8\%$	$-14\%$	$-13\%$

Appendix

Table O.3: Adapter training budget for frozen RR-MoA (MOMENT-small, Top-2, frozen backbone, 3 seeds; mean test MSE).

Method	ETTh1	ETTm1	Weather	Avg. gap
DLinear (scratch)	$\mathit{0.416}$	$\mathit{0.322}$	$\mathit{0.208}$	—
RR-MoA (main)	$0.680$ ( $+63\%$ )	$0.564$ ( $+75\%$ )	$0.276$ ( $+33\%$ )	$+57\%$
Dual-Stream	$\mathbf{0.453{\pm}0.005}$ ( $+9\%$ )	$\mathbf{0.358{\pm}0.002}$ ( $+11\%$ )	$\mathbf{0.198{\pm}0.005}$ ( $\mathbf{-5\%}$ )	$+5\%$
Raw-Input ( $K{=}6$ )	$0.462{\pm}0.025$ ( $+11\%$ )	$0.416{\pm}0.047$ ( $+29\%$ )	$0.205{\pm}0.007$ ( $-2\%$ )	$+13\%$
Multi-Res	$0.460{\pm}0.007$ ( $+11\%$ )	$0.365{\pm}0.008$ ( $+13\%$ )	$0.207{\pm}0.014$ ( $-0.3\%$ )	$+8\%$

Appendix

Table P.1: Closing the DLinear gap (MOMENT-small, strictly frozen, Top-2, 3 seeds, H=96). Dual-stream experts that access both hidden states and raw input narrow the gap from

+33

–

75\%

+9

–

11\%

and beat DLinear on Weather . Learned

\alpha{\approx}0.49

confirms the backbone contributes complementary information when combined with raw access.

Method	ETTh2	ETTm2	Electricity
DLinear (from scratch)	$\mathit{0.341}$	$\mathit{0.200}$	$\mathit{0.158}$
Dual-Stream	$0.452\pm 0.043$ ( $+32\%$ )	$0.252\pm 0.070$ ( $+26\%$ )	$\mathbf{0.171\pm 0.002}$ $\mathbf{(+8\%)}$
Raw-Input Expert	$0.525\pm 0.112$ ( $+54\%$ )	$0.227\pm 0.013$ ( $+13\%$ )	$0.203\pm 0.027$ ( $+28\%$ )
Multi-Resolution	$\mathbf{0.367\pm 0.003}$ $\mathbf{(+7.7\%)}$	$\mathbf{0.223\pm 0.016}$ $\mathbf{(+12\%)}$	$0.179\pm 0.003$ ( $+13\%$ )
FiLM (neg. control)	$0.843\pm 0.289$ ( $+147\%$ )	$0.297\pm 0.013$ ( $+48\%$ )	$0.305\pm 0.014$ ( $+94\%$ )

Appendix

Table P.2: Gap-closing extended to ETTh2, ETTm2, Electricity (MOMENT-small, strictly frozen, Top-2, 3 seeds, H=96, 15 epochs). Bold: best gap-closing variant per dataset. Multi-resolution wins on ETTh2 at

+7.7\%

, the closest new gap-closing result.

Panel of Figure 43 — Figure P.1: Residual-IA + architecture (dual-stream extension). Stream 1 (blue): the frozen TSFM produces hidden states $\mathbf{H}$ , fed into each expert’s Adapter head. Stream 2 (green): a shared NLinear branch reads $\mathbf{X}_{\mathrm{raw}}$ directly, producing $\mathbf{y}_{\mathrm{raw}}$ — DLinear-equivalent and shared across all $K$ experts. Each expert combines them as $\mathbf{y}_{k}=\mathbf{y}_{\mathrm{raw}}+\sigma(\mathrm{blend}_{k}(\mathbf{X}_{\mathrm{raw}}))\cdot\mathrm{Adapter}_{k}(\mathbf{H})$ , with the blend gate initialized at $b{=}{-}2$ so the model starts as effectively-DLinear and learns to add backbone signal where useful. Self-routing $\sigma$ gates (orange) on $\mathbf{X}_{\mathrm{raw}}$ select which experts contribute. Result: 5/6 match-or-beat at $H{=}96$ , $6/6$ at $H{=}192$ on MOMENT-small; 107/123 cells ( $87\%$ ) match-or-beat across 6 backbones $\times$ 6 datasets $\times$ 4 horizons (Tables P.5 , P.8 ).

Config	ETTh1	ETTh2	ETTm1	ETTm2	Weather	Elec.
Dual-stream ( $\alpha_{k}$ )	$0.49$	$0.50$	$0.49$	$0.49$	$0.49$	$0.49$
Residual-IA (gate-init $b{=}0$ )	$0.47$	$0.50$	$0.48$	$0.49$	$0.49$	$0.47$
Residual-IA + cos+wd ( $b{=}0$ )	$0.50$	$0.51$	$0.49$	$0.52$	$0.50$	$0.49$

Appendix

Table P.3: Gate-failure diagnostic. Mean per-sample gate activation

g_{k}(\mathbf{X})

averaged across experts, seeds (3), and test windows for each architecture–training-recipe combination. Across the three rows, all six datasets converge to

g\approx 0.5

, indicating the gate provides no per-window discriminative signal at 15 epochs.

Dataset	DLinear	Residual-IA (ours)	Gap	$p$	Verdict
ETTh1	$0.417\pm 0.003$	$0.438\pm 0.007$ (n=10)	$+5.1\%$	$0.0001$	sig loss
ETTh2	$0.341\pm 0.014$	$0.377\pm 0.018$ (n=10)	$+10.6\%$	$0.019$	sig loss
ETTm1	$0.322\pm 0.005$	$0.328\pm 0.007$ (n=10)	$+1.8\%$	$0.190$	parity
ETTm2	$0.200\pm 0.010$	$\mathbf{0.197\pm 0.004}$ (n=5)	$-1.6\%$	$0.622$	win (mean)
Weather	$0.208\pm 0.004$	$\mathbf{0.195\pm 0.004}$ (n=5)	$-6.6\%$	$0.007^{\ast}$	sig win
Electricity	$0.158\pm 0.002$	$0.159\pm 0.002$ (n=5)	$+0.6\%$	$0.457$	parity

Appendix

Table P.4: Residual-IA (intermediate) (MOMENT-small, frozen, Top-2, H=96; Welch

t

-test vs. DLinear). Achieves 4/6 match-or-beat : one sig win (Weather), one mean win (ETTm2), two parity, two sig losses. Avg gap

+1.6\%

. Table P.5 shows the improved Residual-IA + (5/6).

Dataset	DLinear ( $n{=}10$ )	Res-IA	Res-IA + ( $n{=}10$ )	Gap	$p$	Verdict
ETTh1	$0.419\pm 0.005$	$0.438\pm 0.007$	$0.428\pm 0.012$	$+2.2\%$	$0.037$	sig loss
ETTh2	$0.352\pm 0.016$	$0.377\pm 0.018$	$\mathbf{0.346\pm 0.010}$	$-1.6\%$	$0.444$	win
ETTm1	$0.326\pm 0.006$	$0.328\pm 0.007$	$0.327\pm 0.007$	$+0.4\%$	$0.678$	parity
ETTm2	$0.211\pm 0.016$	$0.197\pm 0.004$	$\mathbf{0.188\pm 0.002}$	$-10.8\%$	$0.002^{\ast}$	sig win
Weather	$0.207\pm 0.003$	$0.195\pm 0.004$	$\mathbf{0.197\pm 0.003}$	$-5.1\%$	${<}0.001^{\ast}$	sig win
Electricity	$0.159\pm 0.002$	$0.159\pm 0.002$	$\mathbf{0.156\pm 0.001}$	$-1.7\%$	$0.002^{\ast}$	sig win

Appendix

Table P.5: Residual-IA + : final six-dataset gap-closing result at H

{=}96

(MOMENT-small, strictly frozen, Top-2,

n{=}10

seeds for both Residual-IA + and DLinear; Welch two-sided

t

-test). Residual-IA + achieves 5/6 match-or-beat with three significant wins (

p{\leq}0.002

): ETTm2 (

-10.8\%

p{=}0.002

), Weather (

-5.1\%

p{<}0.001

), Electricity (

-1.7\%

p{=}0.002

); one mean win (ETTh2

-1.6\%

); one parity (ETTm1

+0.4\%

p{=}0.68

); ETTh1 alone remains

+2.2\%

(

p{=}0.037

). Mean gap:

\mathbf{-2.8\%}

(net win).

Dataset	H ${=}96$	H ${=}192$	H ${=}336$	H ${=}720$	Match-or-beat
ETTh1	$+2.2\%$	$+1.6\%$ parity	$+4.4\%$ marg.	$+12.8\%$ sig loss	$1/4$
ETTh2	$\underline{-1.6\%}$	$\underline{-12.2\%}$	$\mathbf{-16.6\%}$	$\mathbf{-39.6\%}$	$4/4$
ETTm1	$+0.4\%$ parity	$+1.0\%$ parity	$+0.7\%$ parity	$+2.4\%$ sig loss	$3/4$
ETTm2	$\mathbf{-10.8\%}$	$\mathbf{-8.7\%}$	$\mathbf{-13.6\%}$	$\mathbf{-27.0\%}$	$4/4$
Weather	$\mathbf{-5.1\%}$	$\mathbf{-4.0\%}$	$\mathbf{-2.9\%}$	$\underline{-0.4\%}$	$4/4$
Electricity	$\mathbf{-1.7\%}$	$\underline{-0.5\%}$	$\underline{-0.9\%}$	$+1.6\%$ sig loss	$3/4$

Appendix

Table P.6: Multi-horizon gap-closing (MOMENT-small, Res-IA + , frozen,

H{\in}\{96{-}720\}

; Welch

t

-test vs. DLinear). Bold: sig win; underline: mean win. At

H{=}192

: 6/6 match-or-beat . Pooled:

\mathbf{20/24}

. ETTh1 is the only dataset without parity at

H{=}96

, consistent with DLinear’s short-horizon advantage on the smallest LTSF dataset.

Backbone	ETTh1	ETTh2	ETTm1	ETTm2	Weather	Elec	MoB
Moirai	$\mathbf{-9.3\%}$	$-8.5\%$	$\mathbf{-13.3\%}$	$\mathbf{-29.2\%}$	$\mathbf{-29.3\%}$	$\mathbf{-24.8\%}$	$6/6$
Chronos	$\mathbf{-11.0\%}$	$\mathbf{-13.8\%}$	$\mathbf{-12.2\%}$	$\mathbf{-30.5\%}$	$\mathbf{-27.3\%}$	$\mathbf{-28.4\%}$	$6/6$
MM-large	$\mathbf{-9.5\%}$	$\mathbf{-14.8\%}$	$\mathbf{-15.4\%}$	$\mathbf{-30.6\%}$	$\mathbf{-28.2\%}$	$\mathbf{-27.8\%}$	$6/6$
Timer-XL	$\mathbf{-10.4\%}$	—	$\mathbf{-16.8\%}$	—	$\mathbf{-27.9\%}$	—	$3/3$
Moirai-MoE	$\mathbf{-10.3\%}$	$\mathbf{-15.9\%}$	$\mathbf{-14.9\%}$	$\mathbf{-31.7\%}$	$\mathbf{-28.1\%}$	$\mathbf{-28.5\%}$	$6/6$

Appendix

Table P.7: Cross-backbone Residual-IA + at H

{=}96

(strictly frozen, Top-2,

n{=}5

seeds, Welch

t

-test). All five backbones achieve significant wins on all six datasets (

27/27

). Moirai-MoE confirms adapter-level MoE stacks with backbone-internal MoE.

Backbone	Dataset	H ${=}96$	H ${=}192$	H ${=}336$	H ${=}720$	MoB
Moirai	ETTh1	$\mathbf{*{-}9.3}$	$={+}1.4$	X ${+}8.8$	X ${+}19.0$	$2/4$
	ETTh2	$*{-}8.5$	${+}2.5$	${+}43.3$	${+}25.8$	$1/4$
	ETTm1	$\mathbf{*{-}13.3}$	$*{-}0.5$	$*{-}0.9$	X ${+}2.1$	$3/4$
	ETTm2	$\mathbf{*{-}29.2}$	$\mathbf{*{-}8.5}$	$\mathbf{*{-}11.9}$	$*{-}18.8$	$4/4$
	Weather	$\mathbf{*{-}29.3}$	$\mathbf{*{-}2.5}$	$*{-}0.8$	$={+}0.7$	$4/4$
	Elec	$\mathbf{*{-}24.8}$	X ${+}6.5$	X ${+}5.9$	X ${+}7.0$	$1/4$

Appendix

Table P.8: Multi-horizon cross-backbone Residual-IA + (

5

backbones

\times

up to

6

datasets

\times

4

horizons,

n{=}3

–

5

seeds).

*{=}

mean win,

\mathbf{*}{=}

sig win (

p{<}0.05

={=}

parity, X

{=}

sig loss. MOMENT-large achieves

\mathbf{23/24}

(

96\%

); Timer-XL

\mathbf{12/12}

(

100\%

); Moirai-MoE

\mathbf{15/15}

(

100\%

); Chronos

22/24

(

92\%

); Moirai

15/24

(

62\%

). Pooled:

\mathbf{87/99}

Dataset	SR-RIA +	DLinear	RIA +	$\Delta$ vs DL
ETTh1	$0.423\pm 0.004$	$0.416$	$0.428$	$+1.6\%$
ETTh2	$0.345\pm 0.005$	$0.341$	$0.346$	$+1.1\%$
ETTm1	$\mathbf{0.321\pm 0.004}$	$0.322$	$0.327$	$\mathbf{-0.4\%}$
ETTm2	$\mathbf{0.188\pm 0.001}$	$0.200$	$0.188$	$\mathbf{-6.0\%}$
Weather	$\mathbf{0.192\pm 0.002}$	$0.208$	$0.197$	$\mathbf{-7.5\%}$
Electricity	$0.160\pm 0.000$	$0.158$	$0.156$	$+1.2\%$

Appendix

Table P.9: SR-RIA + vs. DLinear (MOMENT-small, frozen,

K{=}5

H{=}96

, 5 seeds). SR-RIA + achieves match-or-beat on 6/6 datasets , with 3 wins (ETTm1

-0.4\%

, ETTm2

-6.0\%

, Weather

-7.5\%

) and 3 within

+1.6\%

Dataset	Raw-MLP MoE	Dual-Stream	DLinear	$\Delta$ vs DS	$\Delta$ vs DL
ETTh1	$0.483\pm 0.019$	$\mathit{0.453}$	$\mathit{0.416}$	$+6.6\%$	$+16.1\%$
ETTh2	$\mathbf{0.426\pm 0.077}$	$\mathit{0.452}$	$\mathit{0.341}$	$\mathbf{-5.7\%}$	$+25.0\%$
ETTm1	$0.370\pm 0.014$	$\mathit{0.358}$	$\mathit{0.322}$	$+3.4\%$	$+15.0\%$
ETTm2	$\mathbf{0.207\pm 0.004}$	$\mathit{0.252}$	$\mathit{0.200}$	$\mathbf{-17.9\%}$	$+3.4\%$
Weather	$0.209\pm 0.007$	$\mathit{0.198}$	$\mathit{0.208}$	$+5.5\%$	$+0.5\%$
Electricity	$\mathbf{0.170\pm 0.002}$	$\mathit{0.171}$	$\mathit{0.158}$	$\mathbf{-0.7\%}$	$+7.5\%$

Appendix

Table P.10: Pure Raw-MLP MoE vs. Dual-Stream and DLinear (5 seeds, mean

\pm

std, H

=

96). The Raw-MLP MoE has no TSFM at all . It matches Dual-Stream within

\pm 7\%

on every dataset, and beats Dual-Stream outright on

\mathbf{3}

\mathbf{6}

datasets (ETTh2, ETTm2, Electricity). Mean delta to Dual-Stream is

-1.5\%

: the TSFM’s contribution is dataset-dependent and negligible in the average case.

Explore similar work

Jun 10, 2026cs.LG

TimeRouter: Efficient and Adaptive Routing of Time-Series Foundation Models

Time-series foundation models (TSFMs) are increasingly explored as predictive experts within emerging agentic time-series systems. However, TSFMs exhibit heterogeneous inductive biases, and no single model consistently dominates across forecasting regimes, making expert selection a critical challenge. Existing systems often delegate this decision to LLM-based controllers, incurring substantial inference overhead. We present TimeRouter, an efficient routing framework that leverages empirical complementarity across a pool of pretrained TSFMs through lightweight discriminative routing, selective gating, and ensemble fallback. Concretely, TimeRouter combines a learned routing head, a selective gate, and an ensemble fallback, enabling adaptive expert selection without invoking an LLM at inference time. TimeRouter achieves state-of-the-art performance on the GIFT-EVAL leaderboard, with an LB MASE of 0.6765. Beyond benchmark performance, our ablation studies provide empirical insights into TSFM routing design, highlighting the importance of pool composition and selective gating. Taken together, these results position TimeRouter as a modular and lightweight routing layer for future agentic time-series systems built upon foundation-model pools. Our code is available at https://github.com/UConn-DSIS/TimeRouter.

Kanghui Ning, Yushan Jiang, Kashif Rasul +3

School of Computing, University of Connecticut, Storrs, USA · Department of Machine Learning Research, Morgan Stanley, New York, USA.

Time Series Foundation Models Timestep Embeddings Wireless Foundation Models Foundation Model Gating

Ask PDF

May 24, 2026cs.LG

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting

Time series forecasting models are increasingly scaled through large Transformer backbones, yet most existing approaches process all series through a shared dense computation path despite substantial heterogeneity in temporal structure. Mixture-of-Experts (MoE) offers a natural alternative by enabling conditional computation, but standard MoE routing leaves expert specialization weakly identified and often unstable during downstream adaptation. We propose AME-TS, a structure-guided sparse time series foundation model that aligns expert routing with interpretable temporal structure. AME-TS first uses a lightweight regime predictor to estimate series-level descriptors, including forecastability, seasonality, trend, and sparsity, and maps them to a soft structural prior over experts. This series-level prior guides token-level routing during training, encouraging structure-aligned specialization. On the GIFT-Eval benchmark, AME-TS delivers a strong accuracy-efficiency tradeoff across model scales: it substantially outperforms existing time series foundation models at small model scales and remains competitive with the strongest models at larger scales, while activating substantially fewer parameters through sparse routing. We further show that AME-TS learns more interpretable routing geometry and substantially more stable expert specialization than standard MoE during fine-tuning on the M5 dataset. These results suggest that structure-aware routing is an effective and reliable way to realize the benefits of sparse expert models for time series forecasting.

Rui Wang, Renhao Xue, Ray Razi +2

Amazon Web Services

Time Series Foundation Models Time Series Mixture-Of-Experts Timestep Embeddings

Ask PDF

Jul 22, 2026cs.LG

Post-Training in Time Series Foundation Models: A Unifying Framework

Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment. Bridging this gap requires further intervention to handle domain shift, task heterogeneity, limited supervision, and computational constraints, which motivates post-training as a broad class of methods to adapt, augment, compose, calibrate, or specialize pretrained TSFMs for downstream tasks. In this work, we analyze TSFM post-training methods based on their locus of intervention in the prediction pipeline, yielding five categories: parameter adaptation, context augmentation, model composition, output processing and uncertainty control, and compression and specialization. Within each category, we study main representative methods and discuss their current limitations. We further identify future directions toward controlled adaptation, reliable context construction, uncertainty-aware model composition, calibrated output processing, and deployment-aware specialization. Overall, by providing a unifying framework for the emerging TSFM post-training landscape, this work aims to support future research to navigate the design space between a pretrained TSFM and its reliable downstream deployment.

Shifeng Xie, Ambroise Odonnat, Zehao Xiao +7

Noah’s Ark Lab · Université Paris Cité · Inria +2

Time Series Foundation Models Pretraining Post-Training Uncertainty-Aware Classification

Ask PDF