CATF: A Manager-Worker Framework for Context-Aware Time-Series Forecasting

Yueqiao Chen; Dongyu Liu

cs.LGNov 3, 2026

CATF: A Manager-Worker Framework for Context-Aware Time-Series Forecasting

Authors: Yueqiao Chen, Dongyu Liu

Organizations: UC Davis, Davis, CA, USA

Abstract

We study context-aware time-series forecasting, in which a single target variable is predicted from its own history and a set of contextual variables, such as weather and calendar information. As the context becomes more diverse, a single model struggles to capture all of its regimes, while ensemble learning and Mixture-of-Experts approaches rely on manual data partitioning or soft routing, which increases the inference cost and hinders interpretability. We propose CATF, a Context-Aware Time-Series Forecasting framework built on a Manager-Worker architecture: a Transformer-based Manager routes each input to a single Worker based on the context, using hard routing during both training and inference. The Workers thus specialize without manual data partitioning, and any single-model forecaster can serve as a Worker. On five context-rich datasets, four horizons, and two normalization schemes, CATF improves over its standalone counterpart in 162 of 200 comparisons, the best CATF-enhanced model outperforms the best standalone baseline in 19 of 20 settings, and it outperforms the zero-shot Time-MoE foundation model in 15 of 20 settings. A case study on bike-sharing data shows that the Manager routes inputs by the calendar and that each Worker specializes in a different weekly pattern.

Figures & tables

Figure 1. Manager–Worker architecture of CATF . (a) Given the target history $X$ and the context history $C$ , the manager $\mathcal{M}$ outputs routing probabilities $\hat{P}$ over the $K$ workers. The worker with the highest probability, $\mathcal{W}_{k^{*}}$ , receives the same input $(X,C)$ and alone predicts the future target sequence $\hat{Y}$ . (b) Inside the manager: feature encoding (L1), $L_{m}$ Transformer encoder layers (L2), last-timestep pooling, a decoder (L3), and softmax (L4). Two-panel architecture diagram. The upper panel shows the target history and the context history entering a manager that selects one worker by the largest routing probability. The same historical input is passed directly to that worker, which alone produces the target forecast. The lower panel expands the manager into feature encoding, Transformer encoders, last-timestep pooling, decoder, and softmax.

Figure 2. Iterative training of CATF , illustrated for one sample. (a) All workers first produce forecasts and losses (steps 1–2). (b) Their losses and training volumes define a pseudo target (step 3) that supervises the manager update (step 4). (c) The updated manager then selects one worker (step 5), and only that worker is trained on the sample (step 6). Three-phase training diagram with six chronological steps. All workers first predict and are scored against the target. Worker losses and training volumes define a pseudo target that supervises the manager. The updated manager then selects one worker for the sample, and only that worker receives a forecasting update. A return arrow repeats the process with updated workers and training volumes.

Dataset	$L_{y}$	iTransformer ( ( 2024a ) )				TimesNet ( ( 2023a ) )				FEDformer ( ( 2022 ) )				Autoformer ( ( 2021 ) )				Informer ( ( 2021 ) )
		Global		Sample		Global		Sample		Global		Sample		Global		Sample		Global		Sample
		Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF	Base	+ CATF
Weather	96	204.7	194.9	447.0	178.3	245.6	215.9	418.6	158.5	28,741	351.3	1,208,418	181.7	39,860	419.5	1,933,907	210.1	1,733	442.7	4,402	155.7
	192	249.3	226.7	637.6	231.6	271.1	234.8	410.6	226.1	28,712	548.9	10,628,690	236.8	230,293	380.4	151,937	285.4	6,645	426.5	7,309	220.8
	336	254.6	232.3	749.2	254.8	264.8	243.3	456.7	241.8	50,859	559.7	380,781	267.3	70,468	279.8	157,994	270.1	77,472	460.2	3,729	240.1
	720	270.1	235.9	748.3	330.0	269.1	241.7	963.3	291.8	41,001	640.8	798,364	268.3	209,899	259.7	29,367	296.1	4,335	527.5	3,637	290.7

Table 1. Forecasting results (MSE on the original scale, mean over 5 runs) of each baseline (Base) and its CATF -enhanced version (+ CATF ) under global and sample normalization; look-back length

L=96

. The better of each baseline and its CATF -enhanced version is in bold ; the best result in each row is marked with a dark orange background and the second best with a light orange background. CATF Win: number of the 20 settings (5 datasets

\times

4 horizons) in which CATF -

<

baseline

>

has a lower mean MSE than

<

baseline

>

. Impr.: relative MSE reduction of CATF -

<

baseline

>

over

<

baseline

>

, averaged over the 20 settings. The very large MSE of some baselines on Weather is caused by missing-value codes in the training target (Appendix B.6 ). Mean

\pm

SD for every setting is listed in Appendix Tables 9 and 10 .

Dataset	$L_{y}$	Best baseline		Best CATF -enhanced		Time-MoE ( ( 2025 ) )
Dataset	$L_{y}$	Model	MSE	Model	MSE	MSE
Weather	96	iTransformer	204.7 G	CATF -Informer	155.7 S	164.9 S
	192	iTransformer	249.3 G	CATF -Informer	220.8 S	215.0 S
	336	iTransformer	254.6 G	CATF -iTransformer	232.3 G	225.6 S
	720	TimesNet	269.1 G	CATF -iTransformer	235.9 G	233.5 S
PM2.5-LA	96	iTransformer	36.08 S	CATF -iTransformer	35.20 S	39.66 S

Table 2. Best standalone baseline vs. best CATF -enhanced model vs. zero-shot Time-MoE ( Shi et al., 2025 ) . For each setting, the best baseline (best CATF -enhanced model) is the lowest mean MSE over the five baselines ( CATF -enhanced models) and the two normalization schemes; for Time-MoE (single deterministic run), the better of the two normalization schemes is used. The superscript gives the normalization (G: global, S: sample). The best result in each row is in bold with a light orange background. CATF Win: number of the 20 settings in which the best CATF -enhanced model has a lower MSE than the best baseline (Time-MoE); Impr.: relative MSE reduction of the best CATF -enhanced model over the best baseline (Time-MoE), averaged over the 20 settings.

Setting		MSE	$\Delta$ (%)	$p$
Default		80.52 $\pm$ 1.02	–	–
Loss	KL only	81.87 $\pm$ 2.75	$+1.7$	0.347
Loss	Wasserstein only	81.56 $\pm$ 1.10	$+1.3$	0.157
Reg.	No reg.	80.94 $\pm$ 1.19	$+0.5$	0.559
Manager	Small ( $d_{model}{=}128$ )	81.23 $\pm$ 0.78	$+0.9$	0.252
Manager	Large ( $d_{model}{=}768$ )	81.05 $\pm$ 0.81	$+0.7$	0.386

Table 3. Ablation study of CATF -iTransformer on PM2.5-RV (global normalization,

L=L_{y}=96

). MSE is mean

\pm

SD over 5 runs without fixed random seeds; the default setting is an independent re-run of the configuration in Table 1 (MSE

81.07\pm 1.18

there).

\Delta

is the relative change in mean MSE w.r.t. the default (positive is worse);

p

is from Welch’s

t

-test against the default (bold:

p<0.05

). The results on Bike Share are reported in Appendix Table 17 .

Figure 3. Routing of CATF -iTransformer on Bike Share (mean over 5 runs). (a) Test MSE of the same trained workers under the learned routing, the best single worker for all samples, an untrained manager, and random routing with the same worker shares. (b) Share of routing decisions that change when one variable is shuffled across test samples in the manager input only (top 8 of 12 inputs; “All context” shuffles all context variables at once), for the trained and an untrained manager. (c) MSE of each worker on the samples routed to each worker, divided by the lowest MSE in the row and averaged over runs.

Figure 4. What each worker of CATF -iTransformer handles on Bike Share (one run). (a) Share of the forecasts made on each day of the week that are routed to each worker. (b) Mean bike rentals in the 96-hour forecast window for the forecasts made at midnight, grouped by the selected worker. $W_{1}$ handles the forecasts made on Sunday to Tuesday, whose window consists mostly of working days (29 forecasts, on average 3.4 working days in the window); $W_{2}$ handles the forecasts made on Wednesday to Friday, whose window reaches into the weekend (29 forecasts, 2.3 working days); $W_{3}$ handles the forecasts made on Friday and Saturday, whose window starts with the weekend (11 forecasts, 1.9 working days).

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Variables	Target	Input Length ( $L$ )	Output Length ( $L_{y}$ )	Frequency	Dataset Size	Domain
Weather	21	CO 2 (ppm)	96	{96, 192, 336, 720}	10 minutes	52,696	Meteorology
PM2.5-LA	15	PM2.5 ( $\mu$ g/m 3 )	96	{96, 192, 336, 720}	1 hour	35,063	Air quality
PM2.5-RV	15	PM2.5 ( $\mu$ g/m 3 )	96	{96, 192, 336, 720}	1 hour	35,063	Air quality
Bike Share	12	Rental count	96	{96, 192, 336, 720}	1 hour	17,379	Transportation
Appliances	26	Energy use (Wh)	96	{96, 192, 336, 720}	10 minutes	19,735	Energy

Appendix

Table 4. Statistics of the datasets used in the forecasting tasks. #Variables counts all numeric columns, including the target. Dataset Size is the number of time steps.

Model	#Layers	#Model Dimension	#Feedforward Dimension	#Attention Heads	#Dropout	#Attention Factor	#Learning Rate
i-Transformer	2	128	128	8	0.1	3	0.0001
TimesNet	2	32	32	8	0.1	3	0.0001
FEDformer	2	512	2048	8	0.1	3	0.0001
Autoformer	2	512	2048	8	0.1	3	0.0001
Informer	2	512	2048	8	0.1	3	0.0001

Appendix

Table 5. Summary of worker models and baselines’ hyperparameters used in our CATF framework.

Model Type	#Layers	#Model Dimension	#Feedforward Dimension	#Attention Heads	#Dropout
Transformer	2	512	2048	8	0.05

Appendix

Table 6. Default setting of the manager model hyperparameter that is used in our CATF framework.

Model	Manager LR	Worker LR
CATF-iTransformer	0.00001	0.0001
CATF-TimesNet	0.00001	0.0001
CATF-FEDformer	0.001	0.0001
CATF-Autoformer	0.00001	0.0001
CATF-Informer	0.00001	0.0001

Appendix

Table 7. Learning rates of the manager and worker models in CATF -enhanced models (identical for all datasets). Both are optimized with Adam and a weight decay of

10^{-5}

Shared Settings
Hyperparameter	Baseline	CATF Models
Batch Size	32	32
Training Size	0.8	0.8
Validation Size	0.1	0.1
Test Size	0.1	0.1
Epochs	10	30

Appendix

Table 8. Training hyperparameters for CATF and Baseline models.

Dataset	$L_{y}$	iTransformer ( ( 2024a ) )	CATF - iTransformer	TimesNet ( ( 2023a ) )	CATF - TimesNet	FEDformer ( ( 2022 ) )	CATF - FEDformer	Autoformer ( ( 2021 ) )	CATF - Autoformer	Informer ( ( 2021 ) )	CATF - Informer
Weather	96	204.7 $\pm$ 2.29	194.9 $\pm$ 5.19	245.6 $\pm$ 10.69	215.9 $\pm$ 21.53	28,741 $\pm$ 16,160	351.3 $\pm$ 97.78	39,860 $\pm$ 35,693	419.5 $\pm$ 104.1	1,733 $\pm$ 830.7	442.7 $\pm$ 35.52
	192	249.3 $\pm$ 3.11	226.7 $\pm$ 8.12	271.1 $\pm$ 13.23	234.8 $\pm$ 8.91	28,712 $\pm$ 19,726	548.9 $\pm$ 19.67	230,293 $\pm$ 340,357	380.4 $\pm$ 53.86	6,645 $\pm$ 10,171	426.5 $\pm$ 44.44
	336	254.6 $\pm$ 2.15	232.3 $\pm$ 3.39	264.8 $\pm$ 3.40	243.3 $\pm$ 6.20	50,859 $\pm$ 39,591	559.7 $\pm$ 79.00	70,468 $\pm$ 97,935	279.8 $\pm$ 13.24	77,472 $\pm$ 170,210	460.2 $\pm$ 41.99
	720	270.1 $\pm$ 2.85	235.9 $\pm$ 6.54	269.1 $\pm$ 5.97	241.7 $\pm$ 4.23	41,001 $\pm$ 22,172	640.8 $\pm$ 66.33	209,899 $\pm$ 267,336	259.7 $\pm$ 12.00	4,335 $\pm$ 4,652	527.5 $\pm$ 59.48
PM2.5-LA	96	37.05 $\pm$ 0.39	37.02 $\pm$ 0.28	37.32 $\pm$ 1.68	44.42 $\pm$ 3.64	57.75 $\pm$ 4.31	42.29 $\pm$ 1.29	86.25 $\pm$ 24.26	71.18 $\pm$ 6.13	45.34 $\pm$ 4.37	45.04 $\pm$ 6.22
	192	40.64 $\pm$ 0.30	41.17 $\pm$ 0.24	40.16 $\pm$ 1.94	42.52 $\pm$ 2.36	65.63 $\pm$ 1.73	45.25 $\pm$ 0.86	66.43 $\pm$ 8.03	64.94 $\pm$ 4.65	46.80 $\pm$ 3.08	42.66 $\pm$ 0.25

Appendix

Table 9. Long-term forecasting results under Global normalization. MSE is computed on the original (de-normalized) scale and reported as MEAN

\pm

SD over 5 runs. The look-back length is 96 for all settings. The better of each baseline and its CATF -enhanced version is in bold ; the best result in each row is marked with a dark orange background and the second best with a light orange background. CATF Win: number of the 20 settings (5 datasets

\times

4 horizons) in which CATF -

<

baseline

>

has a lower mean MSE than

<

baseline

>

. Impr.: relative MSE reduction of CATF -

<

baseline

>

over

<

baseline

>

, averaged over the 20 settings.

Dataset	$L_{y}$	iTransformer ( ( 2024a ) )	CATF - iTransformer	TimesNet ( ( 2023a ) )	CATF - TimesNet	FEDformer ( ( 2022 ) )	CATF - FEDformer	Autoformer ( ( 2021 ) )	CATF - Autoformer	Informer ( ( 2021 ) )	CATF - Informer
Weather	96	447.0 $\pm$ 47.15	178.3 $\pm$ 9.00	418.6 $\pm$ 150.0	158.5 $\pm$ 10.52	1,208,418 $\pm$ 1,398,675	181.7 $\pm$ 6.57	1,933,907 $\pm$ 1,800,987	210.1 $\pm$ 24.66	4,402 $\pm$ 3,262	155.7 $\pm$ 6.54
	192	637.6 $\pm$ 71.19	231.6 $\pm$ 4.39	410.6 $\pm$ 120.8	226.1 $\pm$ 6.49	10,628,690 $\pm$ 15,360,229	236.8 $\pm$ 9.04	151,937 $\pm$ 325,720	285.4 $\pm$ 31.47	7,309 $\pm$ 2,449	220.8 $\pm$ 15.75
	336	749.2 $\pm$ 64.09	254.8 $\pm$ 8.98	456.7 $\pm$ 167.3	241.8 $\pm$ 6.77	380,781 $\pm$ 420,438	267.3 $\pm$ 16.56	157,994 $\pm$ 251,704	270.1 $\pm$ 12.01	3,729 $\pm$ 1,411	240.1 $\pm$ 5.54
	720	748.3 $\pm$ 46.15	330.0 $\pm$ 35.88	963.3 $\pm$ 360.3	291.8 $\pm$ 17.00	798,364 $\pm$ 652,673	268.3 $\pm$ 18.87	29,367 $\pm$ 28,419	296.1 $\pm$ 33.97	3,637 $\pm$ 1,223	290.7 $\pm$ 16.44
PM2.5-LA	96	36.08 $\pm$ 0.96	35.20 $\pm$ 0.96	40.18 $\pm$ 4.20	35.43 $\pm$ 1.30	60.91 $\pm$ 9.88	42.11 $\pm$ 2.02	57.60 $\pm$ 6.91	58.89 $\pm$ 3.52	41.30 $\pm$ 2.24	37.90 $\pm$ 1.28
	192	40.93 $\pm$ 0.48	40.81 $\pm$ 0.46	42.62 $\pm$ 3.67	39.25 $\pm$ 1.73	64.07 $\pm$ 2.94	43.77 $\pm$ 1.81	61.37 $\pm$ 14.40	65.62 $\pm$ 8.96	42.06 $\pm$ 1.10	39.62 $\pm$ 0.90

Appendix

Table 10. Long-term forecasting results under Sample normalization. MSE is computed on the original (de-normalized) scale and reported as MEAN

\pm

\times

4 horizons) in which CATF -

<

baseline

>

has a lower mean MSE than

<

baseline

>

. Impr.: relative MSE reduction of CATF -

<

baseline

>

over

<

baseline

>

, averaged over the 20 settings.

Dataset	$L_{y}$	iTransformer		TimesNet		FEDformer		Autoformer		Informer
		G	S	G	S	G	S	G	S	G	S
Weather	96	+4.8	+60.1	+12.1	+62.1	+98.8	+100.0	+98.9	+100.0	+74.5	+96.5
	192	+9.1	+63.7	+13.4	+44.9	+98.1	+100.0	+99.8	+99.8	+93.6	+97.0
	336	+8.7	+66.0	+8.1	+47.1	+98.9	+99.9	+99.6	+99.8	+99.4	+93.6
	720	+12.7	+55.9	+10.2	+69.7	+98.4	+100.0	+99.9	+99.0	+87.8	+92.0
PM2.5-LA	96	+0.1	+2.4	-19.0	+11.8	+26.8	+30.9	+17.5	-2.3	+0.7	+8.2

Appendix

Table 11. MSE reduction (%) of CATF -

<

baseline

>

over

<

baseline

>

, computed on the mean MSE over 5 runs as (baseline

-

CATF )/baseline. Positive values mean CATF is better. G = global normalization, S = sample normalization. Significance is tested with a two-sided Welch’s

t

-test over the 5 runs of each model: bold = CATF is significantly better (

p<0.05

), underlined = CATF is significantly worse (

p<0.05

Dataset

L_{y}

( Liu et al., 2024a ) iTransformer

CATF - iTransformer

( Wu et al., 2023a ) TimesNet

CATF - TimesNet

( Zhou et al., 2022 ) FEDformer

CATF - FEDformer

( Wu et al., 2021 ) Autoformer

CATF - Autoformer

( Zhou et al., 2021 ) Informer

CATF - Informer

TimeMoE (zero-shot)

Weather

10.31

\pm

0.07

10.05

\pm

0.16

11.41

\pm

0.25

10.23

\pm

0.49

133.1

\pm

42.16

14.26

\pm

1.53

154.9

\pm

88.36

15.97

\pm

2.15

32.48

\pm

8.31

16.52

\pm

0.83

21.61

192

11.64

\pm

0.07

11.04

\pm

0.17

12.16

\pm

0.27

11.30

\pm

0.17

133.1

\pm

64.82

18.50

\pm

0.29

241.3

\pm

233.3

14.93

\pm

1.06

32.27

\pm

12.88

15.93

\pm

1.12

24.60

336

12.03

\pm

0.05

11.43

\pm

0.11

12.25

\pm

0.08

11.72

\pm

0.10

195.6

\pm

75.97

18.37

\pm

1.31

150.0

\pm

121.0

12.79

\pm

0.34

46.55

\pm

63.01

16.70

\pm

0.97

26.44

720

12.71

\pm

0.08

11.74

\pm

0.20

12.64

\pm

0.17

11.92

\pm

0.11

174.9

\pm

58.48

20.08

\pm

1.08

274.6

\pm

233.7

12.26

\pm

0.26

34.14

\pm

6.19

18.16

\pm

1.20

28.63

PM2.5-LA

3.91

\pm

0.01

3.93

\pm

0.00

3.91

\pm

0.06

4.13

\pm

0.19

5.77

\pm

0.24

4.72

\pm

0.07

6.80

\pm

1.00

6.27

\pm

0.26

5.07

\pm

0.26

4.97

\pm

0.22

4.41

192

4.20

\pm

0.02

4.24

\pm

0.02

4.18

\pm

0.09

4.24

\pm

0.09

6.19

\pm

0.09

4.88

\pm

0.05

6.02

\pm

0.39

5.99

\pm

0.22

5.21

\pm

0.14

4.96

\pm

0.05

4.92

Appendix

Table 12. Long-term forecasting results under Global normalization. MAE is computed on the original (de-normalized) scale and reported as MEAN

\pm

SD over 5 runs (TimeMoE: single zero-shot run). The look-back length is 96 for all settings. The better of each baseline and its CATF -enhanced version is in bold ; the best result in each row is marked with a dark orange background and the second best with a light orange background. CATF Win: number of the 20 settings (5 datasets

\times

4 horizons) in which CATF -<baseline> has a lower mean MAE than <baseline>. Impr.: relative MAE reduction of CATF -<baseline> over <baseline>, averaged over the 20 settings.

Dataset

L_{y}

( Liu et al., 2024a ) iTransformer

CATF - iTransformer

( Wu et al., 2023a ) TimesNet

CATF - TimesNet

( Zhou et al., 2022 ) FEDformer

CATF - FEDformer

( Wu et al., 2021 ) Autoformer

CATF - Autoformer

( Zhou et al., 2021 ) Informer

CATF - Informer

TimeMoE (zero-shot)

Weather

14.13

\pm

0.67

9.28

\pm

0.20

11.84

\pm

0.57

8.61

\pm

0.25

725.8

\pm

476.1

9.45

\pm

0.19

983.5

\pm

593.3

10.04

\pm

0.43

28.87

\pm

8.79

8.53

\pm

0.20

9.05

192

17.43

\pm

1.09

10.89

\pm

0.12

12.80

\pm

0.83

10.71

\pm

0.10

1,691

\pm

2,008

11.22

\pm

0.19

185.3

\pm

305.8

12.06

\pm

0.46

32.24

\pm

6.16

10.65

\pm

0.25

10.63

336

19.16

\pm

0.73

11.76

\pm

0.17

13.89

\pm

1.76

11.44

\pm

0.19

380.7

\pm

239.9

12.32

\pm

0.37

194.7

\pm

123.2

12.12

\pm

0.22

29.58

\pm

7.54

11.47

\pm

0.10

11.14

720

19.89

\pm

0.54

13.34

\pm

0.57

17.53

\pm

2.17

12.36

\pm

0.35

588.6

\pm

274.4

12.52

\pm

0.49

118.6

\pm

83.34

13.18

\pm

0.89

28.66

\pm

4.30

12.62

\pm

0.35

11.55

PM2.5-LA

3.93

\pm

0.06

3.87

\pm

0.07

4.02

\pm

0.12

3.85

\pm

0.06

5.21

\pm

0.46

4.24

\pm

0.12

4.89

\pm

0.24

5.07

\pm

0.20

4.33

\pm

0.18

4.12

\pm

0.10

4.10

192

4.25

\pm

0.04

4.26

\pm

0.03

4.26

\pm

0.15

4.14

\pm

0.06

5.45

\pm

0.10

4.39

\pm

0.12

5.13

\pm

0.48

5.37

\pm

0.31

4.40

\pm

0.07

4.26

\pm

0.07

4.41

Appendix

Table 13. Long-term forecasting results under Sample normalization. MAE is computed on the original (de-normalized) scale and reported as MEAN

\pm

\times

4 horizons) in which CATF -<baseline> has a lower mean MAE than <baseline>. Impr.: relative MAE reduction of CATF -<baseline> over <baseline>, averaged over the 20 settings.

Model	Weather	PM2.5-LA	PM2.5-RV	Bike Share	Appliances	#Global better	Median (all)
iTransformer	$+166$	$+0.5$	$-1.5$	$-2.1$	$+97$	11/20	$+0.5$
TimesNet	$+71$	$+3.6$	$+1.5$	$-5.7$	$+112$	13/20	$+6.9$
FEDformer	$+2{,}976$	$+5.7$	$+0.4$	$+52$	$+249$	17/20	$+52$
Autoformer	$+45$	$-3.6$	$+3.5$	$+11$	$+471$	14/20	$+14$
Informer	$-3.1$	$+0.3$	$+3.5$	$-0.1$	$+405$	14/20	$+5.5$
CATF -iTransformer	$+5.9$	$-0.7$	$-1.3$	$-1.4$	$+35$	9/20	$-0.4$

Appendix

Table 14. Global vs. sample normalization. For each model and dataset, the cell is the median over the four horizons of the relative difference in mean MSE, (sample

-

global)/global (%); positive values mean that global normalization is better. The last two columns give the number of the 20 settings (5 datasets

\times

4 horizons) in which global normalization is better, and the median relative difference over all 20 settings.

		iTransformer		TimesNet		FEDformer		Autoformer		Informer
Dataset	$L_{y}$	Base	CATF	Base	CATF	Base	CATF	Base	CATF	Base	CATF
Weather	96	204.7 G	178.3 S	245.6 G	158.5 S	28,741 G	181.7 S	39,860 G	210.1 S	1,733 G	155.7 S
	192	249.3 G	226.7 G	271.1 G	226.1 S	28,712 G	236.8 S	151,937 S	285.4 S	6,645 G	220.8 S
	336	254.6 G	232.3 G	264.8 G	241.8 S	50,859 G	267.3 S	70,468 G	270.1 S	3,729 S	240.1 S
	720	270.1 G	235.9 G	269.1 G	241.7 G	41,001 G	268.3 S	29,367 S	259.7 G	3,637 S	290.7 S
PM2.5-LA	96	36.08 S	35.20 S	37.32 G	35.43 S	57.75 G	42.11 S	57.60 S	58.89 S	41.30 S	37.90 S

Appendix

Table 15. Best normalization per model. For each model, we take the lower mean MSE of global (G) and sample (S) normalization (Tables 9 and 10 ), separately for the standalone baseline (Base) and its CATF -enhanced version; the superscript gives the selected normalization. Bold marks the better of each baseline and its CATF -enhanced version. CATF Win: number of the 20 settings in which CATF improves over its baseline; Impr.: relative MSE reduction (%), averaged over the 20 settings.

	$L_{y}=96$		$L_{y}=192$		$L_{y}=336$		$L_{y}=720$
Model	Time (s/ep.)	Mem. (GB)	Time (s/ep.)	Mem. (GB)	Time (s/ep.)	Mem. (GB)	Time (s/ep.)	Mem. (GB)	Epochs
iTransformer	17	0.04	19	0.04	33	0.09	23	0.05	7.1
CATF -iTransformer	283	0.46	239	0.46	398	0.48	250	0.48	26.7
TimesNet	76	0.13	95	0.15	728	0.20	205	0.29	7.0
CATF -TimesNet	725	0.57	559	0.65	2,227	0.79	922	1.00	26.1
FEDformer	301	1.62	329	1.78	1,443	2.07	499	2.68	6.7

Appendix

Table 16. Computational cost of the baselines and CATF -enhanced models (3 workers): training time per epoch (s), peak GPU memory during training (GB), and the average number of trained epochs, for each forecasting horizon

L_{y}

. Values are averaged over the 5 datasets, 2 normalization schemes, and 5 runs. Several runs shared one GPU, so the times are indicative. For the zero-shot Time-MoE (50M), we report the total inference time on the test split and the peak memory.

		Bike Share				PM2.5-RV
Setting		MSE	MAE	$\Delta$ (%)	$p$	MSE	MAE	$\Delta$ (%)	$p$	#Params (M)
Default (KL + Wass., reg., $d_{model}{=}512$ , $K{=}3$ )		15,097 $\pm$ 241	78.56 $\pm$ 0.67	–	–	80.52 $\pm$ 1.02	6.18 $\pm$ 0.03	–	–	7.3
Manager loss	KL only	15,452 $\pm$ 282	79.36 $\pm$ 0.71	$+2.4$	0.065	81.87 $\pm$ 2.75	6.15 $\pm$ 0.03	$+1.7$	0.347	7.3
Manager loss	Wasserstein only	15,407 $\pm$ 372	80.48 $\pm$ 0.83	$+2.1$	0.163	81.56 $\pm$ 1.10	6.21 $\pm$ 0.04	$+1.3$	0.157	7.3
Regularization	No reg.	15,169 $\pm$ 184	79.01 $\pm$ 0.40	$+0.5$	0.609	80.94 $\pm$ 1.19	6.14 $\pm$ 0.03	$+0.5$	0.559	7.3
Manager arch.	Small ( $d_{model}{=}128$ , $d_{ff}{=}512$ , 4 heads)	15,155 $\pm$ 298	79.32 $\pm$ 0.77	$+0.4$	0.743	81.23 $\pm$ 0.78	6.19 $\pm$ 0.04	$+0.9$	0.252	1.1

Appendix

Table 17. Full ablation results of CATF -iTransformer on Bike Share and PM2.5-RV (global normalization,

L=L_{y}=96

). MSE and MAE are on the original scale, mean

\pm

SD over 5 runs without fixed random seeds.

\Delta

is the relative change in mean MSE w.r.t. the default setting of the same dataset (positive is worse);

p

is from Welch’s

t

-test against the default (bold:

p<0.05

). #Params counts the manager and all workers on Bike Share.

Figure 5. Routing on synthetic data over the first 600 test steps (one run). Top: target series. Middle: true regime of the forecast window. Bottom: worker selected by the manager. The selected worker follows the regime switches; the agreement is 99.4% over the whole test set.

Figure 6. Routing on synthetic data, averaged over 3 runs. (a) Share of the samples of each true regime that the trained manager routes to each worker. (b) The same for an untrained manager with the same architecture and the same trained workers. (c) MSE of each worker on each regime. (d) Permutation importance: share of routing decisions that change when one input variable is shuffled across test samples in the manager input only (workers receive the original input), for the trained and the untrained manager. Workers are relabeled in each run so that $W_{k}$ is the worker that receives regime $k$ most often.

Same trained workers, different routing
Model / routing	MSE	NMI
iTransformer (baseline)	35.98 $\pm$ 0.34	–
CATF -iTransformer	33.39 $\pm$ 0.51	0.85 $\pm$ 0.15
Random routing (same worker shares)	50.69 $\pm$ 4.18	–
Best single worker for all samples	43.77 $\pm$ 1.76	–
Untrained manager	58.13 $\pm$ 6.06	0.04 $\pm$ 0.04

Appendix

Table 18. Synthetic data (

L=L_{y}=96

, global normalization, 3 runs, mean

\pm

SD). The routing variants reuse the trained workers and change only the routing. NMI: between the routing and the true regime.

Figure 7. Counterfactual analysis of the routing of CATF -iTransformer on Bike Share (mean over 5 runs). For each shift

d

, either the calendar of the input window (day of the week and working-day indicator) is shifted by

d

days (blue) or the target history is replaced with that of the forecast made

d

days later (orange), with all other inputs unchanged. (a) Percentage of test forecasts whose selected worker differs from the one selected for the original input. (b) Percentage of test forecasts whose selected worker is the typical worker of the new day, i.e., the majority worker of the real forecasts made

d

days later at the same hour; the dashed line is this percentage for the original input before any change.

Figure 8. Routing of CATF -iTransformer on Bike Share over four weeks of the test period (one run). Top: hourly bike rentals, with weekends in gray and holidays in red. Bottom: the worker selected for the forecast that starts at each hour. The routing follows the weekly cycle:

W_{1}

for forecasts made on Sunday to Tuesday,

W_{2}

for Wednesday to Friday, and

W_{3}

for Friday and Saturday. Thanksgiving (Thursday, Nov 22) is routed to

W_{3}

, like a Saturday.

Figure 9. Context conditions of the Bike Share test samples routed to each worker (one run; the same run as Figure 8 ). Each line is the mean over the samples routed to a worker at each hour relative to the forecast origin (dashed: all samples); the gray area is the forecast window, which the manager does not see. Rentals are shown as a trailing 24-hour mean. The workers correspond to different phases of the week (working-day share) and therefore to different demand trajectories, whereas the weather conditions are similar for all workers.

Figure 10. Routing of the Bike Share test forecasts by each input variable (one run; the same run as Figure 9 ). The variables are taken at the forecast origin; continuous variables are split into tertiles over the test period, and the number of forecasts in each bin is given in parentheses. Each row gives the percentage of the forecasts in the bin that are routed to each worker (rows sum to 100%; first row: all forecasts). Left: day of the week, working day, holiday, and month; right: weather and recent demand. The column “Variable (NMI)” gives the variable of each group of rows and, in parentheses, the normalized mutual information (NMI) between its bin and the selected worker (0: unrelated; 1: the bin determines the worker). The routing depends strongly on the day of the week and the working-day indicator, and only weakly on the other variables.

Figure 11. Example forecasts on Bike Share for typical forecast origins of each worker (one run; the same run as Figure 8 ), together with the standalone iTransformer. For each case, the forecast with the median CATF error among all forecasts starting at midnight on that day of the week (without holidays in the forecast window) is shown. The gray area is the forecast window. The largest difference occurs for the forecast starting on Saturday, for which

W_{3}

captures the midday weekend peak that the standalone iTransformer underestimates.

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 894 papers

Aug 4, 2026

CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting

Time series forecasting is fundamental to decision-making in complex systems, where future dynamics are influenced not only by historical observations but also by evolving contextual features. Recent advances in large language models (LLMs) have extended forecasting beyond numerical extrapolation toward context-aware reasoning. However, existing approaches often lack explicit mechanisms to identify relevant contexts, reason about their impacts, and validate forecasts against temporal and domain constraints. In this work, we propose CastFSR, an agentic framework that formulates context-aware forecasting as a Fast--Slow--Reflect workflow. In fast thinking, CastFSR profiles observations and selects lightweight forecasters to construct a data-driven forecast prior. In slow deliberation, it retrieves contextual evidence, adaptively determines informative look-back windows, and reasons about how contexts reshape future dynamics. In reflection, it iteratively refines forecasts to ensure temporal, contextual, and domain consistency. CastFSR supports both training-free inference with off-the-shelf LLMs and efficient deployment through a two-stage SFT and reinforcement learning strategy that transfers its orchestration capability to compact LLMs. Extensive experiments on public datasets demonstrate that CastFSR consistently outperforms representative baselines. Our code is available at https://github.com/Xiaoyu-Tao/CastFSR.

Xiaoyu Tao, Mingyue Cheng, Bokai Pan +6

State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, Anhui Province, China

Time Series Forecasting LLM-Based Time Series Forecasting Agentic Time Series Forecasting Retrieval-Augmented Time Series Forecasting

Ask PDF

Sep 30, 2026

OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning

Real-world time-series applications increasingly require models that can handle time series forecasting, context-conditioned prediction, and language-based temporal reasoning. Yet current time-series foundation models remain fragmented across these capabilities: numerical specialists often provide the strongest forecasts, while language-based models offer broader contextual understanding and analysis. A central challenge is to unify these heterogeneous capabilities without reducing their individual performance. We introduce OpenTSLM TeeMoE, a generalist time-series language model that can forecast directly from observed time series, reason over textual context and temporal patterns, and synthesize and refine predictions from external numerical forecasting specialists. We independently train three low-rank experts for forecast aggregation, native forecasting, and temporal analysis over a shared backbone. A learned LoRA mixture-of-experts controller then weights their frozen parameter updates for each request. Our proposed model achieves strong performance on widely used benchmarks for time series forecasting, context-conditioned prediction, and language-based temporal reasoning, ranking among the top three on GIFT-Eval by mean MASE rank, Context is Key by RCRPS, and TimeSeriesExam by accuracy.

Tony Chen, Timo Stoffregen, Maxwell Xu +8

Columbia University, USA · Stanford University, USA · Aionic Labs, Switzerland +4

Temporal Reasoning in Language Models LLM-Based Time Series Forecasting Mixture-of-Experts Models Time-Series Foundation Models Time Series Forecasting Time Series Modeling+2

Ask PDF

May 24, 2026

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting

Time series forecasting models are increasingly scaled through large Transformer backbones, yet most existing approaches process all series through a shared dense computation path despite substantial heterogeneity in temporal structure. Mixture-of-Experts (MoE) offers a natural alternative by enabling conditional computation, but standard MoE routing leaves expert specialization weakly identified and often unstable during downstream adaptation. We propose AME-TS, a structure-guided sparse time series foundation model that aligns expert routing with interpretable temporal structure. AME-TS first uses a lightweight regime predictor to estimate series-level descriptors, including forecastability, seasonality, trend, and sparsity, and maps them to a soft structural prior over experts. This series-level prior guides token-level routing during training, encouraging structure-aligned specialization. On the GIFT-Eval benchmark, AME-TS delivers a strong accuracy-efficiency tradeoff across model scales: it substantially outperforms existing time series foundation models at small model scales and remains competitive with the strongest models at larger scales, while activating substantially fewer parameters through sparse routing. We further show that AME-TS learns more interpretable routing geometry and substantially more stable expert specialization than standard MoE during fine-tuning on the M5 dataset. These results suggest that structure-aware routing is an effective and reliable way to realize the benefits of sparse expert models for time series forecasting.

Rui Wang, Renhao Xue, Ray Razi +2

Amazon Web Services

Sparse Mixture-of-Experts Time Series Forecasting Expert Routing Time-Series Foundation Models Time Series Modeling

Ask PDF

Abstract

Time Series Forecasting

Explore similar work

CastFSR: A Fast--Slow--Reflect Agentic Reasoning Framework for Context-Aware Time Series Forecasting

OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning

AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting