TSFM Meets LLM: Context as Covariates

Hemanth Ram; Sai Shankar Narasimhan; Shubhankar Agarwal; Sandeep Chinchali

cs.LGNov 3, 2026

TSFM Meets LLM: Context as Covariates

Authors: Hemanth Ram, Sai Shankar Narasimhan, Shubhankar Agarwal, Sandeep Chinchali

Organizations: The University of Texas at Austin, Austin, TX, 78712.

Abstract

Textual information is ubiquitous in real-world time series datasets, appearing as news, reports, and annotations that hold key predictive signals not deducible from numerical history alone. Large Language Models (LLMs) interpret text but lack numerical precision, while Time Series Foundation Models (TSFMs) are quantitatively accurate but cannot reason about the effects of text. Moreover, the lack of large-scale paired multimodal datasets limits progress in bridging this gap. In this work, we address this challenge by providing a scalable approach for generating realistic multimodal datasets by annotating real-world time series with LLMs. Building on this, we propose Context as Covariates (CoCo), a forecasting framework in which an LLM distills its textual reasoning into numerically grounded forecast and confidence covariates that guide a TSFM backbone, thereby exploiting the strengths of both LLMs and TSFMs. We align the two models in stages, first fine-tuning the LLM via Group Relative Policy Optimization (GRPO) with a reward tied to covariate-induced improvement in forecasting performance, and then fine-tuning the TSFM on covariates generated by the aligned LLM. Our experiments show that models trained on our synthetic multimodal corpus generalize to real benchmarks spanning finance, economics, weather, traffic, and security. Notably, CoCo with Qwen3-4B-Instruct-2507 as the LLM and Chronos2 as the TSFM achieves up to 12% MAE improvement over state-of-the-art TSFMs and LLM forecasters, such as GPT-4o. Code, checkpoints, and the evaluation datasets are publicly available.

Figures & tables

Figure 1: Effective use of context is essential for accurate time series forecasting. In crude oil price forecasting, textual context (e.g., recession fears on March 9) aligns with sudden price shifts. Our approach, Co ntext as Co variates (CoCo), leverages this context to produce accurate forecasts, while unimodal models such as Chronos2 fail to capture context-induced trends.

Figure 2: CoCo outperforms SOTA TSFMs (Chronos2) and closed-source LLMs (GPT-4o) across datasets from diverse domains. CoCo effectively utilizes LLMs to distill textual annotations into usable forecast covariates that guide powerful TSFMs to generate accurate forecasts.

Figure 3: Training on synthetic annotations yields real-world multimodal gains. Using pharmaceutical sales data annotated with both real news and synthetic text, a SARIMA model with sentiment features achieves similar MAE when trained on synthetic versus real annotations, as both encode the same domain-agnostic forecast trends, thereby enabling sim-to-real transfer.

Figure 4: Context as Covariates (CoCo) Description. The LLM $\theta_{\mathrm{LLM}}$ takes the numerical history $X^{H}_{t}$ (blue) and textual context $S_{t}$ (violet) as input and produces a forecast covariate $M_{t}$ (Equation 5 , brown) and a rollout-based confidence covariate $C_{t}$ (Equation 6 , maroon), which are concatenated with $X^{H}_{t}$ and passed to Chronos2 for multivariate forecasting. Our key technical intuition is to utilize the confidence covariate (obtained from the per-step variance in the LLM’s predictions) to teach Chronos2 when to trust the LLM and when not to trust it (low variance $\rightarrow$ trust the LLM, as text induces a more decisive forecast).

Figure 5: CoCo improves over unimodal and multimodal fine-tuning approaches, including SOTA TSFMs on real-world multimodal datasets. CoCo (ours) achieves the best average rank and mean MAE across 30 datasets (95 settings) spanning diverse domains and context lengths against 26 (TimeOmni omitted for scale) approaches. These gains demonstrate the effectiveness of GRPO-based fine-tuning along with our novel two-step fine-tuning.

	Mean MAE ( $\downarrow$ )										Aggregate Performance
	64–16 Setting			128–16 Setting			256–16 Setting			Native	Overall	Average	ELO	CoCo	Wilcoxon
Model	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	TimeMMD ∗	MAE $\downarrow$	Rank $\downarrow$	Rating $\uparrow$	Wins	$p$
CoCo (Ours)	0.809	0.907	0.811	0.567	0.613	0.672	0.446	0.453	0.678	1.329	0.719	5.24	1838	–	–
Zero-shot Unimodal
TiRex1	0.835	0.915	0.824	0.580	0.616	0.696	0.450	0.450	0.738	1.352	0.735	7.26	1750	60/95	5.0e-04
Chronos2	0.847	0.942	0.832	0.590	0.631	0.701	0.461	0.462	0.718	1.362	0.745	10.18	1643	81/95	2.0e-12

Table 1: CoCo outperforms SOTA zero-shot unimodal and multimodal forecasting approaches on real-world multimodal datasets. CoCo (ours) achieves the best mean MAE across 30 datasets (95 settings) spanning diverse domains and context lengths. It outperforms SOTA TSFMs such as TiRex1 in over

63\%

(60/95) of settings. Similarly, CoCo surpasses constructed SOTA zero-shot multimodal baselines (TimesFM2.5 + GPT-4o) in 60/95 settings with a significant improvement in the aggregate MAE and the average rank metrics, highlighting effective sim-to-real transfer.

	Mean MAE ( $\downarrow$ )										Aggregate Performance
	64–16 Setting			128–16 Setting			256–16 Setting			Native	Overall	Average	ELO	CoCo	Wilcoxon
Model	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	TimeMMD ∗	MAE $\downarrow$	Rank $\downarrow$	Rating $\uparrow$	Wins	$p$
CoCo (Ours)	0.809	0.907	0.811	0.567	0.613	0.672	0.446	0.453	0.678	1.329	0.719	5.24	1838	–	–
Unimodal Fine-tuning
Chronos2	0.851	0.924	0.810	0.583	0.637	0.670	0.460	0.469	0.675	1.389	0.738	9.18	1677	64/95	1.2e-05
AutoTimes	0.851	0.916	0.901	0.585	0.633	0.773	0.515	0.473	0.784	1.337	0.763	11.75	1588	78/95	3.2e-10

Table 2: CoCo outperforms SOTA compute-matched and fine-tuned forecasting approaches on real-world multimodal datasets. Similar to Table 1 , here, we show CoCo’s improvements over fine-tuned and compute-matched baselines. Observe how, even after fine-tuning, both unimodal and multimodal baselines struggle to match CoCo’s performance.

Fine-tuning	$M_{t}$	Both $M_{t}$ + $C_{t}$
Only $\theta_{\mathrm{LLM}}$	0.21	1.64
Only $\theta_{\mathrm{M-TSFM}}$	2.45	2.55
Both $\theta_{\mathrm{LLM}}$ and $\theta_{\mathrm{M-TSFM}}$	2.6	3.36

Table 3: Using both

M_{t}

and

C_{t}

consistently outperforms

M_{t}

alone across all fine-tuning configurations . We show the % MAE improvement over SOTA unimodal models. Sequential fine-tuning of both LLM

\theta_{\mathrm{LLM}}

and M-TSFM

\theta_{\mathrm{M-TSFM}}

yields the best % MAE improvement (Table 22 ).

Figure 6: CoCo generalizes across model choices. MAE improvement over each numerical-only M-TSFM baseline on Media Cloud (64-16 setting).

Figure 7: (Left) CoCo consistently outperforms SOTA unimodal models (Chronos2) when text is useful (Q5–Q9) and underperforms when it is not (Q1–Q3). Text usefulness is defined as the relative MAE improvement of the LLM forecast when the context $S_{t}$ is provided versus withheld. (Right) Furthermore, for the set of windows where text is useful, CoCo significantly outperforms unimodal models when the rollout variance from the LLM (Confidence $C_{t}$ ) is very low.

Model	Mean RCRPS $\downarrow$
Qwen3-4B	0.358
Qwen3-4B + Reasoning	0.189
Qwen3-4B (CoCo)	0.301
Qwen3-4B (CoCo) + Reasoning	0.161
Chronos2 (Unimodal)	0.265
CoCo	0.256

Table 4: LLMs fine-tuned with CoCo have improved text-time series alignment and outperform their pretrained counterparts. Notably, Qwen3-4B fine-tuned with CoCo, with reasoning, matches the zero-shot results from GPT-4o. We refer to Table 8 for the detailed results.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Real and synthetic annotations contain similar predictive powers to estimate the future. The significant overlap of cosine similarity distributions between the true and the predicted forecast descriptions for synthetic and real annotations validates our synthetic annotation strategy. For the real cosine similarity distribution, we compare the normalized BERT embeddings for the domain-agnostic forecast predictions from real text with the true domain-agnostic forecast description. Similarly, for the synthetic cosine similarity distribution, we compare the normalized BERT embeddings for the domain-agnostic forecast predictions from synthetic text with the true domain-agnostic forecast description. For this experiment, we consider the Media Cloud datasets for which the real annotations are available, and we generated synthetic annotations for those datasets.

Figure 9: Real and synthetic annotations contain similar domain-agnostic descriptions, but differ in semantics. Here, we show the t-SNE plots for concatenated embeddings of real and synthetic annotations. We consider the Media Cloud datasets for which the real annotations are available, and we generated synthetic annotations for those datasets. On the left, we compare the embeddings of real domain-agnostic predictions concatenated with true forecast descriptions against the synthetic counterpart. Similarly, on the right, we compare the embeddings of the real domain-specific prediction concatenated with the true forecast descriptions against the synthetic counterpart. Note that there is near-perfect overlap in the domain-agnostic case and clear separation in the domain-specific case. This conveys that for the same true forecast description (through concatenation), the domain-agnostic predictions overlap indicating that real and synthetic annotations provide similar domain-agnostic prediction. In contrast, the domain-specific predictions vary significantly, indicating that both real and synthetic domain-specific predictions have different semantic constructions.

	MAE ( $\downarrow$ )
Dataset	LLM Forecast ( $\theta_{\mathrm{LLM}}(X^{H}_{t},S_{t})$ )	CoCo ( $\theta_{\mathrm{M-TSFM}}{}(X^{H}_{t}\|\|M_{t}\|\|C_{t})$ )
Raw	0.878	0.858
Rephrased	0.875	0.859

Appendix

Table 5: CoCo is less sensitive to text rephrasing. We observe that the MAE of the LLM and the MAE of CoCo change marginally with rephrasing as long as the text conveys the same information. This validates Assumption 4.2 (smoothness assumption) indicating that small perturbations in the context

S_{t}

do not significantly affect the forecasting performance.

Perturbation	% Change in the LLM Forecast ( $\theta_{\mathrm{LLM}}(X^{H}_{t},S_{t})$ ) MAE ( $\downarrow$ )
Rephrased	0.3
Random Word Deletion ( $5\%$ )	0.8
Less Verbose Annotations	1.1
Sentence Reordering	1.02
Using Annotations From Other Datasets	5.0
Misleading Text	7.9

Appendix

Table 6: CoCo exhibits local smoothness with respect to text perturbations. For simple perturbations that do not affect the forecast trend indicators (first 4 rows), note that the change in the LLM forecast MAE is less than or around

1\%

, indicating local smoothness towards mild perturbations. However, for perturbations that change the underlying forecast (last two rows), the change is significant.

Domain	Mean Distance Between Real and True Forecast Description Representation	Mean Distance Between Synthetic and True Forecast Description Representation	$\boldsymbol{\Delta}$ Mean
All domains	0.3715	0.3774	+0.0059
Agriculture	0.3718	0.3735	+0.0017
Energy	0.3744	0.3745	+0.0001
Macro Indicators	0.3720	0.3805	+0.0085
Retail	0.3759	0.3744	-0.0015
Finance	0.3707	0.3775	+0.0068

Appendix

Table 7: Our synthetic annotations encode similar information about the true future as the real annotations. We empirically show this in Figure 8 . However, as Assumption 4.1 uses the Euclidean distance measure, we also provide a per-domain breakdown of the

l_{2}

norm between the real and synthetic predicted representations and the representation of the true forecast description. Here, the representation space used is the unit normalized BERT embedding space.

	Average	Intemporal	Historical	Future	Covariate	Causal
Model	RCRPS $\downarrow$	Information	Information	Information	Information	Information
Qwen3-4b (Base)	0.358 ± 0.031	0.390 ± 0.045	0.247 ± 0.023	0.301 ± 0.042	0.310 ± 0.035	0.700 ± 0.149
Qwen3-4b (CoCo Fine-tuned)	0.301 ± 0.027	0.306 ± 0.040	0.219 ± 0.019	0.276 ± 0.036	0.280 ± 0.033	0.675 ± 0.142
Qwen3-4b (Base) + Reasoning	0.189 ± 0.036	0.159 ± 0.011	0.134 ± 0.011	0.088 ± 0.012	0.204 ± 0.047	0.523 ± 0.214
Qwen3-4b (CoCo Fine-tuned) + Reasoning	0.161 ± 0.013	0.154 ± 0.010	0.130 ± 0.012	0.090 ± 0.012	0.171 ± 0.016	0.378 ± 0.064
Chronos2 (Unimodal)	0.265 ± 0.016	0.254 ± 0.021	0.166 ± 0.017	0.293 ± 0.031	0.239 ± 0.018	0.438 ± 0.066

Appendix

Table 8: RCRPS on the CiK benchmark [ 12 ] . We provide a breakdown of RCRPS by information type, and evaluate the effect of fine-tuning with CoCo on the reasoning capacity of the LLM. Bold denotes the best result among our models per column. Reference numbers for GPT-4o and Llama-3.1-405B are from the official CiK leaderboard ( https://github.com/servicenow/context-is-key-forecasting ) and shown in italics for context. Lower is better. Our key takeaway from the results is that LLMs fine-tuned with CoCo consistently outperform their base, pre-trained counterpart. Additionally, a Qwen3-4B model that is fine-tuned with CoCo, when prompted for reasoning, can match the performance of powerful closed-source LLMs, such as GPT-4o.

Figure 10: Average GRPO training reward $r_{t}$ (Eq. 7 ) over training steps, computed as the mean over $N_{\text{rollouts}}$ rollouts per prompt in a batch. The reward increases steadily reflecting that the LLM progressively learns to generate forecast covariates $M_{t}$ that provide better forecasts ( $\theta_{\mathrm{M-TSFM}}(X^{H}_{t}||M_{t})$ ) relative to the unimodal baseline ( $\theta_{\mathrm{M-TSFM}}(X^{H}_{t})$ ).

$\sigma_{\epsilon}$	CoCo MAE ( $\downarrow$ )
0.00	0.810
0.05	0.801
0.25	0.802
0.50	0.805
1.00	0.814

Appendix

Table 9: Effect of noise standard deviation

\sigma_{\epsilon}

on CoCo’s forecasting performance.

\sigma_{\epsilon}

controls the noise added to the history portion of the forecast covariate

M_{t}

(Eq. 5 ) and the confidence covariate

C_{t}

(Eq. 6 ). Results are for 16-step forecasts with context length 64 on the Media Cloud dataset, using the base pretrained Chronos2 as the TSFM backbone. Bold denotes the best result.

\sigma_{\epsilon}=0.05

is selected and used in all experiments.

Figure 11: Qwen3 provides realistic textual annotations. Here, we show samples of annotations generated by Qwen3-30b-A3B-Instruct-2507 [ 9 ] for gasoline prices in the United States for the year 2022. Note that the text generated is relevant to the domain queried and foreshadows the trend in the numerical data. In the plot, the green portion indicates an increasing trend and the red portion indicates a decreasing trend.

Source	Frequency	Datasets	Samples
FRED	Daily	95	107,993
	Weekly	91	97,010
	Monthly	807	433,750
	Quarterly	93	18,891
	Annual	52	3,176
EIA Data	Daily	118	252,581

Appendix

Table 10: Summary of the training datasets used. Datasets from FNSPID contain real annotations

Figure 12: Our approach to generate textual annotations, for real-world time series datasets, extracted from the news articles that are scraped from the internet. Let us consider the task of generating textual annotations for the NVIDIA stock price dataset. First, we use the Media Cloud API to extract the relevant news articles between two successive timestamps $t-1$ and $t$ . These news articles are summarized into Key Events and Facts and Predictive Information by an LLM. This summary is treated as the text annotation for the timestamp $t$ .

Text Source	Dataset	Target Source	Frequency	Date Range	Total	$\geq$ 2024-06
Media Cloud	Apple Stock Price	Google Finance	Daily	2024-01 – 2025-12	503	398
	Copper Futures Price	Google Finance	Daily	2025-01 – 2026-01	258	258
	Crude Oil Price	FRED	Daily	2024-01 – 2026-01	500	395
	Egg Price	Trading Economics	Daily	2025-02 – 2025-12	223	223
	Google Stock Price	Google Finance	Daily	2024-01 – 2025-12	502	398
	NASDAQ Electricity	FRED	Daily	2024-01 – 2025-12	523	413

Appendix

Table 11: Summary of the multimodal evaluation datasets grouped by source.

Forecast	MAE (mean $\pm$ std)
LLM Rollout ( $\times 15$ )	$0.7787\pm 0.0489$
LLM Rollout Median ( $\times 3$ )	$0.7860\pm 0.0044$
CoCo ( $\times 3$ )	$0.7194\pm 0.0001$

Appendix

Table 12: CoCo’s robustness to LLM rollout stochasticity. Overall MAE across all 95 settings, evaluated across three random seeds. Although individual LLM rollouts exhibit substantial variation, taking the median across five rollouts substantially reduces the variability, and the subsequent CoCo forecast is highly stable.

		Mean MAE ( $\downarrow$ )
Group	Dataset	CoCo	TiRex1	Chronos2	Moirai	TimesFM2.5	Toto	TabPFN2.5	TimeMoE	PANDA
Media Cloud	Crude Oil Price	0.700	0.682	0.690	0.687	0.663	0.703	0.777	0.770	0.774
	S&P500 Index	0.594	0.601	0.641	0.625	0.652	0.667	0.778	0.835	0.816
	Apple Stock	0.637	0.666	0.666	0.651	0.676	0.683	0.760	0.797	0.846
	Copper Futures	0.682	0.735	0.727	0.679	0.751	0.701	0.781	0.775	0.941
	Egg Price	0.604	0.587	0.583	0.607	0.587	0.583	0.655	0.598	0.865

Appendix

Table 13: Dataset-level MAE for zero-shot unimodal methods. Each dataset reports the mean over the 64-16, 128-16, and 256-16 settings. TimeMMD Daily, Weekly, and Monthly retain their native settings as separate blocks. Summary statistics are computed across all 95 settings. CoCo outperforms multiple state-of-the-art TSFMs. Bold and underlined values denote the best and second-best method in each row.

		Mean MAE ( $\downarrow$ )
Group	Dataset	CoCo	TimesFM2.5+GPT4o	Chronos2+GPT4o	Chronos2 MV	Toto MV	GPT-4o	Qwen3-30B	TimeOmni
Media Cloud	Crude Oil Price	0.700	0.625	0.645	0.707	0.646	0.713	0.842	0.887
	S&P500 Index	0.594	0.634	0.649	0.693	0.724	0.723	0.655	0.785
	Apple Stock	0.637	0.682	0.692	0.715	0.706	0.770	0.754	0.876
	Copper Futures	0.682	0.685	0.685	0.748	0.740	0.725	0.736	0.853
	Egg Price	0.604	0.579	0.572	0.594	0.577	0.599	0.672	0.809

Appendix

Table 14: Dataset-level MAE for zero-shot multimodal methods. Each dataset reports the mean over the 64-16, 128-16, and 256-16 settings. TimeMMD Daily, Weekly, and Monthly retain their native settings as separate blocks. Summary statistics are computed across all 95 settings. Mean ensembles are included because they combine zero-shot forecasts. CoCo outperforms state-of-the-art LLM forecasters as well as multivariate TSFMs that directly incorporate Sentence Transformer text embeddings as covariates. Bold and underlined values denote the best and second-best method in each row.

		Mean MAE ( $\downarrow$ )
Group	Dataset	CoCo	Chronos2 (FT)	AutoTimes	PatchTST	S2IP-LLM	iTransformer	TEST
Media Cloud	Crude Oil Price	0.700	0.681	0.691	0.707	0.723	0.763	0.769
	S&P500 Index	0.594	0.660	0.652	0.626	0.661	0.740	0.764
	Apple Stock	0.637	0.683	0.669	0.683	0.698	0.772	0.827
	Copper Futures	0.682	0.729	0.731	0.717	0.776	0.779	0.767
	Egg Price	0.604	0.623	0.705	0.698	0.774	0.765	0.805

Appendix

Table 15: Dataset-level MAE for unimodal fine-tuning methods. Each dataset reports the mean over the 64-16, 128-16, and 256-16 settings. TimeMMD Daily, Weekly, and Monthly retain their native settings as separate blocks. Summary statistics are computed across all 95 settings. CoCo outperforms unimodal fine-tuning approaches, indicating that exogenous textual context is essential for improved forecasting and is difficult to recover from numerical history alone. Bold and underlined values denote the best and second-best method in each row.

		Mean MAE ( $\downarrow$ )
Group	Dataset	CoCo	Chronos2 MV (FT)	VoT	TaTS	S2IP-LLM	AutoTimes	TEST
Media Cloud	Crude Oil Price	0.700	0.680	0.667	0.698	0.715	0.699	0.704
	S&P500 Index	0.594	0.641	0.679	0.694	0.666	0.682	0.727
	Apple Stock	0.637	0.657	0.661	0.702	0.698	0.730	0.699
	Copper Futures	0.682	0.679	0.707	0.730	0.755	0.755	0.732
	Egg Price	0.604	0.667	0.709	0.592	0.737	0.781	0.749

Appendix

Table 16: Dataset-level MAE for multimodal fine-tuning methods. Each dataset reports the mean over the 64-16, 128-16, and 256-16 settings. TimeMMD Daily, Weekly, and Monthly retain their native settings as separate blocks. Summary statistics are computed across all 95 settings. The CoCo fine-tuning approach for incorporating text into forecasting outperforms other multimodal fine-tuning methods that jointly incorporate numerical history and text. Bold and underlined values denote the best and second-best method in each row.

				Mean MAE ( $\downarrow$ )
Group	Setting	Dataset	N	CoCo	TimesFM2.5+GPT4o	Chronos2+GPT4o	Chronos2 MV	Toto MV	GPT-4o	Qwen3-30B	TimeOmni
Media Cloud	64-16	Crude Oil Price	316	0.802	0.752	0.758	0.817	0.760	0.877	0.924	1.082
		S&P500 Index	280	0.759	0.797	0.805	0.904	0.929	0.883	0.859	0.978
		Apple Stock	319	0.903	0.977	0.977	1.043	1.014	1.089	1.059	1.467
		Copper Futures	179	0.775	0.784	0.769	0.893	0.914	0.824	0.824	1.004
		Egg Price	144	0.701	0.651	0.669	0.673	0.644	0.714	0.778	0.902

Appendix

Table 17: Dataset-level MAE for samples strictly after June 2024. We evaluate CoCo against multimodal baselines in a regime where the LLMs have limited or no prior exposure to the test period. The gains are consistent with those in Table 14 , confirming that the improvements stem from the ability of the model to extract predictive signals from text rather than from memorized knowledge. Bold and underlined values denote the best and second-best method in each row.

	Geometric Mean MASE ( $\downarrow$ )
	64–16 Setting			128–16 Setting			256–16 Setting			Native
Model	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	Media Cloud	FNSPID	TimeMMD	TimeMMD ∗	Overall MASE	Avg. Rank	ELO	CoCo Wins	Wilcoxon $p$
CoCo
CoCo	3.064	3.136	1.605	3.009	3.056	1.533	2.817	2.961	1.530	1.915	2.455	5.21	1845	–	–
Zero-shot Unimodal
TiRex1	3.136	3.175	1.621	3.075	3.064	1.609	2.857	2.950	1.662	1.936	2.506	6.62	1782	59/95	9.4e-04

Appendix

Table 18: MASE comparison across all 28 evaluated configurations and 95 settings. Each setting/group column reports the geometric mean over its constituent datasets; TimeMMD ∗ pools the Daily, Weekly, and Monthly native-frequency benchmarks. The relative performance of CoCo under MASE is consistent with its performance under MAE in Tables 1 and 2 . Bold and underlined values denote the best and second-best models.

CoCo
	Average Rank ( $\downarrow$ )
Model	MAE	MSE	MAPE	MAE Unscaled	MASE
CoCo	5.24	5.98	6.39	5.81	5.21
Zero-shot Unimodal
TiRex1	7.26	8.63	6.92	6.97	6.62
Chronos2	10.18	11.16	9.81	10.11	9.56

Appendix

Table 19: CoCo performs consistently across raw and normalized evaluation metrics. Average ranks are computed over the same 95 settings, similar to Tables 1 , 2 , and 18 . Bold and underlined values denote the best and second-best method for each metric.

				Mean MAE ( $\downarrow$ )
Group	Setting	Dataset	N	CoCo	TimesFM2.5+GPT4o	Chronos2+GPT4o	Chronos2 MV	Toto MV	GPT-4o	Qwen3-30B	TimeOmni	GPT-5
Media Cloud	64-16	Crude Oil Price	421	0.877	0.817	0.831	0.853	0.816	0.930	1.027	1.160	0.859
		S&P500 Index	384	0.797	0.827	0.843	0.944	0.998	0.896	0.855	0.979	0.766
		Apple Stock	424	0.872	0.948	0.945	1.008	1.003	1.043	0.961	1.325	0.854
		Copper Futures	179	0.775	0.784	0.769	0.893	0.914	0.824	0.824	1.004	0.787
		Egg Price	144	0.701	0.651	0.669	0.673	0.644	0.714	0.778	0.902	0.729

Appendix

Table 20: Dataset-level MAE for 16-step forecasts across context lengths 64, 128, and 256 on the Media Cloud subset, including GPT-5 as an additional LLM baseline. Media Cloud, comprising datasets from 2024–2025, is the only subset where GPT-5 has limited prior exposure to test trends. GPT-5 performs competitively with strong TSFMs, yet CoCo outperforms GPT-5 in 70% (21/30) of settings, demonstrating that a fine-tuned 4B LLM providing forecast-space covariates to Chronos2 surpasses a closed-source model used as a standalone forecaster. Bold and underlined values denote the best and second-best method in each row.

MAE ( $\downarrow$ )
CoCo Fine-tuned	CoCo	CoCo
Chronos2	(With Misleading Text)	(With Truthful Text)
2.929	3.025	2.802

Appendix

Table 21: CoCo’s performance degrades when text is adversarial. We evaluate on a synthetically generated dataset of 1000 samples where the same numerical series is paired with two kinds of annotations: truthful text describing the correct future trend, and misleading text deliberately describing the opposite trend. CoCo improves over the fine-tuned Chronos2 baseline with truthful text but degrades with misleading text, consistent with the gating analysis in Figure 7 (right), where confidently wrong LLM forecasts are the most harmful regime.

	LLM Forecast		Chronos2 Forecast		CoCo		CoCo
	$\theta_{\mathrm{LLM}}(X^{H}_{t},S_{t})$		$\theta_{\mathrm{M-TSFM}}(X^{H}_{t})$		$\theta_{\mathrm{M-TSFM}}(X^{H}_{t}\|\|M_{t})$		$\theta_{\mathrm{M-TSFM}}(X^{H}_{t}\|\|M_{t}\|\|C_{t})$
LLM	MAE ( $\downarrow$ )	Rank ( $\downarrow$ )	MAE ( $\downarrow$ )	Rank ( $\downarrow$ )	MAE ( $\downarrow$ )	Rank ( $\downarrow$ )	MAE ( $\downarrow$ )	Rank ( $\downarrow$ )
Zeroshot	1.270	8.842	0.980	-	1.028	2.579	1.110	4.676
Qwen3-4b Fine-tuned	1.078	3.816	0.980	-	0.978	2.632	0.964	2.684
Chronos2 Fine-tuned	1.270	8.842	0.959	2.342	0.956	2.459	0.955	2.459
Both Fine-tuned	1.078	3.816	0.959	2.342	0.954	2.263	0.947	2.184

Appendix

Table 22: Effect of fine-tuning and the usefulness of the covariates. Extended version of Table 3 showing MAE and average rank over the 30 datasets for the 64-16 setting and the 8 TimeMMD datasets with their native settings (38 settings in total) for different fine-tuning settings and covariate inputs. ‘-’ indicates that the rank is not valid since Chronos2 is one of the baselines.

	CoCo Forecast
	$\theta_{\mathrm{M-TSFM}}(X^{H}_{t}\|\|M_{t}\|\|C_{t})$
$N_{\mathrm{rollouts}}$	MAE ( $\downarrow$ )	Rank ( $\downarrow$ )
5	0.845	1.833
9	0.845	1.867
15	0.846	1.900

Appendix

Table 23: Effect of

N_{\mathrm{rollouts}}

, the number of rollouts used to compute confidence covariate on performance. The MAE and the average rank are shown over the 30 datasets for 16-step forecasts with a context length of 64.

	LLM Forecast ( $\theta_{\mathrm{LLM}}(X^{H}_{t},S_{t})$ )
Qwen3-4b Fine-tuning	Overall MAE ( $\downarrow$ )	TimeMMD MAE ( $\downarrow$ )	FNSPID MAE ( $\downarrow$ )	Rank ( $\downarrow$ )
Supervised Fine-tuning	1.187	1.635	1.050	8.289
CoCo Fine-tuning without Text	1.172	1.543	0.954	4.395
CoCo Fine-tuning with Synthetic Text	1.087	1.365	0.903	3.974
CoCo Fine-tuning with Real Text	1.140	1.528	0.891	3.605
CoCo Fine-tuning with Real and Synthetic Text	1.078	1.354	0.894	3.737

Appendix

Table 24: Effect of using text and the kind of text used in fine-tuning the LLM. The results are for the Qwen3-4B-Instruct-2507 model. Real text here denotes FNSPID data and synthetic text refers to the dataset described in Appendix E.2 . Though the model trained completely with FNSPID data has a marginally better rank (the wins are concentrated in the FNSPID subset, which was seen during training), we observe that it fails in the TimeMMD dataset, which is out of distribution and hence does not generalize. On the other hand, models trained on completely synthetic text or a mixture of both provide improved performance in TimeMMD as well. The MAE and average ranks reported are over the same 38 settings as Table 22 .

		Mean MAE ( $\downarrow$ )
M-TSFM	LLM	$\theta_{\mathrm{LLM}}(X^{H}_{t},S_{t})$	$\theta_{\mathrm{\mathrm{M-TSFM}}}(X^{H}_{t})$	$\theta_{\mathrm{\mathrm{M-TSFM}}}(X^{H}_{t}\\|M_{t})$	$\theta_{\mathrm{\mathrm{M-TSFM}}}(X^{H}_{t}\\|M_{t}\\|C_{t})$
Chronos2	GPT-OSS-120B	0.885	0.847	0.828	0.842
	Qwen3-30B	0.949	0.847	0.816	0.851
	Gemma3-27B	0.871	0.847	0.821	0.827
	Qwen3-4B	1.019	0.847	0.824	0.895
	Qwen3-4B (FT)	0.862	0.847	0.818	0.801

Appendix

Table 25: CoCo generalizes across LLM and M-TSFM choices. Mean MAE is reported on the Media Cloud datasets for the 64-16 context–horizon setting. Across five LLM configurations and three M-TSFM backbones, the forecast covariate

M_{t}

improves over the corresponding M-TSFM baseline in every evaluated combination. The confidence covariate

C_{t}

provides further gains with the zero-shot Toto backbone and after fine-tuning Chronos2. Importantly, Chronos2 is fine-tuned using rollouts pre-generated only by the fine-tuned Qwen3-4B, yet it generalizes to covariates from every evaluated zero-shot LLM. These results demonstrate that CoCo is not reliant on properties specific to Qwen3-4B or Chronos2. Bold denotes the best result in each row.

				Mean MAE ( $\downarrow$ )
				TSFM	TSFM MV	CoCo (With Toto)	CoCo (With Toto)
Group	Setting	Dataset	N	Toto	Toto	Only Forecast Covariate	Both Covariates
Suite	64-16	Crude Oil Price	421	0.900	0.816	0.899	0.899
		S&P500 Index	384	0.926	0.998	0.959	0.948
		Apple Stock	424	0.959	1.003	0.951	0.941
		Copper Futures	179	0.827	0.914	0.814	0.815

Appendix

Table 26: Results with Toto as the TSFM backbone, using covariates generated by the fine-tuned Qwen3-4b LLM without any Toto-specific fine-tuning. CoCo (with Toto) outperforms Toto in

73\%\ (22/30)

of datasets and outperforms sentence transformer augmented Toto (TSFM MV) in

70\%\ (21/30)

of the datasets, demonstrating that the covariate design transfers across TSFM backbones and is not specific to Chronos2. Bold and underlined indicate the best and second best per dataset respectively.

Annotations	Direction	Magnitude Tertile	Argmax Bucket
Media Cloud (Real)	0.533	0.344	0.301
FNSPID (Real)	0.544	0.340	0.279
Synthetic (LLM generated with ground truth future values)	0.555	0.316	0.272
Synthetic (LLM generated without ground truth future values)	0.491	0.376	0.346

Appendix

Table 27: Synthetic textual annotations that depend on numeric history alone provide incorrect information about the true future. In this table, we show the accuracy of linear probes for predicting the domain-agnostic forecast statistics, such as the direction, magnitude tertile, and the argmax location/bucket. Note that the linear probes take an LLM’s embeddings of the context as input. In our training, we utilize synthetic annotations/context that is synthetically generated (Section 4 ). Here, we ablate the synthetic annotation generation strategy. We show two approaches: one where the numeric future is used to generate the annotation (row 3) and one where the numeric future is not used (row 4). We compare the accuracies of predicting domain-agnostic forecast statistics from these two variants of synthetic annotations with the accuracies of predicting the same from real-annotations. Prefix-only (without the numeric future, row 4) annotations are generated using only domain information and the observed history, without access to forecast values. Compared with real and future-conditioned annotations, they yield lower direction accuracy but substantially higher magnitude-tertile and argmax-bucket accuracies, indicating an out-of-distribution probe profile relative to the real text. This mismatch suggests that prefix-only generation may weaken sim-to-real transfer and motivates our randomized-context design (Section 4 ) as a middle ground between limiting future leakage and preserving real-world text statistics. However, one key limitation of this experiment is that the probes are linear, and more non-linear approaches could extract better forecast statistics from prefix-only annotations as well.

Fine-tuned Components	Mean MAE ( $\downarrow$ )	Avg. Rank among Baselines ( $\downarrow$ )	ELO among Baselines ( $\uparrow$ )	Avg. Rank among Variants ( $\downarrow$ )	ELO among Variants ( $\uparrow$ )
M-TSFM & LLM	0.719	5.24	1829	1.87	1535
LLM	0.732	6.44	1785	1.93	1508
M-TSFM	0.724	7.16	1758	2.19	1457

Appendix

Table 28: Ablation of fine-tuned components in CoCo across all 95 settings. We compare variants with only the LLM fine-tuned, only the M-TSFM fine-tuned, and both components fine-tuned, evaluating each against all the 27 methods as well as against the other CoCo variants. Fine-tuning both components jointly (in a sequential manner as described in Section 5 ) achieves the best mean MAE, average rank, and ELO in both comparisons. Fine-tuning either component individually also outperforms the zero-shot baselines, demonstrating that both stages contribute independently. Notably, the advantage of jointly fine-tuning both components is more pronounced when evaluated against the full baseline pool than in the direct three-way comparison.

Figure 13: Comparison of standalone forecasting performance of LLMs. The plot shows the %-improvement in the MAE over the base Qwen3-4B model across the same 38 settings as Table 22 . We observe that the CoCo fine-tuned Qwen3-4B outperforms larger and closed-source LLMs as a standalone forecaster.

Model	Latency (ms/sample)
Chronos2	11.43 $\pm$ 7.18
Toto	31.15 $\pm$ 39.67
TimesFM 2.5	37.08 $\pm$ 52.43
Chronos MV	75.77 $\pm$ 54.26
TabPFN 2.5	270.17 $\pm$ 28.62
CoCo	280.07 $\pm$ 21.96

Appendix

Table 29: Inference latency (ms/sample) for CoCo and all TSFM baselines, measured on a single H100 GPU. Latency is computed by processing 128 samples (16-step forecasts with a context length of 64) in batches of 16 and averaging over batches. For CoCo, the fine-tuned Qwen3-4b-Instruct-2507 is served via a vLLM inference server on the same GPU, with

k=5

rollouts generated per sample for the confidence covariate

C_{t}

. Despite requiring LLM inference with multiple rollouts, CoCo achieves

280

ms/sample, comparable to TabPFN 2.5. The latency overhead is a direct consequence of incorporating an LLM, which we view as necessary for extracting predictive signals from text, a capability that unimodal TSFMs fundamentally lack.

Time Series Forecasting

Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.

See all 910 papers

Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting
May 28, 2026Haoxin Liu, Yichen Zhou, Rajat Sen +2Multivariate Time Series Forecasting LLM Post-Training
Georgia Institute of Technology · Google Research
LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models
Jul 27, 2026Huu Hiep Nguyen, Dung Nguyen, Minh Hoang Nguyen +2Time Series Forecasting LLM-Based Time Series Forecasting
Applied Artificial Intelligence Initiative Deakin University Geelong, Australia
ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset
May 15, 2025Chengsen Wang, Qi Qi, Zhongwen Rao +2Cross-Modal Learning Multivariate Time Series Forecasting
Beijing University of Posts and Telecommunications · Huawei Noah’s Ark Lab

May 28, 2026J/K move · Enter open · S save

Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting

Haoxin Liu, Yichen Zhou, Rajat Sen, B. Aditya Prakash+1

Georgia Institute of Technology · Google Research

Multivariate Time Series Forecasting LLM Post-Training Time-Series Foundation Models

Open paper AskPDF arXiv

Time-Series Foundation Models (TSFMs) excel at zero-shot unimodal forecasting using numerical data, but unlike LLMs they cannot consume multimodal, non-numerical context that often shape real-world trajectories. In this work, we bridge this gap and argue for a multimodal time-series forecasting approach that post-trains LLMs to act as context-guided revisors over strong numerical TSFM priors. We introduce PostTime, a post-training recipe combining Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR), along with a methodology to generate automated reasoning traces for forecast revisions. PostTime teaches an LLM to generate context-conditioned forecast interventions -- decisions to revise, preserve, or ignore the TSFM prior based on the multimodal context. We evaluate this approach on the TimesX multimodal forecasting benchmark using a Gemma-3-4B LLM and TimesFM-2.5 TSFM, and show that it significantly outperforms standalone TSFMs, LLM-only baselines, and existing multimodal forecasting approaches.

Abstract

Time Series Forecasting

Explore similar work

Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting

LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models

ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset