In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks

Shangqing Xu; Harshavardhan Kamarthi; Haoxin Liu; B. Aditya Prakash

cs.LGFeb 23, 2026

In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks

Authors: Shangqing Xu, Harshavardhan Kamarthi, Haoxin Liu, B. Aditya Prakash

Organizations: Georgia Institute of Technology, Atlanta, GA, USA

Abstract

Time-series foundation models (TSFMs) have demonstrated strong generalization capabilities across diverse datasets and tasks. However, existing foundation models are typically pre-trained to enhance performance on specific tasks and often struggle to generalize to unseen tasks without fine-tuning. To address this limitation, we propose augmenting TSFMs with In-Context Learning (ICL) capabilities, enabling them to perform test-time inference by dynamically adapting to input-output relationships provided within the context. Our framework, In-Context Time-series Pre-training (ICTP), restructures the original pre-training data to equip the backbone TSFM with ICL capabilities, enabling adaptation to unseen tasks. Experiments demonstrate that ICT improves the performance of state-of-the-art TSFMs by approximately 11.4% on unseen tasks without requiring fine-tuning.

Figures & tables

Figure 1. Similar to LLMs, existing TSFMs are pre-trained on single-task objectives using large-scale data. Yet, due to the inherent differences between time-series and language data, TSFMs are restricted to single-task adaptation. Instead, we propose In-Context Time-series Pre-training ( ICTP ), reformatting the original dataset into a multi-task context-following structure, enabling TSFMs to gain the in-context learning ability which finally leads to non-fine-tuning adaptation to unseen tasks.

Backbone	Evaluated on	ICTP	ETTh1		ETTm1		Exchange		Weather		PEMS-Bay		METR-LA
Backbone	Evaluated on	ICTP	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
MOMENT	Forecasting	No	0.813	0.629	0.724	0.588	0.228	0.285	0.215	0.345	2.623	0.899	1.291	0.765
	Forecasting	Yes	0.433	0.458	0.496	0.541	0.236	0.279	0.163	0.206	1.625	0.595	1.169	0.728
	BackTracing	No	0.834	0.643	0.75	0.595	0.229	0.289	0.222	0.354	2.594	0.915	1.284	0.767
	BackTracing	Yes	0.439	0.454	0.502	0.527	0.241	0.305	0.165	0.254	1.773	0.613	1.315	0.780
TimesFM	BackTracing	No	0.518	0.464	0.402	0.427	0.118	0.238	0.182	0.207	2.993	0.879	1.477	0.742

Table 1. ICTP significantly improved the performance of backbone TSFMs on unseen tasks of output length 96, on all datasets.

Backbone	Evaluated on	ICTP	ETTh1		ETTm1		Exchange		Weather		PEMS-Bay		METR-LA
Backbone	Evaluated on	ICTP	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
MOMENT	Forecasting	No	0.938	0.691	0.849	0.644	0.337	0.365	0.506	0.547	2.247	0.821	1.336	0.780
	Forecasting	Yes	0.514	0.527	0.618	0.544	0.332	0.345	0.357	0.427	2.198	0.698	1.310	0.760
	BackTracing	No	0.944	0.704	0.886	0.654	0.337	0.358	0.556	0.581	2.296	0.861	1.319	0.797
	BackTracing	Yes	0.526	0.512	0.603	0.531	0.330	0.347	0.402	0.495	2.593	0.847	1.211	0.728
TimesFM	BackTracing	No	0.584	0.509	0.496	0.467	0.278	0.361	0.238	0.277	3.315	0.966	1.755	0.836

Table 2. ICTP significantly improved the performance of backbone TSFMs on unseen tasks of output length 192, on all datasets.

Time-Series Foundation Models

Time-series foundation models are pretrained sequence models designed to transfer across forecasting, classification, anomaly detection, and other temporal tasks.

See all 298 papers

Post-Training in Time Series Foundation Models: A Unifying Framework
Jul 22, 2026Shifeng Xie, Ambroise Odonnat, Zehao Xiao +7Model Compression Time-Series Foundation Models
Noah’s Ark Lab · Université Paris Cité · Inria +2
SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning
Sep 26, 2026Yi Tang, Tengxue Zhang, Yang Shu +3Time-Series Foundation Models Time Series Forecasting
East China Normal University, Shanghai, China · Chang’an University, Xi’an, China
A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks
Mar 23, 2026Anish Saha, Konstantin ShmakovZero-Shot Time Series Forecasting Meta-Learning
Walmart Sunnyvale, USA

Jul 22, 2026J/K move · Enter open · S save

Post-Training in Time Series Foundation Models: A Unifying Framework

Shifeng Xie, Ambroise Odonnat, Zehao Xiao, Lei Zan+6

Noah’s Ark Lab · Université Paris Cité · Inria+2

Model Compression Time-Series Foundation Models

Open paper AskPDF arXiv

Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment. Bridging this gap requires further intervention to handle domain shift, task heterogeneity, limited supervision, and computational constraints, which motivates post-training as a broad class of methods to adapt, augment, compose, calibrate, or specialize pretrained TSFMs for downstream tasks. In this work, we analyze TSFM post-training methods based on their locus of intervention in the prediction pipeline, yielding five categories: parameter adaptation, context augmentation, model composition, output processing and uncertainty control, and compression and specialization. Within each category, we study main representative methods and discuss their current limitations. We further identify future directions toward controlled adaptation, reliable context construction, uncertainty-aware model composition, calibrated output processing, and deployment-aware specialization. Overall, by providing a unifying framework for the emerging TSFM post-training landscape, this work aims to support future research to navigate the design space between a pretrained TSFM and its reliable downstream deployment.

Abstract

Time-Series Foundation Models

Explore similar work

Post-Training in Time Series Foundation Models: A Unifying Framework

SIFT: Enhancing Time Series Foundation Models via Semantic Invariance and Structural Fidelity Fine-Tuning

A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks