Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting

Filippos Bellos; NaveenJohn Premkumar; Yannis Avrithis; Nam H. Nguyen; Jason J. Corso

cs.LGFeb 18, 2026

Deep TPC: Temporal-Prior Conditioning for Time Series Forecasting

Authors: Filippos Bellos, NaveenJohn Premkumar, Yannis Avrithis, Nam H. Nguyen, Jason J. Corso

Organizations: University of Michigan, Ann Arbor, MI, USA · Independent Scientist · Capital One, McLean, VA, USA

Abstract

LLM-for-time series (TS) methods typically treat time shallowly, injecting positional or prompt-based cues once at the input of a largely frozen decoder, which limits temporal reasoning as this information degrades through the layers. We introduce Temporal-Prior Conditioning (TPC), which elevates time to a first-class modality that conditions the model at multiple depths. TPC attaches a small set of learnable time series tokens to the patch stream; at selected layers these tokens cross-attend to temporal embeddings derived from compact, human-readable temporal descriptors encoded by the same frozen LLM, then feed temporal context back via self-attention. This disentangles time series signal and temporal information while maintaining a low parameter budget. We show that by training only the cross-attention modules and explicitly disentangling time series signal and temporal information, TPC consistently outperforms both full fine-tuning and shallow conditioning strategies, achieving state-of-the-art performance in long-term forecasting across diverse datasets. Code available at: https://github.com/fil-mp/Deep_tpc

Figures & tables

Figure 1: TPC Overview. TPC modules are inserted at L layers of a frozen LLM. TS-tokens (yellow) mediate between time series patch embeddings (blue) and temporal embeddings (green) by performing cross-attention to gather temporal information, which is then distributed to patches via self-attention.

Method	TPC (ours)		AutoTimes		OFA		TimeLLM		$S^{2}$ IP-LLM		PatchTST		Dlinear
Method	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	0.399	0.422	0.409	0.438	0.415	0.429	0.451	0.451	0.425	0.440	0.444	0.453	0.418	0.439
ETTh2	0.355	0.399	0.365	0.403	0.367	0.402	0.355	0.398	0.358	0.403	0.381	0.411	0.502	0.481
ETTm1	0.346	0.379	0.358	0.382	0.355	0.386	0.349	0.382	0.347	0.382	0.363	0.391	0.357	0.389
ETTm2	0.265	0.323	0.281	0.330	0.265	0.328	0.271	0.332	0.261	0.326	0.267	0.325	0.275	0.340
Weather	0.230	0.267	0.243	0.278	0.237	0.270	0.230	0.268	0.229	0.267	0.225	0.264	0.248	0.300

Table 1: Long-term forecasting results for {96, 192, 336, 720} horizons. A lower value indicates a better performance. All results are averaged from four forecasting horizons {96, 192, 336, 720}. Underlined : second best. Bold : best. We reproduced AutoTimes, TimeLLM, OFA,

S^{2}

IP-LLM, PatchTST, and Dlinear results using their official open-source implementations.

Method	ETTh1		ETTm1		Weather
Method	MSE	MAE	MSE	MAE	MSE	MAE
TPC (ours)	0.399	0.422	0.346	0.379	0.230	0.267
Pos. Embeddings	0.409	0.438	0.358	0.382	0.243	0.278
Prefix-prompts	0.414	0.441	0.352	0.390	0.243	0.279
Full Fine-Tuning	0.404	0.435	0.357	0.381	0.235	0.273
Partial Fine-Tuning	0.407	0.437	0.365	0.393	0.234	0.270

Table 2: Ablation study on temporal modality and training strategies across diverse datasets. Underlined : second best. Bold : best

LLM-Based Time Series Forecasting

LLM-based time series forecasting uses large language models to predict future values from numerical histories and contextual data.

See all 100 papers

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting
May 19, 2025Siming Sun, Kai Zhang, Xuejun Jiang +2Time Series Forecasting LLM-Based Time Series Forecasting
Zhejiang University
From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Jun 20, 2026Xin Qiu, Junlong Tong, Yao Zhang +3Time Series Reasoning Time Series QA
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU
Representing Time Series as Structured Programs for LLM Reasoning
Jun 10, 2026Jaeho Kim, Changhun Oh, Seokhyun Lee +2Time Series Reasoning Time Series QA
Korea University · Mila, University of Montreal

May 19, 2025J/K move · Enter open · S save

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

Siming Sun, Kai Zhang, Xuejun Jiang, Wenchao Meng+1

Zhejiang University

Time Series Forecasting LLM-Based Time Series Forecasting

Open paper AskPDF arXiv

Adapting pretrained Large Language Models (LLMs) for time series forecasting primarily relies on token-level linguistic-temporal alignment, leading to the stacking of logically disjointed tokens as input. While empirically effective, these methods overlook a fundamental capability of LLMs: modeling linguistic logic and structure, rather than merely processing token features. To address this limitation, we propose the Markovian-Guided Structure-Aware Alignment (MGSAA). Our core contribution is a framework that transcends pointwise feature matching to achieve global structural isomorphism between the linguistic and temporal domains. Specifically, MGSAA distills latent evolutionary patterns of language within LLMs into a Markovian state transition graph, which is transferred as a structural prior to the time series domain. Under this prior, time series patches are decoded into latent states and then aligned via state-constrained cross-attention. Ultimately, MGSAA generates a token sequence topologically isomorphic to the LLM's inherent mental structure, reactivating its reasoning capabilities for forecasting. Comprehensive evaluations across multiple benchmarks demonstrate that MGSAA achieves state-of-the-art performance, providing an innovative solution for cross-modal alignment in LLM for time series forecasting. Code is available at https://github.com/sunzju/MGSAA.

Abstract

LLM-Based Time Series Forecasting

Explore similar work

Markovian Linguistic-Temporal Bridge: Unlocking the Potential of LLMs for Time Series Forecasting

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Representing Time Series as Structured Programs for LLM Reasoning