Thoth: Mid-Training Bridges LLMs to Time Series Understanding

Jiafeng Lin; Yuxuan Wang; Jialong Wu; Huakun Luo; Zhongyi Pei; Jianmin Wang

cs.CLMar 1, 2026

Thoth: Mid-Training Bridges LLMs to Time Series Understanding

Authors: Jiafeng Lin, Yuxuan Wang, Jialong Wu, Huakun Luo, Zhongyi Pei, Jianmin Wang

Organizations: School of Software, BNRist, Tsinghua University.

Abstract

Large Language Models (LLMs) have demonstrated remarkable success in general-purpose reasoning. However, they still struggle to understand and reason about time series data, which limits their effectiveness in decision-making scenarios that depend on temporal dynamics. In this paper, we propose Thoth, the first family of mid-trained LLMs with general-purpose time series understanding capabilities. As a pivotal intermediate stage, mid-training achieves task- and domain-agnostic alignment between time series and natural language, for which we construct Book-of-Thoth, a high-quality, time-series-centric mid-training corpus. Book-of-Thoth enables both time-series-to-text and text-to-time-series generation, equipping LLMs with a foundational grasp of temporal patterns. To better evaluate advanced reasoning capabilities, we further present KnoTS, a novel benchmark of knowledge-intensive time series understanding, designed for joint reasoning over temporal patterns and domain knowledge. Extensive experiments demonstrate that mid-training with Book-of-Thoth enables Thoth to significantly outperform its base model and advanced LLMs across a range of time series question answering benchmarks. Moreover, Thoth exhibits superior capabilities when fine-tuned under data scarcity, underscoring the effectiveness of mid-training for time series understanding. Code is available at: https://github.com/thuml/Thoth.

Figures & tables

Figure 1: Pivotal role of mid-training in bridging LLMs to time series understanding. Mid-training expands the capabilities of pre-trained LLMs while providing a critical transition and warm-up for subsequent post-training on specific time series question answering tasks.

Figure 2: The automated pipeline starts by synthesizing time series through combinations of diverse base functions. For each generated series, both the raw values and their visualizations are provided to GPT‑5.2 to produce a diverse range of time-series-to-text descriptions. These textual descriptions are then paired with the corresponding time series to construct complementary text-to-time-series generation data. In addition, a curated corpus of textual time series knowledge is incorporated to preserve the general-purpose capabilities of the pre-trained models during mid-training.

Figure 3: Components of Book of Thoth with 26.6M tokens.

Figure 4: A brief case of a reasoning problem from the KnoTS benchmark, the complete case can be found in the Appendix E .

Proprietary Models
Model	ChatTime Benchmark				Time-MQA Benchmark
	Trend $\uparrow$	Volatility $\uparrow$	Seasonality $\uparrow$	Outliers $\uparrow$	Forecast $\downarrow$	Impu. $\downarrow$	Clas. $\uparrow$	Anom. $\uparrow$	Judge. $\uparrow$	MCQ $\uparrow$	Open. $\uparrow$
Gemini-3-Flash-Preview	0.837	0.863	0.520	0.747	0.642	0.263	0.34	0.70	0.62	0.52	6.60
GPT-4o-mini	0.850	0.433	0.427	0.673	0.546	0.252	0.16	0.80	0.76	0.60	7.32
Grok-4.1-Fast	0.647	0.383	0.620	0.747	0.503	0.303	0.14	0.68	0.78	0.60	7.62
Open-source Large Language Models

Table 1: Performance comparison across existing benchmarks. For the Time-MQA tasks, we report evaluation metrics including SMAPE for forecasting and imputation tasks, LLM-as-a-judge for open-ended tasks, and Accuracy (ACC) for others. For the ChatTime benchmark, the table reports the average results across all input lengths. The best results are highlighted in bold and the second best are underlined .

Figure 5: Performance of various models in reasoning and decision-making on the KnoTS benchmark. All evaluations are conducted using Gemini-3-Pro-Preview with scores ranging from 0.0 to 10.0.

Figure 6: Performance of Supervised Fine-tuning on ChatTime and Time-MQA benchmark using a 5% sampled training set. For each benchmark, an equal proportion of data is randomly selected from each sub-task for training, with no separate model trained per sub-task.

Figure 7: Aggregate performance of Thoth-8B on ChatTime benchmark at different mid-training data scales. As the data scale increases, Thoth-8B exhibits stronger average performance.

Model	ChatTime Benchmark				Time-MQA Benchmark							KnoTS Benchmark
	Trend $\uparrow$	Volatility $\uparrow$	Seasonality $\uparrow$	Outliers $\uparrow$	Forecast $\downarrow$	Impu. $\downarrow$	Clas. $\uparrow$	Anom. $\uparrow$	Judge. $\uparrow$	MCQ $\uparrow$	Open. $\uparrow$	Reason $\uparrow$	Decision $\uparrow$
Qwen3-8B	0.163	0.220	0.100	0.447	0.671	0.533	0.02	0.48	0.62	0.54	6.28	3.94	1.91
– TS-to-Text	0.360	0.487	0.347	0.453	0.495	0.286	0.14	0.48	0.68	0.46	3.32	3.13	1.45
– Text-to-TS	0.967	0.677	0.630	0.733	0.532	0.476	0.12	0.56	0.68	0.58	5.10	5.23	3.04
– General	0.963	0.547	0.387	0.763	0.538	0.363	0.16	0.58	0.70	0.60	6.12	5.30	3.06
Thoth-8B	0.973	0.750	0.603	0.753	0.522	0.333	0.100	0.58	0.72	0.64	5.98	5.54	3.21

Table 2: Ablation results on Book-of-Thoth . We separately remove the three types of corpora, Time series-to-text pairs (TS-to-Text), Text-to-time series pairs (Text-to-TS), and General pre-training data (General), for training and compare them with Thoth-8b.

Figure 8: Case study of Thoth-30B-A3B model inference on the KnoTS benchmark. Detailed comparative results of various models can be found in Appendix F for further reference.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Model	ChatTime Benchmark				Time-MQA Benchmark							KnoTS Benchmark
	Trend $\uparrow$	Volatility $\uparrow$	Seasonality $\uparrow$	Outliers $\uparrow$	Forecast $\downarrow$	Impu. $\downarrow$	Clas. $\uparrow$	Anom. $\uparrow$	Judge. $\uparrow$	MCQ $\uparrow$	Open. $\uparrow$	Reason $\uparrow$	Decision $\uparrow$
Thoth-8B	0.973 $\pm$ 0.003	0.750 $\pm$ 0.005	0.603 $\pm$ 0.007	0.753 $\pm$ 0.003	0.522 $\pm$ 0.019	0.333 $\pm$ 0.013	0.10 $\pm$ 0.00	0.58 $\pm$ 0.01	0.72 $\pm$ 0.02	0.64 $\pm$ 0.02	5.98 $\pm$ 0.17	5.54 $\pm$ 0.24	3.21 $\pm$ 0.18
Thoth-30B-A3B	0.957 $\pm$ 0.003	0.770 $\pm$ 0.010	0.573 $\pm$ 0.010	0.740 $\pm$ 0.007	0.432 $\pm$ 0.021	0.247 $\pm$ 0.014	0.02 $\pm$ 0.00	0.70 $\pm$ 0.00	0.78 $\pm$ 0.01	0.72 $\pm$ 0.02	7.42 $\pm$ 0.11	8.05 $\pm$ 0.21	4.96 $\pm$ 0.31

Appendix

Table 3: Robustness of Thoth-30B-A3B and Thoth-8B performance on all benchmarks. The results are obtained from three random runs.

Figure 9: Examples of carefully designed prompts used for constructing structured data of Book-of-Thoth .

Figure 10: Examples of carefully designed prompts used for constructing unstructured data of Book-of-Thoth .

Dataset	TTR $\uparrow$	1-Ent $\uparrow$	2-Ent $\uparrow$	3-Ent $\uparrow$	Avg-Len
KnoTS (Ours)	0.2109	9.69	13.02	13.67	169.8
MTBench ( 2025 )	0.1131	7.54	10.69	12.00	77.5
Time-MQA ( 2025 )	0.0877	7.35	8.90	9.25	49.7
ChatTime ( 2025a )	0.0098	5.56	6.86	7.40	90.4
ChatTS ( 2024 )	0.0309	7.16	8.89	9.45	293.7
TS-Insights ( 2025 )	0.0118	4.81	5.05	5.10	37.0

Appendix

Table 4: Diversity and complexity analysis of time series-centric questions in different benchmarks.

Figure 11: A complete example of reasoning questions in the KnoTS benchmark. It is the full version of Figure 4 in the main text.

Figure 12: A complete example of reasoning questions in the KnoTS benchmark.

Figure 13: A complete example of decision-making questions in the KnoTS benchmark.

Figure 14: A showcase of seasonality recognition question from various LLMs in the ChatTime benchmark.

Figure 15: A showcase of an open-ended question from three LLMs in the Time-MQA benchmark.

Figure 16: A showcase of reasoning question from Grok-4.1-Fast and Thoth-30B-A3B in the KnoTS benchmark.

Proprietary Models
Model	Trend				Volatility				Seasonality				Outliers
	64	128	256	Avg	64	128	256	Avg	64	128	256	Avg	64	128	256	Avg
Gemini-3-Flash-Preview	0.970	0.800	0.840	0.870	0.880	0.860	0.920	0.887	0.550	0.550	0.460	0.520	0.700	0.800	0.740	0.747
GPT-4o-mini	0.890	0.790	0.870	0.850	0.380	0.500	0.420	0.433	0.540	0.370	0.370	0.427	0.660	0.660	0.700	0.673
Grok-4.1-Fast	0.680	0.700	0.560	0.647	0.460	0.380	0.310	0.383	0.590	0.730	0.540	0.620	0.700	0.750	0.790	0.747
Open-source Large Language Models

Appendix

Table 5: Full results of ChatTime benchmark ( Wang et al., 2025a ) . There are three fixed time series lengths of 64, 128, and 256 of the four time series pattern recognition tasks. Avg means the average results from the three lengths.

Temporal Reasoning in Language Models

Temporal reasoning in language models studies how models represent, order, update, and apply facts and events across time.

See all 92 papers

TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale
Apr 11, 2026Malgorzata Gwiazda, Yifu Cai, Mononito Goswami +2Time Series Reasoning LLM Evaluation
Technical University of Munich, Munich, Germany · Auton Lab, School of Computer Science, Carnegie Mellon University, Pittsburgh, US
From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Jun 20, 2026Xin Qiu, Junlong Tong, Yao Zhang +3Time Series Reasoning Time Series QA
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU
Representing Time Series as Structured Programs for LLM Reasoning
Jun 10, 2026Jaeho Kim, Changhun Oh, Seokhyun Lee +2Time Series Reasoning Time Series QA
Korea University · Mila, University of Montreal

Apr 11, 2026J/K move · Enter open · S save

TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale

Malgorzata Gwiazda, Yifu Cai, Mononito Goswami, Arjun Choudhry+1

Technical University of Munich, Munich, Germany · Auton Lab, School of Computer Science, Carnegie Mellon University, Pittsburgh, US

Time Series Reasoning LLM Evaluation Synthetic Benchmark Generation Time Series Modeling

Open paper AskPDF arXiv

Large Language Models (LLMs) have shown promising performance in time series modeling tasks, but do they truly understand time series data? While multiple benchmarks have been proposed to answer this fundamental question, most are manually curated and focus on narrow domains or specific skill sets. To address this limitation, we propose scalable methods for creating comprehensive time series reasoning benchmarks that combine the flexibility of templates with the creativity of LLM agents. We first develop TimeSeriesExam, a multiple-choice benchmark using synthetic time series to evaluate LLMs across five core reasoning categories: pattern recognitionnoise understandingsimilarity analysisanomaly detection, and causality. Then, with TimeSeriesExamAgent, we scale our approach by automatically generating benchmarks from real-world datasets spanning healthcare, finance and weather domains. Through multi-dimensional quality evaluation, we demonstrate that our automatically generated benchmarks achieve diversity comparable to manually curated alternatives. However, our experiments reveal that LLM performance remains limited in both abstract time series reasoning and domain-specific applications, highlighting ongoing challenges in enabling effective time series understanding in these models. TimeSeriesExamAgent is available at https://github.com/magwiazda/TimeSeriesExamAgent.

Abstract

Temporal Reasoning in Language Models

Explore similar work

TimeSeriesExamAgent: Creating Time Series Reasoning Benchmarks at Scale

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Representing Time Series as Structured Programs for LLM Reasoning