Perceive, Solve, Reason: Progressive Capability Formation in Time Series-LLMs

Xin Qiu; Junlong Tong; Wei Zhang; Xiaoyu Shen

cs.CLOct 16, 2026

Perceive, Solve, Reason: Progressive Capability Formation in Time Series-LLMs

Authors: Xin Qiu, Junlong Tong, Wei Zhang, Xiaoyu Shen

Organizations: EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Zhejiang University · Shanghai Jiao Tong University

Abstract

Large language models (LLMs) can now perceive, describe, and reason about images and audio, yet time series understanding remains underestimated: most work uses LLMs as forecasters and classifiers, reasoning studies are few and narrow, and no suite trains and evaluates the full range of capabilities. In this paper, we build the first such suite, with 337K training and 44K evaluation samples across three progressive capabilities, Perceive, Solve, and Reason, and use it to ask how time series should enter an LLM that must master all three, and in what order. We find that a dedicated encoder is essential: with one, a 4B model surpasses GPT-4o on every perception attribute we test, whereas textualizing the series costs 8--10 points in reasoning accuracy. The capabilities then build on one another: learning them in order beats mixing the same data and rivals reasoning-only training with over three times the reasoning data, but only when an encoder is used. Analyses further reveal that capabilities are transferred through distinct network depths rather than simply stacked, and that a stage converged on its own benchmark is not necessarily ready for the next. Distilled into a simple recipe, these findings yield 4B TS-LLMs that, without any target-domain data, outperform four recent 7B TS-LLMs on seven of eight out-of-domain metrics, by up to 11 points. We therefore advocate treating time series as a first-class modality for LLMs, read through dedicated encoders and taught to perceive before they reason.

Figures & tables

Figure 1: Three Model Architecture.

Figure 2: Categorical and numerical perception comparison across different methods.

Model	Training Setting	Fore. MSE $\downarrow$	Impu. MSE $\downarrow$	Ano Det. Acc. $\uparrow$	Class. Acc. $\uparrow$
Qwen3-4B	No fine-tuning	1.970	0.0286	0.333	0.296
Pre-align	Stage 1 only	1.842	0.0194	0.587	0.459
	Stage 2 only	1.368	0.0136	0.682	0.567
	Stage (1 $\rightarrow$ 2)	1.328	0.0132	0.739	0.628
Post-align	Stage 1 only	1.716	0.0217	0.612	0.431
	Stage 2 only	1.423	0.0146	0.660	0.529

Table 1: Results of four task solving evaluation sets.

Model	Training Setting	Decoding	Grounding	Inferring	Extrapolating	Acting
Pre-align	Stage 3 Only	72.2	68.5	61.3	57.2	50.7
	Stage (1 $\rightarrow$ 3)	77.8	81.5	63.3	62.2	51.8
	Stage (2 $\rightarrow$ 3)	75.1	72.0	65.8	59.5	53.9
	Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	82.5	79.1	70.7	65.4	57.2
Post-align	Stage 3 Only	68.5	66.2	58.9	58.8	49.2
	Stage (1 $\rightarrow$ 3)	75.6	73.8	61.2	64.1	54.3

Table 2: Accuracy

\uparrow

(%) of different strategies on the Evaluation Set B of high-level reasoning.

Model	Training Setting	Ture/False QA	Multi-Choice QA	Open-end QA	Open-end QA
Model	Training Setting	Acc. $\uparrow$	Acc. $\uparrow$	ROUGE-L $\uparrow$	BLEU $\uparrow$
Pre-align	Stage 3 Only	71.4	58.6	43.8	24.7
	Stage (1 $\rightarrow$ 3)	78.3	62.2	45.5	25.8
	Stage (2 $\rightarrow$ 3)	74.5	65.1	47.2	26.7
	Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	80.6	68.8	51.6	31.5
Post-align	Stage 3 Only	66.3	60.3	42.1	22.3

Table 3: Performance on the Evaluation Set A of high-level reasoning.

Figure 3: Accuracy across different training data scales.

Figure 4: Reason under the Textualization paradigm across training data scales of high-level reasoning.

Training Setting	Training Data	Evaluation Performance	Relative Perf.	I $\rightarrow$ II $\rightarrow$ III
Stage 1 only	(40% $\rightarrow$ 100%) I	0.613, 0.662, 0.738 , 0.745, 0.751	98.3%	0.743
Stage (1 $\rightarrow$ 2)	70% I, (40% $\rightarrow$ 100%) II	0.542, 0.603, 0.644 , 0.649, 0.652	98.8%	0.684
Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	70% I, 70% II, (40% $\rightarrow$ 100%) III	0.537, 0.569, 0.616, 0.663 , 0.682	97.2%	0.710

Table 4: Stage-wise performance of the Pre-align model under increasing training data scales. For each stage, we progressively increase the training data while fixing preceding stages at their identified saturation points. Stage 1 uses Numerical Perception Relative Accuracy; Stage 2 uses Anomaly Detection and Classification Accuracy; Stage 3 uses the average Accuracy on Evaluation Set B. For convenient evaluation, we select accuracy-based tasks as measures. Bold denotes the local saturation points.

Figure 5: Optimization strength dynamics across capability stages.

Figure 12

Models	Weather		Finance (7 days)			Finance (30 days)
Models	MCQ. $\uparrow$	Pre. $\uparrow$	3-way $\uparrow$	5-way $\uparrow$	MSE $\downarrow$	3-way $\uparrow$	5-way $\uparrow$	MSE $\downarrow$
TimeOmni-1-7B	41.67	51.40	40.76	34.97	0.622	37.39	20.00	1.490
ITFormer-7B	57.14	36.26	40.14	33.57	0.174	51.03	31.72	0.907
ChatTS-7B	56.21	28.79	19.71	16.09	2.047	19.38	13.46	3.342
PATRA-7B	60.46	52.04	63.33	60.80	0.191	54.39	43.70	0.822
TS-LLM (Pre-align)	67.24	57.17	68.19	57.82	0.187	60.29	41.51	0.837

Table 7: Out-of-domain results on reasoning tasks. Red / blue indicate the 1st/2nd best results.

Method	TimeSeriesExam					CaTS-Bench
Method	Ano. Det.	Causality	Noise	Pattern	Similarity	Caption Ret.	TS Ret.	Amp. Comp.	Mean Comp.
Original Paper	28.0	52.0	26.0	47.0	45.0	44.0	29.0	43.0	35.0
TimeOmni-1	42.6	37.5	56.0	56.4	46.7	57.0	37.0	92.5	67.5
TS-LLM (Pre-align)	49.1	47.2	60.7	61.6	55.0	52.0	46.0	82.5	77.5

Table 8: Results on TimeSeriesExam and CaTS-Bench. Bold indicates the best results.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Split	Source	Samples	Task Type	Metric
Training	ChatTS	$\sim$ 105K	Temporal Perception	–
Evaluation A	Manual	525	Categorical / Numerical	F1 / Relative Acc.
Evaluation B	Synthetic	1,616	Categorical / Numerical	F1 / Relative Acc.

Appendix

Table 9: Statistics of the Stage 1 perception datasets.

Source	Task	Total Samples	Train Samples	Test Samples	Metric
TSQA	Forecasting	42,557	34,046	8,511	MSE
	Imputation	38,657	30,926	7,731	MSE
	Anomaly Detection	37,000	29,600	7,400	Accuracy
	Classification	37,000	29,600	7,400	Accuracy
TS Instruct	Classification	2,945	2,356	589	Accuracy
Total	–	158,159	126,528	31,631	–

Appendix

Table 10: Detailed statistics of the Stage 2 task-solving datasets.

Split	Source	# Samples	Metric
Training	TSQA Reasoning QA	26,163	–
	ChatTS TSEvol + Instruct Follow	29,320	–
	TSCognition	32,869	–
	TS Instruct	12,852	–
Evaluation A	Binary QA	4,547	Accuracy
	Multiple-Choice QA		Accuracy

Appendix

Table 11: Detailed statistics of the Stage 3 high-level reasoning datasets.

Figure 10: Distribution of TSCognition samples across eight domains and five hierarchical tasks.

Tasks	Variable Numbers			Time Series Length			Choices
Tasks	Min	Max	Avg.	Min	Max	Avg.	Min	Max	Avg.
Decoding Task	4	5	4.7	437	982	742	4	5	4.7
Grounding Task	3	6	4.4	656	1018	871	4	5	4.3
Inferring Task	3	6	5.1	882	1059	912	4	5	4.6
Extrapolating Task	3	6	5.2	601	1052	846	4	5	4.5
Acting Task	4	6	5.5	468	1047	795	4	5	4.2

Appendix

Table 12: Task-level statistics of time series in TSCognition.

Category	Samples
Anomaly Detection	108
Causality	72
Noise Understanding	84
Pattern Recognition	362
Similarity Analysis	120
Total	746

Appendix

Table 13: Statistics of TimeSeriesExam.

Task	Samples
Caption Retrieval	100
Time Series Retrieval	100
Amplitude Comparison	40
Mean Comparison	40
Total	280

Appendix

Table 14: Statistics of CaTS-Bench used in our evaluation.

Domain	Evaluation Tasks
Weather	MCQA / Prediction
Finance (7 days)	3-way / 5-way / Forecasting
Finance (30 days)	3-way / 5-way / Forecasting

Appendix

Table 15: Evaluation settings of MTBench.

Method	Trend		Season		Noise		Local		Corr.	Clus.
Method	Cate.	Num.	Cate.	Num.	Cate.	Num.	Cate.	Num.	Cate.	Cate.
Pre-align	0.937	0.865	0.911	0.872	0.857	0.534	0.799	0.700	0.831	0.783
GPT-4o (Text)	0.664	0.839	0.751	0.546	0.654	0.287	0.237	0.165	0.361	0.294
GPT-4o (Vision)	0.691	0.423	0.907	0.582	0.659	0.361	0.415	0.302	0.394	0.411
Post-align	0.922	0.880	0.931	0.834	0.900	0.479	0.824	0.733	0.771	0.804

Appendix

Table 16: Detailed categorical and numerical perception results.

Model	Training Setting	15%	25%	50%	75%	100%
Pre-align	Stage 3 Only	0.414	0.464	0.527	0.586	0.619
	Mixed Training	0.479	0.520	0.583	0.673	0.689
	Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	0.528	0.574	0.598	0.664	0.710
Post-align	Stage 3 Only	0.403	0.453	0.524	0.585	0.617
	Mixed Training	0.461	0.502	0.570	0.664	0.678
	Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	0.520	0.561	0.614	0.680	0.701

Appendix

Table 17: Detailed reasoning accuracy under different Stage 3 training data scales.

Training Setting	Data Scale	Evaluation Performance	Relative Perf.	Full Progressive
Stage 1	40%	0.613	–	0.743
	55%	0.662	–
	70%	0.738	98.3%
	85%	0.745	–
	100%	0.751	100%
Stage (1 $\rightarrow$ 2)	40%	0.542	–	0.684

Appendix

Table 18: Detailed stage-wise performance under increasing training data scales. For Stage (1

\rightarrow

2), Stage 1 is fixed at its 70% saturation point; for Stage (1

\rightarrow

\rightarrow

3), Stages 1 and 2 are both fixed at 70%. Bold denotes the identified local saturation points.

Model	Trend		Season		Noise		Local		Multivariate
Model	Cate.	Num.	Cate.	Num.	Cate.	Num.	Cate.	Num.	Corr.	Clus.
Pre-align	0.894	0.843	0.859	0.843	0.822	0.558	0.786	0.727	0.788	0.761
Post-align	0.873	0.846	0.892	0.834	0.861	0.506	0.752	0.709	0.755	0.747

Appendix

Table 19: Stage 1 perception results on Qwen3-1.7B.

Model	Training Setting	Fore. MSE $\downarrow$	Impu. MSE $\downarrow$	Ano. Det. Acc. $\uparrow$	Class. Acc. $\uparrow$
Qwen3-1.7B	No Fine-tuning	1.884	0.0312	0.303	0.234
Pre-align	Stage 1 Only	1.813	0.0218	0.548	0.421
	Stage 2 Only	1.412	0.0151	0.651	0.542
	Stage (1 $\rightarrow$ 2)	1.438	0.0144	0.684	0.576
Post-align	Stage 1 Only	1.782	0.0235	0.574	0.408
	Stage 2 Only	1.469	0.0162	0.648	0.507

Appendix

Table 20: Stage 2 task-solving results on Qwen3-1.7B.

Model	Training Setting	Decoding	Grounding	Inferring	Extrapolating	Acting
Pre-align	Stage 3 Only	65.8	63.7	54.2	53.6	46.6
	Stage (1 $\rightarrow$ 3)	73.8	76.9	60.7	58.9	49.4
	Stage (2 $\rightarrow$ 3)	74.9	69.5	65.6	58.1	52.1
	Stage (1 $\rightarrow$ 2 $\rightarrow$ 3)	78.1	74.2	63.1	65.2	55.6
Post-align	Stage 3 Only	62.9	62.8	53.1	55.4	47.7
	Stage (1 $\rightarrow$ 3)	72.5	70.3	59.4	61.5	50.9

Appendix

Table 21: Stage 3 reasoning results on Evaluation Set B using Qwen3-1.7B.

Configuration	Setting
Primary backbone	Qwen3-4B
Additional backbone	Qwen3-1.7B
GPUs	2 $\times$ NVIDIA A800
Framework	PyTorch + HuggingFace Transformers
Distributed training	DeepSpeed + Accelerate
Precision	bfloat16

Appendix

Table 22: Implementation and training configurations used in our experiments.

Time Series Reasoning

Time series reasoning studies how models interpret temporal patterns, answer questions about sequences, and use evidence to make forecasts.

See all 44 papers

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Jun 20, 2026Xin Qiu, Junlong Tong, Yao Zhang +3Time Series Reasoning Time Series QA
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU
Thoth: Mid-Training Bridges LLMs to Time Series Understanding
Mar 1, 2026Jiafeng Lin, Yuxuan Wang, Jialong Wu +3Temporal Reasoning in Language Models LLM Training
School of Software, BNRist, Tsinghua University.
Representing Time Series as Structured Programs for LLM Reasoning
Jun 10, 2026Jaeho Kim, Changhun Oh, Seokhyun Lee +2Time Series Reasoning Time Series QA
Korea University · Mila, University of Montreal

Jun 20, 2026J/K move · Enter open · S save

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Xin Qiu, Junlong Tong, Yao Zhang, Yunpu Ma+2

Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU

Time Series Reasoning Time Series QA Temporal Reasoning in Language Models

Open paper AskPDF arXiv

Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities, yet gains remain limited. We attribute this to a fundamental mismatch between existing task formulations and LLM strengths: most settings reduce time series understanding to curve-fitting systems, focusing on low-level prediction while ignoring the semantic, contextual, and reasoning-intensive nature of real-world temporal decision-making.To address these limitations, we introduce TSCognition, a multimodal benchmark for multi-dimensional time series reasoning. It collects real-world time series and textual information from 15 public sources and constructs approximately 41K QA samples around five cognitive reasoning tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting. Building on this, we further propose TSAlign, a unified framework that encodes time series into compact patch-level representations and aligns them with semantic directions in the LLM embedding space via gated residual injection and multivariate fusion.Experiments show that TSAlign outperforms existing LLM, VLM, and time series QA baselines on TSCognition and the publicly available TimerBed benchmark while substantially reducing computational cost.Code is available at: https://github.com/EIT-NLP/CognitiveTSR

Abstract

Time Series Reasoning

Explore similar work

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Thoth: Mid-Training Bridges LLMs to Time Series Understanding

Representing Time Series as Structured Programs for LLM Reasoning