Organizations: CNIC, CAS, Beijing, China · University of Chinese Academy of Sciences · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences · ByteDance, Beijing, China · Tsinghua University, Beijing, China · ByteDance, San Jose, USA · Nanjing University, Nanjing, China
Driven by the increasingly complex and decision-oriented demands of time series analysis, we introduce the Semantic-Conditional Time Series Reasoning task, which extends conventional time series analysis beyond purely numerical modeling to incorporate contextual and semantic understanding. To further enhance the mode's reasoning capabilities on complex time series problems, we propose a two-round reinforcement learning framework: the first round strengthens the mode's perception of fundamental temporal primitives, while the second focuses on semantic-conditioned reasoning. The resulting model, KairosVL, achieves competitive performance across both synthetic and real-world tasks. Extensive experiments and ablation studies demonstrate that our framework not only boosts performance but also preserves intrinsic reasoning ability and significantly improves generalization to unseen scenarios. To summarize, our work highlights the potential of combining semantic reasoning with temporal modeling and provides a practical framework for real-world time series intelligence, which is in urgent demand.
Figures & tables
Figure 1. Comparison of time series task formulations from the perspectives of Numerical Data Processing and Semantic-Conditional Reasoning (taking the real-world time series anomaly diagnosis task as an example). While the majority of existing studies remain confined to the former, real-world applications also urge for the latter, an end-to-end reasoning paradigm.
Figure 2. Details of KairosDataPipe and KairosDataset.
Table 1. Overall Question-Answer Accuracy (%) of different models on both Real-World datasets (Dataset A ) and KairosBench (Dataset B ). The highest, the second-best scores are shown in bold and underlined, respectively. For each sample in the dataset, we sample the answer five times and compute the mean accuracy value. The margin of errors can be found in Appendix E .
Figure 4. Ablation studies on different training pipeline.
Figure 5. Visualization of some metrics in training process.
Figure 6. Ablation studies on different KL loss settings.
Figure 7. Case Study
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Scenario #1
Scenario #2
Scenario #3
Donut ( Xu et al., 2018 )
26.5
27.3
21.6
LSTMAD ( Malhotra et al., 2015 )
32.9
21.3
27.9
MatrixProfile ( Zhu et al., 2018 )
30.4
35.6
25.4
TranAD ( Tuli et al., 2022 )
34.8
23.2
25.6
KairosVL
88.9
70.4
73.3
Appendix
Table 2. Action Accuracy (%) of Time-Series-Only Anomaly Detection Methods on Dataset A .
Models
Scen. #1
Scen. #2
Scen. #3
GPT-4o (text)
33.3
41.6
17.8
GPT-4o (image)
83.3
64.5
77.3
Gemini-2.5 Flash (text)
72.6
42.5
17.3
Gemini-2.5 Flash (image)
79.4
62.5
48.7
GLM-4V-Plus (text)
38.9
30.8
0.0
GLM-4V-Plus (image)
66.7
70.0
49.3
Appendix
Table 3. Comparison of Question-Answer Accurary (%) on real world time series reasoning tasks between "Time Series as Text" and "Time Series as Image" paradigms. Since each sample in Scenario #3 contains a larger amount of data, GLM-4V-Plus struggles to produce valid responses within the limited output token budget when handling such long inputs. It often generates irrelevant gibberish or text unrelated to the question, resulting in an accuracy of zero for this scenario.
Figure 8. Comparison of token consumption between the “Time Series as Text” and “Time Series as Image” paradigms when using GPT-4o. Generally, when using GPT-4o, representing time series in textual form consumes about 5–8 tokens per numerical value. In contrast, even a high-resolution image of 5400 × 1800 pixels—which can visually encode tens of thousands of values—requires only around 1,400 tokens.
Figure 9. Drawbacks of existing time series reasoning tasks, and the ideal reasoning task.
Figure 10. Rewards On Different Tasks During Training Process.
Dataset
Margin of Error (%)
Scenario #1
±0.9
Scenario #2
±1.4
Scenario #3
±2.1
Fact-Adherent
±0.8
Predictive
±0.6
Event-Aware
±0.6
Appendix
Table 4. KairosVL’s margin of error of five experimental results relative to the mean.
Figure 11. System and user prompts sent to MLLMs.
Figure 12. The task specific descriptions for the Generation Agent (Part 1).
Figure 13. The task specific descriptions for the Generation Agent (Part 2).
Figure 14. The task shared requirements for the Generation Agent.
Figure 15. The prompt for the Data Necessity Judge.
Figure 16. The prompt for the QA-Data Consistency Judge.
Figure 17. The prompt for the Data & Plot Requirement Judge.
Figure 18. The python function that used to execute visualization code to generate time series plot.
Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities, yet gains remain limited. We attribute this to a fundamental mismatch between existing task formulations and LLM strengths: most settings reduce time series understanding to curve-fitting systems, focusing on low-level prediction while ignoring the semantic, contextual, and reasoning-intensive nature of real-world temporal decision-making.To address these limitations, we introduce TSCognition, a multimodal benchmark for multi-dimensional time series reasoning. It collects real-world time series and textual information from 15 public sources and constructs approximately 41K QA samples around five cognitive reasoning tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting. Building on this, we further propose TSAlign, a unified framework that encodes time series into compact patch-level representations and aligns them with semantic directions in the LLM embedding space via gated residual injection and multivariate fusion.Experiments show that TSAlign outperforms existing LLM, VLM, and time series QA baselines on TSCognition and the publicly available TimerBed benchmark while substantially reducing computational cost.Code is available at: https://github.com/EIT-NLP/CognitiveTSR
Xin Qiu, Junlong Tong, Yao Zhang +3
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU
Time series reasoning tasks often start with a natural language question and require targeted analysis of a time series. Evidence may span the full series or appear in a few short intervals, so the model must decide what to inspect. Most existing approaches encode the entire time series into a fixed representation before inference, regardless of whether or not the entire sequence is relevant. We introduce ARTIST, which formulates time-series reasoning as a sequential decision problem. ARTIST interleaves reasoning with adaptive temporal segment selection. It adopts a controller-reasoner architecture and uses reinforcement learning to train the controller role to select informative segments and the reasoner role to generate segment-conditioned reasoning traces and final answers. During inference, the model actively acquires task-relevant information instead of relying on a static summary of the full sequence. We use a novel hierarchical policy optimization approach for post-training that allows the model to excel in both segment selection and question-answering behavior. We evaluate ARTIST on six time-series reasoning benchmarks and compare it with large language models, vision-language models, and prior time-series reasoning systems. ARTIST improves average accuracy by 6.46 absolute percentage points over the strongest baseline. The largest gains appear on rare event localization and multi-segment reasoning tasks. Supervised fine-tuning improves performance, and reinforcement learning provides additional gains by optimizing question-adaptive segment selection. These results show that selective data use drives effective time-series reasoning.
Shvat Messica, Jiawen Zhang, Kevin Li +2
Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · The Hong Kong University of Science and Technology (Guangzhou) · Massachusetts Institute of Technology +1
Timeseries multimodal large language models (TS-MLLMs) have recently begun leveraging the reasoning capabilities of large language models (LLMs) for question-answering tasks. However, these models often fail to capture dynamic temporal patterns, providing only implicit reasoning that lacks the underlying explanations critical for high-stakes applications like healthcare. While reinforcement learning (RL)-based timeseries language models aim to address this, they often fall short because they are trained on narrow, in-distribution data and struggle with out-of-distribution compositional questions. To address these challenges, we present TimeThink, a synthetic framework for eliciting compositional timeseries reasoning. Core timeseries primitives (e.g., trend, seasonality) are domain-independent and can be deterministically generated. Guided by this premise, TimeThink first designs a synthetic data generator that produces atomic and composite question-answer pairs, providing objective ground truth with reasoning traces. Building on this framework, TimeThink employs a reinforcement learning with verifiable rewards (RLVR) training strategy that encourages explicit reasoning. Unlike template-reliant methods, this approach enables the model to learn the underlying logic of composition rather than simply imitating traces. Extensive experiments show that TimeThink, trained only on synthetic data, significantly outperforms strong baselines on both synthetic and real-world benchmarks.