KairosVL: Orchestrating Time Series and Semantics for Unified Reasoning

Haotian Si; Changhua Pei; Xiao He; Zeyan Li; Zhe Xie; Zexin Wang; Jiyao Hu; Zhaoyang Yu

cs.AIFeb 24, 2026

KairosVL: Orchestrating Time Series and Semantics for Unified Reasoning

Authors: Haotian Si, Changhua Pei, Xiao He, Zeyan Li, Zhe Xie, Zexin Wang, Jiyao Hu, Zhaoyang Yu,

Organizations: CNIC, CAS, Beijing, China · University of Chinese Academy of Sciences · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences · ByteDance, Beijing, China ·

Abstract

Driven by the increasingly complex and decision-oriented demands of time series analysis, we introduce the Semantic-Conditional Time Series Reasoning task, which extends conventional time series analysis beyond purely numerical modeling to incorporate contextual and semantic understanding. To further enhance the mode's reasoning capabilities on complex time series problems, we propose a two-round reinforcement learning framework: the first round strengthens the mode's perception of fundamental temporal primitives, while the second focuses on semantic-conditioned reasoning. The resulting model, KairosVL, achieves competitive performance across both synthetic and real-world tasks. Extensive experiments and ablation studies demonstrate that our framework not only boosts performance but also preserves intrinsic reasoning ability and significantly improves generalization to unseen scenarios. To summarize, our work highlights the potential of combining semantic reasoning with temporal modeling and provides a practical framework for real-world time series intelligence, which is in urgent demand.

Figures & tables

Figure 1. Comparison of time series task formulations from the perspectives of Numerical Data Processing and Semantic-Conditional Reasoning (taking the real-world time series anomaly diagnosis task as an example). While the majority of existing studies remain confined to the former, real-world applications also urge for the latter, an end-to-end reasoning paradigm.

Figure 2. Details of KairosDataPipe and KairosDataset.

Figure 3. One-round reinforcement learning pipeline v.s. two-round reinforcement learning pipeline.

Method	Open-Source	Dataset $\mathcal{A}$			Dataset $\mathcal{B}$
Method	Open-Source	Scenario #1	Scenario #2	Scenario #3	Fact-Adherent	Predictive	Event-Aware	Counterfactual
InternVL3-8B	✓	57.4	53.0	62.0	41.3	53.6	40.4	42.7
Qwen2.5VL-7B	✓	75.6	53.3	45.3	47.6	50.0	36.9	42.6
Qwen2.5VL-32B	✓	79.6	56.8	46.7	61.9	51.9	49.6	46.2
Qwen2.5VL-72B	✓	87.8	60.0	70.7	71.2	42.4	54.4	64.7
GLM4V-Plus	✗	66.7	70.0	49.3	60.8	54.0	50.0	50.5

Table 1. Overall Question-Answer Accuracy (%) of different models on both Real-World datasets (Dataset

\mathcal{A}

) and KairosBench (Dataset

\mathcal{B}

). The highest, the second-best scores are shown in bold and underlined, respectively. For each sample in the dataset, we sample the answer five times and compute the mean accuracy value. The margin of errors can be found in Appendix E .

Figure 4. Ablation studies on different training pipeline.

Panel of Figure 5 — Figure 5. Visualization of some metrics in training process.

Figure 6. Ablation studies on different KL loss settings.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Scenario #1	Scenario #2	Scenario #3
Donut ( Xu et al., 2018 )	26.5	27.3	21.6
LSTMAD ( Malhotra et al., 2015 )	32.9	21.3	27.9
MatrixProfile ( Zhu et al., 2018 )	30.4	35.6	25.4
TranAD ( Tuli et al., 2022 )	34.8	23.2	25.6
KairosVL	88.9	70.4	73.3

Appendix

Table 2. Action Accuracy (%) of Time-Series-Only Anomaly Detection Methods on Dataset

\mathcal{A}

Models	Scen. #1	Scen. #2	Scen. #3
GPT-4o (text)	33.3	41.6	17.8
GPT-4o (image)	83.3	64.5	77.3
Gemini-2.5 Flash (text)	72.6	42.5	17.3
Gemini-2.5 Flash (image)	79.4	62.5	48.7
GLM-4V-Plus (text)	38.9	30.8	0.0
GLM-4V-Plus (image)	66.7	70.0	49.3

Appendix

Table 3. Comparison of Question-Answer Accurary (%) on real world time series reasoning tasks between "Time Series as Text" and "Time Series as Image" paradigms. Since each sample in Scenario #3 contains a larger amount of data, GLM-4V-Plus struggles to produce valid responses within the limited output token budget when handling such long inputs. It often generates irrelevant gibberish or text unrelated to the question, resulting in an accuracy of zero for this scenario.

Figure 8. Comparison of token consumption between the “Time Series as Text” and “Time Series as Image” paradigms when using GPT-4o. Generally, when using GPT-4o, representing time series in textual form consumes about 5–8 tokens per numerical value. In contrast, even a high-resolution image of 5400 × 1800 pixels—which can visually encode tens of thousands of values—requires only around 1,400 tokens.

Figure 9. Drawbacks of existing time series reasoning tasks, and the ideal reasoning task.

Figure 10. Rewards On Different Tasks During Training Process.

Dataset	Margin of Error (%)
Scenario #1	$\pm 0.9$
Scenario #2	$\pm 1.4$
Scenario #3	$\pm 2.1$
Fact-Adherent	$\pm 0.8$
Predictive	$\pm 0.6$
Event-Aware	$\pm 0.6$

Appendix

Table 4. KairosVL’s margin of error of five experimental results relative to the mean.

Figure 11. System and user prompts sent to MLLMs.

Panel of Figure 12 — Figure 12. The task specific descriptions for the Generation Agent (Part 1).

Figure 13. The task specific descriptions for the Generation Agent (Part 2).

Figure 14. The task shared requirements for the Generation Agent.

Figure 15. The prompt for the Data Necessity Judge.

Figure 16. The prompt for the QA-Data Consistency Judge.

Figure 17. The prompt for the Data & Plot Requirement Judge.

Figure 18. The python function that used to execute visualization code to generate time series plot.

Time Series Reasoning

Time series reasoning studies how models interpret temporal patterns, answer questions about sequences, and use evidence to make forecasts.

See all 44 papers

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs
Jun 20, 2026Xin Qiu, Junlong Tong, Yao Zhang +3Time Series Reasoning Time Series QA
Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU
Adaptive Time Series Reasoning via Segment Selection
Feb 20, 2026Shvat Messica, Jiawen Zhang, Kevin Li +2Time Series Reasoning Time Series QA
Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · The Hong Kong University of Science and Technology (Guangzhou) · Massachusetts Institute of Technology +1
TimeThink: Eliciting Compositional Reasoning in Timeseries Large Language Models
Sep 11, 2026Sudarshan Regmi, Arvind Pillai, Yu Yvonne Wu +7Time Series Reasoning Time Series QA
Dartmouth College

Jun 20, 2026J/K move · Enter open · S save

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Xin Qiu, Junlong Tong, Yao Zhang, Yunpu Ma+2

Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Zhejiang University · Munich Center for Machine Learning, LMU

Time Series Reasoning Time Series QA Temporal Reasoning in Language Models

Open paper AskPDF arXiv

Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities, yet gains remain limited. We attribute this to a fundamental mismatch between existing task formulations and LLM strengths: most settings reduce time series understanding to curve-fitting systems, focusing on low-level prediction while ignoring the semantic, contextual, and reasoning-intensive nature of real-world temporal decision-making.To address these limitations, we introduce TSCognition, a multimodal benchmark for multi-dimensional time series reasoning. It collects real-world time series and textual information from 15 public sources and constructs approximately 41K QA samples around five cognitive reasoning tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting. Building on this, we further propose TSAlign, a unified framework that encodes time series into compact patch-level representations and aligns them with semantic directions in the LLM embedding space via gated residual injection and multivariate fusion.Experiments show that TSAlign outperforms existing LLM, VLM, and time series QA baselines on TSCognition and the publicly available TimerBed benchmark while substantially reducing computational cost.Code is available at: https://github.com/EIT-NLP/CognitiveTSR

Abstract

Time Series Reasoning

Explore similar work

From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs

Adaptive Time Series Reasoning via Segment Selection

TimeThink: Eliciting Compositional Reasoning in Timeseries Large Language Models