cs.LGNov 11, 2026

CastWorld: Benchmarking Context-Aware Agentic Time Series Forecasting Beyond Predictive Accuracy

Authors: Bokai Pan, Ruijie Yang, Mingyue Cheng, Jiahao Wang, Xiaoyu Tao

Organizations: State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, Anhui Province, China

Abstract

Agentic time series forecasting extends conventional approaches beyond one-shot numerical prediction toward autonomous, context-aware interaction. However, existing benchmarks offer limited integrated support for diverse contextual information, forecasting-specific adaptive interactions, and systematic evaluation of forecasting agents. In this paper, we introduce CastWorld, an agentic benchmark for context-aware time series forecasting. First, its context-rich dataset suite spans 8 domains and 25 real-world datasets, covering external variables, events, and historical cases under a unified context specification. Second, a lightweight, modular forecasting-native harness provides 49 core tools and a state-adaptive loop that tracks forecast versions and enables iterative evidence gathering and forecast revision. Intermediate forecasts can expose information gaps, while newly acquired evidence guides subsequent revisions and final version selection. Third, model-level evaluation assesses forecast quality, process execution, behavioral trustworthiness, and resource efficiency through decision trajectories and forecast-version histories. Also, systems-level evaluation compares predictive accuracy across forecasting systems. Experiments reveal distinct decision behaviors among agents with comparable accuracy. Additional interactions do not necessarily improve forecasts, and agents may fail to select their best previously generated predictions. Allowing evidence acquisition during forecast refinement improves task-aware forecast scores across all three evaluated models. CastWorld provides a reusable foundation for evaluating agentic time series forecasting under controlled conditions.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 10, 2026

World2TS: Real-World Event Context for Time-Series Forecasting

Time-series forecasting plays a vital role in a wide range of real-world applications. Recent approaches have achieved strong performance by modeling temporal correlations and cross-channel dependencies, yet most forecasting benchmarks still represent time series primarily as numerical histories. In practice, time series are observations of complex real-world processes, and their trajectories may reflect events, interventions, disruptions, and changes in measurement or reporting that cannot always be inferred from past values alone. A central barrier to studying context-aware forecasting is therefore the lack of large-scale datasets that align real-world events with the individual series they affect. We present World2TS, a scalable construction framework and large-scale dataset for time-series forecasting with real-world event context. We first develop world2ts-core, a reusable tool that combines agent-led semantic reasoning and Web search, command-line execution and validation, and human review within a feedback-driven discovery workflow. Given heterogeneous time series with semantic descriptions, the workflow discovers candidate events, consolidates retrieved evidence into verified events, and links them to relevant series under explicit temporal and semantic scopes. Using this framework, we construct world2ts-data, which contains 31 datasets across six domains, comprising 4,179 series, 97.1 million observations, 1,159 events, and 56,846 event-series links. We further conduct extensive data analyses and controlled comparative experiments to characterize the dataset and demonstrate the utility of the collected context for improving forecasting accuracy. World2TS provides a reusable data foundation for developing and evaluating context-aware forecasting models and agents. The project homepage is available at https://world2ts.github.io/.
Aug 4, 2026

CastFSR: Fast--Slow--Reflect Agentic Reasoning for Context-Aware Time Series Forecasting

Time series forecasting is fundamental to temporal data mining, yet numerical models often struggle to account for changing contextual factors. A key challenge is determining when contextual evidence should guide forecast revisions while maintaining consistency with temporal patterns and domain knowledge. To address this, we propose CastFSR, a fast--slow--reflect agentic framework for context-aware time series forecasting. Fast-thinking forecasting analyzes numerical patterns, selects a suitable forecaster, and constructs a data-driven forecast prior. Slow deliberative reasoning incorporates relevant contextual evidence to selectively refine this prior, while reflective evaluation checks the revised forecast against temporal patterns, contextual evidence, and domain constraints. A shared state maintains forecasting priors, contextual evidence, intermediate forecasts, and feedback across stages. CastFSR supports both training-free LLM coordination and two-stage post-training for compact LLM coordinators. Workflow-aware trajectory distillation (WATD) learns state-conditioned tool use and forecasting decisions from teacher trajectories, while trajectory-level policy optimization (TLPO) further improves the coordinator through reinforcement learning guided by forecasting accuracy and workflow feedback. Experiments on multiple real-world datasets under an offline evaluation protocol using historical context demonstrate competitive performance against representative baselines. Our code is available at https://github.com/Xiaoyu-Tao/CastFSR.
Oct 13, 2026

GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation

Contextual forecasting seeks to improve time series forecasting by incorporating forward-looking information, such as planned events, external conditions, and expert expectations. However, reliable evaluation remains difficult: existing benchmarks often have limited domain coverage, lack standardized history-future splits, or provide text that is not guaranteed to inform the future. We introduce GIFT-Eval-CTX, a benchmark that isolates whether forecasting methods can translate verified future information into accurate numerical trajectories. Built upon GIFT-Eval, GIFT-Eval-CTX contains 10,000 instances across 20 datasets. Each instance provides structured effects derived from the realized future: Macro effects describe horizon-wide behavior, while Micro effects specify localized changes. We construct these annotations with an agentic pipeline combining LLM-based discovery and deterministic code validation. We benchmark TSFMs, frontier LLMs, and tool-using agentic systems. The results show that structured effects reduce aggregate relative MASE by 40.7% for GPT-5.6 Sol and 29.6% for Claude Opus 4.8 relative to history-only prompting. The evaluated MoiraiAgent system achieves the best overall result. We further analyze the contributions of structured effects and agentic systems to performance, and identify opportunities for future research.