TSFLab: A Self-Improving Agent-Agnostic Workbench for Time Series Forecasting Research

ChengAo Shen; Yanjun Zhao; Haoxin Liu; Bonnie Chen; Miles Ma; Dongjin Song; Hanghang Tong; Qingsong Wen

cs.LGOct 29, 2026

TSFLab: A Self-Improving Agent-Agnostic Workbench for Time Series Forecasting Research

Authors: ChengAo Shen, Yanjun Zhao, Haoxin Liu, Bonnie Chen, Miles Ma, Dongjin Song, Hanghang Tong, Qingsong Wen,

Abstract

As publications and products on time series forecasting (TSF) proliferate, researchers face a growing burden of analyzing existing work. First, not all publications release code. For instance, only about 20% of arXiv TSF papers have a code link in the surveyed period. Second, released code also requires manual maintenance. Thus, TSF researchers must constantly adapt or implement existing works. However, independent implementations can differ in experimental settings and are difficult to maintain. We therefore propose to use AI agents for benchmarking TSF research and even supporting autoresearch, e.g., developing new TSF models. Yet directly applying existing coding agents, such as Claude Code and Codex, is suboptimal: prohibitive costs, limited scalability to large research projects (e.g., benchmarking), and lack of self-maintenance. To address these challenges, we propose TSFLab, a self-improving agent-agnostic "workbench" that equips compatible coding agents with shared resources and tools for low-cost TSF research and continuous maintenance of research resources. Our experiments cover reproduction, benchmarking, and new method development. In reproduction and benchmarking, TSFLab improves task completion by 20-163% and cuts fresh tokens per completed task by 11-66% across baselines. A pilot study of agent-driven method development also shows improved forecasting accuracy via TSFLab, with 32.8-73.6% fewer fresh tokens and 9.2-68.8% less development time than the other settings.

Figures & tables

Figure 1: (a) Growth of TSF papers, papers with code, and methods in libraries. (b) General agents access external resources directly for each task. (c) Agents with a workbench share verified resources and reusable experiments.

Figure 2: Overview of how an agent works with the TSFLab workbench.

Level	When to consult	Information provided
Summary	Screening candidates	Category and use cases
Card	Resource selection, configuration, unit reuse	Capabilities and limits, parameters and interfaces, verification records
Source	Checking details, resolving doubts, modifying methods	Original papers, reference code, detailed evidence

Table 1: Three levels of research knowledge.

Paper reproduction
Setting	Completed $\uparrow$	Time (h) $\downarrow$	Tokens (M) $\downarrow$	Calls $\downarrow$	$\Delta$ LOC $\downarrow$	$\Delta$ Funcs $\downarrow$	$\Delta$ CC $\downarrow$
TSFLab	12/12	1.96	0.434	159.67	148.08	8.25	25.83
TSLib	10/12	2.61	0.533	164.60	160.67	9.67	31.33
TFB	7/12	2.46	0.487	131.86	153.42	11.58	36.00
No workbench	8/12	3.50	0.706	212.75	131.00	11.00	34.00
Benchmarking on new data

Table 2: Main results using Codex CLI with GPT-6 Luna. Costs are normalized by completed outcomes. Best results are bold; second-best are underlined. Metric definitions are in App. D.6 .

Figure 3: Research-task quality. (a) Reproduction MSE error (log scale; lower is better); labels show shares within 10% (96 scheduled). (b,c) Development test MSE versus time; bubble areas and labels show fresh tokens.

(a) Workbench Module Ablation
Setting	Completed $\uparrow$	Time (h) $\downarrow$	Tokens (M) $\downarrow$	Calls $\downarrow$	$\Delta$ LOC $\downarrow$	$\Delta$ Funcs $\downarrow$	$\Delta$ CC $\downarrow$
Full TSFLab	12/12	1.96	0.434	159.67	148.08	8.25	25.83
w/o TSF-Domain Guide	3/12	5.73	1.129	348.33	147.33	11.33	39.42
w/o Knowledge Structurer	8/12	2.08	0.438	118.75	171.42	9.42	31.58
w/o Experiment Workflower	5/12	2.24	0.533	153.00	178.75	10.42	36.42
(b) Agent Ablation

Table 3: Module and agent ablations. The full workbench uses Codex CLI with GPT-6 Luna. Best values are bold; second-best distinct values are underlined, including ties.

Figure 4: Workbench coverage and evolution. (a) Top-cited coverage. (b) Median/max model lines. (c) Functions per method, initial (0%) to latest (100%). (d) Intake tokens including cache reads.

Agentic Time Series Forecasting

See all 17 papers

TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent
Sep 23, 2026Jie Yang, Yan Zheng, Jiarui Sun +8Time Series QA LLM Agent Self-Improvement
University of Illinois at Chicago · Visa Research · University of Florida +2
SEA-TS: Self-Evolving Agent for Autonomous Code Generation of Time Series Forecasting Algorithms
Mar 5, 2026Longkun Xu, Xiaochun Zhang, Qiantu Tuo +1Time Series Forecasting Code Generation
Ecoflow Inc.
TempusBench: An Evaluation Framework for Time-Series Forecasting
Apr 13, 2026Denizalp Goktas, Gerardo Riaño-Briceño, Omkar Tekawade +14Time Series Forecasting Forecasting Benchmarks
Simulacrum New York City, NY, USA

Sep 23, 2026J/K move · Enter open · S save

TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent

Jie Yang, Yan Zheng, Jiarui Sun, Xiran Fan+7

University of Illinois at Chicago · Visa Research · University of Florida+2

Time Series QA LLM Agent Self-Improvement LLM Agents Self-Evolving Agents Data Analysis Agents

Open paper AskPDF arXiv

Time series agents answer analytical questions by calling external tools, and which tools they carry is decided by people before the agent runs. However, we identify two failures in this setup. Human-Agent Tool Misalignment: a library of 21 expert-curated tools helps on some tasks and hurts on others, dropping anomaly accuracy under every backbone we test. Silent Harm: one round of generic self-revision changes 147 answers and breaks 56 of them, while the final score moves by less than a point. Both follow from the same gap: whether a tool helps is decided question by question at runtime, while tools are supplied in advance and judged by a single average. To address this, we propose TimeEvo, which clusters an agent's diagnosed failures into capability gaps, plans a measurement for each, synthesizes evidence-only tools that fill them, and admits the candidate library only through a paired admission gate. Experiments on ten time series QA tasks and three backbones show that TimeEvo, starting from an empty library, improves accuracy on every task and every backbone, and that a library grown on a cheap model still gains when it is installed into stronger ones. Code is available at https://github.com/Muyiiiii/TimeEvo.

Abstract

Agentic Time Series Forecasting

Explore similar work

TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent

SEA-TS: Self-Evolving Agent for Autonomous Code Generation of Time Series Forecasting Algorithms

TempusBench: An Evaluation Framework for Time-Series Forecasting