cs.LGNov 1, 2026

TSSDA: An Autonomous Agent for Constructing a Time-Series Text-to-SQL Benchmark

Authors: Zhihan Zheng, Shaoxu Song, Haitao Yuan, Minxiao Chen

Organizations: Tsinghua University, China · NTU, Singapore · BUPT, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

As agents increasingly undertake data querying, Text-to-SQL serves as a foundational capability for database access. However, for text-to-SQL in time-series databases, high-quality benchmarks and large-scale training corpora remain scarce, limiting the systematic evaluation and domain adaptation of existing models and agents. Existing automated SQL generation approaches lack sufficient autonomy, leaving key challenges unresolved. First, numerous combinations of temporal syntax and functions complicate comprehensive coverage of valid queries. Second, as the corpus grows, generating valuable and diverse new data becomes increasingly challenging. Third, schema and data heterogeneity hinder reuse of temporal query structures, limiting generation efficiency. To address these challenges, we propose TSSDA, a Time-Series SQL (TSSQL) Generation Data Agent framework, and construct TERN, a comprehensive Time-sERies text-to-SQL beNchmark. First, to achieve broad coverage of valid queries, we systematically model TSSQL syntax and semantics to construct a data generation space. Next, to explore the space and improve diversity, we develop a Data Agent that autonomously orchestrates data generation. The agent comprises a planning and decision-making module and a query generation module, with online adaptation for continuous self-improvement. Finally, we introduce a temporal-aware memory that enables adaptive reuse of experience across heterogeneous data instances, improving generation efficiency. Comprehensive evaluations on TERN reveal the limitations of existing methods in performing Text-to-TSSQL tasks. Furthermore, extensive experiments demonstrate that TSSDA substantially improves coverage, diversity, and complexity.

Figures & tables

Figure 1. Challenges of Text-to-SQL for TSDBs
Figure 2. Architecture of the TSSDA Framework
Figure 3. Recursive Operator Expression Structure
Figure 4. Overview of the Temporal-Aware Hierarchical Memory Mechanism
Method# Example# DB# Table / DB# Col / DB# Tok. / SQL# Unique Func.Build ModeTSDB -nativeTime-series Analytics
WikiSQL ( Zhong et al., 2017 )80,65426,5211.006.3412.25Hybrid✗✗
Spider 1.0 ( Yu et al., 2018 )10,1812005.1126.8218.55Manual✗✗
ScienceBenchmark ( Zhang et al., 2023 )5,031316.6786.6715.65Hybrid✗✗
BIRD ( Li et al., 2023 )12,751957.6454.5630.924Manual✗✗
Spider 2.0-Snow ( Lei et al., 2025 )54715252.63812.1161.8106Manual✗✗
BIRD-History ( Zhou et al., 2026 )1,393116.872.580.922Automatic✗✗
Table 1. Statistics of different NL2SQL datasets.
MethodTERN-FoundationTERN-StandardTERN-AnalyticalOverall
EXSEREXSEREXSEREXSER
Fine-tuned LLMs ( ≤\leq 32B)
Qwen3.8-27B(ICL)9.17%15.83%4.34%12.10%2.45%8.87%5.33%12.36%
GLM-4-9B(SFT)58.61%73.33%29.22%73.06%21.71%43.73%36.44%64.62%
Meta-Llama-3.1-8B-Instruct(SFT)65.56%75.83%42.01%74.89%27.83%51.07%45.42%68.27%
Qwen2.5-Coder-7B-Instruct(SFT)55.83%75.55%36.07%83.79%27.52%58.72%39.91%73.87%
Table 2. Performance comparison on the TERN benchmark.
Figure 5. Statistics of Errors
Figure 6. Examples of Prediction Errors
Figure 7. Scalability Comparison of Different Methods
MethodHybrid [-2pt] SimilarityVendi [-2pt] ScoreTSSQL [-2pt] CoverageHalstead [-2pt] ComplexityHill [-2pt] Diversity
DeepSeek-V4.10.68275.656363.342438.483484.2069
OmniSQL0.67866.223463.939258.369166.7064
SQL-Factory0.64267.804066.702944.912586.7652
SemanticAgent0.65756.810865.684450.426675.3282
TACO0.69532.788252.969855.526380.2348
NL2InfluxDB0.58246.502360.835945.519597.5521
Table 3. Effectiveness comparison of different methods.
Figure 8. Efficiency Comparison of Different Methods
More in this topic

Text-to-SQL

Text-to-SQL translates natural-language questions into executable database queries, linking user intent to schemas, values, and SQL logic.

See all 185 papers

Explore similar work

CardsList
Sep 28, 2026

TQTS-Bench: A Multi-Syntax Benchmark for Text-to-Query over Time-Series Databases

Large language models (LLMs) have significantly advanced natural language querying over relational databases, yet their ability to query time-series databases (TSDBs) remains largely unassessed. Existing benchmarks fail to adequately capture the non-unified query syntaxes, diverse application domains, and unique time-specific query intents inherent to TSDBs. To address this gap, we introduce TQTS-BENCH, a multi-syntax benchmark for evaluating text-to-query capabilities over TSDBs. TQTS-BENCH contains 6,125 high-quality question-answering (QA) pairs spanning 97 TSDBs, 23 distinct query syntaxes, 22 application domains, and 4 types of time-specific query intents. It is constructed through a human-centric AI-assisted workflow, where all QA pairs are carefully reviewed and revised by domain experts to ensure quality and correctness. Extensive evaluations of advanced LLMs and state-of-the-art text-to-query methods reveal challenges in querying TSDBs. Even the best-performing model evaluated, Claude-Opus-5, achieves only 48.98% execution accuracy, while humans reach 87.34%. Error analysis reveals that this performance gap mainly stems from the heterogeneous query syntaxes across different TSDBs, misinterpretation of time-specific intents, and incorrect schema linking. These findings highlight new opportunities to narrow the gap between current LLM capabilities and the requirements of TSDB queries in real-world applications. The benchmark is available at: https://anonymous.4open.science/r/TQTS-Bench-00CD.

Fei Lyu, Zhiyi Peng, Jiaming Liu +5

College of Computer Science and Electronic Engineering, Hunan University

LLM EvaluationTime Series QA
AskPDF
Apr 24, 2026

PExA: Parallel Exploration Agent for Complex Text-to-SQL

LLM-based agents for text-to-SQL often struggle with latency-performance trade-off, where performance improvements come at the cost of latency or vice versa. We reformulate text-to-SQL generation within the lens of software test coverage where the original query is prepared with a suite of test cases with simpler, atomic SQLs that are executed in parallel and together ensure semantic coverage of the original query. After iterating on test case coverage, the final SQL is generated only when enough information is gathered, leveraging the explored test case SQLs to ground the final generation. We validated our framework on a state-of-the-art benchmark for text-to-SQL, Spider 2.0, achieving a new state-of-the-art with 70.2% execution accuracy.

Tanmay Parekh, Ella Hofmann-Coyle, Shuyi Wang +3

University of California Los Angeles · Bloomberg

LLM Inference EfficiencyText-to-SQLLLM Agents
AskPDF
Apr 23, 2026

SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesis

Existing text-to-SQL synthesis pipelines still conflate executability with semantic validity: syntactic checks and execution-based validation can retain queries that execute successfully while violating database semantics. To address these limitations, we propose SemanticAgent, a semantic-aware synthesis framework. SemanticAgent organizes synthesis around three specialized modules: an analyzer, a synthesizer, and a verifier. Through a three-stage protocol of semantic analysis, stepwise synthesis, and diagnostic refinement, SemanticAgent transforms execution-based validation alone into a traceable reasoning process. Our framework generates synthetic data that consistently outperforms prior synthesis methods under semantic-quality evaluation, leading to stronger downstream fine-tuning performance, especially on semantically demanding benchmarks.

Qiang Gao, Zhenping Li, Anqi Zhuo +3

Center of Information Research, Academy of Military Science, Beijing 100142, China · School of Mathematics and Physics, University of Science and Technology Beijing, Beijing 100083, China

Synthetic Data GenerationText-to-SQLSynthetic Data Evaluation
AskPDF