ConTSG-Bench: A Unified Benchmark for Conditional Time Series Generation
Organizations: School of Information Science and Technology, ShanghaiTech University, Shanghai, China.
Abstract
Conditional time series generation plays a critical role in addressing data scarcity and enabling causal analysis in real-world applications. Despite its increasing importance, the field lacks a standardized and systematic benchmarking framework for evaluating generative models across diverse conditions. To address this gap, we introduce the Conditional Time Series Generation Benchmark (ConTSG-Bench). ConTSG-Bench comprises a large-scale, well-aligned dataset spanning diverse conditioning modalities and levels of semantic abstraction, first enabling systematic evaluation of representative generation methods across these dimensions with a comprehensive suite of metrics for generation fidelity and condition adherence. Both the quantitative benchmarking and in-depth analyses of conditional generation behaviors have revealed the traits and limitations of the current approaches, highlighting critical challenges and promising research directions, particularly with respect to precise structural controllability and downstream task utility under complex conditions.
Figures & tables
| Condition Modality | Condition Semantic | Evaluation Dimensions | |||||||
| Method | Text | Attr | Label | Morph | Concept | Adh. | Fine-gr. | Comp. Gen. | Down. Util. |
| Existing Benchmark | |||||||||
| TSGBench Ang et al. (2023) | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ |
| Label-conditioned | |||||||||
| TimeVQVAE Lee et al. (2023) | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ |
| TTS-CGAN Li et al. (2022) | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ |
Appendix figures & tables49 assets
Supplementary material from the paper’s appendix.
Appendix
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 41.09 | 42.0 | 60 | 8.90 |
| Validation | 41.21 | 42.0 | 60 | 8.94 |
| Test | 41.21 | 42.0 | 60 | 8.92 |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 62.23 | 63.0 | 83 | 8.97 |
| Validation | 62.27 | 63.0 | 83 | 9.10 |
| Test | 62.39 | 63.0 | 82 | 9.17 |
| Attribute | Value Options | Definition |
|---|---|---|
| Particulate_Matter_Profile | Consistently Low, Low with a Significant Spike, Moderate and Fluctuating, High and Worsening, Consistently High, High with Significant Improvement | The overall 24-hour trend and severity of PM2.5 and PM10. |
| Ozone_Peak_Intensity | Suppressed or No Peak, Moderate Peak, Strong/Very High Peak | The strength of the afternoon ozone (O3) peak. |
| Inverse_Relationship_Strength | No Clear Relationship, Weak Relationship, Strong and Clear Relationship | The clarity of the inverse relationship between O3 and primary pollutants (NO2, CO). |
| Primary_Pollution_Event_Timing | No Specific Event, Morning Peak, Midday/Afternoon Peak, Evening/Nighttime Peak | The main time of day when primary pollutants (PM, CO, NO2) are highest. |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 220.00 | 212.0 | 611 | 54.85 |
| Validation | 221.61 | 213.0 | 508 | 55.64 |
| Test | 218.08 | 209.0 | 472 | 54.33 |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 168.14 | 165.0 | 386 | 31.06 |
| Validation | 167.30 | 164.0 | 302 | 30.07 |
| Test | 168.67 | 165.0 | 323 | 31.27 |
| Attribute | Value Options | Definition |
|---|---|---|
| level_shift_presence | multiple_shifts, no_shifts, other, single_downward_shift, single_upward_shift | Indicates if there are abrupt changes in the average level of the series. |
| overall_trend | downward_trend, fluctuating_trend, mixed_trend, no_clear_trend, other, stable_level, upward_trend | Captures the long-term trajectory and persistent directional movement of the signal. |
| periodicity | absent, other, present, unclear | Assesses the existence of repetitive cycles or patterns occurring at fixed time intervals. |
| prominent_features | minor_fluctuations, other, peaks_and_troughs, peaks_only, troughs_only | Characterizes the most distinct visual landmarks or morphological events within the data. |
| volatility_level | decreasing_volatility, high_volatility, increasing_volatility, low_volatility, mixed_volatility, moderate_volatility, other | Evaluates the intensity and temporal evolution of variance and fluctuations in the series. |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 43.37 | 42.0 | 86 | 7.54 |
| Validation | 43.84 | 43.0 | 81 | 8.03 |
| Test | 44.24 | 43.0 | 74 | 7.71 |
| Attribute | Value Options | Definition |
|---|---|---|
| extreme_points | absent, other, present | Denotes the existence of significant local maxima (peaks) or minima (troughs) within the signal. |
| level_shifts | absent, other, present_distinct, present_gradual, step_wise | Specifies the occurrence and specific characteristics of sudden transitions in the series’ baseline. |
| overall_trend | downward, mixed, other, stable, upward | Represents the predominant long-term trajectory of the data over the entire observation window. |
| periodicity | absent, other, present | Evaluates whether the sequence demonstrates rhythmic or recurring temporal structures. |
| volatility_change | decreasing, increasing, other, stable_volatility | Tracks the evolution of fluctuation intensity, indicating if the variance expands or contracts over time. |
| volatility_level | high, low, moderate, no_volatility, other | Quantifies the overall magnitude of oscillations and noise levels present in the time series. |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 45.64 | 44.0 | 288 | 11.60 |
| Validation | 45.40 | 43.0 | 188 | 11.73 |
| Test | 44.41 | 43.0 | 122 | 11.22 |
| Attribute | Value Options | Definition |
|---|---|---|
| rhythm | SR, AFIB, AFLT, STACH, SBRAD, SARRH, PACE, SVARR, BIGU, TRIGU, SVTAC, PSVT, unknown | Primary rhythm code (highest confidence) |
| hr_cat | bradycardia, normal, tachycardia | HR 60 / 60-100 / 100 bpm |
| rr_regularity | regular, mild_irregular, irregular | RR CV 0.05 / 0.05-0.12 / 0.12 |
| qrs_cat | normal, borderline, wide | QRS 100 / 100-120 / 120 ms |
| qtc_cat | normal, borderline, prolonged | QTc 450 / 450-480 / 480 ms |
| st_anterior | normal, mild_elevation, high_elevation, mild_depression, high_depression | ST deviation in V1-V4 |
| Attribute | Value Options | Definition |
|---|---|---|
| age_group | young, middle_aged, elderly | Age 40 / 40-65 / 65 |
| sex | male, female | From metadata |
| diagnosis | 43 diagnostic codes (e.g., NORM, IMI, ASMI, LVH, …) | Primary diagnosis (highest confidence) |
| heart_axis | normal, LAD, RAD, ALAD, ARAD, unknown | From metadata |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 25.94 | 24.0 | 51 | 5.07 |
| Validation | 26.17 | 24.0 | 50 | 5.17 |
| Test | 26.16 | 24.0 | 50 | 4.98 |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 16.29 | 16.0 | 29 | 4.37 |
| Validation | 16.13 | 16.0 | 28 | 4.17 |
| Test | 16.19 | 16.0 | 27 | 4.19 |
| Attribute | Value Options | Definition |
|---|---|---|
| humidity_level | dry, extremely_high, high, low, medium, moderately_dry, saturated, somewhat_humid, unknown, very_high | A qualitative representation of the ambient moisture concentration in the atmosphere. |
| pressure_level | average, high, low, unknown, very_high, very_low | Reflects the barometric status and atmospheric pressure fluctuations. |
| season | autumn, spring, summer, unknown, winter | Identifies the specific climatological period of the year for the given data. |
| sky_condition | broken_clouds, clear, cloudy, fog, other, partly_cloudy, passing_clouds, precipitation, scattered_clouds, sunny | Characterizes the visual appearance of the firmament and the density of cloud coverage. |
| temperature_level | chilly, cool, high, low, medium, unknown, warm | Provides a non-numeric evaluation of the thermal state of the environment. |
| time_of_day | afternoon, early_morning, evening, morning, night, unknown | Categorizes the observation into broad diurnal temporal segments. |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 142.49 | 140.0 | 276 | 28.91 |
| Validation | 146.09 | 143.0 | 251 | 31.04 |
| Test | 144.45 | 142.0 | 269 | 31.14 |
| Set | Average Tokens | Median Tokens | Max Tokens | Std. Dev. |
|---|---|---|---|---|
| Training | 70.10 | 69.0 | 130 | 10.05 |
| Validation | 73.40 | 73.0 | 125 | 10.64 |
| Test | 72.87 | 73.0 | 112 | 10.58 |
| Category | Statistical | Embedding-based |
|---|---|---|
| Generation Fidelity | • MDD (Marginal Distribution Difference) • ACD (Auto-Correlation Difference) • SD / KD (Skewness / Kurtosis Difference) | • FID (Fréchet Inception Distance) • Precision & Recall |
| Condition Adherence | • DTW Score (Alignment Distance) • CRPS (Distribution Calibration and Sharpness) | • CTTP Score (Cosine Similarity) • J-FTSD (Joint-Space FID) • Joint Precision & Recall |
| Dataset | Acc (%) | Dataset | Acc (%) |
|---|---|---|---|
| TelecomTS | 37.19 | ETTm1 | 16.59 |
| TelecomTS-Segment | 16.09 | Istanbul Traffic | 17.45 |
| PTB-XL (Morph.) | 28.31 | Synth-M | 98.35 |
| PTB-XL (Concept) | 33.85 | Synth-U | 92.52 |
| Weather (Morph.) | 20.52 | Air Quality | 18.37 |
| Weather (Concept) | 20.90 |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE | — | — | — | — | — | — | — | — | — | — |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Model | ACD | SD | KD | MDD | FID | J-FTSD | Precision | Recall | Joint Precision | Joint Recall | CTTP Score |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Bridge | |||||||||||
| DiffuSETS | |||||||||||
| T2S | |||||||||||
| TEdit | |||||||||||
| Text2Motion | |||||||||||
| TimeVQVAE |
| Dataset | TEdit | TimeWeaver | DiffuSETS | TimeVQVAE | VerbalTS | WaveStitch | Text2Motion | Bridge | T2S | TTSCGAN |
|---|---|---|---|---|---|---|---|---|---|---|
| AirQuality Beijing | 0.301 | 0.336 | 0.574 | 0.509 | 0.631 | 0.505 | 0.696 | 0.787 | 1.106 | 0.863 |
| ETTm1 | 0.290 | 0.382 | 0.365 | 0.560 | 0.279 | 0.507 | 1.086 | 0.598 | 0.426 | 0.573 |
| Istanbul Traffic | 1.232 | 0.705 | 0.918 | 1.210 | 1.020 | 1.046 | 0.741 | 0.953 | 1.290 | 0.747 |
| PTB-XL (Conceptual) | 0.672 | 0.722 | 0.532 | 0.841 | 0.582 | 0.827 | 1.383 | 0.568 | 1.003 | 1.083 |
| PTB-XL (Morphological) | 0.320 | 0.624 | 0.202 | 0.607 | 0.499 | 0.620 | 0.585 | 0.719 | 0.645 | 0.546 |
| Synth-M | 0.093 | 0.096 | 0.538 | 0.420 | 0.098 | 0.571 | 0.537 | 0.730 | 0.905 | 0.989 |