xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truth

Gregor Baer

cs.LGMar 6, 2026

xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truth

Authors: Gregor Baer

Organizations: Information Systems, Industrial Engineering & Innovation Sciences Eindhoven University of Technology Eindhoven, The Netherlands

Abstract

Evaluating time series attribution methods is difficult because real-world datasets rarely provide ground truth for which time points drive a prediction. A common workaround is to generate synthetic data where class-discriminating features are placed at known locations, but each study currently reimplements this from scratch. We introduce xaitimesynth, a Python package that provides reusable infrastructure for this evaluation approach. The package generates synthetic time series following an additive model where each sample is a sum of background signal and a localized, class-discriminating feature, with the feature window automatically tracked as a ground truth mask. A fluent data generation API and YAML configuration format allow flexible and reproducible dataset definitions for both univariate and multivariate time series. The package also provides standard localization metrics, including AUC-PR, AUC-ROC, Relevance Mass Accuracy, and Relevance Rank Accuracy. xaitimesynth is open source and available at https://github.com/gregorbaer/xaitimesynth.

Figures & tables

Package	XAI methods	Perturbation metrics	Localization metrics	Synth. time series
TSInterpret	✓	—	—	—
Captum	✓	✓	—	—
AIX360	✓	✓	—	—
Quantus	—	✓	✓	—
Time Interpret	✓	✓	✓	—
TimeSynth	—	—	—	✓

Table 1: Comparison of related packages. xaitimesynth combines synthetic time series generation with localization metrics for evaluating attribution correctness.

Figure 1: One example per class from the synthetic dataset defined in Listing 1 . Each row shows one class; columns show the background signal, the localized feature, and their sum. The shaded region in the rightmost column marks the ground-truth feature window. Class 0 contains a Gaussian pulse, class 1 an oscillation burst; both share a Gaussian noise background. Generated with plot_components(train) , a built-in visualization utility.

Metric	Function	Reference
AUC-ROC	auc_roc_score	Fawcett (2006)
AUC-PR	auc_pr_score	e.g., Baer et al. (2025a)
Relevance Mass Accuracy	relevance_mass_accuracy	Arras et al. (2022)
Relevance Rank Accuracy	relevance_rank_accuracy	Arras et al. (2022)
Pointing Game	pointing_game	Zhang et al. (2018)
Normalized Attribution Correspondence	nac_score	Peters et al. (2005)

Table 2: Localization metrics currently included in xaitimesynth .

Explore similar work

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters
May 7, 2026Hugo Cazaux, Eyjólfur Ingi Ásgeirsson, Hlynur StefánssonSynthetic Data Augmentation Synthetic Data Generation
Department of Engineering Reykjavík University Iceland, Menntavegur 1, 102
Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting
Aug 11, 2026Luis Amorim, Vitor Cerqueira, Moises Santos +2Synthetic Data Generation Time Series Forecasting
Faculdade de Engenharia da Universidade do Porto, Porto, Portugal · University of Coimbra, Coimbra, Portugal · Laboratory for Artificial Intelligence and Computer Science (LIACC), Portugal +3
Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Data
Jul 17, 2026Kiwan Kwon, Kangmin Kim, Hojin Lee +5Synthetic Data Evaluation
UNIST · HUFS · JP Morgan AI Research +1

May 7, 2026J/K move · Enter open · S save

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters

Hugo Cazaux, Eyjólfur Ingi Ásgeirsson, Hlynur Stefánsson

Department of Engineering Reykjavík University Iceland, Menntavegur 1, 102

Synthetic Data Augmentation Synthetic Data Generation Time Series Forecasting Time Series Data Augmentation Synthetic Data Evaluation

Open paper AskPDF arXiv

Synthetic data has transformed language model training, yet its role in time series forecasting remains poorly understood. We present a large-scale empirical study: nine experiment groups, 4,218 runs systematically evaluating synthetic time series augmentation across five architectures, four synthetic signals and seven datasets. The effect is sharply architecture-conditional: channel-mixing models (TimesNet, iTransformer) benefit in the majority of trials, while channel-independent models (DLinear, PatchTST) are consistently degraded. In selected low-resource settings the gains are striking: TimesNet trained on only 10% of Weather data with synthetic augmentation surpasses the full-data baseline (4 of 16 sparsity-dataset combinations). Averaged across all architectures, augmentation hurts in 67% of trials. We further find that only the Seasonal-Trend generator reliably helps across the tested benchmarks, and that hard curriculum switching is actively harmful (+24% MSE degradation). These results provide concrete, actionable guidelines on how to use synthetic data: use synthetic augmentation with channel-mixing architectures, use gradual annealing schedules, and treat low-resource augmentation as architecture- and dataset-dependent. Code is available at \href{https://github.com/hugoiscracked/synthetic-ts/tree/main}

Abstract

Explore similar work

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters

Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting

Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Data