Uncovering Memorization in Timeseries Imputation models: LBRM Membership Inference and its link to attribute Leakage
Authors: Faiz Taleb, Ivan Gazeau, Maryline Laurent
Organizations: EDF SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, 91120 Palaiseau, France · EDF, 91120 Palaiseau, France · SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris
Deep learning models for time series imputation are now essential in fields such as healthcare, the Internet of Things (IoT), and finance. However, their deployment raises critical privacy concerns. Beyond the well-known issue of unintended memorization, which has been extensively studied in generative models, we demonstrate that time series models are vulnerable to inference attacks in a black-box setting. In this work, we introduce a two-stage attack framework comprising: (1) a novel membership inference attack based on a reference model that improves detection accuracy, even for models robust to overfitting-based attacks, and (2) the first attribute inference attack that predicts sensitive characteristics of the training data for timeseries imputation model. We evaluate these attacks on attention-based and autoencoder architectures in two scenarios: models that are trained from scratch, and fine-tuned models where the adversary has access to the initial weights. Our experimental results demonstrate that the proposed membership attack retrieves a significant portion of the training data with a tpr@top25% score significantly higher than a naive attack baseline. We show that our membership attack also provides a good insight of whether attribute inference will work (with a precision of 90% instead of 78% in the genral case).
Figures & tables
Figure 1: Illustrative overview of the LBRM attack pipeline: the attacker compares predictions from the target and reference models to infer membership.
Scenario 1
Scenario 2
LSMEC
ASHRAE
LSMEC
ASHRAE
Public O
2226
590
3340
887
Private P
2226
590
1113
295
Test
1113
295
1113
295
Table 1: Division of datasets in different scenarios
Scenario 1
Scenario 2
MAE
SAITS
Transformer
iTransformer
AE
SAITS
Transformer
iTransformer
AE
Model
T
R
T
R
T
R
T
R
T
R
T
R
T
R
T
R
Data Train
0.175
0.18
0.70
0.75
0.51
0.48
0.15
0.17
0.19
0.185
0.68
0.65
0.48
0.49
0.15
0.17
Data Test
0.20
0.21
0.81
0.85
0.55
0.52
0.20
0.21
0.24
0.21
0.75
0.74
0.55
0.52
0.20
0.21
Table 2: Performance Evaluation of SAITS, AE, iTransformer, and Transformer Models Based on Mean Absolute Error (MAE)
Figure 2: MIA + AIA pipeline. The attacker computes LBRM on the seen portion to obtain a series-level risk score, ranks series, selects the Top- q% high-risk set, and runs AIA on unseen regions of those series.
Model
Recall (avg ± std)
Precision (avg ± std)
Transformer
Target
0.87 ± 0.05
0.75 ± 0.06
Evaluation
0.48 ± 0.08
0.40 ± 0.05
SAITS
Target
0.90 ± 0.04
0.78 ± 0.05
Evaluation
0.55 ± 0.07
0.43 ± 0.05
Table 6: Average and standard deviation of recall and precision across all sliding windows (mask size = 24) for the three attention-based architectures. Target and Evaluation models share identical hyperparameters but are trained on disjoint sample sets.
Model
Rec. (r)
p-val
Prec. (r)
p-val
Transformer
-0.10
0.73
0.57
0.03
SAITS
0.18
0.51
0.64
0.02
iTransformer
0.26
0.34
0.71
0.01
Table 7: Correlation between LBRM membership scores (computed on the attacker-visible portion) and AIA performance (evaluated on unseen regions).
Figure 3: AIA precision on unseen parts of the time series. “Top - 25%” indicates selecting whole sequences based on their LBRM score computed from the part accessible to the attacker.