The state-of-the-art Membership Inference (MI) methods calibrate their signal separately for each example using reference models, auxiliary models trained to exclude the target. This paradigm scales poorly to modern large models, however, whose training is too expensive to replicate. This has motivated one-round settings, where only a single trained model is available; but without reference models the per-example calibration that drives the strongest attacks can no longer be estimated, leaving the membership signal weak. We ask whether neighbors of the target point can recover this calibration without training any additional model. Our key observation is that reference models serve only to reveal how an example behaves under models not trained on it, and that querying the target model on nearby samples yields the same information. We propose two complementary ways to obtain such neighbors, and show that querying them against an early training checkpoint further sharpens the signal. We evaluate across three image classification datasets and three training setups, showing that neighbors yield strong membership signals and competitive attack performance at no additional training cost.
Figures & tables
Empirical Epsilon
Steinke et al. (2023)
Ours
LiRA
DP-SGD
CIFAR-10
1.29
1.78
2.99
CIFAR-100
1.24
1.88
3.16
CINIC-10
0.79
1.38
2.37
Table 1 . Comparison between Steinke et al. (2023) (baseline), our Synthetic Neighbors E.C. attack and LiRA (upper bound) against DP-SGD, in terms of empirical ϵ .
TPR@0.1%FPR (%)
Steinke et al. (2023)
Ours
LiRA
Undefended
CIFAR-10
23.38
84.98
97.03
CIFAR-100
99.28
99.03
99.73
CINIC-10
22.88
79.10
91.29
DP-SGD
CIFAR-10
0.65
2.51
12.21
CIFAR-100
2.19
4.43
13.89
CINIC-10
0.33
1.80
4.43
Table 2 . Comparison between Steinke et al. (2023) (baseline), our Synthetic Neighbors E.C. attack and LiRA (upper bound) across all privacy settings, in terms of TPR at 0.1% FPR.
Empirical Epsilon
TPR@0.1%FPR (%)
DP-SGD
CIFAR-10
CIFAR-100
CINIC-10
CIFAR-10
CIFAR-100
CINIC-10
Steinke et al. (2023)
1.2867
1.2427
0.7917
0.6500
2.1875
0.3312
Similarity Neighbors
1.4851
1.4062
1.0305
0.5250
0.8250
0.1875
Sim. Neighbors E.C.
1.5294
1.4981
1.0504
0.7063
2.4188
0.2938
Synthetic Neighbors
1.3926
1.4549
1.1216
1.9688
2.1812
1.1312
Synt. Neighbors E.C.
1.7832
1.8837
1.3849
2.5063
4.4250
1.8000
Table 3 . Experimental results against DP-SGD, across CIFAR-10, CIFAR-100 and CINIC-10. We measure both ϵ and TPR at 0.1% FPR. Comparison between Steinke et al. (2023) (baseline), our Similarity and Synthetic Neighbors attacks with their Early Checkpoint (E.C.) variants and LiRA (upper bound).
Undefended
RelaxLoss
TPR@0.1%FPR (%)
CIFAR-10
CIFAR-100
CINIC-10
CIFAR-10
CIFAR-100
CINIC-10
Steinke et al. (2023)
23.3813
99.2812
22.8813
1.0563
72.6750
10.9625
Similarity Neighbors
78.9062
74.8875
65.2125
20.2437
61.9687
34.1500
Sim. Neighbors E.C.
81.1063
99.0062
72.4125
20.9663
82.0187
34.4750
Synthetic Neighbors
82.6375
95.8625
70.7562
19.9875
86.6188
57.0500
Synt. Neighbors E.C.
84.9813
99.0250
79.1000
21.4500
84.6361
53.7312
Table 4 . Experimental results against undefended models and RelaxLoss, across CIFAR-10, CIFAR-100 and CINIC-10. We evaluate the TPR at 0.1% FPR. Comparison between Steinke et al. (2023) (baseline), our Similarity and Synthetic Neighbors attacks with their Early Checkpoint (E.C.) variants and LiRA (upper bound).