Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions

Najeeb Jebreel; Mona Khalil; David Sánchez; Josep Domingo-Ferrer

cs.CRMar 8, 2026

Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions

Authors: Najeeb Jebreel, Mona Khalil, David Sánchez, Josep Domingo-Ferrer

Organizations: Universitat Rovira i Virgili, Catalonia

Abstract

Membership inference attacks (MIAs) have become the standard tool for evaluating privacy leakage in machine learning (ML). Among them, the Likelihood-Ratio Attack (LiRA) is widely regarded as the state of the art when sufficient shadow models are available. However, prior evaluations have often overstated the effectiveness of LiRA by attacking models overconfident on their training samples, calibrating thresholds on target data, assuming balanced membership priors, and/or overlooking attack reproducibility. We re-evaluate LiRA under a realistic protocol that (i) trains models using anti-overfitting (AOF) and transfer learning (TL), when applicable, to reduce overconfidence as in production models; (ii) calibrates decision thresholds using shadow models and data rather than target data; (iii) measures positive predictive value (PPV, or precision) under shadow-based thresholds and skewed membership priors (pi <= 10%); and (iv) quantifies per-sample membership reproducibility across different seeds and training variations. We find that AOF significantly weakens LiRA, while TL further reduces attack effectiveness while improving model accuracy. Under shadow-based thresholds and skewed priors, LiRA's PPV often drops substantially, especially under AOF or AOF+TL. We also find that thresholded vulnerable sets at extremely low FPR show poor reproducibility across runs, while likelihood-ratio rankings are more stable. These results suggest that LiRA, and likely weaker MIAs, are less effective than previously suggested under realistic conditions, and that reliable privacy auditing requires evaluation protocols that reflect practical training practices, feasible attacker assumptions, and reproducibility considerations. Code is available at https://github.com/najeebjebreel/lira_analysis.

Figures & tables

Dataset	Benchmark	Architecture	FLP	CRP	ROT	JTR	CUT	CMX	DRP (%)	WD
CIFAR-10	baseline	ResNet-18	✓	✓			✓		0	5e-4
	AOF	ResNet-18	✓	✓	✓	✓		✓	10	1e-3
	TL	EfficientNet-V2	✓	✓	✓	✓		✓	25	5e-2
CIFAR-100	baseline	WideResNet	✓	✓			✓		0	5e-4
	AOF	WideResNet	✓	✓	✓	✓		✓	10	1e-3
	TL	EfficientNet-V2	✓	✓	✓	✓		✓	25	5e-2

Table 1. Benchmark configurations. FLP=H.Flip, CRP=R.Crop with padding, ROT=Rotation, JTR=ColorJitter, CUT=Cutout, CMX=CutMix, DRP=dropout ratio, WD=weight decay.

Panel of Figure 1 — Figure 1. Distributions of decision thresholds of the online LiRA variant at nominal FPRs of $0.001\%\,(10^{-5})$ and $0.1\%\,(10^{-3})$ on CIFAR-10 (ResNet-18, AOF). Boxes show interquartile ranges with medians and relative median absolute deviations (rMAD) from 256 shadow models of a single run (a) and 12 independent runs (b). Threshold variability is substantially higher at $0.001\%$ than at $0.1\%$ , indicating unstable calibration at the extreme tail required for high inference precision. Two side-by-side boxplots showing decision threshold distributions for the online LiRA variant at nominal FPRs of $10^{-5}$ ( $0.001\%$ ) and $10^{-3}$ ( $0.1\%$ ). Left: single-run results. Right: aggregated across 12 independent runs. Variability increases as the target FPR decreases, indicating reduced threshold stability at the extreme tail.

Benchmark	Train Loss	Test Loss	Loss Ratio	Train Acc (%)	Test Acc (%)
CIFAR-10 (baseline)	0.0032	0.2272	71.0	99.94	93.63
CIFAR-10 (AOF)	0.1351	0.2535	1.88	98.78	94.09
CIFAR-10 (AOF+TL)	0.1210	0.1647	1.36	98.70	97.00
CIFAR-100 (baseline)	0.1198	1.2172	10.16	96.75	69.30
CIFAR-100 (AOF)	0.8007	1.2037	1.50	79.75	68.24
CIFAR-100 (AOF+TL)	0.3373	0.6214	1.84	92.71	83.56

Table 2. Model utility and loss ratio across benchmarks.

Benchmark	Attack	[email protected]% FPR (%)	[email protected]% FPR (%)	AUC (%)
Baseline	Online	3.956 ±1.061	10.268 ±0.555	76.48 ±0.32
	Online (FV)	2.876 ±1.064	9.135 ±0.508	76.28 ±0.31
	Offline	0.762 ±0.348	3.262 ±0.338	55.58 ±0.92
	Offline (FV)	0.948 ±0.526	4.540 ±0.424	56.64 ±0.89
	Global	0.003 ±0.004	0.097 ±0.027	59.97 ±0.32
AOF	Online	0.248 ±0.198 ( $\times$ 16)	2.723 ±0.683 ( $\times$ 3.8)	65.76 ±1.27

Table 3. CIFAR-10 with proper AOF and TL. TPR reduction factors relative to baseline shown in parentheses for AOF and TL. Standard deviations reported in small font.

Benchmark	Attack	[email protected]% FPR (%)	[email protected]% FPR (%)	AUC (%)
Baseline	Online	4.619 ±1.730	15.791 ±0.976	88.28 ±0.21
	Online (FV)	1.730 ±1.158	11.306 ±1.000	87.81 ±0.21
	Offline	0.659 ±0.502	6.044 ±0.640	72.10 ±0.40
	Offline (FV)	0.253 ±0.234	3.911 ±0.524	71.88 ±0.39
	Global	0.002 ±0.004	0.098 ±0.026	69.86 ±0.28
AOF	Online	0.351 ±0.226 ( $\times$ 13)	3.097 ±0.353 ( $\times$ 5.1)	75.32 ±0.31

Table 4. CIFAR-100 with proper AOF and TL.

Benchmark	Attack	[email protected]% FPR (%)	[email protected]% FPR (%)	AUC (%)
Baseline	Online	0.039 ±0.028	0.274 ±0.059	53.09 ±0.27
	Online (FV)	0.042 ±0.032	0.319 ±0.068	53.10 ±0.28
	Offline	0.002 ±0.004	0.064 ±0.034	49.47 ±0.62
	Offline (FV)	0.005 ±0.007	0.085 ±0.038	49.44 ±0.63
	Global	0.006 ±0.008	0.100 ±0.033	51.10 ±0.29
AOF+TL	Online	0.006 ±0.007 ( $\times$ 6.5)	0.109 ±0.033 ( $\times$ 2.5)	51.68 ±0.40

Table 5. GTSRB with proper AOF and TL.

Benchmark	Attack	[email protected]% FPR (%)	[email protected]% FPR (%)	AUC (%)
Baseline	Online	0.523 ±0.243	4.491 ±0.281	70.16 ±0.29
	Online (FV)	0.180 ±0.110	3.089 ±0.188	69.52 ±0.28
	Offline	0.007 ±0.007	0.500 ±0.077	55.11 ±0.48
	Offline (FV)	0.022 ±0.017	0.713 ±0.078	56.11 ±0.51
	Global	0.001 ±0.001	0.100 ±0.015	54.83 ±0.15
AOF	Online	0.022 ±0.017 ( $\times$ 24)	0.825 ±0.068 ( $\times$ 5.4)	62.64 ±0.16

Table 6. Purchase-100 with proper AOF.

		Performance		PPV (%)
Benchmark	Attack	TPR ′ (%)	FPR ′ (%)	@ $\pi$ =1%	@ $\pi$ =10%	@ $\pi$ =50%
Target-based thresholds (optimistic)
Baseline	Online	3.956 ±1.061	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Online (FV)	2.876 ±1.064	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline	0.762 ±0.348	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline (FV)	0.948 ±0.526	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00

Table 7. CIFAR-10 under target vs. shadow calibration (target FPR

=0.001\%

		Performance		PPV (%)
Benchmark	Attack	TPR ′ (%)	FPR ′ (%)	@ $\pi$ =1%	@ $\pi$ =10%	@ $\pi$ =50%
Target-based thresholds (optimistic)
Baseline	Online	4.619 ±1.730	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Online (FV)	1.730 ±1.158	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline	0.659 ±0.502	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline (FV)	0.253 ±0.234	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00

Table 8. CIFAR-100 under target vs. shadow calibration (target FPR

=0.001\%

		Performance		PPV (%)
Benchmark	Attack	TPR ′ (%)	FPR ′ (%)	@ $\pi$ =1%	@ $\pi$ =10%	@ $\pi$ =50%
Target-based thresholds (optimistic)
Baseline	Online	0.039 ±0.028	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Online (FV)	0.042 ±0.032	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline	0.002 ±0.004	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline (FV)	0.005 ±0.007	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00

Table 9. GTSRB under target vs. shadow calibration (target FPR

=0.001\%

		Performance		PPV (%)
Benchmark	Attack	TPR ′ (%)	FPR ′ (%)	@ $\pi$ =1%	@ $\pi$ =10%	@ $\pi$ =50%
Target-based thresholds (optimistic)
Baseline	Online	0.523 ±0.243	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Online (FV)	0.180 ±0.110	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline	0.007 ±0.007	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00
	Offline (FV)	0.022 ±0.017	0.000 ±0.000	100.00 ±0.00	100.00 ±0.00	100.00 ±0.00

Table 10. Purchase-100 under target vs. shadow calibration (target FPR

=0.001\%

Panel of Figure 2 — Figure 2. Reproducibility, stability, and coverage vs seeds, training variations, and runs (TP $\geq$ 1). Three line plots summarizing reproducibility online LiRA attack results across multiple training runs. Panel (a) shows Jaccard similarity decreases sharply as runs increase. Panel (b) shows intersection shrinks with more runs. Panel (c) shows union expands with more runs and training variations. Together, these plots illustrate that vulnerable sample identification becomes increasingly inconsistent across seeds and configurations.

Figure 3. Reproducibility, stability, and coverage of zero-FP LiRA positives across runs (identical settings). Rows: number of runs combined ( $k$ ). Columns: within-run support threshold $x$ (TP $\geq x$ among 128 IN shadows), all at FP $=0$ . Modest support ( $x\in[2,5]$ ) improves reproducibility, while very strong support ( $x\geq 20$ ) reduces both stability and coverage. A grid of heatmaps showing reproducibility, stability, and coverage of zero-false-positive LiRA detections across multiple runs. Each row corresponds to the number of runs combined (k), and each column to the within-run support threshold x, defined as the number of agreeing shadow models among 128 IN models. Moderate support thresholds between 2 and 5 improve reproducibility, while higher thresholds reduce both stability and coverage.

Figure 4. Top-9 most vulnerable samples in 10 different runs. Upper row and leftmost run in the lower row: six independent runs (identical settings, different seeds). Rest of runs in the lower row, respectively: one run with a slight change in batch size and dropout, one run using MixUp instead of CutMix together with a modest dropout increase, one run using a different but similar architecture (ResNet18 vs. WideResNet28-2), and one run with transfer learning. On each image, TP denotes the number of true detections (true positives) among 128 IN shadow models in that run, while FP denotes the number of false detections among the 128 OUT shadow models (FP $=0$ for all displayed samples). The figure shows multiple grids of small image patches, each grid presenting the nine samples most frequently flagged as vulnerable by the online LiRA attack in a particular training run. Each grid is arranged in a 3×3 layout. For each image, TP denotes the number of true detections among 128 IN shadow models in that run, and FP denotes the number of false detections among the 128 OUT shadow models (FP=0 for all shown samples). The grids in the top row and the leftmost grid of the bottom row come from runs with identical training settings but different random seeds, yet they display noticeably different sets of images with little overlap in sample identity or ranking. The remaining grids on the bottom row correspond to runs with variations in batch size and dropout, a run using MixUp instead of CutMix, a run with a different model architecture, and a run using transfer learning. Overall, the figure illustrates that the samples identified as most vulnerable vary widely across runs and configurations, highlighting the instability of sample-level vulnerability.

		Pairwise ( $k=2$ runs)			All-runs ( $k=12$ runs)
Top- $q\%$	$T_{q}$	Tail $\rho$ (%)	$\|\cap\|/\|\cup\|$	Jaccard (%)	Tail $\rho$ (%)	$\|\cap\|/\|\cup\|$	Jaccard (%)
0.10	50	49.45 ±17.95	26/74 ±7	36.02 ±13.33	49.44	6/158	3.80
0.25	125	46.25 ±15.68	72/178 ±16	41.25 ±12.88	55.85	21/338	6.21
0.50	250	47.74 ±16.24	157/343 ±26	46.63 ±11.29	46.39	56/621	9.02
0.75	375	50.58 ±15.30	242/508 ±38	48.58 ±11.23	46.77	92/892	10.31
1.00	500	53.45 ±13.79	330/670 ±51	50.16 ±11.27	51.67	134/1153	11.62

Table 11. Reproducibility of ranking-based sets (Top-

q\%

by median gap ) and FPR-thresholded sets.

\rho

denotes the Spearman rank correlation, computed on the intersection set. † / ‡ : Top-

q\%

closely matching the all-runs

|\cap|

of FPR

=\!0.001\%

0.1\%

Figure 5. Rank displacement under run-to-run variability. Curves show the fraction of displaced samples whose percentile rank remained within the Top-

(q{+}\Delta)\%

. The figure shows a set of curves of the fraction of displaced samples that remain near the Top-

q\%

cutoff as the tolerance

\Delta

increases. The horizontal axis is

\Delta

(percentage points added to the cutoff), and the vertical axis is the fraction of displaced samples whose percentile rank in the reference run remains within the Top-

(q+\Delta)\%

. Each curve corresponds to a different Top-

q\%

level (as indicated by the legend). Error bars summarize variability across runs (mean

\pm

standard deviation). The curves rise quickly with small

\Delta

, indicating that most displaced samples fall just below the cutoff rather than far down the ranking.

		Performance		PPV (%)
Benchmark	Attack	TPR ′ (%)	FPR ′ (%)	@ $\pi$ =1%	@ $\pi$ =10%	@ $\pi$ =50%
Target-based thresholds (optimistic)
Baseline	Online	10.268 ±0.555	0.098 ±0.001	51.271 ±1.378	92.038 ±0.408	99.048 ±0.053
	Online (FV)	9.135 ±0.508	0.098 ±0.001	48.350 ±1.414	91.137 ±0.461	98.931 ±0.060
	Offline	3.262 ±0.338	0.098 ±0.001	25.031 ±1.963	78.499 ±1.803	97.040 ±0.310
	Offline (FV)	4.540 ±0.424	0.098 ±0.001	31.724 ±2.028	83.573 ±1.313	97.860 ±0.202

Table 12. CIFAR-10 under target vs. shadow calibration (target FPR

=0.1\%

Panel of Figure 6 — Figure 6. Reproducibility, stability, and coverage vs seeds, training variations, and runs (TP $\geq$ 1 for 0.1%FPR). Three line plots summarizing reproducibility online LiRA attack results across multiple training runs for 0.1%FPR. Panel (a) shows Jaccard similarity decreases sharply as runs increase. Panel (b) shows intersection shrinks with more runs. Panel (c) shows union expands with more runs and training variations. Together, these plots illustrate that vulnerable sample identification becomes increasingly inconsistent across seeds and configurations.

Figure 7. Correlation between loss ratio (test loss / train loss, log scale) and online LiRA [email protected]% FPR. Points represent 41 different model configurations The figure is a scatter plot where each point corresponds to one of 41 model configurations evaluated under different datasets, architectures, and regularization settings. The horizontal axis shows the loss ratio on a logarithmic scale, and the vertical axis shows the online LiRA [email protected]% FPR. The points form an upward trend, indicating that configurations with higher loss ratios also tend to yield higher LiRA success. Models with higher loss ratios tend to exhibit higher attack success rates, while models with ratios below 2 are much less vulnerable.

Panel of Figure 8 — Figure 8. In/out distributions for a representative CIFAR-10 sample under the online attack. AOF and TL significantly narrow the gap between members and non-members. The figure shows two sets of distributions for a single CIFAR-10 sample comparing member and non-member scores under different training configurations. In panel (a), the scaled logit score distributions for IN and OUT samples overlap increasingly as AOF and transfer learning are applied, reducing their separation. Panel (b) shows the corresponding log-likelihood ratios, which also collapse into a tightly overlapping range. Together, the plots illustrate that stronger regularization makes IN and OUT signals nearly indistinguishable for this sample.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9. Reproducibility, stability, and coverage of all LiRA positives across runs (identical settings) at a target FPR of 0.001%. Heatmaps summarizing reproducibility, stability, and coverage for all online LiRA detections at 0.001\% FPR across multiple identical-setting runs. Reproducibility decreases quickly as more runs are combined, and higher support thresholds reduce coverage sharply. — Figure 9. Reproducibility, stability, and coverage of all LiRA positives across runs (identical settings) at a target FPR of 0.001%. Heatmaps summarizing reproducibility, stability, and coverage for all online LiRA detections at 0.001% FPR across multiple identical-setting runs. Reproducibility decreases quickly as more runs are combined, and higher support thresholds reduce coverage sharply.

Figure 10. Reproducibility, stability, and coverage of all LiRA positives across runs (identical settings) at a target FPR of 0.1%. Heatmaps showing reproducibility, stability, and coverage for all online LiRA detections at 0.1\% FPR across multiple identical-setting runs. Agreement improves slightly compared to the 0.001\% setting, but coverage remains large and consistency still increases as more runs are combined. — Figure 10. Reproducibility, stability, and coverage of all LiRA positives across runs (identical settings) at a target FPR of 0.1%. Heatmaps showing reproducibility, stability, and coverage for all online LiRA detections at 0.1% FPR across multiple identical-setting runs. Agreement improves slightly compared to the 0.001% setting, but coverage remains large and consistency still increases as more runs are combined.

Figure 11. Top-16 most vulnerable samples across three seeds (columns) under two FPR settings (rows). Substantial cross-FPR disagreement is visible: many samples appear in only one FPR setting, and those shared often change rank. A grid comparing the top sixteen vulnerable samples identified across seeds under the two different FPR settings. Many samples appear only at one FPR, and shared samples frequently change position, indicating substantial instability in which samples are deemed most vulnerable.

Figure 12. Reproducibility, stability, and coverage of zero-FP LiRA positives across runs (identical settings) for 0.1%FPR. The figure presents a grid of heatmaps that summarize reproducibility, stability, and coverage for zero-false-positive LiRA detections at 0.1\% FPR under identical training settings. Each row corresponds to a different number of combined runs, and each column corresponds to a support threshold, defined as the minimum number of agreeing IN shadow models (out of 128) required for a detection within a run. The heatmaps show how reproducibility decreases as more runs are added, and how increasing the support threshold reduces the number of shared detections while also shrinking total coverage. The visualization highlights the trade-off between requiring stronger within-run agreement and maintaining consistent detections across multiple runs. — Figure 12. Reproducibility, stability, and coverage of zero-FP LiRA positives across runs (identical settings) for 0.1%FPR. The figure presents a grid of heatmaps that summarize reproducibility, stability, and coverage for zero-false-positive LiRA detections at 0.1% FPR under identical training settings. Each row corresponds to a different number of combined runs, and each column corresponds to a support threshold, defined as the minimum number of agreeing IN shadow models (out of 128) required for a detection within a run. The heatmaps show how reproducibility decreases as more runs are added, and how increasing the support threshold reduces the number of shared detections while also shrinking total coverage. The visualization highlights the trade-off between requiring stronger within-run agreement and maintaining consistent detections across multiple runs.

		Pairwise ( $k=2$ runs)				All-runs ( $k=12$ runs)
Setting	$T_{q}$	Tail $\rho$ (%)	$\|\cap\|$	$\|\cup\|$	Jaccard (%)	Tail $\rho$ (%)	$\|\cap\|$	$\|\cup\|$	Jaccard (%)
0.10	50	39.41 ±26.57	23 ±9	77 ±9	31.57 ±16.19	55.45	4	171	2.34
0.25	125	43.13 ±17.97	65 ±21	185 ±21	37.31 ±16.84	52.66	16	364	4.40
0.50	250	45.16 ±19.14	137 ±36	363 ±36	39.20 ±15.00	47.11	42	708	5.93
0.75	375	44.03 ±20.75	222 ±48	528 ±48	43.17 ±13.82	46.10	74	975	7.59
1.00	500	45.26 ±20.49	305 ±61	695 ±61	45.09 ±13.49	46.37	111	1255	8.84

Appendix

Table 13. Reproducibility of ranking-based sets (Top-

q\%

by mean gap and FPR-thresholded sets)

Figure 13. Distribution of median gap vulnerability scores within rank-percentile bins (bins computed per run; values pooled over 12 independent runs). Boxplots of per-sample median-gap vulnerability scores grouped into rank-percentile bins (computed per run and pooled across 12 runs). The top 1–2% bins exhibit substantially larger dispersion and right-skew compared to lower-percentile bins, illustrating that extreme-tail vulnerability scores are more sensitive to run-to-run variability.

Figure 14. For each run, the top-1 most vulnerable sample is selected, and its median gap score is evaluated across all 12 retrainings. Large interquartile ranges and strong right-skew indicate substantial cross-run variance in extreme vulnerability magnitude. While the top-1 identity rotates among a small set of recurring candidates, their scores vary markedly across runs, confirming sensitivity at the extreme boundary. Boxplots showing the distribution of median-gap scores for the sample that was ranked top-1 in each run, evaluated across all 12 retrainings. Each boxplot corresponds to one run’s top-1 sample. Large interquartile ranges and right-skew indicate that extreme vulnerability magnitude is not consistently preserved across runs.

Figure 15. Top-1 identity across runs. For each seed (panel), we display the single most vulnerable sample in that run (highest median gap ). Repeated appearances of the same image across seeds indicate overlap in the extreme tail. A 3×4 grid showing the single most vulnerable sample (highest median-gap score) identified in each of 12 independent runs. Each panel corresponds to one run. Repeated images across panels indicate overlap in top-1 identity, while differing images illustrate instability in extreme-tail selection.

Augmentation variations
Augmentation	DRP (%)	WD	Train Loss	Test Loss	Loss Ratio	Test Acc (%)	TPR (%)
CRP+FLP	0	5e-4	0.0004	0.2855	713.75	93.03	8.47
CRP+FLP+ROT	0	5e-4	0.0010	0.2389	238.90	93.38	8.01
CRP+FLP+CUT (LiRA)	0	5e-4	0.0079	0.2061	26.09	93.76	5.93
CRP+FLP+ROT+CUT	0	5e-4	0.0241	0.2206	9.15	93.08	4.20
CRP+FLP+ROT+CUT+JTR+MIX	0	5e-4	0.1221	0.2421	1.98	92.66	1.93

Appendix

Table 14. Complete ablation study: WideResNet-28-2 on CIFAR-10 with online LiRA (fixed variance) at 0.1% FPR, target-based threshold. Configurations sorted by TPR (descending) within each technique category.

Benchmark	[email protected]% FPR (%)	[email protected]% FPR (%)	Loss Ratio	Test Acc (%)
Baseline (RN-18)	3.956 ±1.061	10.268 ±0.555	71.0	93.63
AOF (RN-18)	0.248 ±0.198 ( $\times$ 16)	2.723 ±0.683 ( $\times$ 3.8)	1.88	94.09
AOF+TL (RN-18)	0.076 ±0.055 ( $\times$ 52)	0.782 ±0.171 ( $\times$ 13)	1.24	95.10
AOF+TL (EN-V2)	0.065 ±0.061 ( $\times$ 61)	0.521 ±0.128 ( $\times$ 20)	1.36	97.00

Appendix

Table 15. CIFAR-10 results under the online LiRA attack. The table isolates the effect of TL. Pretrained EfficientNet-V2 (EN-V2) yields a similar effect on attack as pretrained ResNet-18 (RN-18) while providing higher accuracy.

Membership Inference Attacks

Membership inference attacks test whether records or entities were used to train a machine-learning model, revealing potential privacy leakage.

See all 112 papers

Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA
Mar 12, 2026Rickard BrännvallExponential Family Membership Inference Attacks
RISE Research Institutes of Sweden
Estimating Model-Level Membership Inference Vulnerability Without Reference Models
Oct 22, 2025Euodia Dodd, Nataša Krčo, Igor Shilov +2Membership Inference Attacks
Imperial College London
On Reliability of Membership Inference Vulnerability Evaluation
May 25, 2026Joonas Jälkö, Gauri Pradhan, Ossi Räisä +1Data Leakage Privacy Auditing
University of Helsinki, Finland · CISPA Helmholtz Center for Information Security, Germany

Mar 12, 2026J/K move · Enter open · S save

Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

Rickard Brännvall

RISE Research Institutes of Sweden

Exponential Family Membership Inference Attacks Bayesian Inference

Open paper AskPDF arXiv

Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024) -- appear to use distinct scoring strategies, while the recently proposed BASE (Lassila et al., 2025) was shown to be equivalent to RMIA, making it difficult for practitioners to choose among them. We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions and the number of parameters estimated per data point. This unification reveals a hierarchy (BASE1-4) that connects RMIA and LiRA as endpoints of a spectrum of increasing model complexity, and yields a practical rule -- match the attack's complexity to the available shadow-model budget. Within this framework, we identify variance estimation as a primary bottleneck at small shadow-model budgets and propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors. BaVarIA yields a Student-t predictive (BaVarIA-t) or a Gaussian with stabilized variance (BaVarIA-n), providing stable performance without per-dataset hyperparameter tuning. Across 12 testbeds and 7 shadow-model budgets, BaVarIA is a drop-in replacement for LiRA that matches or, on average, improves upon it. The gains are largest in the practically important low-shadow-model and offline regimes: offline, the Bayesian prior replaces LiRA's heuristic and outperforms it on 10 of 12 testbeds.

Abstract

Membership Inference Attacks

Explore similar work

Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

Estimating Model-Level Membership Inference Vulnerability Without Reference Models

On Reliability of Membership Inference Vulnerability Evaluation