Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

Rickard Brännvall

cs.LGMar 12, 2026

Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

Authors: Rickard Brännvall

Organizations: RISE Research Institutes of Sweden

Abstract

Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024) -- appear to use distinct scoring strategies, while the recently proposed BASE (Lassila et al., 2025) was shown to be equivalent to RMIA, making it difficult for practitioners to choose among them. We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions and the number of parameters estimated per data point. This unification reveals a hierarchy (BASE1-4) that connects RMIA and LiRA as endpoints of a spectrum of increasing model complexity, and yields a practical rule -- match the attack's complexity to the available shadow-model budget. Within this framework, we identify variance estimation as a primary bottleneck at small shadow-model budgets and propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors. BaVarIA yields a Student-t predictive (BaVarIA-t) or a Gaussian with stabilized variance (BaVarIA-n), providing stable performance without per-dataset hyperparameter tuning. Across 12 testbeds and 7 shadow-model budgets, BaVarIA is a drop-in replacement for LiRA that matches or, on average, improves upon it. The gains are largest in the practically important low-shadow-model and offline regimes: offline, the Bayesian prior replaces LiRA's heuristic and outperforms it on 10 of 12 testbeds.

Figures & tables

Additional families (Appendix B ):
Model	Statistic $z$	Params	LLR form
Exponential	loss $\ell$	2	linear in $\ell$
Gaussian	log-odds $\varphi$	4	$\varphi,\varphi^{2}$
(equal var.)	log-odds $\varphi$	3	linear in $\varphi$
Gamma	loss $\ell$	4	$\ell,\log\ell$
Beta	confidence $p$	4	$\log p,\log(1{-}p)$

Table 1: Exponential-family LLR-based attacks.

Panel of Figure 1 — Figure 1: Online ROC for the BASE hierarchy (BASE1 $=$ RMIA at $\gamma{=}1$ ; BASE4 $=$ LiRA for $K\geq 64$ ) on CIFAR-100/WRN (left) and Texas/MLP3 (right) at the two budget extremes $K\in\{8,254\}$ , mean over 32 replicates. The method ordering reverses with $K$ : simpler families win at small $K$ , richer families at large $K$ (the intermediate $K=64$ and the continuous crossover are in Figure 2 ).

Figure 2: TPR @ FPR

=0.01

vs. shadow-model budget

K

for the BASE hierarchy (CIFAR-100/WRN and Texas/MLP3), mean

\pm 1

SE over 32 replicates. Simpler families lead at small

K

and richer families at large

K

; BASE2 (separate IN/OUT means, constant variance) is the informative two-parameter intermediate.

	$K=8$ (simple wins)			$K=254$ (complex wins)
Testbed	AUC	[email protected]	[email protected]	AUC	[email protected]	[email protected]
CIFAR-10 / WRN	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆
CIFAR-100 / WRN	+0.72 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆
CINIC-10 / WRN	+0.07 ⋆	+0.74 ⋆	+0.79 ⋆	+1.00 ⋆	+1.00 ⋆	+0.96 ⋆
Location / MLP3	-0.53	+0.75 ⋆	+0.99 ⋆	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆
Purchase / MLP3	+1.00 ⋆	+1.00 ⋆	+1.00 ⋆	+0.53 ⋆	+0.80 ⋆	+0.65 ⋆

Table 2: Weighted concordance

S\in[-1,+1]

of the observed method ordering with the BASE hierarchy’s prediction ( simple wins at

K=8

: BASE1

>

BASE2

>

BASE3

>

BASE4; complex wins at

K=254

: the reverse).

S=+1

: all pairs agree;

-1

: fully reversed. Bold :

S\geq 0.9

; italic :

S\leq-0.5

\star

: ordering prediction is significantly supported at 95% confidence (bootstrap percentile over the 32 replicates lies above 0).

Panel of Figure 3 — Figure 3: Online (top) and offline (bottom) TPR @ FPR $=0.01$ vs. $K$ for LiRA, BaVarIA- $n$ , BaVarIA- $t$ on CIFAR-100/WRN and Texas/MLP3, mean $\pm 1$ SE over 32 replicates. Online, BaVarIA’s gain peaks near the $K=64$ cutover and vanishes as $K\to\infty$ ; offline, the gain persists at every $K$ because the NIG prior substitutes for the absent IN class.

	Online			Offline
Testbed	LiRA	BaVarIA- $n$	BaVarIA- $t$	LiRA	BaVarIA- $n$	BaVarIA- $t$
CIFAR-10 / WRN	0.129	0.134	0.133	0.087	0.074	0.069
CIFAR-100 / WRN	0.362	0.371	0.368	0.308	0.322	0.320
CINIC-10 / WRN	0.180	0.187	0.187	0.123	0.181	0.178
Location / MLP3	0.423	0.440	0.439	0.327	0.412	0.405
Purchase / MLP3	0.031	0.035	0.036	0.023	0.026	0.023

Table 3: Online and offline TPR @ FPR

=0.01

K=64

, per testbed, mean over 32 replicates; best per row within each setting in bold. BaVarIA-

n

beats LiRA on

12/12

testbeds online and

10/12

offline (offline, LiRA wins only CIFAR-10). The BaVarIA-

n

vs. LiRA differences are significant at

95\%

confidence (Appendix F ).

Testbed	$\varepsilon$	LiRA	BaVarIA- $n$	BASE/RMIA
CIFAR-10 / WRN	6.7	.010	.011	.013
CIFAR-100 / WRN	6.1	.009	.009	.012
Location / MLP3	8.9	.012	.012	.015
Purchase / MLP3	5.7	.010	.011	.012

Table 4: Online TPR @ FPR

=0.01

under DP-SGD at noise multiplier

\sigma=1.0

(

K=64

, mean over 32 replicates). Best per row in bold; TPR

=0.01

is the random-guessing level at this FPR. The full grid

\sigma\in\{0,0.01,0.1,1.0\}

, privacy budgets, utility, and ROC curves in Appendix L .

Appendix figures & tables45 assets

Supplementary material from the paper’s appendix.

Appendix

Testbed	BASE1	BASE2	BASE3	BASE4
CIFAR-10 / WRN	0.098	0.086	0.072	0.027
CIFAR-100 / WRN	0.294	0.283	0.223	0.086
CINIC-10 / WRN	0.111	0.156	0.109	0.041
Location / MLP3	0.225	0.297	0.215	0.110
Purchase / MLP3	0.028	0.024	0.019	0.011
Texas / MLP3	0.202	0.191	0.177	0.065

Appendix

Table 5: TPR @ FPR

=0.01

K=8

, redefined BASE hierarchy. Mean over 32 replicates. Best per row in bold. LiRA

=

BASE4, RMIA

=

BASE1.

Testbed	BASE1	BASE2	BASE3	BASE4
CIFAR-10 / WRN	0.111	0.126	0.136	0.128
CIFAR-100 / WRN	0.326	0.345	0.366	0.362
CINIC-10 / WRN	0.129	0.180	0.182	0.179
Location / MLP3	0.274	0.378	0.398	0.434
Purchase / MLP3	0.037	0.037	0.040	0.031
Texas / MLP3	0.253	0.232	0.306	0.305

Appendix

Table 6: TPR @ FPR

=0.01

K=64

, redefined BASE hierarchy. Mean over 32 replicates. Best per row in bold. LiRA

=

BASE4, RMIA

=

BASE1.

Testbed	BASE1	BASE2	BASE3	BASE4
CIFAR-10 / WRN	0.113	0.132	0.142	0.143
CIFAR-100 / WRN	0.330	0.354	0.383	0.385
CINIC-10 / WRN	0.131	0.183	0.189	0.192
Location / MLP3	0.286	0.387	0.421	0.452
Purchase / MLP3	0.038	0.040	0.045	0.043
Texas / MLP3	0.261	0.237	0.320	0.326

Appendix

Table 7: TPR @ FPR

=0.01

K=254

, redefined BASE hierarchy. Mean over 32 replicates. Best per row in bold. LiRA

=

BASE4, RMIA

=

BASE1.

Figure 4: Per-datapoint QQ diagnostic: standardized residuals of shadow log-odds (per data point, per class) pooled across datapoints and plotted against $\mathcal{N}(0,1)$ quantiles. Annotations show per-datapoint Anderson-Darling (AD) rejection rates. CIFAR-10 WRN is well-approximated by a Gaussian ( ${\sim}6\%$ AD rejection); CIFAR-100 ResNet shows heavier tails in the IN class ( $74\%$ AD rejection).

	IN class		OUT class
Dataset	med AD	rej%	med AD	rej%
C10-RN	0.475	22.4	0.438	19.4
C10-WRN	0.342	5.9	0.355	6.5
C100-RN	1.106	74.3	0.388	11.1
C100-WRN	0.357	6.8	0.369	8.2
CN10-RN	0.607	37.6	0.420	18.2

Appendix

Table 8: Per-datapoint Anderson-Darling normality test on shadow log-odds (

K=254

n=1{,}000

datapoints per class). Rejection rate at the

5\%

significance level (critical value

0.748

). Gaussianity holds well for WideResNet on CIFAR-10/100 but is strongly violated for Location MLP3 and CINIC-10 WideResNet.

	Online		Offline
Testbed	$\Delta$ (%pt)	95% CI	$\Delta$ (%pt)	95% CI
CIFAR-10 / WRN	+0.63 ⋆	[+0.52, +0.73]	-1.36 †	[-1.74, -0.97]
CIFAR-100 / WRN	+0.85 ⋆	[+0.71, +0.99]	+1.32 ⋆	[+1.04, +1.60]
CINIC-10 / WRN	+0.71 ⋆	[+0.65, +0.78]	+5.81 ⋆	[+5.54, +6.08]
Location / MLP3	+0.72 ⋆	[+0.40, +1.06]	+7.68 ⋆	[+6.56, +8.88]
Purchase / MLP3	+0.44 ⋆	[+0.42, +0.46]	+0.25 ⋆	[+0.17, +0.33]

Appendix

Table 9: Paired advantage

\Delta=

BaVarIA-

n

-

LiRA in TPR @ FPR

=0.01

K=64

(percentage points), over the 32 replicates (shared-shadows: both methods on one common shadow subsample per replicate). 95% CI is a percentile bootstrap of the mean

\Delta

. ⋆ / † : significantly better/worse at

95\%

confidence (paired Wilcoxon signed-rank, Holm–Bonferroni within each family). BaVarIA-

n

significantly beats LiRA on 12/12 online and 10/12 offline, and is significantly worse on 2/12 offline (both CIFAR-10 testbeds).

Dataset	EXP	GAUSS	GAMMA	BETA	LiRA	RMIA
C10-WRN	0.695	0.720	0.651	0.691	0.720	0.691
C100-WRN	0.883	0.908	0.897	0.903	0.908	0.882
CN10-WRN	0.748	0.756	0.738	0.746	0.756	0.708
Loc-4	0.885	0.929	0.852	0.887	0.929	0.843
Pur-4	0.617	0.649	0.566	0.597	0.649	0.642
Tex-4	0.857	0.895	0.785	0.865	0.895	0.862

Appendix

Table 10: AUC by distributional family at

K=64

, mean over 32 replicates; best per row in bold.

Dataset	EXP	GAUSS	GAMMA	BETA	LiRA	RMIA
C10-WRN	0.018	0.129	0.020	0.018	0.129	0.112
C100-WRN	0.128	0.361	0.060	0.330	0.361	0.326
CN10-WRN	0.168	0.178	0.048	0.146	0.178	0.129
Loc-4	0.041	0.355	0.054	0.112	0.355	0.189
Pur-4	0.008	0.053	0.010	0.012	0.053	0.049
Tex-4	0.060	0.357	0.075	0.151	0.357	0.281

Appendix

Table 11: [email protected] by distributional family at

K=64

, mean over 32 replicates; best per row in bold.

Testbed	LiRA	BaVarIA- $n$	BaVarIA- $t$
CIFAR-10 / WRN	0.686	0.680	0.689
CIFAR-100 / WRN	0.888	0.886	0.887
CINIC-10 / WRN	0.729	0.729	0.732
Location / MLP3	0.933	0.936	0.941
Purchase / MLP3	0.581	0.576	0.585
Texas / MLP3	0.844	0.844	0.854

Appendix

Table 12: Online AUC,

K=8

. Mean over 32 replicates. Best per row in bold.

Testbed	LiRA	BaVarIA- $n$	BaVarIA- $t$
CIFAR-10 / WRN	0.092	0.087	0.070
CIFAR-100 / WRN	0.294	0.293	0.261
CINIC-10 / WRN	0.158	0.154	0.148
Location / MLP3	0.344	0.371	0.351
Purchase / MLP3	0.025	0.021	0.021
Texas / MLP3	0.204	0.226	0.212

Appendix

Table 13: Online TPR @ FPR

=0.01

K=8

. Mean over 32 replicates. Best per row in bold.

Method	C10	C100	CN10	Loc	Pur	Tex
LiRA	.720	.908	.756	.929	.649	.895
BASE3	.723	.908	.753	.928	.654	.894
BASE2	.721	.906	.755	.923	.654	.886
RMIA	.691	.882	.708	.843	.642	.862
ELSA1	.588	.805	.613	.802	.563	.812
ELSA2 $\phi$	.684	.884	.701	.925	.620	.882

Appendix

Table 14: ELSA: AUC at

K=64

, mean over 32 replicates (WideResNet image and MLP4 tabular testbeds). Discriminative ELSA recovers but does not beat the generative Gaussian estimators; best per column in bold.

Method	C10	C100	CN10	Loc	Pur	Tex
LiRA	0.129	0.361	0.178	0.355	0.053	0.357
BASE3	0.135	0.363	0.178	0.366	0.063	0.355
BASE2	0.127	0.345	0.178	0.283	0.059	0.280
RMIA	0.112	0.326	0.129	0.189	0.049	0.281
ELSA1	0.054	0.178	0.071	0.137	0.021	0.213
ELSA2 $\phi$	0.118	0.316	0.180	0.340	0.057	0.320

Appendix

Table 15: ELSA: [email protected] at

K=64

, mean over 32 replicates (WideResNet and MLP4 testbeds). The generative BASE hierarchy leads; ELSA2

\phi

only matches its BASE2 twin; best per column in bold.

Method	Online parameters	Offline fallback
BASE1	$\hat{z}_{i,\text{pool}}$ from all shadows	Same; optional $\alpha$ scaling ( 22 )
BASE2	$\hat{\mu}_{i,0},\hat{\mu}_{i,1}$ ; global $\sigma^{2}$	$\hat{\mu}_{i,1}=\hat{\mu}_{i,0}+\Delta$ ; $\to$ const-var LLR
BASE3	$\hat{\mu}_{i,0},\hat{\mu}_{i,1}$ ; $\hat{\sigma}_{i}^{2}$ pooled	$\hat{\mu}_{i,1}=\hat{\mu}_{i,0}+\Delta$ ; $\to$ eq-var LLR
BASE4	$\hat{\mu}_{i,m},\hat{\sigma}_{i,m}^{2}$ per class	$\sigma_{\text{in}}^{2}=\sigma_{\text{out}}^{2}$ , $\mu_{\text{in}}$ via shift; $\to$ eq-var LLR

Appendix

Table 16: BASE hierarchy: online vs. offline parameter estimation. In the offline setting, methods that estimate separate IN-class parameters collapse toward pooled estimation.

	$K=8$		$K=64$		$K=254$
Method	Off	$\Delta$	Off	$\Delta$	Off	$\Delta$
LiRA	0.755	+0.033	0.772	+0.042	0.775	+0.045
RMIA	0.763	+0.009	0.769	+0.008	0.770	+0.008
BaVarIA- $n$	0.773	+0.013	0.785	+0.029	0.786	+0.034
BaVarIA- $t$	0.778	+0.015	0.784	+0.031	0.784	+0.036

Appendix

Table 17: Offline AUC (mean over 12 testbeds, 32 replicates).

\Delta

= online

-

offline. Bold: best offline value per

K

	$K=8$		$K=64$		$K=254$
Method	Off	$\Delta$	Off	$\Delta$	Off	$\Delta$
LiRA	0.174	+0.034	0.216	+0.050	0.233	+0.051
RMIA	0.144	+0.013	0.180	+0.013	0.185	+0.013
BaVarIA- $n$	0.200	+0.014	0.246	+0.027	0.254	+0.031
BaVarIA- $t$	0.196	+0.005	0.242	+0.029	0.249	+0.035

Appendix

Table 18: Offline TPR at FPR

{=}0.01

(mean over 12 testbeds, 32 replicates).

\Delta

= online

-

offline. Bold: best offline value.

Testbed	BASE	LiRA	RAPID	BaVarIA- $n$
CIFAR-10 / WRN	0.082	0.087	0.115	0.074
CIFAR-100 / WRN	0.218	0.308	0.337	0.322
CINIC-10 / WRN	0.090	0.123	0.158	0.181
Location / MLP3	0.148	0.327	0.393	0.412
Purchase / MLP3	0.034	0.023	0.037	0.026
Texas / MLP3	0.177	0.250	0.238	0.268

Appendix

Table 19: Offline TPR @ FPR

=0.01

K=64

, per testbed (mean over 32 replicates); best per row in bold. RAPID is an offline difficulty-calibration attack: an MLP head on two features—the raw loss and the offline BASE calibrated score (the same OUT-only reference used by BASE/LiRA/BaVarIA-

n

)—trained on shadow-derived member/non-member labels, frozen, and applied to the target.

Testbed	BASE	LiRA	RAPID	BaVarIA- $n$
CIFAR-10 / WRN	0.659	0.677	0.703	0.666
CIFAR-100 / WRN	0.825	0.856	0.893	0.847
CINIC-10 / WRN	0.670	0.691	0.745	0.749
Location / MLP3	0.816	0.902	0.941	0.927
Purchase / MLP3	0.614	0.607	0.622	0.596
Texas / MLP3	0.822	0.816	0.856	0.832

Appendix

Table 20: Offline AUC at

K=64

(RAPID MLP head); best per row in bold.

Calibrated score	Attack	AUC	[email protected]	[email protected]
BASE calibration (RMIA/BASE)	score alone	0.746	0.128	0.041
	$+$ raw loss (RAPID)	0.796	0.207	0.091
LiRA (variance-aware)	score alone	0.772	0.216	0.093
	$+$ raw loss (RAPID)	0.807	0.207	0.075
BaVarIA- $n$ (variance-aware)	score alone	0.785	0.246	0.117
	$+$ raw loss (RAPID)	0.797	0.247	0.118

Appendix

Table 21: Mean over the 12 testbeds (offline,

K=64

): a calibrated score alone vs. RAPID (that score

+

raw loss, MLP head). Adding the raw-loss feature lifts the plain BASE calibration substantially but does almost nothing once the calibrated component is already variance-aware (BaVarIA-

n

)—indicating that it gives “diminishing returns” once variance is modelled.

Testbed	Defence	Test Acc (%)	$\varepsilon$	$\delta$
CIFAR-10 / WRN	Baseline	76.1	—	—
	$\sigma=0.01$	71.6	${>}1$ M	$3.3\!\times\!10^{-5}$
	$\sigma=0.1$	71.2	${>}10$ K	$3.3\!\times\!10^{-5}$
	$\sigma=1.0$	54.1	6.7	$3.3\!\times\!10^{-5}$
CIFAR-100 / WRN	Baseline	40.0	—	—
	$\sigma=0.01$	27.7	${>}1$ M	$3.3\!\times\!10^{-5}$

Appendix

Table 22: Privacy budget and model utility per testbed and noise level (DP-SGD

C=10

and

\delta\approx 1/n_{\mathrm{train}}

);

\varepsilon

from the Opacus accountant, rounded down to the nearest order of magnitude when

{>}100

Testbed	Defense	Acc (%)	LiRA	BaVarIA- $n$	BaVarIA- $t$	BASE
CIFAR-10 WRN	Baseline	76.1	.773	.775	.776	.762
	$\sigma=0.01$	71.6	.674	.678	.678	.653
	$\sigma=0.1$	71.2	.641	.645	.645	.630
	$\sigma=1.0$	54.1	.509	.510	.510	.526
CIFAR-100 WRN	Baseline	40.0	.741	.743	.744	.742
	$\sigma=0.01$	27.7	.600	.603	.604	.620

Appendix

Table 23: AUC across testbeds and defense levels. Mean over 32 replicates,

K=64

, online. Best method per row in bold. BaVarIA-

n

/ BaVarIA-

t

track or beat LiRA at every noise level (16/16 cells in AUC); BASE progressively overtakes the Gaussian methods as

\sigma

grows, winning 4/4 testbeds at

\sigma=1.0

. The crossover mirrors the small-

K

regime in the absence of defense—simpler distributional families dominate when per-sample signal is scarce, whether scarcity comes from few shadow models or from added DP-SGD noise.

Testbed	Defense	Acc (%)	LiRA	BaVarIA- $n$	BaVarIA- $t$	BASE
CIFAR-10 WRN	Baseline	76.1	.150	.156	.156	.148
	$\sigma=0.01$	71.6	.045	.049	.049	.048
	$\sigma=0.1$	71.2	.032	.037	.036	.038
	$\sigma=1.0$	54.1	.010	.011	.011	.013
CIFAR-100 WRN	Baseline	40.0	.082	.087	.086	.093
	$\sigma=0.01$	27.7	.022	.023	.023	.028

Appendix

Table 24: TPR at FPR

=0.01

across testbeds and defense levels. Mean over 32 replicates,

K=64

, online. Best per row in bold. The Gaussian-family BaVarIA-

n

matches or beats LiRA at every noise level; BASE leads at heavy noise. At

\sigma=1.0

all attacks approach random guessing (TPR

\leq 0.015

vs. random at

0.01

Figure 5: Offline TPR @ FPR

=0.01

vs. shadow budget

K

(WRN and MLP3 testbeds), mean over 32 replicates: BASE, LiRA, BaVarIA-

n

, and RAPID (MLP head). RAPID tracks the calibrated attacks and is competitive at small

K

, while BaVarIA-

n

pulls ahead as the budget grows.

Figure 6: Offline TPR @ FPR

=0.01

vs. shadow budget

K

(RN18 and MLP4 testbeds); same methods and protocol as Figure 5 .

Figure 7: Defense-level view, baseline (clipping;

\sigma=0

): all four attacks (LiRA, BASE, BaVarIA-

n

, BaVarIA-

t

) overlaid on each of four testbeds.

K=64

, online, mean ROC over 32 replicates, log–log axes; legend AUC included. Panel titles include mean test accuracy across shadow models.

Figure 10: Defense-level view, DP-SGD with

\sigma=1.0

C=10

(meaningful privacy,

\varepsilon\approx 6

–

9

). LiRA, BASE, BaVarIA-

n

, BaVarIA-

t

overlaid;

K=64

, mean over 32 replicates. All attacks approach random guessing at this noise level.

Figure 11: Attack-method view, LiRA across defense levels

\sigma\in\{0,\,0.01,\,0.1,\,1.0\}

on each testbed. Linestyle distinguishes defense level;

K=64

, online, mean over 32 replicates, log–log axes.

Figure 12: Attack-method view, BASE across defense levels

\sigma\in\{0,\,0.01,\,0.1,\,1.0\}

. Linestyle distinguishes defense level;

K=64

, online, mean over 32 replicates.

Figure 13: Attack-method view, BaVarIA-

n

across defense levels

\sigma\in\{0,\,0.01,\,0.1,\,1.0\}

. Linestyle distinguishes defense level;

K=64

, online, mean over 32 replicates.

Figure 14: Attack-method view, BaVarIA-

t

across defense levels

\sigma\in\{0,\,0.01,\,0.1,\,1.0\}

. Linestyle distinguishes defense level;

K=64

, online, mean over 32 replicates.

Figure 15: Online AUC vs.

K

(WRN/MLP3 testbeds).

Figure 16: Online AUC vs.

K

(RN18/MLP4 testbeds).

Figure 17: Online TPR @ FPR

=0.01

vs.

K

(WRN/MLP3 testbeds).

Figure 18: Online TPR @ FPR

=0.01

vs.

K

(RN18/MLP4 testbeds).

Figure 19: Online AUC vs.

K

for the BASE hierarchy (WRN/MLP3 testbeds). BASE3/BASE4 overtake the simpler families at moderate

K

; BASE2 tracks BASE3.

Figure 20: Online AUC vs.

K

for the BASE hierarchy (RN18/MLP4 testbeds).

Figure 21: Online TPR at FPR

{=}0.01

vs.

K

for the BASE hierarchy (WRN/MLP3 testbeds).

Figure 22: Online TPR at FPR

{=}0.01

vs.

K

for the BASE hierarchy (RN18/MLP4 testbeds).

Figure 23: Online vs. offline AUC as a function of

K

(WRN/MLP3 testbeds). Solid = online, dashed = offline.

Figure 24: Online vs. offline AUC as a function of

K

(RN18/MLP4 testbeds).

Figure 25: Online vs. offline TPR at FPR

{=}0.01

as a function of

K

(WRN/MLP3 testbeds). Solid = online, dashed = offline.

Figure 26: Online vs. offline TPR at FPR

{=}0.01

as a function of

K

(RN18/MLP4 testbeds).

Figure 27: BASE hierarchy ROC at

K=8

(WRN/MLP3 testbeds).

Figure 28: BASE hierarchy ROC at

K=254

(WRN/MLP3 testbeds), ordering reversed relative to

K=8

Figure 29: BaVarIA online ROC at

K=64

(WRN/MLP3 testbeds).

Figure 30: BaVarIA offline ROC at

K=64

(WRN/MLP3 testbeds).

Exponential Family

See all 13 papers

Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions
Mar 8, 2026Najeeb Jebreel, Mona Khalil, David Sánchez +1Membership Inference Attacks Privacy Auditing
Universitat Rovira i Virgili, Catalonia
Estimating Model-Level Membership Inference Vulnerability Without Reference Models
Oct 22, 2025Euodia Dodd, Nataša Krčo, Igor Shilov +2Membership Inference Attacks
Imperial College London
On Reliability of Membership Inference Vulnerability Evaluation
May 25, 2026Joonas Jälkö, Gauri Pradhan, Ossi Räisä +1Data Leakage Privacy Auditing
University of Helsinki, Finland · CISPA Helmholtz Center for Information Security, Germany

Mar 8, 2026J/K move · Enter open · S save

Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions

Najeeb Jebreel, Mona Khalil, David Sánchez, Josep Domingo-Ferrer

Universitat Rovira i Virgili, Catalonia

Membership Inference Attacks Privacy Auditing

Open paper AskPDF arXiv

Membership inference attacks (MIAs) have become the standard tool for evaluating privacy leakage in machine learning (ML). Among them, the Likelihood-Ratio Attack (LiRA) is widely regarded as the state of the art when sufficient shadow models are available. However, prior evaluations have often overstated the effectiveness of LiRA by attacking models overconfident on their training samples, calibrating thresholds on target data, assuming balanced membership priors, and/or overlooking attack reproducibility. We re-evaluate LiRA under a realistic protocol that (i) trains models using anti-overfitting (AOF) and transfer learning (TL), when applicable, to reduce overconfidence as in production models; (ii) calibrates decision thresholds using shadow models and data rather than target data; (iii) measures positive predictive value (PPV, or precision) under shadow-based thresholds and skewed membership priors (pi <= 10%); and (iv) quantifies per-sample membership reproducibility across different seeds and training variations. We find that AOF significantly weakens LiRA, while TL further reduces attack effectiveness while improving model accuracy. Under shadow-based thresholds and skewed priors, LiRA's PPV often drops substantially, especially under AOF or AOF+TL. We also find that thresholded vulnerable sets at extremely low FPR show poor reproducibility across runs, while likelihood-ratio rankings are more stable. These results suggest that LiRA, and likely weaker MIAs, are less effective than previously suggested under realistic conditions, and that reliable privacy auditing requires evaluation protocols that reflect practical training practices, feasible attacker assumptions, and reproducibility considerations. Code is available at https://github.com/najeebjebreel/lira_analysis.

Abstract

Exponential Family

Explore similar work

Revisiting the LiRA Membership Inference Attack Under Realistic Assumptions

Estimating Model-Level Membership Inference Vulnerability Without Reference Models

On Reliability of Membership Inference Vulnerability Evaluation