cs.LGOct 22, 2025

Estimating Model-Level Membership Inference Vulnerability Without Reference Models

Authors: Euodia Dodd, Nataša Krčo, Igor Shilov, Matthew Wicker, Yves-Alexandre de Montjoye

Organizations: Imperial College London

Abstract

Membership inference attacks (MIAs) have emerged as the standard tool for evaluating the privacy risks of AI models. However, state-of-the-art attacks require training numerous, often computationally expensive, reference models, limiting their practicality. We present a novel approach for estimating model-level vulnerability to the Likelihood Ratio Attack (LiRA), the strongest available attack, directly from the train and test loss distributions of the target model and without training any reference models. We show that LiRA's per-sample signal decomposes into a variance-ratio term and a residual mean-shift term, with the relative contribution of each determined by how much training collapses model uncertainty at the trained sample. This places models on a continuum, with different regimes calling for different reference-free loss-based statistics as proxies for LiRA TPR. The shapes of the loss distributions themselves indicate which proxy applies. We instantiate the framework with two natural proxies. At the heavy-tailed end, the LOSS attack TNR predicts LiRA TPR@FPR=10−310^{-3} with RMSE 0.036 across 10 image classification architectures and 4 datasets, outperforming low-cost reference-model attacks such as RMIA. At the symmetric end, the LOSS attack AUC predicts LiRA TPR with RMSE 0.018 across five GPT-2 sizes from 10M to 1B parameters.

Explore similar work

CardsList
  1. CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

    Jun 16, 2026Jeffrey G. Wang, Jason Wang, Marvin Li +1Membership Inference AttacksAttacker Large Language Model

  2. On Reliability of Membership Inference Vulnerability Evaluation

    May 25, 2026Joonas Jälkö, Gauri Pradhan, Ossi Räisä +1Membership Inference AttacksFalse-Negative Rate

  3. Membership Inference via Pairwise Likelihood Ratios

    Sep 14, 2026Shengjie Niu, Zebin Yun, Yeheng Ge +1Membership Inference AttacksMembership Inference