cs.LGNov 7, 2026

Bayesian Low-Rank Posteriors for Scalable Membership Inference

Authors: Khanh Nguyen, Antonio Emanuele Cinà, Andrey Barsky, Alejandra Reinares, Fabio Roli, Dimosthenis Karatzas

Organizations: Computer Vision Center, Universitat Autònoma de Barcelona · University of Trieste, MIGe, Italy · University of Genoa, DIBRIS, Italy · University of Cagliari, DIEE, Italy

Abstract

Membership inference attacks (MIAs) aim to determine whether a sample was used during the training of a target model. Strong MIAs rely on reference (or shadow) models to estimate target-conditional score distributions, but training such ensembles is computationally expensive for modern large-scale Vision--Language Models (VLMs). In this work, we propose a scalable alternative that replaces explicit reference-model training with a Bayesian approximation over low-rank adaptation parameters. Specifically, we model uncertainty only within the LoRA subspace and construct a posterior from a single training run, from which we sample a diverse set of virtual reference models. We instantiate this approach using Stochastic Weight Averaging Gaussian (SWAG), enabling efficient approximation of target-conditional score distributions at a fraction of the cost of conventional shadow-model ensembles. We evaluate the resulting attack on three downstream-adapted VLMs across two privacy-sensitive visual question answering tasks, namely Document VQA and Medical VQA. To isolate the effect of pretraining knowledge, we additionally introduce a controlled synthetic dataset. Our approach outperforms standard reference-based attacks while requiring only a single reference-model fine-tuning run, making membership inference more practical for large VLMs.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration

    Oct 12, 2026Damien Lo, Hong Kyu Lee, Ruixuan Liu +1Membership Inference AttacksLarge Vision-Language Models

  2. CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

    Jun 16, 2026Jeffrey G. Wang, Jason Wang, Marvin Li +1LLM EvaluationMembership Inference Attacks

  3. Estimating Model-Level Membership Inference Vulnerability Without Reference Models

    Oct 22, 2025Euodia Dodd, Nataša Krčo, Igor Shilov +2Membership Inference Attacks