CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training
Authors: Qi Li, Cheng-Long Wang, Yinzhi Cao, Di Wang
Organizations: King Abdullah University of Science and Technology · Provable Responsible AI and Data Analytics Lab · National University of Singapore · Johns Hopkins University
Training models on a carefully chosen portion of data rather than the full dataset is now a standard preprocess for modern ML. From vision coreset selection to large-scale filtering in language models, it enables scalability with minimal utility loss. A common intuition is that training on fewer samples should also reduce privacy risks. In this paper, we challenge this assumption. We show that subset training is not privacy free: the very choices of which data are included or excluded can introduce new privacy surface and leak more sensitive information. Such information can be captured by adversaries either through side-channel metadata from the subset selection process or via the outputs of the target model. To systematically study this phenomenon, we propose CoLA (Choice Leakage Attack), a unified framework for analyzing privacy leakage in subset selection. In CoLA, depending on the adversary's knowledge of the side-channel information, we define two practical attack scenarios: Subset-aware Side-channel Attacks and Black-box Attacks. Under both scenarios, we investigate two privacy surfaces unique to subset training: (1) Training-membership MIA (TM-MIA), which concerns only the privacy of training data membership, and (2) Selection-participation MIA (SP-MIA), which concerns the privacy of all samples that participated in the subset selection process. Notably, SP-MIA enlarges the notion of membership from model training to the entire data-model supply chain. Experiments on vision and language models show that existing threat models underestimate subset-training privacy risks: the expanded privacy surface leaks both training and selection membership, extending risks from individual models to the broader ML ecosystem.
Figures & tables
Figure 1: Privacy surfaces under subset training.
Figure 2: Signal distributions under subset training.
Intensity
Setting
NN
NN_top3
NN_cls
LiRA
CoLA
AUC
TPR@5%FPR
AUC
TPR@5%FPR
AUC
TPR@5%FPR
AUC
TPR@5%FPR
AUC
TPR@5%FPR
Light
SP-MIA
51.23 ± 2.56
5.83 ± 1.34
51.77 ± 3.02
3.34 ± 4.45
51.59 ± 2.66
6.37 ± 2.22
51.26 ± 4.85
6.43 ± 3.70
61.39 ± 2.48
14.24 ± 2.02
TM-MIA
64.00 ± 12.15
12.13 ± 5.96
61.57 ± 15.05
8.93 ± 13.52
67.24 ± 16.18
19.30 ± 17.14
69.86 ± 22.08
15.74 ± 18.19
83.77 ± 2.44
42.19 ± 4.51
Medium
SP-MIA
52.33 ± 3.56
6.31 ± 1.48
53.59 ± 4.30
3.56 ± 2.28
54.51 ± 4.51
6.64 ± 1.58
54.99 ± 4.69
5.31 ± 0.42
81.93 ± 3.50
42.66 ± 5.81
TM-MIA
59.84 ± 12.84
10.80 ± 4.97
60.37 ± 10.53
2.91 ± 3.32
66.84 ± 11.79
12.51 ± 5.85
62.96 ± 13.69
4.61 ± 2.52
88.53 ± 2.55
60.10 ± 7.62
Heavy
SP-MIA
52.21 ± 3.83
12.20 ± 15.48
53.20 ± 4.85
2.80 ± 2.43
53.53 ± 5.94
12.37 ± 15.44
53.69 ± 5.68
4.26 ± 1.77
96.86 ± 2.60
88.60 ± 5.51
Table 1: Results for vision models under the subset-aware side-channel attack setting. Results are averaged over 9 coreset selection methods. Intensity denotes the selection ratio r (Light: r=0.2 , Medium: r=0.4 , Heavy: r=0.6 , Extensive: r=0.8 ). Best results per row are in bold.
Figure 3: The MIA performance of different attack surface on vision models under black-box setting.
Figure 4: Ablation studies on (left) the window size and (right) the embedding model for MIA performance.
Figure 6
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Choice Leakage Attack ( CoLA ) across the data–model supply chain. CoLA augments conventional MIA by exploiting subset selection metadata leaked along the data–model supply chain. By identifying which samples are more likely to pass selection, it not only strengthens membership inference but also enables adversaries to craft tailored threats.
Vision-Language-Action (VLA) models enable end-to-end robot control and have garnered widespread attention. However, the memorization of training data inherent to VLA, coupled with the high cost of robotic data acquisition, raises serious concerns regarding data privacy leakage and intellectual property infringement. Membership inference attacks (MIAs) aim to determine whether a given sample belongs to the training set. While representing a significant privacy threat, this attack remains underexplored in the context of VLA models. To bridge this gap, we propose VLALeaks, which is based on attention discrepancies in VLA models. We reveal, for the first time, the privacy vulnerabilities of VLA models. Specifically, it comprises a two-stage process: (1) membership feature extraction, and (2) attack model construction. Experimental results across multiple VLA benchmarks demonstrate that VLALeaks readily reveals membership information and achieves optimal attack AUC and TPR@1%FPR, highlighting the privacy vulnerabilities in current VLA model deployments. Our work is the first systematic study of MIAs on VLA models, aiming to provide insights for secure and trustworthy VLA models.
Xukun Luan, Jinyan Liu, Xuesong Li +4
School of Computer Science and Technology, Beijing Institute of Technology · School of Computer Science and Engineering, Northeastern University · PRADA Lab, King Abdullah University of Science and Technology
Recent work in the privacy literature shows that sample-targeted membership inference attacks (MIAs) significantly outperform untargeted approaches by a wide margin. Motivated by this observation, we address the following question: can the privacy vulnerability of individual training points be assessed without training shadow models? We show that per-sample exposure to MIA is governed not only by a point's loss, but also by a data-dependent geometric measure. In the linear setting, we derive a closed-form decomposition of individual black-box MIA vulnerability into a population leverage score and a residual loss term, making explicit how sample-dependent geometry translates into privacy exposure. Since the final layer of most modern architectures is linear, we extend this framework to deep networks and propose a surrogate score operating on last-layer representations that requires only a single trained model and no shadow models. Empirical evaluations across diverse datasets and architectures show that our score outperforms loss and gradient-norm baselines at identifying the highest-risk points under state-of-the-art attacks, providing a computationally efficient and theoretically grounded tool for per-sample privacy risk assessment.
Large Language Models are increasingly trained on proprietary or sensitive data, from private healthcare and financial records to user conversations containing secrets. Ensuring the privacy of such data against extraction attacks has become a central concern. In this paper, we ask whether an attacker who can poison a portion of the training data can facilitate the leakage of a separate target record they have no access to. We answer in the affirmative and show that such leakage can be induced by a poisoning mechanism that reshapes the model's local loss landscape around the target completion. Our key insight is that poisoning to create a sharp loss minimum at the target, surrounded by elevated loss on nearby alternatives, forces the model to memorize the target as the unique low-loss solution in its neighborhood. The attack requires no architectural changes, and generalizes across centralized and federated learning settings. We demonstrate that the attack amplifies privacy leakage across language (up to 100% successful extraction), and vision-language models (up 90% successful extraction). We show that the attack is thwarted when the model is trained to be differentially private. However, we introduce a new attack that directly probes the loss landscape bypassing even differential privacy defenses.
Md Abdullah Al Mamun, Ngoc Phu Doan, Pedram Zaree +2
UC Riverside · Riverside, California, USA · Queen’s University Belfast +2