inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

Gonzalo Mancera; Julian Fierrez; Aythami Morales; Ruben Tolosana; Daniel DeAlcala; Francisco Jurado

cs.LGOct 13, 2026

inLoRA-MINT: Internal Representations for Auditing Training Data in Domain-adapted LLMs

Authors: Gonzalo Mancera, Julian Fierrez, Aythami Morales, Ruben Tolosana, Daniel DeAlcala, Francisco Jurado

Organizations: BiometricsAI, Universidad Autonoma de Madrid, Spain · Mathematics Dept., Universidad de Las Palmas de Gran Canaria, Spain · Computer Engineering Dept., Universidad Autonoma de Madrid, Spain

Abstract

This work investigates the application of the Membership Inference Test (MINT) to large language models (LLMs) fine-tuned with LoRA, aiming to determine whether specific samples were used during training and address concerns about data privacy, leakage, and transparency. Building on our previous LoRA-MINT framework, which relies exclusively on perplexity under a strict black-box setting, we propose an approach based on intermediate model representations, including activations and parameter-dependent signals. These representations enable a MINT classifier to distinguish training from non-training samples without requiring full access to model parameters. We evaluated the method on four LLMs (Qwen, Phi-4, DeepSeek-V3, and LLaMA 3.2-3B) fine-tuned on three datasets covering medical, mathematical, and academic reasoning tasks. Results show consistent performance across architectures and domains, achieving scores between 95% and 98%, with robustness to variations in model configurations and data distributions. The proposed approach enables efficient auditing of sensitive training data and supports data provenance and compliance with emerging privacy and AI governance regulations. Although our experiments focus on LoRA-based fine-tuning, the methodology can be extended to other LLM adaptation techniques and, more broadly, to other AI models.

Figures & tables

Fig. 1: MINT discerns whether given data ( d ) were utilized in the training process of an AI Model ( M ) trained with a specific database (

\mathcal{D}

Fig. 2: The MINT model

T(q(d)\mid\boldsymbol{\theta})

is trained to determine whether a given data instance

d

was part of the training data

\mathcal{D}

used to develop a language model

M_{\text{LoRA}}

. The inputs to the MINT model are represented by

q(d)

, which consists of features extracted from

M_{\text{LoRA}}

, including activation-based statistics, gradient-derived sensitivity measures, and output-level features computed with respect to the sample

d

DATASET	PROPOSED METHOD	Qwen3	Phi-4	DeepSeek-V3	LLaMA 3.2-3b
Medical-o1-reasoning-SFT	Loss [ 36 ] (Relevant literature 2022)	0.7508	0.9390	0.8840	0.8000
	MinK [ 37 ] (Relevant literature 2024)	0.7690	0.9440	0.8840	0.7770
	MinK++ [ 38 ] (Relevant literature 2025)	0.7780	0.9590	0.8625	0.7812
	Antepenultimate Layer (L1)	0.8306	0.9800	0.9697	0.8811
	Penultimate Layer (L2)	0.9179	0.9788	0.9840	0.9202
	Last Layer (L3)	0.9272	0.9811	0.9846	0.9210

TABLE I: Performance comparison (AUC) of different MINT-based variants across three datasets and 4 LoRA fine-tuned models. Layer-based configurations are reported as Antepenultimate (L1), Penultimate (L2), and Last Layer (L3).

Target Modules		AUC
q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj		0.6178
q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj		0.6978
q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj		0.7945
q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj		0.9019
q_proj , k_proj , v_proj , o_proj , gate_proj , up_proj , down_proj		0.9819

TABLE II: Ablation study of LoRA target modules using the Phi-4 model evaluated on the Medical-o1-SF dataset. All experiments are conducted using the configuration that aggregates the representations from the last three transformer layers together with the MaxGradProb scoring criterion. Active LoRA injection points are shown in bold, while inactive modules appear in gray. Each configuration isolates a specific subset of attention or MLP pathways to quantify their contribution to detection performance.

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 644 papers

Auditing Training Data in Domain-adapted LLMs: LoRA-MINT
Jun 5, 2026Gonzalo Mancera, Daniel DeAlcala, Aythami Morales +3Membership Inference Attacks LLM Auditing
BiometricsAI, Universidad Autonoma de Madrid, Spain · Department of Mathematics, Universidad de Las Palmas de Gran Canaria, Spain · Computer Engineering Department, Universidad Autonoma de Madrid, Spain
Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2
Jun 5, 2026Daniel DeAlcala, Gonzalo Mancera, Julian Fierrez +3Membership Inference Attacks Privacy Leakage in Language Models
BiometricsAI, Universidad Aut´onoma de Madrid, Spain
Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints
Aug 3, 2026Zirui Huang, Yunlong Mao, Wei Tong +3Language Model Fingerprinting Training Data Attribution
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing, Jiangsu, China

Jun 5, 2026J/K move · Enter open · S save

Auditing Training Data in Domain-adapted LLMs: LoRA-MINT

Gonzalo Mancera, Daniel DeAlcala, Aythami Morales, Julian Fierrez+2

BiometricsAI, Universidad Autonoma de Madrid, Spain · Department of Mathematics, Universidad de Las Palmas de Gran Canaria, Spain · Computer Engineering Department, Universidad Autonoma de Madrid, Spain

Membership Inference Attacks LLM Auditing Privacy Leakage in Language Models

Open paper AskPDF arXiv

We present LoRA-MINT, a new methodology for Membership Inference Test (MINT) applied to recent Large Language Models (LLMs) fine-tuned for specific Natural Language Processing (NLP) tasks through Low-Rank Adaptation (LoRA). The primary goal is to assess whether individual samples were part of the training data of these adapted models, providing a useful auditing tool for the management of intellectual property and sensitive data. Our analysis explores the relationship between model perplexity and membership status, providing a systematic framework for estimating data exposure in fine-tuned LLMs. We conducted experiments on four models and three benchmark datasets, obtaining precision values in determining if given data were used for training ranging from 0.77 to 0.92, which outperform state-of-the-art baselines and demonstrate the robustness and generality of the proposed method. In general, our findings underscore the potential of LoRA-MINT as an effective and scalable framework for auditing LLMs, improving transparency, and fostering the ethical and responsible deployment of AI and NLP technologies. For the sake of concreteness and current relevance, our discussion and experiments are centered on LoRAadjusted LLMs, but note that most of the presented methodology is easily applicable for auditing training data given any other technique for adapting LLMs or, more generally, any other domain-adapted AI models.

Abstract

LLM Auditing

Explore similar work

Auditing Training Data in Domain-adapted LLMs: LoRA-MINT

Is My Vision-Language Data in Your AI? Membership Inference Test (MINT) Demo 2

Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints