From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models

Ruiqi Zhang; Lingxiang Wang; Hainan Zhang; Zhiming Zheng; Yanyan Lan

cs.CLMar 5, 2026

From Unfamiliar to Familiar: Detecting Pre-training Data via Gradient Deviations in Large Language Models

Authors: Ruiqi Zhang, Lingxiang Wang, Hainan Zhang, Zhiming Zheng, Yanyan Lan

Organizations: Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University

Abstract

Pre-training data detection for LLMs is essential for addressing copyright concerns and mitigating benchmark contamination. Existing methods mainly focus on the likelihood-based statistical features or heuristic signals before and after fine-tuning, but the former are susceptible to word frequency bias in corpora, and the latter strongly depend on the similarity of fine-tuning data. From an optimization perspective, we observe that during training, samples transition from unfamiliar to familiar in a manner reflected by systematic differences in gradient behavior. Familiar samples exhibit smaller update magnitudes, distinct update locations in model components, and more sharply activated neurons. Based on this insight, we propose GDS, a method that identifies pre-training data by probing Gradient Deviation Scores of target samples. Specifically, we first represent each sample using gradient profiles that capture the magnitude, location, and concentration of parameter updates across FFN and Attention modules, revealing consistent distinctions between member and non-member data. These features are then fed into a lightweight classifier to perform binary membership inference. Experiments on five public datasets show that GDS achieves state-of-the-art performance with significantly improved cross-dataset transferability over strong baselines. Further interpretability analyses reveal differences in gradient distributions, and the semi-supervised results offer a practical way to detect pre-training data.

Figures & tables

Figure 1: Heat maps of gradient matrices from unfamiliar model(a, c) and familiar model(b, d). Panels a and b display the update magnitude of the FFN module in layer 30, while panels c and d illustrate the update positions, with white for sparse and blue for core updates.

Panel of Figure 2 — Figure 2: Dynamic curves of four parameter update characteristics defined in the Motivation section. $i\to j$ denotes the epoch interval. Ordinate shows values of predefined characteristics, with orange and blue representing Attention and FFN modules.

Figure 3: Overview of Gradient Deviation Scores method.

	WikiMIA					ArXivTection
Method	2.7B	6B	6.7B	6.9B	7B	2.7B	6B	6.7B	6.9B	7B
PPL	0.61/0.13	0.64/0.13	0.60/0.12	0.63/0.13	0.69/0.14	0.62/0.13	0.64/0.13	0.60/0.12	0.63/0.13	0.70/0.14
ZLib	0.58/0.10	0.60/0.11	0.58/0.10	0.59/0.10	0.71/0.22	0.58/0.10	0.60/0.11	0.58/0.09	0.59/0.11	0.71/0.22
Min-k	0.65/0.17	0.67/0.19	0.63/0.15	0.67/0.19	0.73/0.18	0.65/0.17	0.68/0.19	0.63/0.15	0.67/0.19	0.72/0.18
Min-k++	0.67/0.15	0.69/0.19	0.65/0.11	0.70/0.18	0.82/0.22	0.67/0.15	0.69/0.19	0.65/0.10	0.70/0.17	0.83/0.22
FSD	0.92/0.77	0.95/0.78	0.90/0.63	0.90/ 0.66	0.92/0.41	0.91/0.65	0.96/0.79	0.89/0.63	0.95/0.66	0.94/0.81

Table 1: Comparison(AUROC/TPR@5%FPR) on 4 Datasets across 5 target models. Target models are denoted as 2.7B/6B/6.7B/6.9B/7B, corresponding to Neo-2.7B/GPT-J-6b/OPT-6.7b/Pythia-6.9b/LLaMA-7B respectively.

	Wikipedia					Github					Pile CC					PubMed Central
Method	2.7B	6B	6.7B	6.9B	7B	2.7B	6B	6.7B	6.9B	7B	2.7B	6B	6.7B	6.9B	7B	2.7B	6B	6.7B	6.9B	7B
PPL	0.56	0.57	0.56	0.56	0.56	0.78	0.79	0.78	0.90	0.83	0.56	0.57	0.56	0.56	0.52	0.78	0.79	0.78	0.78	0.72
ZLib	0.62	0.65	0.59	0.65	0.58	0.90	0.91	0.79	0.91	0.86	0.55	0.55	0.55	0.55	0.52	0.78	0.78	0.73	0.77	0.71
Min-k	0.66	0.68	0.65	0.68	0.63	0.88	0.89	0.76	0.90	0.83	0.56	0.57	0.56	0.56	0.52	0.81	0.80	0.74	0.79	0.72
Min-k++	0.65	0.68	0.60	0.69	0.56	0.84	0.86	0.56	0.86	0.73	0.54	0.55	0.54	0.56	0.51	0.71	0.72	0.60	0.70	0.59
FSD	0.61	0.67	0.60	0.65	0.60	0.77	0.80	0.62	0.77	0.72	0.55	0.55	0.54	0.55	0.52	0.71	0.79	0.63	0.56	0.63

Table 2: Performance Comparison on Mimir datasets with 7 subsets across 5 target models using AUROC scores.

Dataset	-Magnitude	-Concentrate	-Position
WikiMIA	0.94 (↓0.02)	0.94 (↓0.02)	0.93 (↓0.03)
Arxiv.	0.96 (↓0.01)	0.95 (↓0.02)	0.95 (↓0.02)

Table 3: Ablation results using AUROC on WikiMIA and ArxivTection. - denotes the features removal.

Dataset	ATT-Only	FFN-Only	Origin
WikiMIA	0.94 / 0.68	0.93 / 0.64	0.96 / 0.84
Arxiv.	0.95 / 0.76	0.93 / 0.72	0.97 / 0.85

Table 4: Ablation results using AUROC and TPR@5%FPR. ATT-Only refers to performance validation using only attention module.

	MinK++	FSD	GDS
LLaMA-7B	0.82/0.22	0.79/0.134	0.86/0.40
Pythia-6.9B	0.70/0.18	0.76/0.18	0.80/0.34
GPT-J-6B	0.69/0.19	0.75/0.14	0.79/0.26
OPT-6.7B	0.65/0.11	0.70/0.11	0.81/0.37
GPT-Neo-2.7B	0.67/0.15	0.76/0.17	0.78/0.26

Table 5: Semi-supervised performance (AUROC / TPR@5%FPR) on WikiMIA.

Panel of Figure 4 — Figure 4: Distribution differences of eight gradient features between member(red) and non-member(blue) samples. Dashed lines indicate distribution means. The x-axis shows feature values, and the y-axis shows probability density.

Figure 5: Distribution Difference Statistics of Discriminative Features. The y-axis shows the statistical count.

Method	Wiki (arXiv)	arXiv (Wiki)	Mix
FSD	0.52	0.58	0.92
Ours	0.66	0.68	0.95

Table 6: Cross-Dataset Generalization Performance using AUROC. Wiki(arXiv) indicates training on ArxivTection with test on WikiMIA, while Mix indicates mixed datasets.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Feature Dynamic Changes During LoRA Fine-Tuning (Learning Rate $1e-5$ ). Top row: Features in ATT/MLP modules; Bottom row: Features in low/middle/high layers.

Panel of Figure 7 — Figure 7: Feature Dynamic Changes During LoRA Fine-Tuning (Learning Rate $3e-5$ ).

Panel of Figure 8 — Figure 8: Feature Dynamic Changes During LoRA Fine-Tuning (Learning Rate $5e-5$ ).

Dataset	Sub-feature (Ablation Variant)								Full
Dataset	-Abs_Mean	-Std	-10p_Ratio	-Sparsity	-Row_Ecc	-Col_Ecc	-Row_Max	-Row_Std	Full
WikiMIA	0.95/0.71	0.95/0.72	0.95/0.70	0.94/0.67	0.95/0.70	0.93/0.62	0.95/0.73	0.95/0.72	0.96 / 0.84
ArxivTection	0.96/0.82	0.96/0.81	0.96/0.83	0.96/0.82	0.96/0.83	0.95/0.78	0.96/0.83	0.96/0.83	0.97/0.85

Appendix

Table 7: Ablation Experiment Results of Sub-features (AUC Scores).

Dataset	Sub-module (Ablation Variant)							Full
Dataset	-Q-Proj	-K-Proj	-V-Proj	-Out-Proj	-Gate-Proj	-Up-Proj	-Down-Proj	Full
WikiMIA	0.95/0.72	0.95/0.71	0.94/0.69	0.94/0.67	0.95/0.69	0.94/0.65	0.94/0.61	0.96/0.84
ArxivTection	0.96/0.80	0.96/0.82	0.95/0.79	0.96/0.80	0.96/0.82	0.95/0.78	0.96/0.82	0.97/0.85

Appendix

Table 8: Ablation Experiment Results of Sub-modules (AUC / TPR@5% FPR Scores).

Model	WikiMIA		ArxivTection
Model	GDS	mink++	GDS	mink++
llama	0.90/0.52	0.82/0.22	0.91/0.65	0.83/0.22
pythia	0.81/0.33	0.70/0.18	0.88/0.56	0.70/0.17
opt	0.86/0.37	0.65/0.11	0.88/0.47	0.65/0.10
gpt-j	0.84/0.37	0.69/0.19	0.91/0.60	0.69/0.19
gpt-neo	0.81/0.35	0.67/0.15	0.92/0.67	0.67/0.15

Appendix

Table 9: Detection performance under low-data scenario (10% training data).

LoRA rank $r$	AUC	TPR@5%FPR
16	0.96	0.84
8	0.96	0.82
4	0.95	0.77
Seed	AUC	TPR@5%FPR
99	0.9658	0.8421
88	0.9674	0.8408

Appendix

Table 10: Sensitivity analysis on WikiMIA (LLaMA-7B) using AUROC and TPR@5%FPR.

Method	Origin	Deletion
PPL	0.69	0.62
Min-k	0.73	0.63
Min-k++	0.82	0.59
FSD	0.92	0.76
Ours	0.96	0.84

Appendix

Table 12: Comparison of Temporal Feature Perturbation via AUROC. Deletion denotes the removal of all year timestamps.

Membership Inference Attacks

Membership inference attacks test whether records or entities were used to train a machine-learning model, revealing potential privacy leakage.

See all 116 papers

Calibrating Post-Training Feature Shifts for LLM Data Contamination Detection
Aug 11, 2026Zhen Yang, Mengqi Wang, Gengda Zhao +3Membership Inference Attacks Data Contamination in Language Models
The University of New South Wales
MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models
Jun 6, 2026Kaixin Lan, Mu You, Tao Fang +3Membership Inference Attacks LLM Auditing
Natural Language Processing & Portuguese-Chinese Machine Translation (NLP2CT) Laboratory, University of Macau, Macau SAR, China · Institute of International Language Services Studies, Macau Millennium College, Macau SAR, China · Innovation Research & Development, BoardWare Information System Limited
Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradients
Jun 23, 2026Zhihao Zhu, Hongyi Tang, Yi Yang +1LLM Auditing
Department of Information Systems, Business Statistics and Operations Management (ISOM), Hong Kong University of Science and Technology, Hong Kong, China. · Department of IT, Analytics, and Operations, University of Notre Dame, Notre Dame, Indiana, USA.

Aug 11, 2026J/K move · Enter open · S save

Calibrating Post-Training Feature Shifts for LLM Data Contamination Detection

Zhen Yang, Mengqi Wang, Gengda Zhao, Mo Zhou+2

The University of New South Wales

Membership Inference Attacks Data Contamination in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are trained on massive and largely undisclosed corpora that may contain copyrighted or privacy-sensitive content. Data contamination detection (DCD) therefore aims to determine whether a given text is a member of the pre-training corpus of a target LLM. Recent state-of-the-art DCD methods follow a feature-based paradigm that derives membership features from the input text and the corresponding model output. However, most modern LLMs undergo post-training, such as instruction tuning, preference optimization, and reasoning-oriented training, which can alter model outputs and shift the corresponding membership features, thereby reducing the separability between members and non-members. To address this problem, we propose CalibDCD, a broadly applicable calibration framework for feature-based DCD methods, comprising (1) Multi-View Shift Detection, which identifies recurring feature shifts associated with post-training, and (2) Bounded Feature Correction, which selectively mitigates their influence on membership prediction. Specifically, Multi-View Shift Detection evaluates controlled prompt variants on known non-member texts and consolidates the most informative views to identify recurring feature shifts. Bounded Feature Correction selectively adjusts feature components aligned with the detected shifts and controls the correction extent to preserve useful detection information. Experiments show that CalibDCD consistently improves existing feature-based detectors, with gains of up to 7.0% in AUC and 15.0% in TPR@5%FPR.

Abstract

Membership Inference Attacks

Explore similar work

Calibrating Post-Training Feature Shifts for LLM Data Contamination Detection

MC-PDD: Masked Corpus-Level Pretraining Data Detection for Black-Box Large Language Models

Revealing Training Data Exposure in Vision Language Large Models via Parameter Gradients