Learning the Signature of Memorization in Autoregressive Language Models

David Ilić; Kostadin Cvejoski; David Stanojević; Evgeny Grigorenko

cs.CLApr 3, 2026

Learning the Signature of Memorization in Autoregressive Language Models

Authors: David Ilić, Kostadin Cvejoski, David Stanojević, Evgeny Grigorenko

Organizations: JetBrains Research

Abstract

All prior membership inference attacks for fine-tuned language models use hand-crafted heuristics (e.g., loss thresholding, Min-K%, reference calibration), each bounded by the designer's intuition. We introduce the first transferable learned attack, enabled by the observation that fine-tuning any model on any corpus yields unlimited labeled data, since membership is known by construction. This removes the shadow model bottleneck and brings membership inference into the deep learning era: learning what matters rather than designing it, with generalization through training diversity and scale. We discover that fine-tuning language models produces an invariant signature of memorization detectable across architectural families and data domains. We train a membership inference classifier exclusively on transformer-based models. It transfers zero-shot to Mamba (state-space), RWKV-4 (linear attention), and RecurrentGemma (gated recurrence), achieving 0.963, 0.972, and 0.936 AUC respectively. Each evaluation combines an architecture and dataset never seen during training, yet all three exceed performance on held-out transformers (0.908 AUC). These four families share no computational mechanisms, their only commonality is gradient descent on cross-entropy loss. Even simple likelihood-based methods exhibit strong transfer, confirming the signature exists independently of the detection method. Our method, Learned Transfer MIA (LT-MIA), captures this signal most effectively by reframing membership inference as sequence classification over per-token distributional statistics. On transformers, LT-MIA achieves 2.8 $\times$ higher TPR at 0.1% FPR than the strongest baseline. The method also transfers to code (0.865 AUC) despite training only on natural language texts. Code and trained classifier available at https://github.com/JetBrains-Research/learned-mia.

Figures & tables

Figure 1: LT-MIA transfers from transformers to non-transformer architectures, exceeding transformer performance on every text dataset. Blue bars show held-out transformer results (mean over 7 models), while the remaining bars show zero-shot transfer to Mamba-2.8B (state-space), RWKV-4-3B (linear attention), and RecurrentGemma-2B (gated recurrence). The classifier was trained exclusively on transformers and textual data.

Figure 2: LT-MIA pipeline. Given a text sample and black-box access to a fine-tuned target model and its pre-trained reference, we extract 154-dimensional feature vectors at each token position capturing how the two models’ predictions diverge. A lightweight transformer encoder processes this feature sequence to predict membership.

Method	Transformer	Mamba	RWKV	RecurrentGemma	Mean
Loss	0.622	0.867	0.786	0.860	0.784
Min-K%++	0.585	0.814	0.706	0.813	0.730
Zlib	0.611	0.837	0.761	0.836	0.761
RefLoss	0.760	0.892	0.915	0.800	0.842
EZ-MIA	0.887	0.931	0.971	0.915	0.926
LT-MIA	0.908	0.963	0.972	0.936	0.945

Table 1: AUC by method and architecture. All methods are evaluated on held-out combinations, only LT-MIA involves training. Means for transformers computed over 7 models

\times

4 datasets, non-transformer means computed over 4 datasets each.

TPR @ 1% FPR
Method	Transformer	Mamba	RWKV	RecurrentGemma	Mean
Loss	0.036	0.356	0.108	0.185	0.171
RefLoss	0.056	0.346	0.308	0.094	0.201
EZ-MIA	0.230	0.526	0.439	0.273	0.367
LT-MIA	0.375	0.569	0.593	0.399	0.484
TPR @ 0.1% FPR

Table 2: True positive rate at 1% and 0.1% false positive rate across 4 datasets.

Figure 3: Feature importance measured as AUC drop when each feature group is ablated. Comparison features (relating target to reference model outputs) dominate across all four architectural families. The consistent hierarchy (Comparison $>$ Target-only $>$ Reference-only) confirms the membership signal is relational: what matters is how fine-tuning changed the model, not the model’s behavior in isolation.

Figure 4: Effect of training diversity on generalization, with total samples fixed at 18,000. “Train AUC” is evaluated on held-out samples from training model-dataset combinations (in-distribution), while “Eval AUC” is evaluated on entirely different model-dataset combinations (out-of-distribution). All metrics use samples never seen during classifier training. Training on one combination yields near-perfect in-distribution AUC (0.998) but poor transfer (0.796). With 30 combinations, the gap shrinks from 20.2 points to 0.2 points.

Classifier	Train AUC	Eval AUC
Transformer (sequence)	0.914	0.925
MLP (flattened)	0.885	0.877
MLP (mean pooled)	0.885	0.875
Logistic Regression	0.838	0.839

Table 3: Classifier architecture comparison on held-out combinations. Sequence modeling contributes 5.0 AUC points over pooling.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
GPT-2	AG News	0.745	0.704	0.717	0.790	0.960	0.945
GPT-2	WikiText	0.745	0.696	0.713	0.814	0.971	0.980
GPT-2	XSum	0.768	0.719	0.760	0.956	0.994	0.991
GPT-2	Code	0.618	0.608	0.618	0.608	0.795	0.761
Pythia-2.8B	AG News	0.603	0.565	0.584	0.740	0.872	0.911
Pythia-2.8B	WikiText	0.585	0.558	0.583	0.645	0.867	0.944

Appendix

Table 4: Full results on held-out transformers: AUC.

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
GPT-2	AG News	0.018	0.026	0.014	0.020	0.408	0.274
GPT-2	WikiText	0.084	0.024	0.092	0.018	0.058	0.632
GPT-2	XSum	0.122	0.100	0.104	0.090	0.868	0.756
GPT-2	Code	0.024	0.010	0.010	0.010	0.030	0.054
Pythia-2.8B	AG News	0.014	0.020	0.012	0.042	0.132	0.257
Pythia-2.8B	WikiText	0.014	0.012	0.020	0.010	0.016	0.492

Appendix

Table 5: Full results on held-out transformers: TPR @ 1% FPR.

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
GPT-2	AG News	0.008	0.008	0.006	0.008	0.124	0.044
GPT-2	WikiText	0.008	0.002	0.006	0.000	0.002	0.273
GPT-2	XSum	0.032	0.048	0.016	0.016	0.426	0.340
GPT-2	Code	0.000	0.002	0.000	0.000	0.000	0.006
Pythia-2.8B	AG News	0.006	0.008	0.006	0.000	0.034	0.064
Pythia-2.8B	WikiText	0.002	0.000	0.000	0.004	0.000	0.291

Appendix

Table 6: Full results on held-out transformers: TPR @ 0.1% FPR.

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
Mamba-2.8B	AG News	0.949	0.888	0.918	0.975	0.983	0.969
Mamba-2.8B	WikiText	0.966	0.886	0.897	0.947	0.979	0.995
Mamba-2.8B	XSum	0.988	0.949	0.977	0.999	0.999	0.999
Mamba-2.8B	Code	0.565	0.534	0.555	0.646	0.763	0.887
RWKV-3B	AG News	0.783	0.703	0.748	0.931	0.978	0.976
RWKV-3B	WikiText	0.792	0.704	0.753	0.865	0.965	0.986

Appendix

Table 7: Full results on non-transformer architectures: AUC.

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
Mamba-2.8B	AG News	0.150	0.150	0.114	0.316	0.718	0.206
Mamba-2.8B	WikiText	0.462	0.154	0.544	0.040	0.330	0.900
Mamba-2.8B	XSum	0.774	0.518	0.800	0.998	0.986	0.996
Mamba-2.8B	Code	0.036	0.028	0.022	0.028	0.070	0.174
RWKV-3B	AG News	0.092	0.060	0.026	0.084	0.402	0.430
RWKV-3B	WikiText	0.066	0.048	0.090	0.022	0.050	0.624

Appendix

Table 8: Full results on non-transformer architectures: TPR @ 1% FPR.

Model	Dataset	Loss	Min-K%++	Zlib	RefLoss	EZ-MIA	LT-MIA
Mamba-2.8B	AG News	0.044	0.052	0.012	0.000	0.018	0.018
Mamba-2.8B	WikiText	0.070	0.020	0.038	0.002	0.004	0.300
Mamba-2.8B	XSum	0.480	0.242	0.460	0.592	0.888	0.582
Mamba-2.8B	Code	0.010	0.018	0.002	0.002	0.048	0.058
RWKV-3B	AG News	0.010	0.010	0.008	0.002	0.098	0.060
RWKV-3B	WikiText	0.024	0.012	0.010	0.000	0.000	0.231

Appendix

Table 9: Full results on non-transformer architectures: TPR @ 0.1% FPR.

Training Models (10)	DistilGPT-2 [ Sanh et al., 2019 ] , GPT-2-XL [ Radford et al., 2019 ] , Pythia-1.4B [ Biderman et al., 2023 ] , Cerebras-GPT-2.7B [ Dey et al., 2023 ] , GPT-J-6B [ Wang and Komatsuzaki, 2021 ] , Gemma-2B [ Gemma Team, 2024 ] , Qwen2-1.5B [ Bai et al., 2023 ] , MPT-7B [ MosaicML NLP Team, 2023 ] , Falcon-RW-1B, Falcon-7B [ Penedo et al., 2023 ]
Training Datasets (3)	News Category Dataset [ Misra, 2022 ] , Wikipedia [ Wikimedia Foundation, 2024 ] , CNN/DailyMail [ Hermann et al., 2015 ]
Eval Transformers (7)	GPT-2, Pythia-2.8B, Pythia-6.9B, OPT-2.7B, Llama-2-7B, Phi-2, Mistral-7B
Eval Non-Transformers (3)	Mamba-2.8B (state-space), RWKV-3B (linear attention), RecurrentGemma-2B (gated recurrence)
Eval Datasets (4)	AG News [ Zhang et al., 2015 ] , WikiText-103 [ Merity et al., 2016 ] , XSum [ Narayan et al., 2018 ] , Swallow-Code [ Fujii et al., 2025 ]

Appendix

Table 10: Training and evaluation split. Training uses 10 transformer models

\times

3 datasets = 30 combinations. Evaluation uses 7 held-out transformers + 3 non-transformers

\times

4 datasets = 40 combinations.

Classifier	Train AUC	Eval AUC	Train TPR@1%	Eval TPR@1%
Transformer (sequence)	0.914	0.925	0.601	0.459
MLP (flattened)	0.885	0.877	0.483	0.343
MLP (mean pooled)	0.885	0.875	0.506	0.319
Logistic Regression	0.838	0.839	0.351	0.237

Appendix

Table 11: Classifier architecture ablation. The transformer processes features as a sequence; MLP and logistic regression flatten to a fixed-length vector; mean-pooled MLP averages features across positions before classification.

Combinations	Samples/Combo	Train AUC	Eval AUC	Train TPR@1%	Eval TPR@1%
1	18,000	0.998	0.796	0.992	0.185
10	1,800	0.875	0.843	0.371	0.244
30	600	0.862	0.860	0.409	0.284

Appendix

Table 12: Effect of training diversity on generalization with fixed total samples.

Model	LT-MIA AUC	LT-MIA TPR@1%	EZ-MIA AUC	EZ-MIA TPR@1%
GPT-2	0.980	0.632	0.971	0.058
Pythia-2.8B	0.944	0.492	0.867	0.016
Pythia-6.9B	0.796	0.137	0.911	0.036
OPT-2.7B	0.862	0.262	0.852	0.000
Llama-2-7B	0.781	0.130	0.733	0.020
Phi-2	0.823	0.187	0.859	0.010

Appendix

Table 13: Calibration comparison: LT-MIA vs. EZ-MIA on WikiText (where calibration gap is largest).

Membership Inference Attacks

Membership inference attacks test whether records or entities were used to train a machine-learning model, revealing potential privacy leakage.

See all 111 papers

Jun 16, 2026

CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

Membership inference attacks (MIAs) are a canonical way to assess a machine learning model's privacy properties. Although several attempts have been made to evaluate MIAs on language models, the extant literature has suffered numerous difficulties in constructing clean evaluations to test new techniques. In particular, subtle distribution shifts between member and non-member sets can undermine the statistical validity of MIAs; recent work has underscored this by showing that "blind" methods with no access to the underlying model can perform far better than published methods on the same benchmarks. This paper constructs a benchmark for principled evaluation of MIAs against LLMs, by leveraging the insight that training data before and after a fixed point during training are drawn from the same distribution. Therefore, all open-source models with intermediate checkpoints and public training data can be converted into MIA testbeds. We apply our framework to a half-dozen published attacks on the Pythia and OLMo family of models, from 70M to 7B parameters. To facilitate further privacy research, we open-source a modular library for designing and implementing attacks in this setting: https://github.com/safr-ai-lab/pandora_llm.

Jeffrey G. Wang, Jason Wang, Marvin Li +1

Harvard University · Harvard Business School

LLM Evaluation Membership Inference Attacks LLM Auditing

Ask PDF

Oct 2, 2026

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

Membership inference on large language models (LLMs) aims to determine whether a given text sample was included in an LLM's training data, without access to its training corpus. Despite recent progress, existing benchmarks suffer from three limitations: limited coverage of training stages, insufficient distributional alignment between members and non-members, and lack of rigorous filtering of non-members against the training corpus. To address these limitations, we propose OLMo-Detect, a multi-stage, confounder-controlled benchmark built upon the fully open OLMo 2 pipeline. OLMo-Detect spans pre-training, mid-training, and post-training, explicitly aligns members and non-members on three key axes, and rigorously filters non-members via infini-gram. To assess robustness to distribution shifts, we further introduce OLMo-Detect (Shifted), a variant where members are misaligned with non-members. We evaluate 15 unsupervised and 3 supervised membership inference attacks (MIAs) across the OLMo 2 family, finding that: (i) overall performance is limited: the best unsupervised and supervised MIAs both reach an AUC of only 0.68, and supervised MIAs degrade under cross-domain evaluation; (ii) MIA performance peaks at mid-training and is lower at pre-training and post-training, a pattern driven by data type rather than a stage effect: curated math data is far more detectable than other types; (iii) overall scores improve from 1B to 13B but plateau at 32B; and (iv) no unsupervised MIA is robust to distribution shifts, with AUCs shifting by up to 0.42. Finally, we find that our findings on OLMo 2 generalize to OLMo 3 and non-OLMo models.

Tao Shi, Chaoyi Xiang, Qiongkai Xu +1

The University of Melbourne · Macquarie University

Membership Inference Attacks Privacy Leakage in Language Models LLM Auditing

Ask PDF

May 9, 2026

JUMP: Efficient Membership Inference on Fine-Tuned Diffusion Language Models

Membership inference attacks (MIAs) test whether a candidate example was used to train a language model. Existing attacks on fine-tuned discrete diffusion language models (dLLMs) often aggregate reconstruction signals across many mask configurations, requiring repeated model evaluations. We propose Joint Uncertainty Guided Mask Probing (JUMP), an efficient MIA that exploits the ability of dLLMs to predict masked tokens in parallel. Using the pre-fine-tuning checkpoint as a reference, JUMP selects low-confidence positions, masks them jointly, and aggregates clipped target-reference reconstruction gaps. This focuses the attack on positions that reveal stronger membership signals from fine-tuning. After mask selection, all selected tokens are evaluated with one scoring query per model. Across six MIMIR domains, JUMP improves mean ROC-AUC over a prior multi-mask attack from 0.819 to 0.902 on LLaDA and from 0.851 to 0.942 on Dream. Including mask selection, it requires only three forward passes per example, compared with 32 for the baseline. We further extend JUMP to the target-only setting by replacing target-reference scoring with relative token preference, which compares the observed token with alternative predictions at the same masked position. Target-Only JUMP achieves mean ROC-AUCs of 0.609 and 0.638 on LLaDA and Dream.

Yeachan Jun, Albert No

Department of Artificial Intelligence Yonsei University, Seoul, Korea

Membership Inference Attacks Privacy Leakage in Language Models Diffusion Language Models

Ask PDF

Abstract

Membership Inference Attacks

Explore similar work

CheckMIABench: Firm Foundations For Membership Inference Attacks on Language Models

OLMo-Detect: A Multi-Stage, Confounder-Controlled Benchmark for Membership Inference on Large Language Models

JUMP: Efficient Membership Inference on Fine-Tuned Diffusion Language Models