Leaky Students: Membership Inference against On-Policy Distillation

Zhexi Lu; Mingzhi Zhu; Stacy Patterson; Lei Yu

cs.LGSep 27, 2026

Leaky Students: Membership Inference against On-Policy Distillation

Authors: Zhexi Lu, Mingzhi Zhu, Stacy Patterson, Lei Yu

Organizations: Rensselaer Polytechnic Institute

Abstract

On-policy distillation (OPD) trains a student to match a teacher's next-token distributions on student-generated trajectories. However, privileged information supplied to the teacher for OPD training may contain sensitive data. Whether the student leaks private information about the records supplied to the teacher during distillation remains poorly understood. To the best of our knowledge, we present the first systematic study of membership inference in this setting. We find that fresh student trajectories expose sparse membership signals that fixed reference-answer losses often miss. These signals are mixed with probability changes caused by training on other records. We introduce Leaky, which samples fresh trajectories from the target model and compares its token log-probabilities with the maximum across matched reference models trained without the candidate records. It applies Leaky ReLU to the resulting gaps, preserving positive gaps and downweighting negative gaps as an approximate correction for incidental positive gaps in non-members. Across fifteen targets spanning mathematics, medical question answering, and code generation, Leaky outperforms all evaluated baselines and achieves mean AUROC 0.875, compared with 0.614 for the strongest baseline on each target in the main evaluation. On the same sampled trajectories, the strongest baseline achieves mean AUROC 0.826. These results show that students trained through OPD can expose the membership of records used for teacher supervision, even when fixed reference-answer losses provide little evidence of membership.

Figures & tables

Figure 1: Overview. (a) In OPD with privileged information, the student samples rollouts from the prompt $x$ alone, and the teacher, which also sees the reference answer $r$ , supplies the next-token distributions the student matches. (b) Leaky samples trajectories from the target conditioned on the candidate records, scores each token under the target and $K$ reference models trained without the candidate, and aggregates the resulting gaps with a Leaky ReLU into the membership score .

Figure 2: Membership signals in OPD. (a,b) Mean loss on fixed answers and sampled continuations provides limited separation between members and non-members. (c) Leaky separates the two groups more clearly. (d) Positive token-level gaps account for only 7.08% of all tokens, and larger positive gaps generally have a higher member share.

	Mathematics					Medical QA					Code generation
Method	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B
AUROC
LOSS	.516	.513	.502	.502	.505	.553	.552	.542	.545	.530	.546	.532	.543	.550	.559
ZLIB	.528	.525	.497	.497	.503	.509	.511	.503	.505	.504	.616	.590	.612	.616	.622
Min-K%	.521	.518	.499	.500	.506	.540	.542	.532	.544	.518	.535	.512	.533	.539	.540
Min-K%++	.521	.522	.509	.509	.515	.492	.508	.511	.515	.522	.492	.454	.502	.526	.442

Table 1: Membership inference against OPD: AUROC and TPR @ 1% FPR, leading zeros omitted, best per column in bold. Leaky scores eight target-generated trajectories per candidate. Q3/Q3.5: Qwen3/Qwen3.5; G4: Gemma-4-E2B. Results at 5% and 10% FPR are in Appendix C .

	Mathematics		Medical QA		Code generation
Target model	AUROC	TPR@1% FPR	AUROC	TPR@1% FPR	AUROC	TPR@1% FPR
Qwen3-1.7B	.776	.273	.854	.389	.747	.213
Qwen3-4B	.796	.289	.691	.211	.687	.146
Qwen3.5-2B	.774	.191	.768	.342	.740	.238
Qwen3.5-4B	.812	.309	.737	.189	.788	.412
Gemma-4-E2B	.727	.152	.698	.189	.660	.084

Table 2: Black-box membership inference on the fifteen targets in Table 1 . We report AUROC and TPR at 1% FPR for the text-only prefix-query attack.

	Mathematics			Medical QA			Coding
Training	$\Delta$ Util.	AUROC	TPR	$\Delta$ Util.	AUROC	TPR	$\Delta$ Util.	AUROC	TPR
Full fine-tuning	$+5.00$	0.9158	0.418	$+3.98$	0.9665	0.740	$+0.57$	0.8467	0.434
LoRA	$+2.50$	0.6455	0.098	$+2.46$	0.7868	0.268	$+0.14$	0.6585	0.146
DP-LoRA ( $\varepsilon\approx 8$ )	$+1.67$	0.4903	0.014	$+0.04$	0.5044	0.023	$+0.07$	0.5097	0.027

Table 3: Defenses on Qwen3-1.7B (teacher Qwen3-8B): change in Avg@8 over the base (percentage points), AUROC, and TPR at 1% FPR of Leaky with eight matched reference models. Full fine-tuning is the Table 1 cell; DP-LoRA is at nominal

\varepsilon\approx 8

. Configurations are in Appendix D .

Panel of Figure 3 — Figure 3: Ablations on Qwen3-1.7B. (a) AUROC and (b) TPR at 1% FPR against the number of reference models $K$ in the three domains; $K=8$ is Table 1 , and the other students are in Figures C.1 and C.2 . (c) Mathematics AUROC. (d) Mathematics AUROC across checkpoints for OPD with the Qwen3-8B teacher and for OPSD with the frozen initial student.

Panel of Table 4 — Table 4: Auxiliary data for reference-model training under the Table 1 protocol; the real-data and OpenThoughts-Math columns are the Table 1 cells. (a) Qwen3-1.7B targets, reference models trained on real or Qwen3.5-27B-synthesized records. (b) Mathematics targets, reference models trained on OpenThoughts-Math or DeepMath-103K.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure A.1: Gray-box AUROC against the rectifier slope $\alpha$ for each target, on target-generated (top) and teacher-generated (bottom) trajectories; the dashed line marks $\alpha=0.04$ .

Student	Mathematics teacher	Medical teacher	Coding teacher
Qwen3-1.7B	Qwen3-8B	Qwen3-8B	Qwen3-8B
Qwen3-4B	Qwen3-14B	Qwen3-14B	Qwen3-14B
Qwen3.5-2B	Qwen3.5-27B	Qwen3.5-9B	Qwen3.5-9B
Qwen3.5-4B	Qwen3.5-27B	Qwen3.5-27B	Qwen3.5-27B
Gemma-4-E2B	Gemma-4-26B-A4B	Gemma-4-26B-A4B	Gemma-4-26B-A4B

Appendix

Table 5: Student and teacher models used for OPD in each domain.

Student	Interpolation weight ( $\beta$ )	Learning rate	Training steps	Global batch size
Mathematics
Qwen3-1.7B	0	5e-6	126	32
Qwen3-4B	0	2e-6	126	32
Qwen3.5-2B	0.5	1e-6	126	32
Qwen3.5-4B	0.5	1e-6	126	32
Gemma-4-E2B	0	3e-6	126	32

Appendix

Table 6: OPD training configurations. Training steps indicate the total training budget.

Student	Training steps	Base	Trained	$\Delta$ (pp)
Mathematics: AIME 2024
Qwen3-1.7B	126	47.50	52.50	+5.00
Qwen3-4B	126	72.92	75.00	+2.08
Qwen3.5-2B	126	31.25	42.50	+11.25
Qwen3.5-4B	126	85.00	87.08	+2.08
Gemma-4-E2B	126	41.93	45.00	+3.07

Appendix

Table 7: Downstream utility (Avg@8, %) of the evaluated targets;

\Delta

is the change from the base in percentage points. Training steps indicate the total training budget.

Model family	Decoding	Output tokens
Mathematics: reasoning enabled; 30 AIME questions
Qwen3	$T=1.0$ , top- $p=0.95$ , no top- $k$	38,912
Qwen3.5	$T=1.0$ , top- $p=0.95$ , top- $k=20$ , presence penalty 1.5	81,920
Gemma-4	$T=1.0$ , top- $p=0.95$ , top- $k=64$	32,768
Medical: reasoning disabled; 1,273 MedQA-USMLE questions
All models	$T=0.7$ , top- $p=0.8$ , top- $k=20$ , presence penalty 1.5	32,768

Appendix

Table 8: Utility decoding per base/target pair (not used for attack scoring). Coding uses SGLang ( Zheng et al., 2024 ) for Qwen3-1.7B and Qwen3.5 and vLLM ( Kwon et al., 2023 ) otherwise.

	Mathematics					Medical QA					Code generation
Method	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B
TPR at 5% FPR
LOSS	.045	.035	.049	.070	.055	.039	.053	.047	.049	.041	.033	.043	.033	.035	.047
ZLIB	.043	.039	.059	.051	.045	.051	.049	.041	.051	.049	.049	.057	.055	.082	.082
Min-K%	.043	.039	.049	.070	.055	.043	.035	.043	.039	.029	.031	.037	.041	.045	.053
Min-K%++	.031	.037	.062	.047	.053	.047	.041	.059	.053	.061	.035	.037	.035	.061	.057

Appendix

Table 9: TPR at 5% and 10% FPR for the methods in Table 1 . Leading zeros omitted; Q3/Q3.5: Qwen3/Qwen3.5; G4: Gemma-4-E2B.

	Mathematics					Medical QA					Code generation
Method	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B	Q3 1.7B	Q3 4B	Q3.5 2B	Q3.5 4B	G4 E2B
AUROC
LOSS	.502	.477	.489	.506	.495	.536	.549	.568	.557	.550	.534	.536	.547	.551	.544
ZLIB	.504	.472	.490	.507	.487	.505	.530	.558	.546	.540	.531	.543	.552	.555	.553
Min-K%	.501	.473	.492	.510	.496	.531	.548	.572	.559	.544	.530	.536	.557	.556	.545
Min-K%++	.486	.472	.491	.492	.495	.491	.482	.564	.550	.499	.510	.503	.558	.566	.512

Appendix

Table 10: Baselines applied to the eight target-generated trajectories that Leaky scores, on the candidates of Table 1 : AUROC and TPR at 1%, 5% and 10% FPR, leading zeros omitted, best per column in bold. Q3/Q3.5: Qwen3/Qwen3.5; G4: Gemma-4-E2B.

Reference	Scoring rule	Mean AUROC	Mean TPR at 1% FPR
None	Target log-probability mean	0.529	0.009
None	Min-K% (20%)	0.530	0.009
None	Min-K%++ (20%)	0.511	0.013
Base model	Mean gap	0.520	0.014
Reference mean	Mean gap	0.801	0.282
Reference maximum	Mean gap	0.658	0.068

Appendix

Table 11: Comparison of scoring rules on the same target-generated trajectories. AUROC and TPR at 1% FPR are averaged over fifteen targets. All scores using trained reference models use the same eight models.

Panel of Figure 16 — Figure C.1: AUROC against the number of reference models $K$ for the four students not in Figure 3 (a). Each $K$ uses the first $K$ of the same eight reference models, so $K=8$ is Table 1 .

Panel of Figure 17 — Figure C.2: TPR at 1% FPR along the sweeps of Figure C.1 .

Training step	OPSD AUROC	OPSD TPR	OPD AUROC	OPD TPR
25	0.5935	0.090	0.7259	0.260
50	0.6565	0.090	0.8088	0.336
75	0.6869	0.146	0.8654	0.424
100	0.6987	0.150	0.8771	0.449

Appendix

Table 12: Membership inference under on-policy self-distillation and external-teacher OPD on mathematics Qwen3-1.7B. Leakage increases across the evaluated checkpoints under both training procedures. TPR is measured at 1% FPR.

Student	TPR at 5% FPR	TPR at 10% FPR
Mathematics
Qwen3-1.7B	0.393	0.533
Qwen3-4B	0.453	0.543
Qwen3.5-2B	0.344	0.486
Qwen3.5-4B	0.469	0.568
Gemma-4-E2B	0.283	0.396

Appendix

Table 13: Additional black-box results at 5% and 10% FPR for the fifteen targets in Table 2 .

Panel of Figure 20 — Figure C.3: Effect of the number of target-generated trajectories per candidate across all fifteen targets. Eight reference models and $\alpha=0.04$ are fixed throughout. The eight-trajectory results match Table 2 .

Domain	Training	$\Delta$ Util.	AUROC	TPR@1%	TPR@5%	TPR@10%
Mathematics	Full fine-tuning	$+5.00$	0.9158	0.418	0.662	0.758
	LoRA	$+2.50$	0.6455	0.098	0.180	0.277
	DP-LoRA	$+1.67$	0.4903	0.014	0.049	0.092
Medical QA	Full fine-tuning	$+3.98$	0.9665	0.740	0.865	0.928
	LoRA	$+2.46$	0.7868	0.268	0.396	0.502
	DP-LoRA	$+0.04$	0.5044	0.023	0.070	0.111

Appendix

Table 14: Defenses on Qwen3-1.7B: change in Avg@8 over the base (percentage points), AUROC, and TPR at 1%, 5%, and 10% FPR of Leaky (

K=8

\alpha=0.04

Domain	Attack configuration	LoRA	DP-LoRA
Mathematics	Leaky ( $K=8$ , $\alpha=0.04$ )	0.6455 / 0.098	0.4903 / 0.014
	ReLU ( $K=8$ , $\alpha=0$ )	0.5881 / 0.084	0.4874 / 0.016
	Leaky ( $K=4$ , $\alpha=0.04$ )	0.5866 / 0.076	0.4856 / 0.008
	Leaky ( $K=1$ , $\alpha=0.04$ )	0.5511 / 0.039	0.4903 / 0.008
Medical QA	Leaky ( $K=8$ , $\alpha=0.04$ )	0.7868 / 0.268	0.5044 / 0.023
	ReLU ( $K=8$ , $\alpha=0$ )	0.7689 / 0.240	0.5025 / 0.021

Appendix

Table 15: Attack performance under LoRA and DP-LoRA across four scoring configurations. Entries report AUROC / TPR at 1% FPR.

Membership Inference Attacks

Membership inference attacks test whether records or entities were used to train a machine-learning model, revealing potential privacy leakage.

See all 111 papers

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation
Oct 8, 2026Yilong Yang, Wenzhuo Shang, Yule Liu +2LLM Auditing Membership Inference Attacks
Xidian University · Hong Kong University of Science and Technology (Guangzhou)
Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers
Sep 16, 2026Yihao Ai, Weilong YanLanguage Model Distillation On-Policy Distillation
National University of Singapore
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
May 29, 2026Yanjiang Liu, Jie Lou, Xinyan Guan +7CoT Distillation Language Model Distillation
University of Chinese Academy of Sciences · 2Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China · 3Xiaohongshu

Oct 8, 2026J/K move · Enter open · S save

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

Yilong Yang, Wenzhuo Shang, Yule Liu, Jiale Teng+1

Xidian University · Hong Kong University of Science and Technology (Guangzhou)

LLM Auditing Membership Inference Attacks On-Policy Distillation

Open paper AskPDF arXiv

On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts are often private and costly, creating a need for prompt-level membership auditing. Existing methods mainly rely on likelihood-based confidence signals or student policy drift between checkpoints, but they do not capture the teacher-induced direction of the student update. In this paper, we propose Policy Alignment Membership Auditing (PAMA), a new auditing framework tailored for OPD. Our key observation is that a member prompt directly contributes to the teacher-guided policy update, while a non-member prompt only experiences indirect effects through cross-prompt generalization. Based on this directional trace, PAMA measures whether the student update moves toward reducing the teacher loss on a candidate prompt. Specifically, we introduce Teacher Alignment Gain (TAG) to estimate the teacher-aligned update direction from model outputs, and further combine it with student drift and uncertainty alignment signals for reliable membership auditing. We evaluate PAMA on six datasets and three teacher-student model families. On MATH, the primary evaluation benchmark, PAMA achieves AUC values of 0.791--0.941, improving AUC by 14.6--20.6% over state-of-the-art baselines.

Abstract

Membership Inference Attacks

Explore similar work

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

Trajectory Learnability for Offline On-Policy Distillation with Imperfect Teachers

Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation