OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation

Qi Li; Xinchao Wang

cs.LGOct 22, 2026

OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation

Authors: Qi Li, Xinchao Wang

Organizations: National University of Singapore

Abstract

On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures restart-based output-basin stability as an observable surrogate for the hidden correction process. Across diverse models and distillation schemes, all baseline attacks remain essentially at random guessing even under gray-box access, whereas OP-Print is broadly validated to be effective in both gray-box and black-box settings.

Figures & tables

Panel of Figure 1 — Figure 1: Gray-box MIA signals from rollout-level dynamics and attack behavior (AUC) of Gray-box score.

Panel of Figure 2 — Figure 2: (a). Divergence varies only mildly across token positions, indicating that position alone is not the main source of the signal. (b). Divergence increases with student uncertainty, showing that correction concentrates on difficult reasoning tokens. (c). The reliability map reveals localized rather than uniform divergence, confirming that the supervision imprint is sparse and structured.

Figure 3: An illustration of the black-box attack process of OP-Print. Given multiple initial rollouts, OP-Print truncates each rollout at different prefix ratios, restarts generation from the partial prefix, and scores membership by measuring the similarity between restarted outputs. Members tend to produce more stable and similar restart trajectories, while non-members exhibit lower restart consistency.

Method	Qwen3-1.7B			Qwen3-4B			Qwen3-8B
Method	AUC	Acc.	TPR@5%FPR	AUC	Acc.	TPR@5%FPR	AUC	Acc.	TPR@5%FPR
Gray-box setting
Min-K% Prob	0.512	0.518	0.052	0.497	0.505	0.046	0.529	0.511	0.055
ModRényi ( $\alpha=0.5$ )	0.484	0.506	0.044	0.508	0.508	0.051	0.513	0.514	0.056
ModRényi ( $\alpha=2$ )	0.516	0.514	0.057	0.507	0.512	0.047	0.506	0.502	0.050
Rényi ( $\alpha=0.5$ )	0.503	0.509	0.049	0.517	0.511	0.054	0.495	0.503	0.045

Table 1: Gray-box and black-box results with the JSD objective. Colors indicate the top-2 and worst-2 results in the gray-box setting, and the top-1 and worst-1 results in the black-box setting.

Panel of Figure 4 — Figure 4: Representative gray-box and black-box OP-Print signals. (a)(b) In the gray-box setting, members show lower chunk-level KL gaps and higher scores than non-members. (c)(d) In the black-box setting, members exhibit more stable outputs under prefix restarts, whereas non-members are less stable.

Panel of Figure 5 — Figure 5: Ablation studies of OP-Print on Qwen3-4B. (a) Gray-box performance across rollout numbers and chunk sizes. (b) Black-box performance with varying numbers of prefix ratios. (c) Restart cost-performance trade-off, where bubble size denotes TPR@5%FPR.

Method	Qwen3-1.7B			Qwen3-4B			Qwen3-8B
Method	AUC	Acc.	TPR@5%FPR	AUC	Acc.	TPR@5%FPR	AUC	Acc.	TPR@5%FPR
Gray-box setting
Min-K% Prob	0.509	0.510	0.051	0.491	0.502	0.045	0.522	0.503	0.054
ModRényi ( $\alpha=0.5$ )	0.478	0.504	0.042	0.503	0.500	0.052	0.506	0.507	0.055
ModRényi ( $\alpha=2$ )	0.507	0.505	0.055	0.496	0.508	0.046	0.500	0.501	0.049
Rényi ( $\alpha=0.5$ )	0.495	0.502	0.050	0.510	0.504	0.056	0.484	0.500	0.043

Table 2: Gray-box and black-box results with tinker-style objective. Colors indicate the top-2 and worst-2 results in the gray-box setting, and the top-1 and worst-1 results in the black-box setting.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Configuration	Qwen3-1.7B	Qwen3-4B	Qwen3-8B
Number of epochs	3	3	3
Learning rate	$5\times 10^{-6}$	$5\times 10^{-6}$	$5\times 10^{-6}$
Precision	bfloat16	bfloat16	bfloat16
Attention implementation	SDPA	SDPA	SDPA
Gradient checkpointing	Yes	Yes	Yes
Max completion length	1024	1024	1024

Appendix

Table 3: Training configurations for target models. All models are trained under the same on-policy self-distillation pipeline; only model scale and a few runtime settings differ across runs.

Membership Inference Attacks

Membership inference attacks test whether records or entities were used to train a machine-learning model, revealing potential privacy leakage.

See all 115 papers

Leaky Students: Membership Inference against On-Policy Distillation
Sep 27, 2026Zhexi Lu, Mingzhi Zhu, Stacy Patterson +1Membership Inference Attacks Privacy Leakage in Language Models
Rensselaer Polytechnic Institute
Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation
Oct 8, 2026Yilong Yang, Wenzhuo Shang, Yule Liu +2LLM Auditing Membership Inference Attacks
Xidian University · Hong Kong University of Science and Technology (Guangzhou)
Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement
Aug 31, 2026Yi Ding, Ruqi ZhangRL for Language Models Language Model Distillation
Department of Computer Science, Purdue University, USA

Sep 27, 2026J/K move · Enter open · S save

Leaky Students: Membership Inference against On-Policy Distillation

Zhexi Lu, Mingzhi Zhu, Stacy Patterson, Lei Yu

Rensselaer Polytechnic Institute

Membership Inference Attacks Privacy Leakage in Language Models On-Policy Distillation Knowledge Distillation

Open paper AskPDF arXiv

On-policy distillation (OPD) trains a student to match a teacher's next-token distributions on student-generated trajectories. However, privileged information supplied to the teacher for OPD training may contain sensitive data. Whether the student leaks private information about the records supplied to the teacher during distillation remains poorly understood. To the best of our knowledge, we present the first systematic study of membership inference in this setting. We find that fresh student trajectories expose sparse membership signals that fixed reference-answer losses often miss. These signals are mixed with probability changes caused by training on other records. We introduce Leaky, which samples fresh trajectories from the target model and compares its token log-probabilities with the maximum across matched reference models trained without the candidate records. It applies Leaky ReLU to the resulting gaps, preserving positive gaps and downweighting negative gaps as an approximate correction for incidental positive gaps in non-members. Across fifteen targets spanning mathematics, medical question answering, and code generation, Leaky outperforms all evaluated baselines and achieves mean AUROC 0.875, compared with 0.614 for the strongest baseline on each target in the main evaluation. On the same sampled trajectories, the strongest baseline achieves mean AUROC 0.826. These results show that students trained through OPD can expose the membership of records used for teacher supervision, even when fixed reference-answer losses provide little evidence of membership.

Abstract

Membership Inference Attacks

Explore similar work

Leaky Students: Membership Inference against On-Policy Distillation

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement