Organizations: MIIT Key Laboratory of Pattern Analysis and Machine Intelligence College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing, China · School of Intelligence Science and Technology Nanjing University Suzhou, China · Department of Computer Science and Technology Institute for Artificial Intelligence Tsinghua University Beijing, China
Privacy-preserving continual learning (PPCL) must reduce the reproduction of sensitive content while retaining useful knowledge across sequential tasks. Formal privacy guarantees characterize randomized mechanisms, whereas operational output control concerns whether a trained model selectively reduces the likelihood of sensitive content in its outputs. In this work, we investigate the latter together with continual-learning utility under realistic task evolution. Retention and privacy correction operate at different granularities: task acquisition requires broad preservation of current- and old-task behavior, whereas privacy correction targets sparse annotated positions. Joint optimization leaves the current-task preservation target continually changing. We propose SPARK, a retention-correction decomposition that first freezes the learned post-task distribution and then applies selective correction around this stable reference. Self-Distillation Replay learns the current task while distilling behavior from previous tasks, and Post-Task Privacy Correction reduces annotated-PII likelihood while anchoring current- and old-task non-PII behavior to the resulting checkpoint. Extensive evaluations demonstrate that SPARK achieves effective selective PII suppression while preserving strong continual-learning utility and knowledge retention across diverse settings. Code and data will be released upon publication.
Figures & tables
Figure 1. Motivation for output-selective PPCL. Broad retention and sparse PII correction require different controls: embedding perturbation alone does not establish output selectivity; direct correction needs PII targets and a fixed preservation reference. This motivates retaining knowledge before sparse post-task probability correction. Three panels summarize the motivation. Panel (a) contrasts broad knowledge retention with sparse sensitive-probability correction across sequential tasks. Panel (b) shows embedding noise passing through Transformer layers: at the training scale, PII and non-PII output probabilities change similarly, while larger noise suppresses both. Panel (c) identifies the two requirements for selective output control: an explicit PII target and a fixed preservation reference for useful non-PII behavior.
Figure 2. Overview of SPARK. Phase 1 initializes from θk−1priv , learns the current task with response-only supervision, and reuses this frozen checkpoint as replay teacher to produce θktask . Phase 2 initializes and freezes θktask , corrects PII over the full valid sequence, and anchors current- and old-task non-PII behavior to the same checkpoint, producing θkpriv . A two-phase pipeline across a sequence of tasks. In Phase 1, the previous privacy-corrected checkpoint teaches a student on replay data while the student learns the current task. In Phase 2, token sensitivity separates PII from non-PII positions; PII positions receive direct correction and non-PII positions are anchored to the post-task checkpoint.
Method
FOMC
Yelp
AGNews
Amazon
MentILL
Yahoo
BWT ↑
Last ↑
Avg ↑
Standard continual-learning baselines
SeqFT
0.193
0.000
0.820
0.383
0.293
0.687
− 0.500
0.396
0.707
ER (50% replay)
0.920
0.653
0.920
0.960
0.737
0.700
+ 0.009
0.815
0.830
EWC ( λ=100 )
0.000
0.017
0.770
0.000
0.253
0.690
− 0.621
0.288
0.683
GEM
0.817
0.037
0.570
0.803
0.527
0.007
+ 0.000
0.460
0.548
O-LoRA
0.000
0.000
0.733
0.027
0.283
0.703
− 0.629
0.291
0.672
Table 1. Continual-learning performance across six tasks. Task columns report final post-stream accuracy. PeCL † denotes our controlled reproduction; its complete learning trajectory and BWT interpretation are reported in Appendix A. Blue shading marks SPARK.
Method
Last ↑
BWT ↑
PII NLL ↑
Low-sens. NLL ↓
Canary NLL (macro) ↑
Panel A: structural ablations
ER (no privacy)
0.815
+0.009
2.974
2.076
3.598
PeCL †
0.658
+0.093
2.538
1.643
4.135
SPARK (full)
0.799
−0.009
4.203
2.903
5.423
w/o PTPC
0.804
+0.001
2.739
1.881
3.620
w/o SD-Replay
0.236
−0.684
2.926
2.133
3.484
Table 2. Structural and internal PTPC ablations under the primary full-benchmark evaluation. All unmatched likelihood diagnostics use the same evaluation split and aggregation protocol as the Order 1 results in Appendix G. Panel A separates retention from correction; Panel B isolates correction components using the same full-model reference; comparisons are within panels.
Figure 3. PTPC selectivity ablations: (a) matched Δsel with 95% source-cluster bootstrap CIs; (b) full-vocabulary PII Top-1; (c) suppression versus non-PII drift; and (d) BWT under anchor removal. Four plots summarize selective output suppression and PTPC ablations. Two forest plots show confidence intervals for matched PII-minus-control NLL and PII Top-1 rate. A scatter plot contrasts PII NLL with low-sensitivity NLL, and a bar chart reports backward transfer after removing PTPC components.
Figure 4. Cross-backbone results on Qwen2.5-1.5B . (a–b) SPARK approaches ER utility and improves retention over PeCL. (c) Unmatched NLL supports absolute PII-likelihood suppression, not matched selectivity. Three horizontal plots summarize Qwen2.5 results. Grouped bars report Last and average accuracy, a bar chart reports backward transfer, and a scatter plot contrasts PII NLL with low-sensitivity NLL for ER, ER with DPSGD, PeCL, and SPARK.
Figure 5. Task-order robustness across three continual-learning sequences. Top row: Last, Avg, and BWT. Bottom row: unmatched PII, low-sensitivity, and canary NLL, showing consistent utility and absolute suppression across orders. Six grouped-bar panels compare four methods over three task orders using Last, average accuracy, backward transfer, PII NLL, low-sensitivity NLL, and canary NLL.
Figure 6. PTPC sensitivity to unlikelihood, privacy, and old-task anchor weights and correction steps. Each panel jointly reports Last accuracy and PII NLL, while dashed vertical lines identify the default settings. Four line panels report continual-learning utility and likelihood diagnostics while varying PTPC hyperparameters.
Noise scale σ
PII Δp (%)
Non-PII Δp (%)
Gap (pp)
0.01 (training)
− 4.01
− 3.72
0.29
0.05
− 96.27
− 86.06
10.21
0.10
− 99.64
− 99.65
0.01
0.50
− 99.66
− 99.74
0.08
Table 3. Embedding-noise output changes; negative values denote reductions. Training-scale selectivity is only 0.29 pp.
Figure 7. Case analysis on a FOMC benchmark example with injected synthetic PII. Both methods retain the correct task prediction, while SPARK assigns lower conditional likelihood to the annotated identifiers. A FOMC prompt contains an injected synthetic name, password, and Social Security number. PeCL and SPARK both predict the correct label, dovish. SPARK has a higher injected-PII negative log-likelihood than PeCL, indicating lower conditional likelihood for the annotated identifiers in this example.
Figure 8. Training-time decomposition: SPARK requires 200 min in total, while its PTPC stage accounts for only 4.0%. Most additional cost comes from replay-based distillation. A stacked horizontal bar chart decomposes each method's wall-clock time into task training, replay overhead, and privacy correction.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
T1
T2
T3
T4
T5
T6
After T1
0.843
—
—
—
—
—
After T2
0.813
0.013
—
—
—
—
After T3
0.810
0.027
0.800
—
—
—
After T4
0.827
0.050
0.807
0.943
—
—
After T5
0.830
0.080
0.837
0.943
0.660
—
After T6
0.780
0.460
0.880
0.937
0.667
0.227
Appendix
Table 4. Complete accuracy matrix from the reproduced PeCL implementation. Under the evaluated joint configuration, Yelp accuracy drops to 0.013 immediately after Task 2. This result provides a supporting example of task-sensitive acquisition under the reproduced joint configuration; it does not imply that joint objectives are universally unstable.
Task
PeCL † (After T k )
SPARK (After T k )
FOMC (T1)
0.843
0.837
Yelp (T2)
0.013
0.673
AGNews (T3)
0.800
0.923
Amazon (T4)
0.943
0.973
MentILL (T5)
0.660
0.733
Yahoo (T6)
0.227
0.703
Appendix
Table 5. Per-task accuracy immediately after training on each task. Under the reproduced joint PeCL configuration, Yelp accuracy drops to near-random (0.01); SPARK does not exhibit comparable instability.
Hyperparameter
Value
Model & Training
Base model
LLaMA-2-7B
LoRA rank r
16
LoRA α
32
LoRA targets
Q, K, V, O
Learning rate
5×10−4
Appendix
Table 6. Hyperparameter settings for SPARK.
Order
Method
Last ↑
Avg ↑
BWT ↑
NLL pii ↑
NLL low ↓
Canary NLL ↑
MIA AUC ≈ 0.5
Order 1
ER (no privacy)
0.815
0.830
+0.009
2.974
2.076
3.598
0.509
PeCL †
0.658
0.631
+0.093
2.538
1.643
4.135
0.515
ER+DPSGD
0.633
0.551
+0.016
2.628
1.708
3.762
0.519
SPARK (Ours)
0.799
0.812
-0.009
4.203
2.903
5.423
0.517
Order 2
ER (no privacy)
0.803
0.771
-0.021
3.306
2.439
4.321
0.519
PeCL †
0.746
0.759
-0.033
2.634
2.043
3.622
0.516
Appendix
Table 7. Continual-learning results and unmatched teacher-forced likelihood diagnostics across three task orders. NLL values are not extraction success rates.
Method
Last ↑
Avg ↑
BWT ↑
PII NLL ↑
Low NLL ↓
Canary NLL ↑
ER (no privacy)
0.7933
0.8105
−0.0040
3.4649
2.2066
5.006
ER+DPSGD
0.6789
0.6134
+0.0207
3.2977
1.9244
4.450
PeCL †
0.7089
0.7509
−0.0827
3.2034
1.9663
4.848
SPARK (Ours)
0.7844
0.7931
−0.0107
4.5651
2.0303
4.938
Appendix
Table 8. Cross-backbone evaluation on Qwen2.5-1.5B . Higher PII and canary NLL indicate lower absolute conditional likelihood, while Low NLL diagnoses non-PII drift.
Method
Last ↑
Avg ↑
BWT ↑
PII NLL ↑
Low-sens. NLL ↓
SPARK (full)
0.7994
0.8125
−0.0093
4.2028
2.9031
w/o dKL
0.7844
0.7985
−0.0193
6.3271
3.3000
w/o current anchor
0.7950
0.8105
−0.0187
4.6858
3.4055
w/o old anchor
0.7800
0.8067
−0.0307
4.7117
3.3442
Appendix
Table 9. Internal PTPC ablations. Higher PII NLL can reflect stronger suppression, but must be interpreted jointly with low-sensitivity NLL and continual-learning utility. Blue shading denotes the full model.
Method
NLLPII↑ [95% CI]
NLLMatched [95% CI]
Δsel↑ [95% CI]
Pretrained
2.5634 [2.2030, 2.9432]
2.5635 [2.2072, 2.9405]
−0.0002 [ −0.0136 , 0.0133]
ER
3.2904 [2.8278, 3.8166]
3.1268 [2.6259, 3.7063]
0.1635 [ −0.2845 , 0.6500]
PeCL †
3.2220 [2.7122, 3.9207]
2.9542 [2.4334, 3.5784]
0.2677 [ −0.3107 , 0.9602]
SD-Replay w/o PTPC
2.8806 [2.4456, 3.3629]
2.7638 [2.3557, 3.2463]
0.1168 [ −0.1771 , 0.4470]
ER+PTPC
3.3898 [2.9776, 3.8481]
3.8909 [3.2647, 4.6358]
−0.5010 [ −1.1670 , 0.1249]
SPARK (full)
4.2254 [3.8925, 4.6538]
3.4789 [3.1159, 3.8745]
0.7464 [0.3853, 1.1692]
Appendix
Table 10. Pooled matched-control NLL evaluation. Only SPARK full and the variant without dKL have Δsel intervals entirely above zero.
Method
Rank mean ↑ [95% CI]
Rank median ↑ [95% CI]
Top-1 (%) ↓ [95% CI]
Top-5 (%) ↓ [95% CI]
Top-10 (%) ↓ [95% CI]
Pretrained
47.86 [22.70, 81.68]
2 [1, 3]
47.13 [39.93, 53.36]
71.26 [64.98, 77.06]
76.63 [70.34, 82.56]
ER
89.71 [36.01, 163.73]
2 [2, 3]
41.00 [34.58, 46.46]
64.75 [57.89, 70.67]
75.10 [69.20, 80.56]
PeCL †
85.23 [29.72, 174.75]
2 [2, 3]
40.23 [32.84, 46.78]
66.28 [59.22, 72.20]
73.18 [66.31, 79.15]
SD-Replay w/o PTPC
69.52 [32.54, 116.00]
2 [1, 3]
44.44 [37.29, 50.74]
67.05 [60.66, 73.43]
76.63 [70.54, 82.19]
ER+PTPC
73.25 [36.50, 119.95]
3 [2, 4]
37.55 [31.20, 42.57]
59.39 [53.60, 64.61]
70.88 [64.87, 76.28]
SPARK (full)
75.65 [28.72, 160.37]
3 [3, 5]
23.75 [16.06, 31.51]
62.07 [54.67, 68.64]
69.73 [62.15, 76.17]
Appendix
Table 11. Pooled full-vocabulary ranking of observed PII tokens. Rank uncertainty is wide; the most robust separation for SPARK full occurs at Top-1 rather than across all Top- k thresholds.
Method
Loss AUC
Response AUC
Min-K10 AUC
Response TPR@5% FPR (%)
Pretrained
0.5149
0.4744
0.5213
3.98
ER (no privacy)
0.5008
0.5933
0.4897
5.83
PeCL †
0.5185
0.4801
0.5206
4.35
SD-Replay w/o PTPC
0.5079
0.5178
0.5002
4.40
ER+PTPC
0.5035
0.5912
0.5012
5.28
SPARK (full)
0.5173
0.5693
0.5051
8.03
Appendix
Table 12. Membership-inference diagnostics. AUC values closer to 0.5 and lower TPR are preferable. Loss and Min-K attacks are mostly near chance, but response-based attacks retain non-trivial discrimination for several trained checkpoints.
Method
Full NLL ↑ [95% CI]
Password NLL ↑ [95% CI]
SSN NLL ↑ [95% CI]
Pretrained
3.5178 [3.4600, 3.5701]
3.6276 [3.5300, 3.7236]
1.9964 [1.9743, 2.0165]
ER
3.8907 [3.8159, 3.9570]
4.1108 [3.9981, 4.2217]
1.9749 [1.9483, 2.0036]
PeCL †
4.4528 [4.3782, 4.5228]
4.8657 [4.7464, 4.9804]
2.0094 [1.9915, 2.0279]
SD-Replay w/o PTPC
3.9384 [3.8740, 3.9965]
3.8993 [3.8070, 3.9916]
2.0315 [2.0172, 2.0458]
ER+PTPC
3.9631 [3.8916, 4.0330]
4.1161 [3.9871, 4.2459]
2.0996 [2.0630, 2.1368]
SPARK (full)
5.5716 [5.5322, 5.6103]
5.9906 [5.9305, 6.0508]
3.6541 [3.6311, 3.6780]
Appendix
Table 13. Teacher-forced canary NLL. Higher values indicate lower absolute conditional likelihood of the planted sequence, but do not directly measure its relative rank among plausible candidates.
Method
Normalized full rank ↑ [95% CI]
Exposure ↓ [95% CI]
Full Top-1 (%) ↓ [95% CI]
Full Top-10 (%) ↓ [95% CI]
Pretrained
239.00 [197.90, 278.69]
1.688 [1.233, 2.199]
1.00 [0.00, 3.00]
3.33 [0.00, 8.33]
ER
247.87 [211.55, 282.51]
1.520 [1.152, 1.986]
1.00 [0.00, 3.00]
4.00 [0.00, 9.35]
PeCL †
228.78 [194.73, 264.16]
1.725 [1.286, 2.242]
2.00 [0.00, 5.33]
5.67 [0.00, 13.00]
SD-Replay w/o PTPC
242.64 [204.80, 281.46]
1.630 [1.183, 2.147]
2.00 [0.00, 6.00]
3.00 [0.00, 7.67]
ER+PTPC
241.24 [200.42, 280.07]
1.801 [1.287, 2.398]
2.67 [0.00, 7.00]
8.00 [2.00, 15.33]
SPARK (full)
227.90 [190.13, 264.90]
1.689 [1.288, 2.122]
1.00 [0.00, 3.00]
3.67 [0.00, 9.00]
Appendix
Table 14. Normalized full-canary candidate ranking among 512 candidates. The confidence intervals overlap broadly; unlike the NLL result, this table does not show a general improvement in candidate-ranking attack resistance.
Continual learning (CL) adapts pretrained models to a sequence of tasks while retaining knowledge acquired from earlier tasks. When task data are sensitive, differential privacy (DP) provides record-level protection, but maintaining both adaptation and retention under privacy constraints remains challenging. We study this problem in continual low-rank adaptation (LoRA) and observe that: 1) private training increases forgetting primarily during subsequent adaptation rather than initial task acquisition; 2) jointly perturbing both LoRA factors introduces a non-negligible bilinear noise interaction; and 3) DP aggravates the stability-plasticity dilemma, hindering both knowledge retention and adaptation to new tasks. Based on these observations, we propose DP-SPA, which constructs a stable subspace from previously privatized updates and a plastic subspace from pretrained weights within its orthogonal complement. The method fixes these bases and privately optimizes only the LoRA factor, eliminating the bilinear noise interaction within each step. Before merging each task update, it rescales stable-direction coefficients using historical and current update energies. We establish a privacy guarantee and show that recalibration does not increase historical overlap and tightens a conditional upper bound on old-task loss increase. Experiments across five datasets, eight settings, and multiple privacy budgets show that DP-SPA improves average accuracy and reduces forgetting compared with the evaluated baselines.
Cloud-local LLM inference can use the reasoning capability of large cloud models, but cloud-bound requests must exclude personally identifiable information (PII). Existing privacy-preserving methods modify the prompt or the model, which may distort contextual semantics or require additional training. This paper proposes P2Skill, a prompt-based skill distillation method in which a local small language model (SLM) autonomously performs four skills. A cloud LLM iteratively refines the skills from execution failures to let the local SLM generalize beyond memorized PII patterns, and the final skills are selected at the lowest validation leakage within set bounds. P2Skill requires no privacy-specific fine-tuning or learned auxiliary detectors. On real user queries, P2Skill on the best of three local SLMs transmits fewer annotated identifiers than every existing privacy-preserving method that sends text to the cloud and preserves quality more often than the closest of them in leakage, although fully local answering preserves quality more often.
Myunghoon Ryu, Geunpyo Park, Sungjoon Lee +2
Department of Electrical and Computer Engineering, Korea University, Seoul, Republic of Korea
Contextual Integrity (CI) defines privacy not merely as keeping information hidden, but as governing information flows according to the norms of a given context. As large language models are increasingly deployed as personal agents handling sensitive workflows, adhering to CI becomes critical. However, even frontier models remain unreliable in making disclosure decisions, and existing mitigation strategies often degrade underlying task performance. To overcome this privacy-utility trade-off, we propose SELFCI, a complementary self-distillation framework that decouples information suppression from task resolution. SELFCI jointly optimizes two independent reverse KL divergences over distinct teacher distributions derived from feedback: one encourages preserving task-relevant information for utility, while the other enforces minimal and appropriate disclosure. This complementary formulation induces a Product-of-Experts (PoE) target, aligning the policy with the intersection of capability and privacy requirements. Empirical evaluations demonstrate that SELFCI, without relying on costly external supervision, consistently outperforms competitive baselines such as online reinforcement learning algorithms (e.g., GRPO). These trends further extend to out-of-domain settings involving agentic workflows and accumulated private context, suggesting that SELFCI provides a practical path toward CI alignment.