Multimodal large language models (MLLMs) can reproduce sensitive, copyrighted, or unsafe information from their training data. Effective unlearning must remove target information while preserving non-target content, even within the same response. Existing methods typically apply uniform forgetting objectives to complete responses and rely on fixed, off-policy supervision, resulting in coarse token control and misalignment with the model's current behavior. We propose CUSE, a Counterfactual-guided Unlearning framework via on-policy Self-distillation for multimodal large language models. It first constructs counterfactual inputs by removing target-related evidence and uses the resulting changes in token probabilities to identify which tokens require stronger forgetting. It then distills token-level forgetting and retention targets on responses sampled from the current policy, with a frozen copy of the original model serving as the teacher model augmented with the corresponding forgetting- and retention-oriented privileged contexts. This design jointly determines what to forget and what to preserve. On MLLMU-Bench, CUSE reduces forget-set ROUGE-L by 33.6% while achieving 1.2% higher retain-set ROUGE-L than the strongest baseline.
Figures & tables
Figure 1: Motivation of CUSE and its key components for addressing fine-grained forgetting control and supervision–behavior misalignment in MLLM unlearning.
Figure 2: CUSE consists of (a) on-policy sampling, (b) counterfactual sensitivity estimation, and (c) dual-branch on-policy self-distillation with sensitivity-aware forgetting supervision.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (5% Forget)
Vanilla
59.20
44.80
60.70
40.21
0.645
0.289
0.584
0.312
54.00
26.00
55.47
8.17
GA
38.40
41.60
37.35
26.24
0.427
0.192
0.356
0.226
24.00
18.00
38.91
5.88
GA_Diff
40.80
42.40
49.81
30.68
0.503
0.224
0.547
0.271
32.00
22.00
49.96
6.87
KL_Min
37.60
39.20
45.91
28.72
0.519
0.230
0.552
0.285
28.00
20.00
42.26
6.21
Table 1: Unlearning results on MLLMU-Bench across the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively. The best results are shown in bold.
Figure 3: Behavioral evaluation on MLLMU-Bench using Qwen2.5-VL-7B-Instruct under a 5% forgetting ratio. ( 3(a) ) Forgetting naturalness. ( 3(b) ) Target-information leakage reduction and visual concept preservation.
Models
Classification Accuracy (%)
Generation: ROUGE-L Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Vanilla
59.20
44.80
60.70
40.21
0.645
0.289
0.584
0.312
54.00
26.00
55.47
8.17
Off-policy
57.00
40.00
60.31
37.27
0.372
0.214
0.539
0.301
48.00
24.00
52.76
7.52
Uniform
24.00
22.40
59.67
38.94
0.192
0.177
0.548
0.309
18.00
22.00
53.55
7.52
Random
28.00
20.80
58.65
38.48
0.227
0.238
0.527
0.304
22.00
20.00
54.29
7.19
Ours
11.20
14.40
60.51
39.61
0.129
0.009
0.554
0.310
10.00
6.00
55.89
8.17
Table 2: Ablation study of CUSE on MLLMU-Bench using LLaVA-1.5-7B with a 5% forget ratio. Results are reported on the forget (Fgt), test, retain (Ret), and celebrity (Cele) sets.
Figure 4: Forgetting–utility trade-off on MLLMU-Bench using LLaVA-1.5-7B under 5%, 10%, and 15% forgetting ratios. The two plots on the left present classification results, with the x -axis measuring the drop in forget-set accuracy (Fgt Acc Diff), whereas the two panels on the right present generation results, with the x -axis measuring the decrease in forget-set ROUGE score (Fgt Rouge Diff). The y -axis reports model utility on the retained (Ret) and celebrity (Cele) sets.
Models
Cls: Accuracy (%)
Gen: ROUGE-L Score
Clz: Accuracy (%)
Forget ↓
Retain ↑
Forget ↓
Retain ↑
Forget ↓
Retain ↑
CUSE
11.20
60.51
0.129
0.554
10.00
55.89
Random Prefix
10.40
60.13
0.125
0.553
8.00
55.56
Jailbreak Prompt
12.80
61.92
0.134
0.559
12.00
56.71
Paraphrase
12.00
60.58
0.130
0.561
10.00
54.32
Sampling Attack
11.20
60.67
0.121
0.557
12.00
55.48
Table 3: Performance comparison of different unlearning settings on MLLMU-Bench with LLaVA-1.5-7B at a 5% forgetting ratio. Results are reported on the forget (Fgt) and retain (Ret) sets.
Figure 5: Parameter sensitivity and optimization dynamics on MLLMU-Bench with LLaVA-1.5-7B. ( 5(a) ) and ( 5(b) ) show the impact of γ on ROUGE score and classification accuracy, while ( 5(c) ) presents ROUGE score evolution across optimization steps.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Parameter sensitivity on MLLMU-Bench with Qwen2.5-VL-7B-Instruct. ( 6(a) ) and ( 6(b) ) show the impact of γ on ROUGE score and classification accuracy.
Models
Classification Accuracy (%)
Generation: ROUGE-L Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (5% Forget)-VQA
Vanilla
59.20
44.80
60.70
40.21
0.645
0.289
0.584
0.312
54.00
26.00
55.47
8.17
GA
38.40
41.60
37.35
26.24
0.427
0.192
0.356
0.226
24.00
18.00
38.91
5.88
GA_Diff
40.80
42.40
49.81
30.68
0.503
0.224
0.547
0.271
32.00
22.00
49.96
6.87
KL_Min
37.60
39.20
45.91
28.72
0.519
0.230
0.552
0.285
28.00
20.00
42.26
6.21
Appendix
Table 4: Unlearning performance of LLaVA-1.5-7B on MLLMU-Bench under 5%, 10%, and 15% forgetting ratios. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets across classification, generation, and cloze tasks. ↓ indicates lower is better, while ↑ indicates higher is better. The results of our method are highlighted in bold.
Figure 7: Training procedure of CUSE.
Method
Efficacy ↑
Generality ↑
Divergence ↑
Fluency ↑
Specificity ↑
Capability ↑
LLaVA-1.5-7B (5% Forget)
Vanilla
3.75
5.00
30.60
73.13
97.71
78.46
GA
84.31
78.28
70.14
44.92
81.39
69.65
GA_Diff
77.84
72.55
63.07
46.11
83.25
70.20
KL_Min
73.97
68.92
61.06
47.40
82.23
70.95
NPO
87.43
82.24
72.13
49.20
78.65
69.17
Appendix
Table 5: Unlearning performance on CoVUBench under the 5% forget setting. Results are reported for LLaVA-1.5-7B and Qwen3-VL-4B-Instruct.
Figure 8: Unlearning performance comparison of LLaVA-1.5-7B on MLLMU-Bench. ( 8(a) ) Forget naturalness and ( 8(b) ) visual concept accuracy under 5% forgetting ratio.