Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and harmful content. Existing unlearning approaches for MLLMs typically adapt training-based strategies such as gradient ascent or preference optimization, but these methods are computationally expensive, irreversible, and often distort retained knowledge. In this work, we propose MLLMEraser, an input-aware, training-free framework for test-time unlearning. Our approach leverages activation steering to enable dynamic knowledge erasure without parameter updates. Specifically, we construct a multimodal erasure direction by contrasting adversarially perturbed, knowledge-recall image-text pairs with knowledge-erasure counterparts, capturing both textual and visual discrepancies. To prevent unnecessary interference, we further design an input-aware steering mechanism that adaptively determines when and how the erasure direction should be applied, preserving utility on retained knowledge while enforcing forgetting on designated content. Experiments on LLaVA-1.5 and Qwen-2.5-VL demonstrate that MLLMEraser consistently outperforms state-of-the-art MLLM unlearning baselines, achieving stronger forgetting performance with lower computational cost and minimal utility degradation.
Figures & tables
Figure 1: (a) Comparison between training-based and test-time unlearning paradigms for MLLMs. (b) Illustration of the activation steering process. (c)–(d) Differences between existing methods and ours in constructing and applying the steering vector.
Figure 2: Overview of the proposed MLLMEraser framework. Stage 1 derives a multimodal erasure direction derase from contrastive image-text pairs. Stage 2 introduces an input-aware steering mechanism f(h) that adaptively applies derase to shift the activations of forget samples toward refusal-style responses, while leaving retain samples nearly unaffected to preserve correct responses.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (5% Forget)-VQA
Vanilla
42.40
40.00
45.23
47.00
0.632
0.286
0.585
0.312
54.00
24.00
55.37
8.17
GA
40.08
38.40
39.75
38.77
0.353
0.251
0.389
0.265
14.00
16.00
28.00
5.56
GA_Diff
35.20
29.60
36.00
37.10
0.554
0.265
0.585
0.310
32.00
20.00
52.11
7.84
KL_Min
40.00
36.80
42.87
44.78
0.629
0.309
0.585
0.307
52.00
18.00
53.58
6.86
Table 1: Unlearning results on MLLMU-Bench across the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively. The best results are shown in bold.
Figure 3: Trade-off between forget quality and model utility on LLaVA under 5% and 10% forget ratios. The left two plots correspond to classification task, where the x -axis shows accuracy difference on the forget set (Fgt VQA Acc Diff), and the right two plots correspond to generation, where the x -axis shows ROUGE-L difference on the forget set (Fgt Rouge Diff). The y -axis reports model utility on the retained (Ret) and celebrity (Cele) sets.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Vanilla
67.20
68.80
66.20
78.33
0.642
0.317
0.583
0.453
36.00
22.00
40.42
26.80
Text-only erasure direction
48.40
65.60
65.74
78.06
0.483
0.249
0.568
0.442
11.00
16.00
38.53
25.82
Input-unaware steering
15.20
7.20
13.29
4.05
0.130
0.008
0.121
0.121
8.00
8.00
10.74
12.42
Ours
19.20
38.40
65.86
78.32
0.165
0.175
0.580
0.445
10.00
12.00
39.37
26.47
Table 2: Ablation study on MLLMU-Bench (5% Forget) using the Qwen2.5-VL-7B-Instruct model. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Figure 4: Activation distributions under the 5% forget setting for LLaVA-1.5-7B ( 4(a) ) and Qwen-2.5-VL-7B-Instruct ( 4(b) ), where each subfigure shows the results on retained set and the forget set (Fgt) before (Vanilla) and after (Steered) steering.
Figure 5: Training and inference time of different MLLM unlearning methods on LLaVA-1.5-7B under the 5% forget setting. Inference time is measured on 10 randomly sampled queries.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Case study demonstrating that the unaligned base Qwen-2-VL-7B model naturally refuses unethical or misleading requests.
Figure 7: Sensitivity analysis of the regularization parameter γ on LLaVA-1.5-7B under the 10% forgetting setting. 7(a) reports results on the classification task, 7(b) shows results on the generation task, and 7(c) presents results on the cloze task.
Figure 8: Sensitivity analysis of the steering strength λ on LLaVA-1.5-7B under the 10% forgetting setting. 8(a) reports results on the classification task, 8(b) shows results on the generation task, and 8(c) presents results on the cloze task.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (10% Forget)-VQA
Vanilla
43.60
41.20
45.26
47.00
0.591
0.339
0.591
0.312
57.00
15.00
55.11
8.17
ϵ=25516
2.00
27.20
45.08
46.48
0.176
0.202
0.594
0.314
5.00
6.00
54.33
7.84
ϵ=25532
0.00
14.00
42.94
44.26
0.040
0.133
0.584
0.308
2.00
6.00
52.56
6.86
ϵ=25564
2.00
20.40
44.59
46.21
0.355
0.228
0.593
0.314
14.00
12.00
53.56
7.19
Appendix
Table 3: Unlearning results on MLLMU-Bench (10% Forget) under different perturbation budgets ϵ . Results are evaluated on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
GA
GA_Diff
KL_Min
NPO
MMUnlearner
Ours
Memory Usage
86292MB
101262MB
106482MB
102616MB
78472MB
0MB
Appendix
Table 4: Training memory usage for updating MLLM parameters in different unlearning methods.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (10% Forget)-VQA
Vanilla
43.60
41.20
45.26
47.00
0.591
0.339
0.591
0.312
57.00
15.00
55.11
8.17
GA
40.00
31.20
41.43
40.08
0.510
0.315
0.470
0.262
43.00
14.00
49.22
5.56
KL_Min
42.40
35.20
45.08
46.08
0.588
0.335
0.579
0.306
54.00
15.00
53.89
7.10
GA_Diff
41.20
34.00
39.64
36.16
0.520
0.336
0.517
0.312
23.00
9.00
21.44
4.58
Appendix
Table 5: Unlearning results on MLLMU-Bench (10% Forget) with VQA and QA evaluation using LLaVA-1.5-7B. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Figure 9: Case Study on Forget Set before and after unlearning. The figure shows model responses to a forget-set query about sensitive attribute information. While the majority of training-based methods collapse or continue to expose the forgotten knowledge after unlearning, our method successfully removes the targeted information and produces a refusal-style response.
Figure 10: Case Study on Retain Set before and after unlearning. The figure shows model responses to a retain-set query asking about a person’s residence. While training-based methods (GA, GA_Diff, KL_Min, NPO, and MMUnlearner) either collapse or generate incorrect answers after unlearning, our method preserves the original correct response, demonstrating its superior ability to maintain retained knowledge while performing effective unlearning.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (5% Forget)-VQA
Vanilla
42.40
40.00
45.23
47.00
0.632
0.286
0.585
0.312
54.00
24.00
55.37
8.17
GA
40.08
38.40
39.75
38.77
0.353
0.251
0.389
0.265
14.00
16.00
28.00
5.56
GA_Diff
35.20
29.60
36.00
37.10
0.554
0.265
0.585
0.310
32.00
20.00
52.11
7.84
KL_Min
40.00
36.80
42.87
44.78
0.629
0.309
0.585
0.307
52.00
18.00
53.58
6.86
Appendix
Table 6: Unlearning results on MLLMU-Bench (5% Forget) with VQA and QA evaluation using LLaVA-1.5-7B. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Qwen-2.5-VL-7B (5% Forget)-VQA
Vanilla
67.20
68.80
66.20
78.33
0.642
0.317
0.583
0.453
36.00
22.00
40.42
26.80
GA
64.00
64.80
65.11
74.15
0.419
0.315
0.438
0.335
16.00
14.00
16.21
25.16
GA_Diff
56.00
59.20
64.05
76.76
0.482
0.296
0.566
0.442
16.00
22.00
33.79
24.84
KL_Min
51.20
50.40
50.30
48.57
0.594
0.322
0.578
0.441
32.00
18.00
34.32
25.16
Appendix
Table 7: Unlearning results on MLLMU-Bench (5% Forget) with VQA and QA evaluation using Qwen2.5-VL-7B. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (15% Forget)-VQA
Vanilla
44.80
42.93
45.09
47.00
0.598
0.294
0.586
0.312
53.33
19.33
55.76
8.17
GA
38.67
38.13
36.70
42.74
0.493
0.291
0.523
0.307
25.33
18.00
23.29
5.56
GA_Diff
28.27
36.80
43.68
41.26
0.333
0.303
0.517
0.301
24.00
18.67
45.29
4.28
KL_Min
42.67
37.87
39.95
39.57
0.537
0.316
0.575
0.306
44.00
19.01
43.29
6.21
Appendix
Table 8: Unlearning results on MLLMU-Bench (15% Forget) across the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Figure 11: Layer-wise distributions of L2 norms for forget (blue) and retain (pink) steering vectors across Layers 2–31 of LLaVA-1.5-7B.
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (10% Forget)-VQA
Vanilla
43.60
41.20
45.26
47.00
0.591
0.339
0.591
0.312
57.00
15.00
55.11
8.17
variant-1
33.20
37.20
45.43
47.00
0.572
0.325
0.588
0.309
54.00
13.00
55.11
8.17
variant-2
2.00
24.00
42.85
42.95
0.145
0.170
0.584
0.308
1.00
14.00
52.78
7.52
all
2.00
27.20
45.08
46.48
0.176
0.202
0.594
0.314
5.00
6.00
54.33
7.84
Appendix
Table 9: Unlearning performance on MLLMU-Bench (10% Forget). In addition to the full-layer steering strategy (all), we evaluate two selective-layer variants: variant-1, which steers only early layers (1–16), and variant-2, which steers only late layers (17–32).
Models
Classification Accuracy (%)
Generation: Rouge Score
Cloze: Accuracy (%)
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
Fgt ↓
Test ↓
Ret ↑
Cele ↑
LLaVA-1.5-7B (10% Forget)-VQA
Vanilla
43.60
41.20
45.26
47.00
0.591
0.339
0.591
0.312
57.00
15.00
55.11
8.17
Ours-MLP
29.20
32.40
38.62
42.19
0.363
0.265
0.414
0.311
19.00
8.00
32.00
6.54
Ours
2.00
27.20
45.08
46.48
0.176
0.202
0.594
0.314
5.00
6.00
54.33
7.84
Appendix
Table 10: Comparison of linear and MLP implementations of f(h) on MLLMU-Bench (10% Forget) using LLaVA-1.5-7B. Ours uses the default linear implementation, while Ours-MLP employs a two-layer MLP. Results are reported on the forget (Fgt), test (Test), retain (Ret), and celebrity (Cele) sets. ↓ and ↑ denote lower-is-better and higher-is-better metrics, respectively.
Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands