Organizations: School of Computer Science, Northwestern Polytechnical University, Xi’an · Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen · Independent Researcher · Key Laboratory of Big Data Storage and Management, Northwestern Polytechnical University, Ministry of Industry and Information Technology, Xi’an
Multimodal Large Language Models (MLLMs) require continuous factual updates to stay current, yet lifelong knowledge editing remains a daunting challenge. The primary obstacle is the severe Locality Erosion and Catastrophic Forgetting, where sequential edits inevitably interfere with pre-trained capabilities and prior updates. In this paper, we propose ROSE, a Risk-aware Orthogonal Subspace navigation framework for lifelong knowledge Editing. ROSE addresses forgetting by aiming to restrict parameter updates to the orthogonal complement of previously edited subspaces. Crucially, it introduces a risk-guided mechanism that utilizes a dynamically computed risk mask to safeguard specific factual parameter subspaces essential for model stability. During inference, ROSE employs a multi-granularity knowledge integration strategy, featuring prototype-anchored semantic gating and perturbation-driven subspace fusion to precisely activate edited knowledge for relevant queries while limiting interference on unrelated inputs. Comprehensive experiments on two prominent multimodal benchmarks show that ROSE is competitive with state-of-the-art approaches across five key metrics, while maintaining outstanding stability over 1,000 sequential updates. Our code is available at https://github.com/sunlitsong/ROSE.
Figures & tables
Figure 1: Comparison of optimization trajectories generated by conventional Editors and our ROSE. By navigating parameter updates away from high-risk general subspaces, ROSE achieves a superior balance between incorporating new knowledge and preserving pre-trained stability.
Figure 2: An illustration of the ROSE framework. It mainly consists of two core modules: the Risk-Guided Orthogonal Editing and the Multi-Granularity Knowledge Integration.
Edits
Methods
VLKEB
E-IC
E-VQA
Reliability
Generality
Locality
Reliability
Generality
Locality
Reliability
Generality
Locality
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T=1
FT-L
94.29
87.00
92.22
91.16
91.37
73.48
72.98
65.79
99.28
99.06
93.88
87.98
80.25
99.61
94.78
FT-M
76.31
65.57
59.43
100.00
92.35
56.19
56.55
49.94
100.00
100.00
87.29
76.11
53.23
100.00
96.95
MEND
92.13
91.28
90.22
89.19
90.13
92.82
91.81
90.59
96.38
93.69
91.23
90.05
91.29
91.02
90.22
SERAC
89.77
89.11
87.92
66.68
14.20
88.18
81.03
85.61
84.01
28.58
89.33
83.72
84.97
82.05
23.78
Table 1: Performance comparison (%) on LLaVA-v1.5 (7B), with baseline results taken from Chen et al. [18] . Models are assessed on five primary aspects: Reliability (Rel.), Text Generality (T-Gen.), Multimodal Generality (M-Gen.), Text Locality (T-Loc.), and Multimodal Locality (M-Loc.) after 1, 100, and 1000 sequential edits.
Settings
Reliability
Generality
Locality
Average
T-Gen
M-Gen.
T-Loc.
M-Loc.
Fine Tuning
79.16
76.75
75.94
14.54
7.44
50.77
w/o Multimodal Decoupling
91.05
88.14
85.63
100.00
40.54
81.07
w/o Knowledge Integration
93.15
90.81
90.05
57.38
15.83
69.44
w/o Semantic Gating
91.98
89.71
89.02
76.87
25.97
74.71
w/o Knowledge Fusion
96.28
93.90
83.30
100.00
84.37
91.57
Table 2: Ablation study of ROSE on the VLKEB benchmark.
Table 5Figure 6
Edits
Editing Time (s/edit)
Inference Time (s/query)
Peak VRAM (MB)
1
1.6748
0.9973
16781.52
10
1.7113
1.1849
16826.89
100
1.3975
1.1248
16857.73
1000
1.3140
1.1256
17005.69
Table 5: Computational cost of ROSE across different numbers of sequential edits. Metrics include average editing time per edit, average inference time per query, and peak VRAM consumption.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Impact of the Reference Anchor for Risk Identification.
Edits
Methods
VLKEB
E-IC
E-VQA
Reliability
Generality
Locality
Reliability
Generality
Locality
Reliability
Generality
Locality
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T-Gen.
M-Gen.
T-Loc.
M-Loc.
T=1
FT-L
82.17
81.64
78.45
98.45
75.06
60.40
59.47
52.79
99.95
92.46
93.85
86.25
88.58
99.47
84.50
FT-M
93.51
91.28
68.30
100.00
81.78
71.60
67.34
49.55
100.00
91.69
91.42
85.82
49.09
100.00
86.72
MEND
90.28
89.18
89.55
93.63
86.69
90.28
83.91
79.05
95.92
93.97
93.47
91.32
91.22
81.51
74.97
SERAC
89.28
89.06
87.35
64.88
12.72
85.09
78.96
81.70
83.36
24.50
87.66
69.24
85.96
82.11
24.11
Appendix
Table 6: Performance comparison (%) on MiniGPT-4 (7B), with baseline results taken from Chen et al. [18] . Models are assessed on five primary aspects: Reliability (Rel.), Text Generality (T-Gen.), Multimodal Generality (M-Gen.), Text Locality (T-Loc.), and Multimodal Locality (M-Loc.) after 1, 100, and 1000 sequential edits.
Figure 6: Performance comparison across LLaVA-v1.5-7B, LLaVA-v1.5-13B, and Qwen2-VL-7B backbones after 1,000 sequential edits on the VLKEB benchmark.