When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs

Linyu Li; Zhi Jin; Yichi Zhang; Dongming Jin; Yuanpeng He; Haoran Duan; Gadeng Luosang; Nyima Tashi

cs.CLApr 3, 2026

When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs

Authors: Linyu Li, Zhi Jin, Yichi Zhang, Dongming Jin, Yuanpeng He, Haoran Duan, Gadeng Luosang, Nyima Tashi

Organizations: Key Laboratory of High Confidence Software Technologies (PKU), Ministry of Education School of Computer Science, Peking University Beijing, China · School of Computer Science Zhejiang University Hangzhou, China · School of Cyber Science and Engineering Wuhan University Wuhan, China · School of Information Science and Technology Tibet University Lhasa, China

Abstract

Real-world multimodal knowledge graphs (MMKGs) are dynamic, with new entities, relations, and multimodal knowledge emerging over time. Existing continual knowledge graph reasoning (CKGR) methods focus on structural triples and cannot fully exploit multimodal signals from new entities. Existing multimodal knowledge graph reasoning (MMKGR) methods, however, usually assume static graphs and suffer catastrophic forgetting as graphs evolve. To address this gap, we present a systematic study of continual multimodal knowledge graph reasoning (CMMKGR). We construct several continual multimodal knowledge graph benchmarks from existing MMKG datasets and propose MRCKG, a new CMMKGR model. Specifically, MRCKG employs a multimodal-structural collaborative curriculum to schedule progressive learning based on the structural connectivity of new triples to the historical graph and their multimodal compatibility. It also introduces a cross-modal knowledge preservation mechanism to mitigate forgetting through entity representation stability, relational semantic consistency, and modality anchoring. In addition, a multimodal contrastive replay scheme with a two-stage optimization strategy reinforces learned knowledge via multimodal importance sampling and representation alignment. Experiments on multiple datasets show that MRCKG preserves previously learned multimodal knowledge while substantially improving the learning of new knowledge.

Figures & tables

Figure 1. CMMKG stores knowledge in the form of triplets; however, unlike traditional knowledge graphs, its multimodal information also continuously evolves over time. Three snapshots illustrate a continually evolving multimodal knowledge graph. Entities representing political figures and the United States gain and change structural relations, portrait images, and textual descriptions over time.

Figure 2. Overall framework of MRCKG for continual multimodal knowledge graph reasoning. The MRCKG pipeline combines a scalable multimodal encoding backbone with multimodal structure-aware curriculum learning, cross-modality knowledge preservation, multimodal contrastive replay, and two-stage optimization across graph snapshots.

Dataset	Split	$\mathcal{S}_{0}$			$\mathcal{S}_{1}$			$\mathcal{S}_{2}$			$\mathcal{S}_{3}$			$\mathcal{S}_{4}$
Dataset	Split	#Ent	#Rel	#Triples	#Ent	#Rel	#Triples	#Ent	#Rel	#Triples	#Ent	#Rel	#Triples	#Ent	#Rel	#Triples
DB15K	Entity	4,494	216	50,149	8,501	253	36,735	10,504	261	12,255	11,839	273	5,207	12,842	279	2,012
	Higher	1,884	136	8,616	4,266	185	14,853	6,813	206	19,806	9,895	232	29,718	12,842	279	28,290
	Equal	4,332	181	19,808	7,221	204	22,783	9,459	229	22,783	11,530	256	22,778	12,842	279	11,451
MKG-W	Entity	5,250	126	21,025	9,930	150	15,891	12,270	156	5,111	13,830	166	2,651	15,000	169	1,323
	Higher	1,648	67	3,717	3,891	67	6,411	5,982	100	8,550	9,166	129	12,825	15,000	169	17,095

Table 1. Dataset statistics for all 9 continual MMKG benchmarks (

T\!=\!5

snapshots). #Ent and #Rel denote cumulative entity/relation counts at each snapshot; #Triples denotes the number of triples used at each snapshot after bridge augmentation, rather than only newly introduced triples (split into Train/Valid/Test at a 3:1:1 ratio).

		DB15K						MKG-W						MKG-Y
		Entity		Higher		Equal		Entity		Higher		Equal		Entity		Higher		Equal
Setting	Method	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10	MRR	H@10
Unimodal +FT	TransE+FT	0.1085	0.2355	0.0987	0.2271	0.0913	0.2160	0.0956	0.1909	0.1024	0.2289	0.0867	0.1919	0.1243	0.2212	0.1587	0.2962	0.1462	0.2697
	DistMult+FT	0.1186	0.2092	0.1063	0.1988	0.0978	0.1880	0.1043	0.1693	0.1098	0.1995	0.0942	0.1695	0.1356	0.1961	0.1725	0.2617	0.1589	0.2382
	ComplEx+FT	0.1644	0.2348	0.1512	0.2289	0.1387	0.2159	0.1425	0.1872	0.1536	0.2260	0.1308	0.1905	0.1872	0.2192	0.2316	0.2845	0.2153	0.2614
	TuckER+FT	0.1618	0.2151	0.1489	0.2098	0.1354	0.1962	0.1397	0.1709	0.1503	0.2058	0.1276	0.1730	0.1835	0.2000	0.2274	0.2600	0.2108	0.2382

Table 2. Average MRR and Hits@10 comparison on all nine benchmarks. The best result in each column is shown in bold , and the second best is underlined .

Variant	MRR	H@1	H@10	BWT
Full	0.2856	0.2213	0.4102	$-$ 0.068
w/o CMKP	0.2549	0.1988	0.3673	$-$ 0.126
w/o MMCR	0.2683	0.2056	0.3875	$-$ 0.095
w/o MSCL	0.2715	0.2075	0.3914	$-$ 0.085
w/o Prog.	0.2639	0.2021	0.3749	$-$ 0.102
w/o Visual	0.2671	0.2003	0.3931	$-$ 0.073

Table 3. Ablation study on DB15K-Entity, including component ablation and modality ablation.

Figure 3. Analysis on DB15K-Entity. (a) Per-snapshot MRR on new vs. old triples; (b)

\mathcal{S}_{0}

forgetting curves; (c) MRR–BWT Pareto front; (d) multi-metric radar plot. Four plots compare continual-learning behavior on DB15K-Entity: MRR for new and old triples, forgetting on the first snapshot, the MRR versus backward-transfer frontier, and a radar chart of normalized performance metrics.

Figure 4. Per-snapshot MRR bar plots of MRCKG on the three DB15K settings. Three grouped bar charts show MRR for snapshots zero through four after successive training stages under the Entity, Higher, and Equal DB15K splits.

Figure 5. (a) Error type distribution of MRCKG; (b) Hits@1 of three methods across snapshots. A donut chart partitions MRCKG predictions into forgetting, cold-start, cross-modal ambiguity, other, and correct outcomes. A grouped bar chart compares MRCKG, Structure-CL, and fine-tuning Hits at one across five snapshots.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Entity					Higher					Equal
	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT
TransE+FT	0.1085	0.0427	0.1331	0.2355	$-$ 0.064	0.0987	0.0350	0.1247	0.2271	$-$ 0.067	0.0913	0.0334	0.1165	0.2160	$-$ 0.074
DistMult+FT	0.1186	0.0751	0.1274	0.2092	$-$ 0.057	0.1063	0.0606	0.1176	0.1988	$-$ 0.060	0.0978	0.0576	0.1093	0.1880	$-$ 0.066
ComplEx+FT	0.1644	0.1275	0.1798	0.2348	$-$ 0.024	0.1512	0.1055	0.1703	0.2289	$-$ 0.025	0.1387	0.1000	0.1578	0.2159	$-$ 0.028
TuckER+FT	0.1618	0.1338	0.1724	0.2151	$-$ 0.022	0.1489	0.1108	0.1634	0.2098	$-$ 0.023	0.1354	0.1041	0.1500	0.1962	$-$ 0.025
TransE+EWC	0.1088	0.0477	0.1301	0.2276	$-$ 0.062	0.0994	0.0392	0.1224	0.2204	$-$ 0.065	0.0921	0.0376	0.1145	0.2100	$-$ 0.071

Appendix

Table 4. Full results on DB15K benchmarks (Avg MRR / H@1 / H@3 / H@10 / BWT). Best bold , second best underlined .

Method	Entity					Higher					Equal
	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT
TransE+FT	0.0956	0.0376	0.1138	0.1909	$-$ 0.061	0.1024	0.0351	0.1269	0.2289	$-$ 0.066	0.0867	0.0314	0.1074	0.1919	$-$ 0.072
DistMult+FT	0.1043	0.0660	0.1087	0.1693	$-$ 0.054	0.1098	0.0605	0.1191	0.1995	$-$ 0.059	0.0942	0.0549	0.1022	0.1695	$-$ 0.064
ComplEx+FT	0.1425	0.1105	0.1512	0.1872	$-$ 0.023	0.1536	0.1036	0.1697	0.2260	$-$ 0.025	0.1308	0.0933	0.1445	0.1905	$-$ 0.027
TuckER+FT	0.1397	0.1155	0.1444	0.1709	$-$ 0.021	0.1503	0.1081	0.1617	0.2058	$-$ 0.023	0.1276	0.0971	0.1373	0.1730	$-$ 0.025
TransE+EWC	0.0968	0.0424	0.1123	0.1863	$-$ 0.059	0.1037	0.0396	0.1252	0.2234	$-$ 0.064	0.0879	0.0355	0.1062	0.1876	$-$ 0.069

Appendix

Table 5. Full results on MKG-W benchmarks (Avg MRR / H@1 / H@3 / H@10 / BWT). Best bold , second best underlined .

Method	Entity					Higher					Equal
	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT	MRR	H@1	H@3	H@10	BWT
TransE+FT	0.1243	0.0611	0.1403	0.2212	$-$ 0.056	0.1587	0.0768	0.1811	0.2962	$-$ 0.053	0.1462	0.0719	0.1668	0.2697	$-$ 0.055
DistMult+FT	0.1356	0.1073	0.1340	0.1961	$-$ 0.050	0.1725	0.1344	0.1723	0.2617	$-$ 0.047	0.1589	0.1258	0.1587	0.2382	$-$ 0.049
ComplEx+FT	0.1872	0.1685	0.1884	0.2192	$-$ 0.021	0.2316	0.2084	0.2356	0.2845	$-$ 0.020	0.2153	0.1938	0.2190	0.2614	$-$ 0.021
TuckER+FT	0.1835	0.1651	0.1799	0.2000	$-$ 0.019	0.2274	0.2047	0.2253	0.2600	$-$ 0.018	0.2108	0.1897	0.2089	0.2382	$-$ 0.019
TransE+EWC	0.1258	0.0689	0.1384	0.2158	$-$ 0.055	0.1604	0.0865	0.1784	0.2886	$-$ 0.051	0.1478	0.0810	0.1644	0.2628	$-$ 0.053

Appendix

Table 6. Full results on MKG-Y benchmarks (Avg MRR / H@1 / H@3 / H@10 / BWT). Best bold , second best underlined .

Explore similar work

CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs
May 11, 2026Yousef A. Radwan, Yao Li, Qing Qing +5Continual Learning KG Embedding
Technology, Innovation, Entrepreneurship Department King Abdullah University of Science and Technology · School of Computing and Information Systems The University of Melbourne · College of Computer Science and Technology Jilin University +2
MMKG-R1: Constructing Multimodal Knowledge Graphs via Reinforcement Learning
Oct 14, 2026Zongyu Wu, Yilong Wang, Xiaochen Wang +5KG Construction Multimodal KGs
The Pennsylvania State University
VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings
Mar 2, 2026Athanasios Efthymiou, Stevan Rudinac, Monika Kackovic +2KG Embedding Multimodal KGs
University of Amsterdam, Amsterdam, The Netherlands

May 11, 2026J/K move · Enter open · S save

CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs

Yousef A. Radwan, Yao Li, Qing Qing, Ziqi Xu+4

Technology, Innovation, Entrepreneurship Department King Abdullah University of Science and Technology · School of Computing and Information Systems The University of Melbourne · College of Computer Science and Technology Jilin University+2

Continual Learning KG Embedding Multimodal Graph Learning Multimodal KGs Knowledge Graphs KG Completion+1

Open paper AskPDF arXiv

Biomedical knowledge graphs are increasingly large, dynamic, and multimodal, driven by rapid advances in biotechnology such as high-throughput sequencing. Machine learning models can infer previously unobserved biomedical relationships and characterize biomedical entities in these graphs, but existing knowledge graph embedding methods and their continual learning extensions either assume static graph structure or fail to exploit multimodal information under evolving data distributions. They also apply uniform regularization across all model parameters, ignoring that different modalities may exhibit distinct forgetting dynamics as the graph evolves. We propose the Continual Multimodal Knowledge Graph Learner (CMKL), a CL framework for biomedical KGs that natively encodes structure, text, and molecules, fuses them through a Mixture-of-Experts (MoE) router, and protects previously learned knowledge with standard EWC regularization and a K-means-diverse multimodal replay buffer. We evaluate CMKL on a 129K-entity biomedical continual benchmark with 10 tasks. On continual biomedical entity classification, CMKL reaches AP 0.591 versus 0.370 for the strongest structural baseline, a 60% gain that is driven by access to multimodal features and preserved across the sequence with near-zero forgetting (AF 0.008). On continual relationship prediction, CMKL reaches AP

0.062

, matching Naive Sequential and EWC (0.058) within seed noise and outperforming Joint Training (0.047, p=0.045) and LKGE (0.039). A frozen-text ablation reaches AP 0.136, more than double any jointly trained model, yet that signal is unreachable by margin-ranking gradients: the greedy-modality asymmetry lives at the representation level, not the fusion level, and MoE routing manages it by suppressing the unreachable modality without forcing it through a learned bottleneck. Code: github.com/yradwan147/cmkl-neurips2026

Abstract

Explore similar work

CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs

MMKG-R1: Constructing Multimodal Knowledge Graphs via Reinforcement Learning

VL-KGE: Vision-Language Models Meet Knowledge Graph Embeddings