Aligning Multimodal Patient Evidence with Biomedical Knowledge Graphs for Clinical LLMs

Jiawen Du; Arshan Ali Khan; Chenhao Zhang; Zachary Plotkin; Li Shen; Qi Long; Yun Li; Can Chen

cs.LGOct 5, 2026

Aligning Multimodal Patient Evidence with Biomedical Knowledge Graphs for Clinical LLMs

Authors: Jiawen Du, Arshan Ali Khan, Chenhao Zhang, Zachary Plotkin, Li Shen, Qi Long, Yun Li, Can Chen, +2 more

Organizations: University of North Carolina at Chapel Hill · Siddaganga Institute of Technology · Foci Labs · University of Pennsylvania

Abstract

Clinical questions often depend on linking a patient's multimodal evidence to external biomedical knowledge, yet existing predictive systems rarely represent such links explicitly, so they can neither be traced to their evidence sources nor removed to measure their contributions. We present MM-KG (Multimodal Knowledge Graph), which represents heterogeneous, multimodal patient observations and biomedical concepts as separate layers in one typed graph, joined by explicit alignment edges. First, modality-specific harmonizers convert EHR text, imaging, genomic, and biospecimen data into typed observations mapped to UMLS concepts, which a route-prioritized aligner links to a biomedical knowledge graph. Query-conditioned retrieval then selects a compact subgraph for downstream use by a large language model or a graph neural network. We build MM-KGs for MIMIC-IV and ADNI, and evaluate them with a 2x2 design that separates patient evidence, biomedical knowledge, and their interaction. On questions that require both sources, neither source alone performs far above chance, whereas their combination yields a drug-controlled AUROC interaction of +0.194 on MIMIC and +0.299 on ADNI. On held-out five-candidate ranking, MM-KG outperforms MindMap by +0.131 Hits@1 and leads an adapted GraphCare on the items that require consulting the patient, and deleting the single answer-bearing relation from the retrieved packet returns Hits@1 to the no-knowledge baseline. Finally, query-conditioned retrieval reaches 0.731 AUROC with 6.8x less context than the strongest generic policy, whereas static knowledge graph context gives no consistent gain on ordinary outcome prediction. Knowledge graphs thus benefit clinical LLMs not as background context but as explicit links between multimodal patient evidence and the relation a question requires, and MM-KG makes these links retrievable, traceable, and testable.

Figures & tables

	Patient evidence in the graph					Representation			Consumer
Method	Codes	Labs	Imaging	Text	Genetic	Link	Prov	Q-Ret	LLM	GNN
Multimodal clinical representation
MedFuse ( Hayat et al., 2022 )	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗
MUSE ( Wu et al., 2024 )	✓	✓	✓	✓	✗	✗	✗	✗	✗	✓
MedGTX ( Park et al., 2022 )	✓	✗	✗	✓	✗	✗	✗	$\sim$	✗	✓
Knowledge-enhanced patient representation

Table 1: Capabilities of related methods compared to MM-KG. “

\sim

”: partial. “Link”: patient observations linked to a standard vocabulary; “Prov”: per-observation provenance; “Q-Ret”: question-conditioned retrieval.

Figure 2: Overview of MM-KG. Harmonizers convert each record into typed, provenance-linked observations; an aligner links them to PrimeKG concepts; query-conditioned retrieval supplies a subgraph to an LLM/GNN.

Family	A	B	C	D	Interaction
contra-report	0.496 (.59)	0.519 (.61)	0.546 (.57)	0.751 (.82)	+0.182 ( +0.228)
contra-labs	0.507 (.61)	0.516 (.65)	0.560 (.52)	0.636 (.68)	+0.068 ( +0.121)
mechanism	0.524 (.52)	0.510 (.55)	0.526 (.53)	0.616 (.67)	+0.103 ( +0.118)
drug-effect	0.475 (.37)	0.449 (.31)	0.494 (.49)	0.600 (.88)	+0.132 ( +0.463)
All ( $n{=}3{,}200$ )	0.501 (.52)	0.498 (.51)	0.532 (.53)	0.651 (.72)	+0.122 ( +0.209)
ADNI paired contraindication, test set ( $n{=}70$ )

Table 2: Factorial results in both cohorts: balanced accuracy (chance 0.500) with AUROC in parentheses. A : conventional record; B :

+

patient modalities; C :

+

complete KG list; D : both. Interaction:

I_{M}

Figure 3: Patient–KG interaction across question families, controls, language models, and cohorts: AUROC

I_{M}

with 95% CI; marker area

\propto\sqrt{n}

(Table 18 ).

	Full ( $n{=}1{,}244$ )		Id.-hard ( $n{=}598$ )
Method	Hits@1	MRR	Hits@1
MM-KG (condition D)	0.575	0.738	0.582
GraphCare-BAT adapted	0.609	0.764	0.375
(patient side removed)	0.645	0.787	0.381
MindMap-PrimeKG	0.444	0.646	0.448

Table 3: External methods on held-out five-candidate ranking (MIMIC; chance Hits@1 0.2). “Identity-hard”: items a candidate-only prior ranks wrongly.

	MIMIC (Hits@1)	ADNI (AUROC)
Correct alignment (D)	0.575	0.860
Answer-bearing edge removed	0.299	0.714
Matched control packet a	0.237	0.548
Shuffled packets	0.185	0.531
Node names without relations	—	0.558
No knowledge (A)	0.293	0.558

Table 4: Correspondence disruption. MIMIC: five-candidate ranking (

n{=}1{,}244

); ADNI: test set (

n{=}70

). a on MIMIC: another candidate’s length-matched packet; on ADNI: degree- and relation-matched null packet.

Figure 4: Retrieval accuracy against context cost on the held-out MIMIC split: test AUROC (95% CI) vs. median input tokens.

Channel	LLM (zero-shot)	GNN (trained)
	( $\Delta$ )AUROC	( $\Delta$ )AUROC
Full model	$0.733$	$0.804$
$-$ Laboratory	$-0.049^{***}$	$-0.085^{***}$
$-$ Prior-visit history	$-0.024^{***}$	$-0.005^{***}$
$-$ Medications	$-0.011^{***}$	$-0.001$ n.s.
$-$ Radiology report	$-0.011^{***}$	$+0.001$ n.s.

Table 5: ICU mortality ablations:

\Delta

AUROC when each channel is removed (ablated

-

full). LLM: 28,582 stays; GNN: 5,676-stay test split.

{}^{***}p{<}0.001

{}^{*}p{<}0.05

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Patient-side type	Nodes	PrimeKG type	Nodes
Patient	51,844	Disease	17,080
Admission	219,720	Effect/phenotype	15,311
CXR study	204,016	Drug	7,957
Finding	28,714	Exposure	818
Procedure	9,348
Drug / diagnosis	5,908 / 84

Appendix

Table 6: MIMIC–PrimeKG graph composition. The two sides are connected by 13,352 typed alignment edges.

Family (hops)	A	B	C	C shuf	C corr	$\Delta\!\uparrow$
Contraindication (1)	0.537	0.608	0.958	0.525	0.500	$+0.420$
Shared drug target (2)	0.575	0.571	0.975	0.504	0.500	$+0.400$
Mechanistic overlap (2)	0.592	0.617	0.946	0.533	0.500	$+0.355$
Drug side effect (1)	0.721	0.742	0.983	0.533	0.529	$+0.262$
Drug class / ATC (1)	0.667	0.658	0.871	0.604	0.508	$+0.203$
Three-hop chain (3)	0.604	0.592	0.787	0.525	0.500	$+0.184$

Appendix

Table 7: Biomedical fact-checking with complete retrieval (

n{=}1{,}920

balanced items). A : question only; B : entity names; C : retrieved KG edges; C shuf : same-length neighborhood from another item; C corr : true edge structure with corrupted names.

\Delta

is C

-

A; all

p<0.001

	Qwen3.5-35B		Qwen3.5-9B		Llama-3.1-8B
Family	prose	triples	prose	triples	prose
contra-report	$+0.228$	$+0.208$	$+0.138$	$+0.142$	$+0.025^{\dagger}$
contra-labs	$+0.121$	$+0.101$	$+0.063$	$+0.010^{\dagger}$	$+0.046$
drug-effect	$+0.463$	$+0.471$	$+0.513$	$+0.479$	$+0.524$
mechanism	$+0.118$	$+0.185$	$+0.168$	$+0.110$	$+0.084$
All	$+0.209$	$+0.220$	$+0.201$	$+0.153$	$+0.092$

Appendix

Table 8: Robustness of the join interaction (AUROC) across language models and KG renderings.

{}^{\dagger}p\geq 0.05

; all other entries

p<0.05

Leave-one-out (KG on)				Occultness split
Family	$-$ labs	$-$ report	$-$ image	Subset ( $n$ )	C	Interaction
contra-labs	$+0.090^{*}$	$+0.004$	$+0.000$	Strictly occult (654)	0.483	$+0.126^{*}$
contra-report	$-0.017$	$+0.162^{*}$	$+0.005$	Readable in codes (146)	0.783	$+0.040$ n.s.
mechanism	$-0.022$	$+0.087^{*}$	$-0.026^{*}$
drug-effect	$-0.119^{*}$	$+0.087^{*}$	$-0.010$

Appendix

Table 9: Mechanism checks. Left: loss in balanced accuracy after dropping one patient channel with the KG on. Right: the AUROC interaction appears mainly when the state is absent from coded text.

{}^{*}p<0.05

, n.s. not significant.

Model	A	B	C	D	Interaction [95% CI]
Qwen2.5-7B-Instruct	0.491	0.563	0.479	0.679	$+0.1285$ $[+0.0185,+0.2372]$
Qwen3.5-9B	0.5636	0.5534	0.5506	0.8590	$+0.3186$ $[+0.1806,+0.4539]$

Appendix

Table 10: ADNI paired contraindication benchmark across language models. Both models are evaluated on the same 114-item validation bank using the same item keys, arm definitions, KG packets, and non-KG inputs. Interactions are

I_{M}

with participant-clustered 95% confidence intervals.

Condition	AUROC	Participant-clustered 95% CI
A: base	0.5584	$[0.4050,0.7151]$
B: patient only	0.5706	$[0.4016,0.7305]$
C: knowledge only	0.5494	$[0.3946,0.6988]$
D: patient + dynamic KG	0.8604	$[0.7433,0.9557]$
Shuffled KG	0.5314	$[0.3949,0.6741]$
Degree/relation null	0.5478	$[0.3979,0.6933]$

Appendix

Table 11: ADNI paired contraindication test-set results (

n=70

items; 35 positive–negative pairs from 39 participants). AUROC intervals are participant-clustered 95% confidence intervals. The factorial interaction is

I_{M}=+0.2988

(95% CI

[+0.0650,+0.5319]

; predeclared

p=0.0135

Contrast	$\Delta$ AUROC	95% CI	Holm $p$
D $-$ shuffled KG	$+0.3290$	$[+0.1484,+0.4887]$	0.0339
D $-$ degree/relation null	$+0.3127$	$[+0.1742,+0.4445]$	0.0290
D $-$ decisive support removed	$+0.1461$	$[+0.0789,+0.2192]$	$0.0002$
D $-$ node names only	$+0.3020$	$[+0.1637,+0.4534]$	0.0200

Appendix

Table 12: Destructive-control contrasts on the ADNI test set. Differences are D minus each control; Holm adjustment covers the four prespecified destructive-control comparisons.

Method	AUROC
MM-KG (condition D) a	0.8590
MindMap v2	0.9674
Direct PrimeKG lookup	0.9561
GraphCare GIN adapted	0.9437
GraphCare BAT adapted	0.8122
Candidate-only prior	0.5000

Appendix

Table 13: Exploratory external-method comparison on the ADNI paired contraindication benchmark validation bank (

n=114

). Methods use their method-specific evidence construction; the direct PrimeKG lookup is the primary no-model comparator. a MM-KG condition D is the previously frozen Qwen3.5-9B patient-plus-KG result on the same validation bank; its complete candidate KG packet differs from the evidence representations used by MindMap and GraphCare, so the row is shown for within-bank context rather than as a controlled architecture-only comparison.

Relation family	Hops	Closed book	Static context	Query loop	Gold path
Drug–protein	1	0.760	0.370	0.920	100%
Protein–disease	1	0.570	0.400	0.970	100%
Disease–disease	1	0.820	0.290	0.980	100%
Drug–protein–disease	2	0.450	0.430	0.670	100%
Drug–protein–disease–disease	3	0.470	0.450	0.590	90%

Appendix

Table 14: Balanced accuracy for three graph-access regimes in the ADNI-derived hop ladder. Gold-path presence is measured in the static packet.

Policy	Configuration	Input	Path recall	AUROC [95% CI]	$\Delta$	Holm $p$	Failures (retrieval / use)
R5	Gold-path oracle	660	0.997	0.917 $[0.906,0.927]$	$+0.186$	$<0.001$	4 / 251
R4	MM-KG	1,206	0.997	0.731 $[0.710,0.750]$	ref.	—	4 / 442
R2	SapBERT, KG-only, 8K	8,157	0.746	0.712 $[0.691,0.733]$	$-0.019$	0.010	305 / 263
MindMap	evidence + yes-token	4,448	0.726	0.686 $[0.665,0.708]$	$-0.044$	$<0.001$	329 / 401
R2	SapBERT, multi-query, 8K	8,157	0.649	0.669 $[0.647,0.691]$	$-0.062$	$<0.001$	421 / 216
R3	Typed one-hop, 32K	32,353	0.747	0.655 $[0.633,0.677]$	$-0.076$	$<0.001$	304 / 373

Appendix

Table 15: Retrieval-policy results on the retrospective MIMIC test split (

n{=}2{,}398

items; 1,079 patients). R0–R5 are the frozen retrieval policies; MindMap-PrimeKG is included as an external comparator evaluated on the same test items. Input is the median number of model-input tokens. Path recall is the fraction of the 1,199 positive items for which the gold patient evidence is visible under the frozen patient-side caps and every edge of at least one gold answer path is retrieved.

\Delta

and Holm-adjusted

p

values compare each policy with R4. The final column gives retrieval/use failures among the 1,199 positive items.

Figure 6: All sixteen frozen retrieval policies and two MindMap comparator configurations, ordered by the proportion of positive items resolved. Each bar decomposes the 1,199 positive items of the frozen MIMIC test split into the answer path never being retrieved, the path being present while the model still answers negatively, and the remainder, resolved correctly, ordering rows by the share of the 1,199 positive items each policy resolves correctly rather than by AUROC. The two orderings are not equivalent: the 32K full-admissible policy resolves more positive items than the strongest vector policy despite a lower AUROC, because this decomposition counts only positive items at the decision threshold whereas AUROC is threshold-free and also reflects performance on the negatives.

Group	Condition / policy	Hits@1 [95% CI]	$\Delta$ vs. D [95% CI]	Holm $p$	MRR	Identity-hard
Factorial	A: neither channel	0.293 $[0.269,0.319]$	$-0.281$ $[-0.318,-0.245]$	$<0.001$	0.539	0.268
Factorial	B: patient only	0.304 $[0.277,0.330]$	$-0.271$ $[-0.307,-0.236]$	$<0.001$	0.555	0.276
Factorial	C: PrimeKG only	0.278 $[0.254,0.304]$	$-0.297$ $[-0.328,-0.265]$	$<0.001$	0.523	0.296
Factorial	D: patient + PrimeKG (R4q)	0.575 $[0.548,0.603]$	ref.	—	0.738	0.582
Disruption	Gold edge removed	0.299 $[0.273,0.323]$	$-0.276$ $[-0.305,-0.250]$	$<0.001$	0.540	0.279
Disruption	Matched wrong packet	0.237 $[0.213,0.261]$	$-0.338$ $[-0.371,-0.303]$	$<0.001$	0.484	0.253

Appendix

Table 16: Frozen candidate-ranking results on MIMIC (

n{=}1{,}244

independent patients; chance Hits@1 0.200). MM-KG uses the per-candidate query-conditioned R4q formulation. Marginal Hits@1 intervals and paired

\Delta

intervals use 2,000 bootstrap resamples of the 1,244 independent patients; Holm-adjusted

p

values compare each row with condition D. The identity-hard subset (

n{=}598

) contains items for which a candidate-only prior based on training-bank drug frequencies ranks the wrong drug. Structured-explanation diagnostics were evaluated separately under the earlier five-packet rendering and are not mixed with the R4q results here.

Condition	Hits@1 [95% CI]	Hits@3	MRR	Candidate AUROC
A: neither channel	0.342 $[0.297,0.390]$	0.726	0.567	0.581
B: patient only	0.250 $[0.208,0.292]$	0.766	0.520	0.606
C: PrimeKG only	0.274 $[0.226,0.318]$	0.747	0.532	0.564
D: patient + PrimeKG	0.408 $[0.361,0.458]$	0.818	0.626	0.672

Appendix

Table 17: Strict-matching d8 sensitivity (

n{=}380

; chance Hits@1 0.200). The ordering is preserved, but the D–A comparison is not conclusive at this sample size (

p{=}0.060

Analysis	$n$	Interaction	95% CI	Source
Primary
Pooled, all items	3,200	$+0.209^{***}$	$[+0.191,+0.227]$	§ 5.1
Drug-controlled	1,895	$\mathbf{+0.194}^{***}$	$[+0.170,+0.217]$	§ 5.1
Leave-one-family-out c	2,400	$+0.179^{***}$	$[+0.160,+0.197]$	§ 5.1
Behaviorally corroborated contraindications d	152	$+0.145^{***}$	$[+0.074,+0.224]$	App. G
Behaviorally contradicted contraindications d	580	$+0.150^{***}$	$[+0.111,+0.190]$	App. G

Appendix

Table 18: The join interaction across every subset, model, rendering and cohort tested. All entries are AUROC interactions

I_{M}

as defined in § 3.5 ; positive means the two evidence channels are superadditive.

{}^{***}p<0.001

;

{}^{*}p<0.05

; n.s. not significant. a § 5.1 reports the interaction as positive and significant for every family on both metrics without printing per-family

p

values. b Per the Table 8 caption, every entry without a dagger satisfies

p<0.05

. c Excluding the drug-effect family. d Behavioral subsets retain a positive item when its answer-bearing contraindication edge belongs to the independently defined prescribing-behavior partition and retain that item’s same-admission matched negative. The corroborated subset contains 76 positive–negative pairs; the contradicted subset contains 290 pairs.

Condition	AUROC $\uparrow$	Evidence available or disrupted
A: base	0.5636	Neither patient nor KG
B: patient only	0.5534	Patient record, no KG relation
C: PrimeKG only	0.5506	KG relation, no patient condition
D: patient + PrimeKG	0.8590	Both aligned channels
Shuffled PrimeKG	0.5131	Wrong candidate packet
Degree/relation-matched null	0.5260	Nuisance structure without true content

Appendix

Table 19: ADNI validation set (

n{=}114

items; 57 matched pairs from 61 participants). A–D form the factorial design; remaining rows disrupt the evidence. Rows are point estimates; the participant-clustered interaction is

+0.3186

(95% CI

[+0.1806,+0.4539]

Endpoint	Patient representation	$+$ targeted KG	$\Delta$ AUROC [95% CI]
Amyloid status	0.777	0.773	$-0.004$ $[-0.027,+0.018]$
Tau status	0.708	0.687	$-0.021$ $[-0.058,+0.017]$

Appendix

Table 20: Targeted APOE-relation augmentation does not improve ADNI biomarker prediction. Intervals are 95% confidence intervals for the AUROC change.

Block	Content (exact header)	A	B	C	D
demo	“A {age}-year-old {sex} patient in intensive care.”	✓	✓	✓	✓
codes	“Coded diagnoses on record: …” (prior admissions; $\leq 25$ names), or “No coded diagnoses on record.”	✓	✓	✓	✓
meds	“Current medications ({n}): …” (first 48 h; $\leq 40$ names)	✓	✓	✓	✓
labs	“Laboratory values, first 24 hours: creatinine {min–max}, BUN …” (up to 17 analytes; legacy header, see below)		✓		✓
report	“Radiology report for the chest film describes: …” ( $\leq 20$ UMLS finding names)		✓		✓
imaging	“Chest radiograph, automated image analysis reports: …” (BiomedCLIP findings)		✓		✓

Appendix

Table 21: Prompt blocks included in factorial conditions A–D.

Explore similar work

Apr 29, 2026cs.AI

Unifying biomedical knowledge in a modern multimodal graph

Biomedical knowledge graphs (KGs) are widely used in the life sciences, yet many are derived from unstructured documents and therefore lack schema-level constraints, whereas graphs assembled from structured resources are difficult to harmonize into a unified representation. We present OptimusKG, a multimodal biomedical labeled property graph (LPG) built from structured and semi-structured resources to preserve factual, type-specific metadata across molecular, anatomical, clinical, and environmental domains. OptimusKG contains 190,939 nodes across 10 entity types, 21,818,752 edges across 27 edge types, and 67,070,490 property instances encoding 109,665,797 values across 145 distinct property keys, derived from 18 ontologies and controlled vocabularies. The graph enforces a top-level schema for nodes and edges and retains granular, type-specific properties, cross-references, and provenance. We assessed the validity of OptimusKG by evaluating whether graph relationships are supported by evidence from the scientific literature using a multimodal agent, PaperQA3. PaperQA3 identified supporting evidence for 70.0% of sampled edges, whereas 83.4% of sampled false edges received no supporting evidence. Edges without literature support were concentrated in associations derived from experimental and functional genomics resources, suggesting that OptimusKG captures biomedical knowledge that may precede synthesis in the scientific literature. OptimusKG is distributed as Apache Parquet files, providing a standardized resource for graph-based machine learning, knowledge-grounded retrieval with large language models, and biomedical discovery use cases such as hypothesis generation.

Lucas Vittor, Ayush Noori, Iñaki Arango +5

Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA · Department of Engineering Science, University of Oxford, Oxford, UK · Edison Scientific Inc., San Francisco, CA, USA +4

Biomedical Knowledge Graph Biomedical Graph Foundation Model Graph Representations Metadata

Ask PDF

May 21, 2026cs.CL

ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning

Biomedical knowledge graphs (KGs) treat disease associations as static facts, but temporal information is crucial for clinical reasoning, e.g., a symptom diagnostic of one disease at age 3 may imply a different disease at age 13. Existing KGs such as PrimeKG, Hetionet, and iKraph do not encode when a finding becomes clinically relevant over the course of a disease. This limits their usefulness for longitudinal clinical reasoning and retrieval augmentation. We introduce ChronoMedKG, a temporal biomedical knowledge graph that contains 460,497 evidence-linked triples (filtered from 13M raw extractions) covering 13,431 diseases. Each association is tied to temporal components like onset window or progression stage, which are backed by PMID-traceable evidence and a multi-signal credibility score. The graph is constructed through a disease-autonomous multi-agent pipeline in which multiple frontier LLMs independently extract knowledge from PubMed and PMC literature. Only those relations are kept that are supported by multi-model consensus, survive credibility filtering, as well as ontology alignment. ChronoMedKG scored 92.7% agreement against Orphadata and adds temporal grounding for 6,250 diseases absent from HPOA, Orphadata, and Phenopackets, including 1,657 Orphanet-coded rare diseases. We further introduce ChronoTQA, a benchmark of 3,341 questions across eight task types (six temporal plus two static controls), with a 12-question supplementary probe. Frontier LLMs lose roughly 30 points moving from static to temporal questions; ChronoMedKG retrieval rescues 47-65% of their long-tail failures, against 17-29% for HPOA-RAG. As such, ChronoMedKG provides a crucial temporal axis for retrieval-augmented clinical systems that was previously absent.

Md Shamim Ahmed, Farzaneh Firoozbakht, Lukas Galke Poech +2

University of Southern Denmark · University of Hamburg

Biomedical Knowledge Graph Clinical Reasoning Training Longitudinal Patient Records

Ask PDF

May 11, 2026cs.LG

CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs

Biomedical knowledge graphs are increasingly large, dynamic, and multimodal, driven by rapid advances in biotechnology such as high-throughput sequencing. Machine learning models can infer previously unobserved biomedical relationships and characterize biomedical entities in these graphs, but existing knowledge graph embedding methods and their continual learning extensions either assume static graph structure or fail to exploit multimodal information under evolving data distributions. They also apply uniform regularization across all model parameters, ignoring that different modalities may exhibit distinct forgetting dynamics as the graph evolves. We propose the Continual Multimodal Knowledge Graph Learner (CMKL), a CL framework for biomedical KGs that natively encodes structure, text, and molecules, fuses them through a Mixture-of-Experts (MoE) router, and protects previously learned knowledge with standard EWC regularization and a K-means-diverse multimodal replay buffer. We evaluate CMKL on a 129K-entity biomedical continual benchmark with 10 tasks. On continual biomedical entity classification, CMKL reaches AP 0.591 versus 0.370 for the strongest structural baseline, a 60% gain that is driven by access to multimodal features and preserved across the sequence with near-zero forgetting (AF 0.008). On continual relationship prediction, CMKL reaches AP

0.062

, matching Naive Sequential and EWC (0.058) within seed noise and outperforming Joint Training (0.047, p=0.045) and LKGE (0.039). A frozen-text ablation reaches AP 0.136, more than double any jointly trained model, yet that signal is unreachable by margin-ranking gradients: the greedy-modality asymmetry lives at the representation level, not the fusion level, and MoE routing manages it by suppressing the unreachable modality without forcing it through a learned bottleneck. Code: github.com/yradwan147/cmkl-neurips2026

Yousef A. Radwan, Yao Li, Qing Qing +5

Technology, Innovation, Entrepreneurship Department King Abdullah University of Science and Technology · School of Computing and Information Systems The University of Melbourne · College of Computer Science and Technology Jilin University +2

Knowledge Graph Embeddings Biomedical Knowledge Graph Medical Ontology Modalities

Ask PDF