DeKoR: Decomposed Knowledge Routing for Efficient GNN--LLM Fusion

Peiyuan Li; Yongqi Huang; Jitao Zhao; Di Jin; Dongxiao He

cs.LGOct 10, 2026

DeKoR: Decomposed Knowledge Routing for Efficient GNN--LLM Fusion

Authors: Peiyuan Li, Yongqi Huang, Jitao Zhao, Di Jin, Dongxiao He

Organizations: School of Computer Science and Technology, Tianjin University, Tianjin, China

Abstract

Large Language Models (LLMs) have shown strong potential for text-attributed graph learning by leveraging rich pre-trained knowledge and language understanding capabilities. Combining LLMs with Graph Neural Networks (GNNs) offers a promising paradigm for expanding the modeling capabilities of graph learning. However, LLMs and GNNs follow distinct learning paradigms, leading to a pronounced mismatch between their knowledge spaces. Such discrepancies make effective knowledge fusion nontrivial. Moreover, existing methods often incur substantial additional computation for only limited performance gains. To address these challenges, we propose DeKoR, an efficient framework for GNN and LLM knowledge fusion. To enable efficient knowledge transfer, DeKoR decomposes LLM knowledge into complementary local and global views and uses the LLM as an offline teacher to distill them into lightweight experts. To enable selective knowledge fusion, an adaptive router learns how much each node should rely on each expert. Extensive experiments on multiple benchmarks demonstrate that DeKoR outperforms existing baselines on both node and subgraph classification, achieving superior effectiveness and efficiency.

Figures & tables

Figure 1: UMAP visualizations of GCN, GCNII, and Qwen-8B on PubMed and arXiv.

Dataset	GCN vs. GCNII	GCN vs. Qwen-8B	GCNII vs. Qwen-8B
Cora	$0.05$	$0.26$	$\mathbf{0.29}$
CiteSeer	$0.05$	$\mathbf{0.24}$	$0.21$
PubMed	$0.14$	$\mathbf{0.33}$	$0.26$
WikiCS	$0.19$	$\mathbf{0.37}$	$0.30$
arXiv	$0.20$	$\mathbf{0.38}$	$0.37$
Average	$0.13$	$\mathbf{0.32}$	$0.29$

Table 1: Representation gaps among GCN, GCNII, and Qwen-8B across five datasets.

Figure 2: Overall framework of DeKoR. LLM semantic knowledge is decomposed into complementary global and local views and distilled into lightweight experts. During routing, a node-adaptive router dynamically combines the two experts for final prediction.

Category	Method	Cora	CiteSeer	PubMed	WikiCS	arXiv
GNNs	GCN ICLR’17	$87.40\pm 0.96$	$74.95\pm 1.38$	$88.70\pm 0.35$	$82.25\pm 0.36$	$66.57\pm 0.45$
	GAT ICLR’18	$87.99\pm 0.47$	$75.83\pm 0.96$	$87.51\pm 0.77$	$82.71\pm 0.67$	$65.64\pm 0.99$
	GraphSAGE NeurIPS’17	$88.53\pm 1.26$	$76.87\pm 1.34$	$90.60\pm 0.43$	$84.01\pm 0.45$	$68.40\pm 0.63$
	GIN ICLR’19	$86.46\pm 0.87$	$73.65\pm 1.32$	$84.85\pm 1.06$	$61.08\pm 1.51$	$55.05\pm 2.70$
GTs	DIFFormer ICLR’23	$87.15\pm 1.41$	$77.12\pm 0.69$	$91.37\pm 0.27$	$83.29\pm 0.72$	$70.43\pm 0.21$
	NodeFormer NeurIPS’22	$86.71\pm 1.82$	$76.91\pm 1.03$	$91.12\pm 0.24$	$80.44\pm 0.29$	$48.25\pm 0.85$

Table 2: Accuracy(%) of the node classification task across five datasets. Best and runner-up methods are bolded and underlined , respectively.

Category	Method	Cora	CiteSeer	PubMed	WikiCS	arXiv
GNNs	GCN ICLR’17	$87.30\pm 0.59$	$74.74\pm 0.84$	$84.14\pm 0.75$	$79.37\pm 0.93$	$67.67\pm 0.49$
	GAT ICLR’18	$86.95\pm 0.45$	$74.24\pm 1.47$	$84.08\pm 0.92$	$79.06\pm 0.85$	$67.75\pm 0.56$
	GraphSAGE NeurIPS’17	$87.25\pm 0.60$	$74.99\pm 1.17$	$83.77\pm 0.68$	$79.49\pm 0.55$	$68.20\pm 0.42$
	GIN ICLR’19	$86.46\pm 1.07$	$74.49\pm 0.69$	$84.17\pm 0.38$	$78.23\pm 0.98$	$66.20\pm 0.31$
GTs	DIFFormer ICLR’23	$87.30\pm 0.77$	$75.74\pm 1.70$	$84.10\pm 0.49$	$79.36\pm 0.72$	$67.82\pm 0.61$
	NodeFormer NeurIPS’22	$87.54\pm 0.39$	$73.99\pm 0.92$	$83.76\pm 1.06$	$75.47\pm 1.93$	$66.33\pm 0.20$

Table 3: Accuracy(%) of the subgraph classification task across five datasets. Best and runner-up methods are bolded and underlined , respectively.

Backbone	Avg. Accuracy
GCN	$82.09\pm 1.25$
GraphSAGE	$78.02\pm 4.53$
GCNII	$82.86\pm 1.22$
GCN (Ours)	$83.99\pm 1.46$
GraphSAGE (Ours)	$85.97\pm 2.05$
GCNII (Ours)	$85.78\pm 1.18$

Table 4: Accuracy(%) of DeKoR with different GNN and LLM backbones on node classification.

Figure 3: Ablation studies of key components in DeKoR.

Figure 4: Routing weights of local and global semantic experts grouped by expert classification correctness on WikiCS.

Method	Time/Memory	Cora	CiteSeer	PubMed	WikiCS	arXiv	Avg. Accuracy(%)
UniGraph	Time	$1363594.3$	$680603.5$	$775438.3$	$1172503.5$	$2426704.3$	$72.84$
UniGraph	Memory	$14363$	$14368$	$14338$	$14367$	$14596$	$72.84$
OFA	Time	$55285.8$	$54932.2$	$69428.5$	$70333.5$	$65201.4$	$80.03$
OFA	Memory	$13027$	$13027$	$13028$	$13026$	$13027$	$80.03$
GraphGPT	Time	$768910.8$	$495775.1$	$471579.6$	$1548980.4$	$5966955.7$	$68.20$
GraphGPT	Memory	$15351$	$15359$	$16213$	$16253$	$15821$	$68.20$

Table 5: Inference time (ms) and GPU memory usage (MiB) across five datasets.

Figure 5: Sensitivity analysis of the number of prototypes

K

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	Original Feature Dim.	#Classes
Cora	2,708	5,429	1,433	7
CiteSeer	3,186	4,277	3,703	6
PubMed	19,717	44,335	500	3
WikiCS	11,701	215,863	300	10
arXiv	169,343	1,166,243	128	40

Appendix

Table 6: Summary statistics of the datasets used in our experiments.

Text-Attributed Graph Learning

Text-attributed graph learning combines node text and graph structure to improve classification, retrieval, anomaly detection, and reasoning.

See all 66 papers

Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
Jun 10, 2026Zhuoyi Peng, Hanlin Gu, Lixin Fan +1Graph Representation Learning Text-Attributed Graph Learning
1The Hong Kong University of Science and Technology · 2WeBank
PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning
Jun 29, 2026Zhifei Hu, Alexandra I. CristeaText-Attributed Graph Learning Multimodal Fusion
Durham University, Durham, United Kingdom
LoReC: Rethinking Large Language Models for Graph Data Analysis
Apr 20, 2026Hongyu Zhan, Qixin Wang, Yusen Tan +6Graph Representation Learning LLMs for Graph Learning
The Hong Kong University of Science and Technology (Guangzhou) · Tongji University · Westlake University +2

Jun 10, 2026J/K move · Enter open · S save

Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching

Zhuoyi Peng, Hanlin Gu, Lixin Fan, Yi Yang

1The Hong Kong University of Science and Technology · 2WeBank

Graph Representation Learning Text-Attributed Graph Learning Pseudo-Labeling LLMs for Graph Learning

Open paper AskPDF arXiv

Text-attributed graphs (TAGs) underlie real-world applications such as citation networks, social media, and e-commerce. Few-shot graph learning on TAGs is hard: with only a handful of labels per class and the rest of the graph unannotated, neither GNNs nor LLMs can learn well on their own. GNNs read topology and fail on cold nodes; LLMs read text and fail on text-ambiguous nodes. Existing LLM-GNN methods all follow the same recipe: designate one model as the golden teacher and use its outputs (e.g., features or pseudo-labels) to supervise the other. We argue this golden-teacher assumption breaks under sparse supervision: neither model is golden, and treating either as such transfers its blind spots into the student. We therefore ask: can we avoid designating either model as the golden teacher, and still perform effective graph learning? We answer with LLM-GNN Co-Teaching, a bidirectional co-teaching framework in which neither model is fixed as teacher. The GNN and LLM exchange their most confident pseudo-labels under an architecture-specific small-loss criterion, and both update every round. Supervision is then mined from the trajectory: whenever a node moves from cross-model contradiction at round t to cross-model agreement at round t+1, the LLM's two answers on the same input form a preference pair (old contradicting self < new peer-endorsed self) for DPO training. We call this Round-based Pseudo-Label Preference Optimization (RPL-PO). On six benchmarks, LLM-GNN Co-Teaching consistently outperforms GNN-as-Judge and all prior methods, with absolute 3-shot gains of 7.86% on Cora and 7.73% on ogbn-arxiv; improvements carry over to 5-shot and to zero-shot cross-dataset transfer. Error-structure analysis further shows that abandoning the golden-teacher assumption substantially improves the LLM's graph learning capability on challenging samples.

Abstract

Text-Attributed Graph Learning

Explore similar work

Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching

PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning

LoReC: Rethinking Large Language Models for Graph Data Analysis