Organizations: Jiajun Zhou, Yadong Li, Xuanze Chen, Chen Ma, Shanqing Yu and Qi Xuan are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China, and with the Soovar Technologies Co., Ltd., Hangzhou 310056, China. · Chuang Zhao is with the Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China.
Mixture-of-Experts (MoE) architectures offer a scalable path for Graph Neural Networks (GNNs) in node classification tasks but typically rely on static and rigid routing strategies that enforce a uniform expert budget or coarse-grained expert toggles on all nodes. This limitation overlooks the varying discriminative difficulty of nodes and leads to under-fitting for hard nodes and redundant computation for easy ones. To resolve this issue, we propose D2MoE, a novel framework that shifts the focus from static expert selection to node-wise expert resource allocation. By using predictive entropy as a real-time proxy for difficulty, D2MoE employs a difficulty-driven top-p routing mechanism to adaptively concentrate expert resources on hard nodes while reducing overhead for easy ones, achieving continuous and fine-grained expert budget scaling for node classification. Experiments on 13 benchmarks demonstrate that D2MoE achieves consistent state-of-the-art performance, surpassing leading baselines by up to 7.92% in accuracy on heterophilous graphs. Notably, on large-scale graphs, it reduces memory consumption by up to 73.07% and training time by 46.53% compared to the best-performing Graph MoE, thereby validating its superior efficiency.
Figures & tables
Fig. 1 : Impact of static expert budgets (top- k ) on node classification performance across predictive entropy intervals.
Fig. 2 : The architecture of D 2 MoE. The complete workflow proceeds as follows: (1) D 2 MoE first quantifies node-wise difficulty via predictive entropy; (2) Top- p router adaptively scales the expert resources, activating a sparse set of experts for easy nodes while mobilizing a comprehensive ensemble for hard nodes; (3) Selected expert outputs are fused to update node representations, which generate predictions and initiate the next difficulty quantification.
TABLE I : Node classification results: average test accuracy (%) ± standard deviation. Boldface letters mark the best performance while underlined letters indicate the second best.
Fig. 3 : Fine-grained performance comparison across different node difficulty levels. We stratify test nodes into 10 equal-sized intervals (deciles) based on their predictive entropy (calculated by a fixed proxy model). The x-axis represents entropy percentiles (from Easy to Hard), and the y-axis denotes classification accuracy. D 2 MoE demonstrates superior robustness, particularly in high-entropy (hard) intervals.
Fig. 4 : Ablation analysis on D 2 MoE.
Fig. 5 : Node density distribution and average number of activated experts across varying levels of discriminative difficulty.
Fig. 6 : Visualization of learned node representations via t-SNE across 12 datasets. For each dataset, nodes are colored by Ground-Truth Labels (Top) and Activated Expert Counts (Bottom). The results highlight a clear boundary-focused resource allocation pattern.
Fig. 7 : Visualization of expert activation patterns across 12 datasets. The heatmaps illustrate the average routing weights assigned to different experts (Y-axis) across four quartiles of node predictive entropy (X-axis, ranging from low to high). Brighter colors indicate higher activation weights.
Fig. 8 : Efficiency analysis on large-scale graphs.
Fig. 9 : All results for Ogbn-arxiv: (a) Visualization of average routing weights; (b) Label distribution vs. expert activation count distribution; (c) Performance comparison across node difficulty levels; (d) Ablation analysis on D 2 MoE; (e) Node density distribution and average number of acti- vated experts across varying levels of discriminative difficulty.