Organizations: College of Information Science and Engineering, Ningbo University, Ningbo, 315211, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, CB3 0FD, UK · School of Artificial Intelligence, Jilin University, Changchun, 130015, China
In recent years, Graph Neural Networks (GNNs) and architecture search frameworks have gained extensive application in non-Euclidean data processing, attributable to their superior capacity in managing unstructured data. Nevertheless, traditional approaches typically apply uniform convolution operations to all nodes, regardless of their varying structural and feature characteristics, which can undermine model performance and result in over-smoothing issues as the number of layers increases. To overcome this limitation, in this work, we propose a \textbf{N}ode-Level \textbf{G}raph \textbf{N}eural \textbf{A}rchitecture \textbf{S}earch (N-GNAS) algorithm. It can automatically choose an appropriate network architecture for each subset of nodes when updating node features. N-GNAS also introduces a contrastive learning loss to separate sample features from different categories and vice versa. In experiments conducted on eight datasets for node and graph classification, our methodology outperforms current leading GNAS techniques and traditional human-designed GNNs. For example, it achieves an accuracy rate of 78.26% on the CiteSeer dataset.
Figures & tables
Figure 1: An example of a graph neural network discovered by N-GNAS. Part I illustrates an L -layer architecture. Part II demonstrates the search space across three stages, and Part III provides an example of search outcomes. Each layer comprises two stages: node selection operation search and graph neural network operation search, denoted by the symbols S and G with subscripts reflecting their diversity. Every layer comprises an input cell, an output cell, and operation cells. In this example, seven cells and five operation are used in the first and the second stage respectively. The variable X denotes the respective features obtained from these cells, with subscripts distinguishing stages and superscripts defining layers and cells. Links indicate possible operations.
Def.
Operation
fG1
GCN [ 18 ]
fG2
GAT [ 35 ]
fG3
GraphSAGE [ 16 ]
fG4
GIN [ 41 ]
fG5
GatedGCN [ 2 ]
Table 1: Some popular graph neural network operation within our search space. Def. denotes different graph neural network operations. Operation denotes the corresponding concrete graph neural network operation respectively.
Figure 2: The specific process of N-GNAS in graph classification task. Select Node refers to the process of searching for node selection operations. In contrast, Search GNNs indicates the process of searching for graph neural network operations. A Readout operation is required for the result of each layer, and the Readout result is used as the output of the current layer.
Task
Dataset
Graphs
Nodes
Edges
Features
Classes
Node Classification
Cora
-
2708
10556
1433
7
Citeseer
-
3327
9104
3703
6
PubMed
-
19717
88648
500
3
Graph Classification
D&D
1178
384.3
715.7
89
2
PROTEINS
1113
39.1
72.8
3
2
IMDB-MULTI
1500
13
65.9
0
3
Table 2: Statistics of the datasets used in the experiments. For node classification tasks, the terms ”Nodes” and ”Edges” correspond to the cumulative count of nodes and all edges within the datasets. Conversely, ”Nodes” and ”Edges” signify the mean number of nodes and edges across the graphs for graph classification tasks.
Methods
Cora
CiteSeer
PubMed
GCN [ 18 ]
86.09±0.50
74.64±0.20
88.96±0.29
GIN [ 41 ]
85.68±0.61
73.40±0.14
88.23±0.28
GraphSAGE [ 16 ]
85.66±0.52
74.59±0.63
89.21±0.29
GAT [ 35 ]
85.92±0.72
74.26±0.13
88.67±0.19
SGC [ 40 ]
85.31±0.86
72.94±0.98
88.40±0.25
PNA [ 7 ]
85.06±0.72
75.06±0.61
87.18±0.30
Table 3: Node classification task accuracy on the Cora, CiteSeer, and PubMed datasets. The top three are emphasized by first , second , and third.
Methods
D&D
PROTEINS
IMDB-MULTI
COX2
MR
GCN [ 18 ]
76.98±4.43
72.94±1.82
50.25±3.42
78.68±1.92
75.62±0.85
GIN [ 41 ]
73.95±2.98
73.68±2.78
50.04±2.75
80.52±3.41
76.05±0.74
GraphSAGE [ 16 ]
76.78±4.06
72.58±2.43
49.62±4.58
79.63±2.63
76.85±0.63
GAT [ 35 ]
75.14±2.84
74.29±1.69
49.85±3.65
81.16±3.64
76.92±1.02
manually crafted
DGCNN [ 48 ]
76.66±4.03
73.28±3.16
49.63±3.73
80.16±3.23
77.06±0.68
GraphNAS [ 14 ]
73.56±2.47
73.12±4.27
47.23±4.59
78.91±2.37
76.37±2.17
Table 4: Graph classification task accuracy on the D&D, PROTEINS, IMDB-MULTI, COX2, and MR datasets. The top three are emphasized by first , second , and third.
Figure 3: Results of various methods on the Cora, CiteSeer, and PubMed datasets. Different colors represent different methods. It can be found that our work achieves the best results on both datasets
Figure 4: Performance of the model with different numbers of layers. Different colors represent different methods. It can be found that our work achieves the best results on both datasets
Figure 5: Node selection results obtained by the first and the second layer of graph neural networks optimized with N-GNAS. Red nodes (gate value >0.5 ) are strongly selected for GNN processing, while green nodes (gate value ≤0.5 ) mostly bypass GNN operations via residual connections.
Cora
Layer
Stage II
N-GNAS
1
76.93±0.49
78.51±1.13
2
86.36±1.12
88.59±0.63
3
84.43±0.71
87.92±0.68
4
82.71±0.83
85.43±0.66
5
79.57±0.55
83.76±0.47
Table 5: Performance of N-GNAS with varying layers on the Cora dataset. We present the test accuracy results for N-GNAS without Node Selection Operations and for N-GNAS with these operations.
Methods
Cora
CiteSeer
PubMed
Random
85.74
74.63
88.62
Lce
86.94
76.52
87.98
N-GNAS
88.59
78.26
90.27
Table 6: Ablation study on search space and loss function.
Graph neural networks have achieved strong performance in node classification by aggregating information from graph neighborhoods. However, a single aggregation mechanism may be insufficient to capture diverse structural patterns across graph datasets. Moreover, independently training multiple structural branches can introduce substantial overhead without necessarily producing stable node representations. To address these issues, this paper proposes PreGS, a parameter-transfer-based multi-expert graph neural network framework. PreGS first pretrains a multi-head graph attention network (GAT) and transfers the linear transformation weights of its first-layer attention heads to multiple GraphSAGE experts. The transferred experts are frozen and used as complementary structural branches. The fused raw node features, GAT head representations, and GraphSAGE expert representations are fed into a multilayer perceptron (MLP), whose output is further fused with the pretrained GAT logits. Based on PreGS, we further develop PreGSv2, which introduces source-level weighting and a structural gating mechanism for adaptive multi-source feature integration. Experiments on eight public graph datasets show that PreGS and PreGSv2 achieve competitive performance against representative graph neural network baselines. Ablation, parameter-transfer, sensitivity, aggregator, visualization, and training-time analyses further validate the effectiveness and stability of the proposed framework. The code and datasets are available at https://github.com/LH-Czc/PreGS.
Zhicong Cai, Yinglong Zhang, Xiaoying Hong +2
College of Physics and Information Engineering, Minnan Normal University, Zhangzhou 363000, China
Hypergraph neural networks have achieved significant success in recent years. However, manual architecture crafting is labor-intensive and often fails to capture complex, higher-order relations, making the automation of hypergraph neural network structure design crucial. To improve the automation and adaptability of hypergraph learning, this paper proposes AutoHGNN, a neural architecture search framework tailored for hypergraph neural networks. First, we introduce a Hyper-Interaction Module (HIM) into the search space to address the mismatch between conventional graph neural network designs and hypergraph data. Second, we propose Hypergraph Stable Topological Distance (HyperSTD) as a structural selection criterion to identify architectures that best preserve the intrinsic structural affinities of the original hypergraph during differentiable search. Extensive experiments on various benchmark datasets demonstrate that AutoHGNN consistently outperforms manually designed and automatically searched baselines in classification accuracy and time efficiency, proving that the discovered architectures are significantly more effective.
Sirui Li, Pietro Liò b, Xinsheng Li +2
Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, 315211, Zhejiang, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, CB2 1TN, Cambridgeshire, United Kingdom
The reasoning process of Graph Neural Networks is complex and considered opaque, limiting trust in their predictions. To alleviate this issue, prior work has proposed concept-based explanations, extracted from clusters in the model's node embeddings. However, a limitation of concept-based explanations is that they only explain the node embedding space and are obscured by pooling in graph classification. To mitigate this issue and provide a deeper level of understanding, we propose the Subgraph Concept Network. The Subgraph Concept Network is the first graph neural network architecture that distils subgraph and graph-level concepts. It achieves this by performing soft clustering on node concept embeddings to derive subgraph and graph-level concepts. Our results show that the Subgraph Concept Network allows to obtain competitive model accuracy, while discovering meaningful concepts at different levels of the network.
Lucie Charlotte Magister, Alexander Norcliffe, Iulia Duta +1