Organizations: College of Information Science and Engineering, Ningbo University, Ningbo, 315211, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, CB3 0FD, UK · School of Artificial Intelligence, Jilin University, Changchun, 130015, China
In recent years, Graph Neural Networks (GNNs) and architecture search frameworks have gained extensive application in non-Euclidean data processing, attributable to their superior capacity in managing unstructured data. Nevertheless, traditional approaches typically apply uniform convolution operations to all nodes, regardless of their varying structural and feature characteristics, which can undermine model performance and result in over-smoothing issues as the number of layers increases. To overcome this limitation, in this work, we propose a \textbf{N}ode-Level \textbf{G}raph \textbf{N}eural \textbf{A}rchitecture \textbf{S}earch (N-GNAS) algorithm. It can automatically choose an appropriate network architecture for each subset of nodes when updating node features. N-GNAS also introduces a contrastive learning loss to separate sample features from different categories and vice versa. In experiments conducted on eight datasets for node and graph classification, our methodology outperforms current leading GNAS techniques and traditional human-designed GNNs. For example, it achieves an accuracy rate of 78.26% on the CiteSeer dataset.
Figures & tables
Figure 1: An example of a graph neural network discovered by N-GNAS. Part I illustrates an L -layer architecture. Part II demonstrates the search space across three stages, and Part III provides an example of search outcomes. Each layer comprises two stages: node selection operation search and graph neural network operation search, denoted by the symbols S and G with subscripts reflecting their diversity. Every layer comprises an input cell, an output cell, and operation cells. In this example, seven cells and five operation are used in the first and the second stage respectively. The variable X denotes the respective features obtained from these cells, with subscripts distinguishing stages and superscripts defining layers and cells. Links indicate possible operations.
Def.
Operation
fG1
GCN [ 18 ]
fG2
GAT [ 35 ]
fG3
GraphSAGE [ 16 ]
fG4
GIN [ 41 ]
fG5
GatedGCN [ 2 ]
Table 1: Some popular graph neural network operation within our search space. Def. denotes different graph neural network operations. Operation denotes the corresponding concrete graph neural network operation respectively.
Figure 2: The specific process of N-GNAS in graph classification task. Select Node refers to the process of searching for node selection operations. In contrast, Search GNNs indicates the process of searching for graph neural network operations. A Readout operation is required for the result of each layer, and the Readout result is used as the output of the current layer.
Task
Dataset
Graphs
Nodes
Edges
Features
Classes
Node Classification
Cora
-
2708
10556
1433
7
Citeseer
-
3327
9104
3703
6
PubMed
-
19717
88648
500
3
Graph Classification
D&D
1178
384.3
715.7
89
2
PROTEINS
1113
39.1
72.8
3
2
IMDB-MULTI
1500
13
65.9
0
3
Table 2: Statistics of the datasets used in the experiments. For node classification tasks, the terms ”Nodes” and ”Edges” correspond to the cumulative count of nodes and all edges within the datasets. Conversely, ”Nodes” and ”Edges” signify the mean number of nodes and edges across the graphs for graph classification tasks.
Methods
Cora
CiteSeer
PubMed
GCN [ 18 ]
86.09±0.50
74.64±0.20
88.96±0.29
GIN [ 41 ]
85.68±0.61
73.40±0.14
88.23±0.28
GraphSAGE [ 16 ]
85.66±0.52
74.59±0.63
89.21±0.29
GAT [ 35 ]
85.92±0.72
74.26±0.13
88.67±0.19
SGC [ 40 ]
85.31±0.86
72.94±0.98
88.40±0.25
PNA [ 7 ]
85.06±0.72
75.06±0.61
87.18±0.30
Table 3: Node classification task accuracy on the Cora, CiteSeer, and PubMed datasets. The top three are emphasized by first , second , and third.
Methods
D&D
PROTEINS
IMDB-MULTI
COX2
MR
GCN [ 18 ]
76.98±4.43
72.94±1.82
50.25±3.42
78.68±1.92
75.62±0.85
GIN [ 41 ]
73.95±2.98
73.68±2.78
50.04±2.75
80.52±3.41
76.05±0.74
GraphSAGE [ 16 ]
76.78±4.06
72.58±2.43
49.62±4.58
79.63±2.63
76.85±0.63
GAT [ 35 ]
75.14±2.84
74.29±1.69
49.85±3.65
81.16±3.64
76.92±1.02
manually crafted
DGCNN [ 48 ]
76.66±4.03
73.28±3.16
49.63±3.73
80.16±3.23
77.06±0.68
GraphNAS [ 14 ]
73.56±2.47
73.12±4.27
47.23±4.59
78.91±2.37
76.37±2.17
Table 4: Graph classification task accuracy on the D&D, PROTEINS, IMDB-MULTI, COX2, and MR datasets. The top three are emphasized by first , second , and third.
Figure 3: Results of various methods on the Cora, CiteSeer, and PubMed datasets. Different colors represent different methods. It can be found that our work achieves the best results on both datasets
Figure 4: Performance of the model with different numbers of layers. Different colors represent different methods. It can be found that our work achieves the best results on both datasets
Figure 5: Node selection results obtained by the first and the second layer of graph neural networks optimized with N-GNAS. Red nodes (gate value >0.5 ) are strongly selected for GNN processing, while green nodes (gate value ≤0.5 ) mostly bypass GNN operations via residual connections.
Cora
Layer
Stage II
N-GNAS
1
76.93±0.49
78.51±1.13
2
86.36±1.12
88.59±0.63
3
84.43±0.71
87.92±0.68
4
82.71±0.83
85.43±0.66
5
79.57±0.55
83.76±0.47
Table 5: Performance of N-GNAS with varying layers on the Cora dataset. We present the test accuracy results for N-GNAS without Node Selection Operations and for N-GNAS with these operations.
Methods
Cora
CiteSeer
PubMed
Random
85.74
74.63
88.62
Lce
86.94
76.52
87.98
N-GNAS
88.59
78.26
90.27
Table 6: Ablation study on search space and loss function.
Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, 315211, Zhejiang, China · Department of Computer Science and Technology, University of Cambridge, Cambridge, CB2 1TN, Cambridgeshire, United Kingdom