Disentangled Graph Prompting for Out-Of-Distribution Detection
Authors: Cheng Yang, Yu Hao, Qi Zhang, Chuan Shi
Organizations: the Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications, China · China Mobile Group Shaanxi Co., Ltd.
When testing data and training data come from different distributions, deep neural networks (DNNs) will face significant safety risks in practical applications. Therefore, out-of-distribution (OOD) detection techniques, which can identify OOD samples at test time and alert the system, are urgently needed. Existing graph OOD detection methods usually characterize fine-grained in-distribution (ID) patterns from multiple perspectives, and train end-to-end graph neural networks (GNNs) for prediction. However, due to the unavailability of OOD data during training, the absence of explicit supervision signals could lead to sub-optimal performance of end-to-end encoders. To address this issue, we follow the pre-training+prompting paradigm to utilize pre-trained GNN encoders, and propose Disentangled Graph Prompting (DGP), to capture fine-grained ID patterns with the help of ID graph labels. Specifically, we design two prompt generators that respectively generate class-specific and class-agnostic prompt graphs by modifying the edge weights of an input graph. We also design several effective losses to train the prompt generators and prevent trivial solutions. We conduct extensive experiments on ten datasets to demonstrate the superiority of our proposed DGP, which achieves a relative AUC improvement of 3.63% over the best graph OOD detection baseline. Ablation studies and hyper-parameter experiments further show the effectiveness of DGP. Code is available at https://github.com/BUPT-GAMMA/DGP.
Figures & tables
Fig. 1: An illustration of our Disentangled Graph Prompting (DGP) method. DGP can take advantage of pre-trained GNN encoders, and generate prompt graphs from both class-specific and class-agnostic views for fine-grained ID pattern mining. The parameters of GNN encoders are frozen at the prompting stage.
Method
Fine-grained modeling
No OOD exposure
Pre-trained encoder utilized
OCGIN
GLocalKD
✓
GraphDE
✓
✓
GOOD-D
✓
✓
AAGOD
✓
✓
DGP
✓
✓
✓
Table I: Comparison among graph OOD detection work.
Fig. 2: An illustration of the training process of our proposed DGP. For each input graph, we generate two prompt graphs to capture ID patterns from class-specific and class-agnostic views, respectively. We design several losses to supervise and regularize the training of prompt generators. Here the parameters of pre-trained GNN encoder are frozen in our method.
ID-dataset
BZR
PTC_MR
AIDS
ENZYMES
IMDB-M
Tox21
FreeSolv
BBBP
ClinTox
Esol
Avg.
OOD-dataset
COX2
MUTAG
DHFR
PROTEIN
IMDB-B
SIDER
ToxCast
BACE
LIPO
MUV
NegLabel
41.18
56.61
34.79
54.53
58.38
51.65
45.59
46.85
44.97
48.81
48.34
AdaNeg
47.69
54.06
72.15
58.90
49.72
43.25
49.82
42.42
55.68
27.08
50.08
Local-Prompt
73.09
59.90
68.61
43.09
76.14
25.00
25.20
44.30
43.30
10.00
46.86
PFSOOD
64.08
61.57
10.06
50.84
72.29
50.45
69.76
46.01
42.49
47.12
51.47
PRO
52.05
40.86
86.64
55.71
37.52
46.00
36.00
44.00
57.00
24.00
47.98
Table II: OOD detection results in terms of AUC(%). The best results are highlighted in boldface, and the second-best results are underlined.
Fig. 3: Decision score distributions of ID and OOD graphs on ten datasets. Smaller distribution overlap indicates better detection ability. The two images in the same column respectively represent the pre-trained (left), pre-trained+prompted right.
Fig. 4: Graph OOD detection results (AUC values) of different ablated variants and GNN encoder initialization strategies.
Fig. 5: Graph OOD detection results (AUC values) on BZR-COX2 and PTC_MR-MUTAG with respect to four hyper-parameters and different MLP layers.
Fig. 6: Running time compared with other SOTA models.
Fig. 7: Visualizations of two types of prompt graphs as case study analysis. Here darker edges indicate larger learned weights. The two images in the same column respectively represent the class-specific prompt graph (top) and the class-agnostic prompt graph (bottom).
Figure 10Figure 11Figure 12Figure 13
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
ID-dataset
BZR
PTC_MR
AIDS
ENZYMES
IMDB-M
Tox21
FreeSolv
BBBP
ClinTox
Esol
Avg.
OOD-dataset
COX2
MUTAG
DHFR
PROTEIN
IMDB-B
SIDER
ToxCast
BACE
LIPO
MUV
NegLabel
45.35
35.24
20.16
67.87
43.26
14.76
51.05
37.09
67.80
57.94
44.05
AdaNeg
61.85
58.41
64.49
55.64
49.67
49.45
49.96
47.41
56.78
49.16
54.28
Local-Prompt
91.21
75.34
76.85
50.60
72.02
46.43
45.56
54.14
53.91
26.30
59.24
PFSOOD
64.92
63.14
35.28
54.94
69.87
50.26
61.78
48.51
46.13
49.77
54.46
PRO
45.47
41.12
74.70
54.14
47.56
44.98
40.10
47.91
58.86
36.13
49.10
Appendix
Table III: OOD detection results in terms of AUPR(%). The best results are highlighted in boldface, and the second-best results are underlined.
ID-dataset
BZR
PTC_MR
AIDS
ENZYMES
IMDB-M
Tox21
FreeSolv
BBBP
ClinTox
Esol
Avg.
OOD-dataset
COX2
MUTAG
DHFR
PROTEIN
IMDB-B
SIDER
ToxCast
BACE
LIPO
MUV
NegLabel
87.65
65.99
97.70
94.67
94.93
94.97
91.59
91.52
89.10
81.38
88.95
AdaNeg
57.41
72.82
84.45
95.50
92.87
91.29
95.79
86.81
78.67
93.00
84.86
Local-Prompt
88.89
100.00
95.45
100.00
87.86
100.00
100.00
100.00
100.00
100.00
97.22
PFSOOD
54.17
90.00
96.10
88.39
84.00
92.31
97.79
98.03
99.76
99.51
90.01
PRO
73.17
91.43
25.50
98.33
99.00
100.00
100.00
100.00
90.00
100.00
87.74
Appendix
Table IV: OOD detection results in terms of FPR95(%). The best results are highlighted in boldface, and the second-best results are underlined.
College of Artificial Intelligence, Dalian Maritime University · Computer Vision Center, Universitat Aut`onoma de Barcelona · School of Computing Technologies, RMIT University +1