GPromptShield: Elevating Resilience in Graph Prompt Tuning against Adversarial Attacks
Authors: Shuhan Song, Ping Li, Ming Dun, Maolei Huang, Huawei Cao, Xiaochun Ye
Organizations: State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Fliggy, Alibaba Group · Zhongguancun Laboratory, Beijing, China
The paradigm of ``pre-training and prompt-tuning", with its effectiveness and lightweight characteristics, has rapidly spread from the language field to the graph field. Several pioneering studies have designed specialized prompt functions for diverse downstream graph tasks based on various graph pre-training strategies. These prompts concentrate on the compatibility between the pre-training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly to adapt to downstream tasks based on this concept neglects crucial security issues. By conducting covert attacks on downstream graph data, we find that even when the downstream task data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the former directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph's feature space, while the latter adopts a training strategy to bypass the invalid components and amplifies valid part. We provide a theoretical derivation that proves their feasibility, indicating that unbiased prompts exist under certain conditions on unreliable data. Extensive experiments across various scenarios of adversarial attacks indicate that the prompts within our defense system exhibit enhanced resilience and superiority. This paper explores a new perspective in graph prompt learning, offering a novel option for robust prompt tuning in downstream tasks.
Figures & tables
Figure 1
Figure 3: The performance of existing prompt functions on clean graphs and graphs attacked by MetaAttack with a 20% attack ratio. The scale of the radar chart represents classification accuracy.
Figure 4: Robust Prompt Optimization Workflow for Hybrid Multi-Defense Prompt.
Pre-Training Strategies
Prompts
Cora
Citeseer
CoraML
Clean
M-0.25
D-0.5
R-0.5
H-0.5
Clean
M-0.25
D-0.5
R-0.5
H-0.5
Clean
M-0.25
D-0.5
R-0.5
H-0.5
GraphPrompt
57.68
34.06
36.01
42.68
35.19
67.20
40.22
34.24
34.29
31.68
67.17
41.96
36.75
44.68
44.32
MultiGPrompt
48.34
39.37
32.97
43.71
29.52
50.37
41.99
38.09
42.31
48.34
56.24
36.99
39.75
46.68
26.90
GraphCL
All-in-one
50.93
31.02
17.91
34.29
28.34
43.96
37.18
37.29
27.72
21.31
37.43
28.38
26.50
35.35
27.46
GPF
54.06
33.83
27.62
29.43
15.92
56.46
38.78
44.07
14.37
22.12
41.95
30.26
28.58
39.31
24.46
GPF-plus
66.39
36.83
30.52
42.22
26.53
66.19
45.30
38.68
31.68
14.37
71.30
35.43
38.59
30.74
26.30
Table 1: Performance of Different Prompts in 5-shot scenario under Non-Adaptive Attacks.
Pre-Training Strategies
Prompts
CoraML
Citeseer
GCNSVD
GRAND
GNNGuard
GCNJaccard
GCNSVD
GRAND
GNNGuard
GCNJaccard
GraphPrompt
47.66
35.24
37.64
41.90
48.72
34.99
40.71
38.25
MultiGPrompt
24.22
30.20
15.56
14.97
25.16
25.96
31.41
33.55
GraphCL
ProG
37.32
31.02
52.78
32.38
23.93
30.56
33.92
31.14
GPF
30.88
23.08
27.39
35.42
29.59
21.85
18.43
25.53
GPF-plus
52.29
28.93
41.41
38.37
24.95
21.85
24.95
18.38
Table 2: Performance of Different Prompts in 5-shot scenario under Adaptive Attacks.
Ablation Variants
Cora 5-shot
Citeseer 5-shot
Cora 10-shot
Citeseer 10-shot
Meta 0.25
DICE 0.5
Meta 0.25
DICE 0.5
Meta 0.25
DICE 0.5
Meta 0.25
DICE 0.5
MD-PT
60.12%
54.55
64.20%
52.39
71.02%
62.30%
67.04%
55.35
MD-PT w/o pd
62.31%
53.05%
63.68%
51.76%
67.87%
61.36%
66.67%
56.31%
MD-PT w/o ps
58.46%
52.02%
58.97%
53.21%
69.92%
59.48%
61.57%
54.39%
MD-PT w/o po
59.32%
55.37%
60.42%
51.98%
69.09%
60.99%
65.69%
55.18%
MD-PT w/o Lkl
59.86%
55.28%
60.42%
52.14%
69.83%
60.81%
64.42%
54.23%
Table 3: Performance of Hybrid Multi-Defense Prompt by Ablating Different Modules.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Datasets
NLCC
ELCC
Features
Classes
Citeseer
2110
3668
3703
6
Cora
2485
5069
1433
7
Cora-ML
2810
7981
2879
7
Appendix
Table 4: Details of the largest connected component (LCC) for each dataset.
Datasets
Graphs
Nodes
Edges
Features
Classes
Task
Catagory
Cora
1
2708
5429
1433
7
N / L
Homophilic
Pubmed
1
19717
88648
2879
3
N / L
Homophilic
Citeseer
1
3327
9104
3703
6
N / L
Homophilic
Wisconsin
1
251
515
1703
5
N / L
Heterophilic
ogbn-arxiv
1
169343
1166243
128
40
N / L
Homophilic & Large scale
Appendix
Table 5: Statistics of all datasets for node classification and link prediction tasks.
Datasets
Graphs
Avg. Nodes
Avg. Edges
Features
Classes
Task
Catagory
MUTAG
188
17.9
19.8
7
2
G
small molecule
COLLAB
5000
74.5
2457.8
0
3
G
social network
PROTEINS
1113
39.1
72.8
3
2
G
proteins
IMDB-BINARY
1000
19.8
96.53
0
2
G
social network
Appendix
Table 6: Statistics of all datasets for graph classification tasks.
Pre-Training Strategies
Prompts
Cora
Citeseer
CoraML
Clean
M-0.25
D-0.5
R-0.5
H-0.5
Clean
M-0.25
D-0.5
R-0.5
H-0.5
Clean
M-0.25
D-0.5
R-0.5
H-0.5
GraphPrompt
72.15
40.48
39.83
52.30
39.79
73.44
49.21
30.84
41.73
32.57
64.80
42.21
39.01
41.08
39.78
MultiGPrompt
52.53
46.88
41.08
49.22
33.44
50.24
47.59
38.16
42.84
45.47
48.99
37.31
33.86
46.76
24.98
GraphCL
All-in-one
69.78
45.26
14.03
11.64
18.95
67.05
42.17
34.04
45.75
17.40
49.68
27.09
19.10
32,08
24.21
GPF
60.99
35.65
35.05
39.37
29.85
73.66
41.63
42.17
37.40
21.90
44.48
15.98
30.45
34.71
24.49
GPF-plus
70.61
41.63
37.21
51.01
36.89
69.16
42.38
42.60
45.58
26.02
65.69
34.83
36.01
24.70
31.63
Appendix
Table 7: Performance of Different Prompts in 10-shot scenario under Non-Adaptive Attacks.
Pre-Training Strategies
Prompts
CoraML
Citeseer
GCNSVD
GRAND
GNNGuard
GCNJaccard
GCNSVD
GRAND
GNNGuard
GCNJaccard
GraphPrompt
53.77
38.78
51.01
35.60
55.99
32.25
45.64
53.77
MultiGPrompt
29.99
32.38
19.09
13.25
31.49
25.04
34.20
33.39
GraphCL
All-in-one
40.48
25.44
36.71
21.90
41.25
33.06
37.13
42.33
GPF
40.80
43.79
16.65
26.91
38.59
37.78
18.92
26.29
GPF-plus
38.96
36.02
45.77
40.39
45.47
22.49
53.17
22.49
Appendix
Table 8: Performance of Different Prompts in 10-shot scenario under Adaptive Attacks.
Table 9: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on 5-shot Node Classification and Link Prediction Tasks.
Table 10: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on 10-shot Node Classification and Link Prediction Tasks.
Table 11: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on Graph Classification Tasks.
Prompts
Attacks
Cora
Citeseer
CoraML
M-0.25
D-0.5
R-0.5
H-0.5
M-0.25
D-0.5
R-0.5
H-0.5
M-0.25
D-0.5
R-0.5
H-0.5
GPF
M-0.25
31.44
32.47
20.23
26.26
20.26
20.85
32.80
35.79
28.99
32.07
33.07
30.42
D-0.5
33.97
23.38
28.21
32.70
36.22
25.37
25.37
20.75
36.75
33.79
34.79
31.71
R-0.5
36.92
36.37
36.24
34.37
25.37
21.96
21.96
21.96
34.63
34.47
39.31
39.95
H-0.5
29.16
35.85
29.30
29.02
31.26
25.37
20.15
19.86
31.39
21.74
34.19
32.54
GPF-plus
M-0.25
35.56
35.51
34.92
36.33
25.37
24.46
38.76
40.48
33.15
35.67
35.15
36.75
Appendix
Table 12: Performance of Proposed Prompts and GPF/GPF-plus on Different Attack Transfer Scenarios.
Graph Neural Networks (GNNs) have emerged as a cornerstone for representing complex relational dependencies in diverse multimedia tasks, particularly in cross-platform user interest modeling and cross-modal semantic alignment. In the real world, a practical defense against graph adversarial perturbations is needed. However, we observe that the prevailing adversarial purification methods are essentially domain-restricted defenses, which leads to the following shortcomings: (1) single-domain data provides insufficient structural and semantic diversity for learning robust purification criteria; (2) training of domain-specific defense strategies from scratch consumes substantial computational cost. To address the above limitations, we propose a transferable graph purification scheme, named ProGAP, to bridge adversarial defense knowledge via vulnerability-aware graph prompt learning. Firstly, to capture universal adversarial patterns, a perturbation-capture edge detector is pretrained on data-rich graphs by jointly modeling topological and semantic information. Subsequently, to achieve more knowledge transfer w.r.t. robustness, vulnerability-aware prompts are designed that inject targeted purification guidance into biased nodes, during which the pretrained detector adapts to distribution shifts in downstream graphs without parameter-laborious updates. Experimental results demonstrate that compared with state-of-the-art baselines, our ProGAP achieves 1%-9% improvement, and reduces the time consumption by up to 2.2x. The code for ProGAP is available at https://github.com/Lieyoufffff/ProGAP.
Shuomin Xue, Jingyuan Li, Ju Jia +2
Southeast University Nanjing, China · Nanyang Technological University Singapore, Singapore
Prompt tuning has become a key mechanism for adapting pre-trained Graph Neural Networks (GNNs) to new downstream tasks. However, existing approaches are predominantly supervised, relying on labeled data to optimize the prompting parameters and typically fine-tuning a task-specific prediction head -- practices that undermine the promise of parameter-efficient adaptation. We propose Unsupervised Graph Prompting Problem (UGPP), a challenging new setting where the pre-trained GNN is kept entirely frozen, labels on the target domain are unavailable, the source data is inaccessible, and the target distribution exhibits covariate shift. To address this, we propose UGPrompt, the first fully unsupervised GNN prompting framework. UGPrompt leverages consistency regularization and pseudo-labeling to train a prompting function, complemented with diversity and domain regularization to mitigate class imbalance and distribution mismatch. Our extensive experiments demonstrate that UGPrompt consistently outperforms state-of-the-art supervised prompting methods with access to labeled data, demonstrating the viability of unsupervised prompting as a practical adaptation paradigm for GNNs.
Peyman Baghershahi, Sourav Medya
Department of Computer Science University of Illinois Chicago
Graph Data Prompt (GDP), which introduces specific prompts in graph data for efficiently adapting pre-trained GNNs, has become a mainstream approach to graph fine-tuning learning problem. However, existing GDPs have been respectively designed for distinct graph component (e.g., node features, edge features, edge weights) and thus operate within limited prompt spaces for graph data. To the best of our knowledge, it still lacks a unified prompter suitable for targeting all graph components simultaneously. To address this challenge, in this paper, we first propose to reinterpret a wide range of existing GDPs from an aspect of Graph Message Prompt (GMP) paradigm. Based on GMP, we then introduce a novel graph prompt learning approach, termed Low-Rank GMP (LR-GMP), which leverages low-rank prompt representation to achieve an effective and compact graph prompt learning. Unlike traditional GDPs that target distinct graph components separately, LR-GMP concurrently performs prompting on all graph components in a unified manner, thereby achieving significantly superior generalization and robustness on diverse downstream tasks. Extensive experiments on several graph benchmark datasets demonstrate the effectiveness and advantages of our proposed LR-GMP.
Beibei Wang, Bo Jiang, Ziyan Zhang +1
School of Computer Science and Technology, Anhui University, Hefei 230601, China. · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, School of Computer Science and Technology, Anhui University, Hefei 230601, China.