cs.LGSep 27, 2026

ZeroGAR: Benchmarking the Adversarial Robustness of Zero-Shot Graph Models

Authors: Zhongjian Zhang, Xiao Wang, Busheng Zhang, Bo Yan, Xingtong Yu, Yue Gao, Jia Li, Chuan Shi

Organizations: Beijing University of Posts and Telecommunications · Beihang University · Tsinghua University · The Chinese University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Zero-shot graph models (ZGMs), which learn transferable knowledge from source graphs and directly apply to unseen target graphs without any adaptation, have achieved promising performance and attracted considerable attention. Despite their proliferation, existing ZGMs are predominantly evaluated on clean graphs, while existing graph robustness benchmarks mainly focus on supervised settings, leaving a fundamental question largely unexplored: How robust are ZGMs when their unseen target graphs are exposed to adversarial manipulation? In this paper, we answer this question by proposing ZeroGAR, the first systematic benchmark for evaluating the adversarial robustness of ZGMs. ZeroGAR evaluates 13 representative ZGMs from 3 different paradigms on 8 graph datasets across 4 domains, covering both in-domain and cross-domain transfer under structural, textual, and node injection attacks with multiple perturbation budgets. It further investigates whether existing graph defenses remain effective in the zero-shot setting. Extensive experiments reveal that strong clean zero-shot performance does not guarantee adversarial robustness, with three key findings: (1) Vulnerability patterns are related to model prediction mechanisms: GNN-based methods are particularly vulnerable to structural and node injection attacks, whereas LLM-based methods are more vulnerable to textual attacks; (2) Stronger LLM backbones introduce a structure-text robustness trade-off; (3) Existing graph defense methods do not consistently improve zero-shot robustness and may compromise clean performance. We hope that ZeroGAR will facilitate rapid, equitable evaluation and inspire further innovative research in ZGM security.

Figures & tables

Figure 1: Overview of 3 paradigms of ZGMs.
BenchmarkData domainsAttack typeVictim modelZero-shot setting
StructureTextualNode injectionGNNPredGTextLLMGTokenLLM
Zheng et al.20212✓✗✓✗✗✗✗
Gosch et al.20234✓✗✗✗✗✗✗
Guo et al.20243✓✓✗✗✓✗✗
Zhang et al.2025a2✓✓✗✗✗✓✗
Olatunji et al.20251✓✓✗✗✗✓✗
Table 1: Comparison of existing graph robustness benchmarks.
Dataset#Nodes#EdgesAvg. # DegAvg. # Tok#CTrain/Val/Test (%)Node TextDomainUsage
OGBN-Arxiv169,3431,157,79913.77232.614053.7/17.6/28.7Paper contentAcademicGs\mathcal{G}^{s}
OGBN-Products2,449,02961,859,01250.52153.41478.0/1.6/90.4Product descriptionE-commerceGs\mathcal{G}^{s}
Cora2,7085,2783.90189.54760.0/20.0/20.0Paper contentAcademicGt\mathcal{G}^{t}
Citeseer3,1864,2252.65193.11660.0/20.0/20.0Paper contentAcademicGt\mathcal{G}^{t}
History41,551251,59012.11298.971260.0/20.0/20.0Product reviewsE-commerceGt\mathcal{G}^{t}
Child76,8751,162,52230.24283.002460.0/20.0/20.0Product reviewsE-commerceGt\mathcal{G}^{t}
Table 2: Statistics of the source and target graph datasets.
Figure 2: Overview of the ZeroGAR evaluation pipeline.
MethodCoraCiteseerHistoryChildInstagramWikiCSRank
GNNPred
OFA26.0145.305.990.2748.679.9910.67
G2P245.7653.7613.0811.7045.4841.299.00
ZeroG52.0352.6637.5812.8956.5037.636.83
GraphCLIP63.1067.5556.8521.1164.8269.452.67
GTextLLM
Table 3: Clean performance and overall rank comparison across target datasets. Lower ranks are better.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA25.46 ↓\downarrow 2.1124.72 ↓\downarrow 4.9624.17 ↓\downarrow 7.0742.01 ↓\downarrow 7.2640.44 ↓\downarrow 10.7339.50 ↓\downarrow 12.806.63 ↑\uparrow 10.687.26 ↑\uparrow 21.206.82 ↑\uparrow 13.860.23 ↓\downarrow 14.810.18 ↓\downarrow 33.330.20 ↓\downarrow 25.9348.61 ↓\downarrow 0.1247.60 ↓\downarrow 2.2045.61 ↓\downarrow 6.2910.35 ↑\uparrow 3.6010.19 ↑\uparrow 2.0010.12 ↑\uparrow 1.3021.67 ↓\downarrow 2.00
G2P245.57 ↓\downarrow 0.4239.67 ↓\downarrow 13.3138.19 ↓\downarrow 16.5447.34 ↓\downarrow 11.9440.44 ↓\downarrow 24.7838.71 ↓\downarrow 27.9911.80 ↓\downarrow 9.7911.32 ↓\downarrow 13.4611.42 ↓\downarrow 12.6910.81 ↓\downarrow 7.6110.29 ↓\downarrow 12.0510.10 ↓\downarrow 13.6845.47 ↓\downarrow 0.0245.31 ↓\downarrow 0.3742.62 ↓\downarrow 6.2940.50 ↓\downarrow 1.9139.82 ↓\downarrow 3.5639.20 ↓\downarrow 5.0631.59 ↓\downarrow 10.38
ZeroG45.20 ↓\downarrow 13.1333.39 ↓\downarrow 35.8323.62 ↓\downarrow 54.6047.18 ↓\downarrow 10.4141.69 ↓\downarrow 20.8336.99 ↓\downarrow 29.7620.62 ↓\downarrow 45.1317.11 ↓\downarrow 54.4711.83 ↓\downarrow 68.528.18 ↓\downarrow 36.546.69 ↓\downarrow 48.105.71 ↓\downarrow 55.7037.59 ↓\downarrow 33.4731.51 ↓\downarrow 44.2320.10 ↓\downarrow 64.4234.03 ↓\downarrow 9.5728.82 ↓\downarrow 23.4125.57 ↓\downarrow 32.0526.44 ↓\downarrow 40.16
GraphCLIP53.51 ↓\downarrow 15.2046.31 ↓\downarrow 26.6144.10 ↓\downarrow 30.1162.07 ↓\downarrow 8.1161.76 ↓\downarrow 8.5758.93 ↓\downarrow 12.7636.42 ↓\downarrow 35.9434.30 ↓\downarrow 39.6729.00 ↓\downarrow 48.9918.32 ↓\downarrow 13.2217.91 ↓\downarrow 15.1617.71 ↓\downarrow 16.1164.30 ↓\downarrow 0.8064.57 ↓\downarrow 0.3964.43 ↓\downarrow 0.6059.89 ↓\downarrow 13.7755.46 ↓\downarrow 20.1452.85 ↓\downarrow 23.9046.77 ↓\downarrow 20.93
Table 4: Performance comparison under structural attacks across different perturbation sizes. Each cell reports the ACC (top) and RDA (%, bottom). Best scores are bolded and runner-up scores are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA24.91 ↓\downarrow 4.2323.06 ↓\downarrow 11.3423.99 ↓\downarrow 7.7742.32 ↓\downarrow 6.5837.46 ↓\downarrow 17.3128.21 ↓\downarrow 37.735.86 ↓\downarrow 2.175.96 ↓\downarrow 0.505.85 ↓\downarrow 2.340.26 ↓\downarrow 3.700.25 ↓\downarrow 7.410.24 ↓\downarrow 11.1145.70 ↓\downarrow 6.1044.69 ↓\downarrow 8.1841.96 ↓\downarrow 13.799.65 ↓\downarrow 3.409.41 ↓\downarrow 5.819.32 ↓\downarrow 6.7119.95 ↓\downarrow 7.95
G2P245.53 ↓\downarrow 0.5045.02 ↓\downarrow 1.6244.10 ↓\downarrow 3.6351.10 ↓\downarrow 4.9548.75 ↓\downarrow 9.3247.34 ↓\downarrow 11.9412.84 ↓\downarrow 1.8312.57 ↓\downarrow 3.9012.30 ↓\downarrow 5.9611.33 ↓\downarrow 3.1611.23 ↓\downarrow 4.0210.77 ↓\downarrow 7.9544.34 ↓\downarrow 2.5143.68 ↓\downarrow 3.9642.44 ↓\downarrow 6.6841.19 ↓\downarrow 0.2441.05 ↓\downarrow 0.5839.03 ↓\downarrow 5.4733.59 ↓\downarrow 4.40
ZeroG52.01 ↓\downarrow 0.0451.85 ↓\downarrow 0.3551.73 ↓\downarrow 0.5852.04 ↓\downarrow 1.1849.84 ↓\downarrow 5.3649.22 ↓\downarrow 6.5334.87 ↓\downarrow 7.2132.62 ↓\downarrow 13.2028.02 ↓\downarrow 25.4412.74 ↓\downarrow 1.1612.55 ↓\downarrow 2.6412.48 ↓\downarrow 3.1855.44 ↓\downarrow 1.8853.02 ↓\downarrow 6.1651.34 ↓\downarrow 9.1335.54 ↓\downarrow 5.5532.63 ↓\downarrow 13.2927.76 ↓\downarrow 26.2338.65 ↓\downarrow 8.39
GraphCLIP59.78 ↓\downarrow 5.2655.54 ↓\downarrow 11.9848.52 ↓\downarrow 23.1163.79 ↓\downarrow 5.5757.52 ↓\downarrow 14.8544.51 ↓\downarrow 34.1155.69 ↓\downarrow 2.0454.55 ↓\downarrow 4.0550.86 ↓\downarrow 10.5419.49 ↓\downarrow 7.6718.06 ↓\downarrow 14.4514.30 ↓\downarrow 32.2663.46 ↓\downarrow 2.1063.42 ↓\downarrow 2.1662.01 ↓\downarrow 4.3464.73 ↓\downarrow 6.8058.77 ↓\downarrow 15.3844.24 ↓\downarrow 36.3049.96 ↓\downarrow 12.38
Table 5: Performance comparison under textual attacks across different perturbation sizes. Each cell reports the ACC (top) and RDA (%, bottom). Best scores are bolded and runner-up scores are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA25.83 ↓\downarrow 0.6925.38 ↓\downarrow 2.4225.01 ↓\downarrow 3.8443.73 ↓\downarrow 3.4743.10 ↓\downarrow 4.8640.60 ↓\downarrow 10.385.95 ↓\downarrow 0.675.72 ↓\downarrow 4.515.05 ↓\downarrow 15.690.12 ↓\downarrow 55.560.12 ↓\downarrow 55.560.12 ↓\downarrow 55.5650.42 ↑\uparrow 3.6051.74 ↑\uparrow 6.3153.55 ↑\uparrow 10.038.64 ↓\downarrow 13.518.36 ↓\downarrow 16.327.82 ↓\downarrow 21.7222.29 ↓\downarrow 13.25
G2P244.28 ↓\downarrow 3.2342.80 ↓\downarrow 6.4740.59 ↓\downarrow 11.3048.28 ↓\downarrow 10.1947.34 ↓\downarrow 11.9444.36 ↓\downarrow 17.4911.14 ↓\downarrow 14.8311.88 ↓\downarrow 9.1712.57 ↓\downarrow 3.909.95 ↓\downarrow 14.969.33 ↓\downarrow 20.267.78 ↓\downarrow 33.5045.29 ↓\downarrow 0.4245.21 ↓\downarrow 0.5945.16 ↓\downarrow 0.7040.74 ↓\downarrow 1.3340.36 ↓\downarrow 2.2538.96 ↓\downarrow 5.6432.56 ↓\downarrow 9.06
ZeroG50.00 ↓\downarrow 3.9048.89 ↓\downarrow 6.0338.75 ↓\downarrow 25.5247.96 ↓\downarrow 8.9340.75 ↓\downarrow 22.6231.50 ↓\downarrow 40.1812.75 ↓\downarrow 66.076.88 ↓\downarrow 81.694.43 ↓\downarrow 88.217.69 ↓\downarrow 40.344.70 ↓\downarrow 63.543.02 ↓\downarrow 76.5741.82 ↓\downarrow 25.9836.23 ↓\downarrow 35.8835.43 ↓\downarrow 37.2933.83 ↓\downarrow 10.1036.00 ↓\downarrow 4.3328.37 ↓\downarrow 24.6128.28 ↓\downarrow 41.59
GraphCLIP60.33 ↓\downarrow 4.3957.38 ↓\downarrow 9.0645.94 ↓\downarrow 27.1965.52 ↓\downarrow 3.0162.23 ↓\downarrow 7.8860.82 ↓\downarrow 9.9646.65 ↓\downarrow 17.9441.41 ↓\downarrow 27.1636.16 ↓\downarrow 36.3920.24 ↓\downarrow 4.1220.46 ↓\downarrow 3.0820.22 ↓\downarrow 4.2262.85 ↓\downarrow 3.0462.98 ↓\downarrow 2.8462.14 ↓\downarrow 4.1363.95 ↓\downarrow 7.9262.48 ↓\downarrow 10.0460.03 ↓\downarrow 13.5650.66 ↓\downarrow 12.97
Table 6: Performance comparison under node injection attacks across different perturbation sizes. Each cell reports the ACC (top) and RDA (%, bottom). Best scores are bolded and runner-up scores are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA7.8011.3512.7711.6615.1917.3119.6817.8722.0938.1045.2445.244.007.6713.2213.4916.0818.6718.75
G2P26.4520.5627.8213.3927.3830.9517.9420.7023.9216.4523.1827.357.547.7812.443.155.758.2316.72
ZeroG16.6741.4959.2212.9523.8033.1346.9757.0669.6440.9752.0258.6839.6352.3576.1312.6327.6336.5742.09
GraphCLIP15.7927.1931.098.648.8813.0839.2442.7751.1732.4535.3536.301.611.611.5316.9022.6025.9822.90
Table 7: Attack success rate (ASR, %) under structural attacks across different perturbation sizes. Lower values indicate stronger robustness. Best results are bolded and runner-up results are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA5.6713.4814.187.2718.3440.487.0310.8421.6911.9023.8130.955.4811.8024.1710.9615.5825.0016.59
G2P22.025.659.685.5410.5013.393.777.8212.886.2810.1216.845.148.3313.084.357.7914.298.75
ZeroG1.774.264.963.578.3312.2010.2818.0931.326.8110.6016.3510.3024.5731.209.9522.7737.0014.68
GraphCLIP5.5612.5724.275.8015.0834.573.015.8413.428.3815.5934.180.550.692.837.2116.0337.2813.49
Table 8: Attack success rate (ASR, %) under textual attacks across different perturbation sizes. Lower values indicate stronger robustness. Best results are bolded and runner-up results are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA3.555.678.517.2712.1117.3024.3030.7240.1654.7657.1457.148.5014.6419.2617.9822.2627.7423.83
G2P28.0614.5219.7612.2416.6221.2823.6427.1431.4628.4637.0851.3614.9216.9624.036.3811.8921.0421.49
ZeroG9.5713.8339.0112.8029.4649.4068.6883.8991.2645.5168.2181.8446.1865.5267.8618.0526.7751.5048.30
GraphCLIP5.8510.8229.535.8011.3715.5524.3233.0641.9726.4427.8330.604.146.0812.5013.1716.5519.7218.63
Table 9: Attack success rate (ASR, %) under node injection attacks across different perturbation sizes. Lower values indicate stronger robustness. Best results are bolded and runner-up results are underlined.
MethodCoraCiteseerHistoryChildInstagramWikiCSAvg.
LowMidHighLowMidHighLowMidHighLowMidHighLowMidHighLowMidHigh
GNNPred
OFA24.72 ↓\downarrow 4.9623.62 ↓\downarrow 9.1923.06 ↓\downarrow 11.3443.89 ↓\downarrow 3.1142.48 ↓\downarrow 6.2338.56 ↓\downarrow 14.886.12 ↑\uparrow 2.176.20 ↑\uparrow 3.516.27 ↑\uparrow 4.670.27 0.000.29 ↑\uparrow 7.410.28 ↑\uparrow 3.7046.76 ↓\downarrow 3.9246.19 ↓\downarrow 5.1045.84 ↓\downarrow 5.819.65 ↓\downarrow 3.409.47 ↓\downarrow 5.218.79 ↓\downarrow 12.0121.25 ↓\downarrow 2.78
G2P245.39 ↓\downarrow 0.8144.65 ↓\downarrow 2.4343.17 ↓\downarrow 5.6651.72 ↓\downarrow 3.7952.51 ↓\downarrow 2.3350.00 ↓\downarrow 6.9912.89 ↓\downarrow 1.4512.83 ↓\downarrow 1.9112.63 ↓\downarrow 3.4411.47 ↓\downarrow 1.9711.29 ↓\downarrow 3.5010.51 ↓\downarrow 10.1745.57 ↑\uparrow 0.2045.48 0.0044.78 ↓\downarrow 1.5441.29 0.0041.06 ↓\downarrow 0.5639.97 ↓\downarrow 3.2034.29 ↓\downarrow 2.75
ZeroG51.29 ↓\downarrow 1.4251.29 ↓\downarrow 1.4251.11 ↓\downarrow 1.7752.35 ↓\downarrow 0.5953.45 ↑\uparrow 1.5053.13 ↑\uparrow 0.8937.25 ↓\downarrow 0.8836.60 ↓\downarrow 2.6136.54 ↓\downarrow 2.7713.03 ↑\uparrow 1.0913.14 ↑\uparrow 1.9411.99 ↓\downarrow 6.9856.54 ↑\uparrow 0.0755.80 ↓\downarrow 1.2455.84 ↓\downarrow 1.1737.64 ↑\uparrow 0.0337.40 ↓\downarrow 0.6137.18 ↓\downarrow 1.2041.20 ↓\downarrow 1.13
GraphCLIP60.70 ↓\downarrow 3.8057.01 ↓\downarrow 9.6553.69 ↓\downarrow 14.9165.05 ↓\downarrow 3.7064.11 ↓\downarrow 5.0961.29 ↓\downarrow 9.2754.12 ↓\downarrow 4.8051.14 ↓\downarrow 10.0444.30 ↓\downarrow 22.0820.16 ↓\downarrow 4.5019.00 ↓\downarrow 10.0016.29 ↓\downarrow 22.8363.82 ↓\downarrow 1.5463.86 ↓\downarrow 1.4863.55 ↓\downarrow 1.9666.84 ↓\downarrow 3.7662.70 ↓\downarrow 9.7253.48 ↓\downarrow 22.9952.28 ↓\downarrow 9.71
Table 10: Performance comparison under TextFooler attacks across different perturbation sizes. Each cell reports the ACC (top) and RDA (%, bottom). Best scores are bolded and runner-up scores are underlined.

Explore similar work

CardsList

  1. Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation

    May 7, 2026Tran Gia Bao Ngo, Zulfikar Alom, Federico Errica +2Graph Neural NetworksGraph Representations

    Department of Computer Science, University of Manitoba · University of Toledo · NEC Laboratories Europe +2

  2. On the Safety of Graph Representation Learning

    May 7, 2026Xiaoguang Guo, Zehong Wang, Ziming Li +5Graph Representation LearningGraph Neural Networks

    University of Connecticut · University of Notre Dame

  3. COPYCOP: Ownership Verification for Graph Neural Networks

    May 6, 2026Rahul Nandakumar, Deepayan ChakrabartiGraph Neural NetworksWatermarking

    McCombs School of Business University of Texas at Austin Austin, TX