Understanding and Mitigating Under-Confidence in GNNs from the Final Layer

Jincheng Huang; Jie Xu; Xiaoshuang Shi; Ping Hu; Lei Feng; Xiaofeng Zhu

cs.LGMay 16, 2025

Understanding and Mitigating Under-Confidence in GNNs from the Final Layer

Authors: Jincheng Huang, Jie Xu, Xiaoshuang Shi, Ping Hu, Lei Feng, Xiaofeng Zhu

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China · Southeast University

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable effectiveness on graph-based tasks. However, their predictive confidence is often miscalibrated, typically exhibiting under-confidence, which harms the reliability of their decisions. Existing calibration methods for GNNs normally introduce additional calibration components, which fail to capture the intrinsic relationship between the model and the prediction confidence, resulting in limited theoretical guarantees and increased computational overhead. To address this issue, we propose a simple yet efficient graph calibration method. We establish a unified theoretical framework revealing that model confidence is jointly governed by class-centroid-level and node-level calibration at the final layer. Based on this insight, we theoretically show that reducing the weight decay of the final-layer parameters alleviates GNN under-confidence by acting on the class-centroid level, while node-level calibration acts as a finer-grained complement to class-centroid-level calibration, which encourages each test node to be closer to its predicted class prototype in the final-layer representations. Extensive experiments validate the superiority of our method. The code is released at https://github.com/huangJC0429/SCAR.

Figures & tables

Datasets	L/C	Uncal.	TS	MS	CaGCN	GATS	AU-LS	DCGC	GETS	SCAR
Datasets	L/C	GCN
Cora	20	13.47 $\pm$ 0.63	4.88 $\pm$ 0.55	4.14 $\pm$ 0.57	4.01 $\pm$ 0.67	3.96 $\pm$ 0.46	4.32 $\pm$ 0.23	3.91 $\pm$ 0.36	3.83 $\pm$ 0.41	3.35 $\pm$ 0.53
	40	11.34 $\pm$ 0.47	4.17 $\pm$ 0.72	3.72 $\pm$ 0.46	4.07 $\pm$ 0.54	3.82 $\pm$ 0.74	3.69 $\pm$ 0.38	3.53 $\pm$ 0.65	3.72 $\pm$ 0.39	2.67 $\pm$ 0.33
	60	9.37 $\pm$ 0.49	3.55 $\pm$ 0.54	3.64 $\pm$ 0.61	3.74 $\pm$ 0.44	3.56 $\pm$ 0.36	4.09 $\pm$ 0.81	3.42 $\pm$ 0.66	3.26 $\pm$ 0.67	2.60 $\pm$ 0.62
Citeseer	20	12.48 $\pm$ 0.71	6.41 $\pm$ 0.87	6.44 $\pm$ 0.37	5.95 $\pm$ 0.72	7.26 $\pm$ 0.63	6.69 $\pm$ 1.76	6.24 $\pm$ 0.41	6.72 $\pm$ 0.78	3.43 $\pm$ 0.58
	40	9.57 $\pm$ 0.77	6.01 $\pm$ 0.42	5.38 $\pm$ 0.57	5.45 $\pm$ 0.55	6.82 $\pm$ 0.25	7.56 $\pm$ 2.19	6.16 $\pm$ 0.52	6.01 $\pm$ 0.63	4.17 $\pm$ 0.45

Table 1: ECE (%) with 20 bins on different models for citation networks, considering various numbers of labels per class (L/C). “-” denotes the result is not meaningful. Uncal. represents the uncalibrated model. The best results are highlighted in black.

Datasets	L/C	Uncal.	TS	MS	CaGCN	GATS	AU-LS	DCGC	GETS	SCAR
Datasets	L/C	GAT
Cora	20	15.58 $\pm$ 0.89	7.17 $\pm$ 0.98	5.44 $\pm$ 0.94	4.50 $\pm$ 0.56	4.25 $\pm$ 0.29	5.51 $\pm$ 0.83	4.10 $\pm$ 0.53	4.53 $\pm$ 0.48	3.52 $\pm$ 0.74
	40	13.40 $\pm$ 0.54	4.85 $\pm$ 0.77	4.91 $\pm$ 0.60	3.65 $\pm$ 0.56	3.68 $\pm$ 0.67	3.79 $\pm$ 0.58	4.28 $\pm$ 0.42	4.01 $\pm$ 0.68	3.52 $\pm$ 0.74
	60	12.01 $\pm$ 0.33	3.93 $\pm$ 0.61	4.11 $\pm$ 0.53	3.13 $\pm$ 0.32	2.76 $\pm$ 0.31	3.44 $\pm$ 0.95	2.64 $\pm$ 0.63	3.16 $\pm$ 0.58	2.59 $\pm$ 0.43
Citeseer	20	15.34 $\pm$ 0.50	9.16 $\pm$ 0.87	6.33 $\pm$ 0.98	5.72 $\pm$ 0.68	6.01 $\pm$ 0.65	8.12 $\pm$ 0.13	5.88 $\pm$ 0.15	5.85 $\pm$ 0.33	4.37 $\pm$ 0.83
	40	12.52 $\pm$ 0.87	7.97 $\pm$ 0.31	5.90 $\pm$ 0.54	5.32 $\pm$ 0.54	6.19 $\pm$ 0.85	7.24 $\pm$ 0.87	5.26 $\pm$ 0.48	5.43 $\pm$ 0.47	3.54 $\pm$ 0.65

Table 2: ECE (%) with 20 bins on different models for citation networks, considering various numbers of labels per class (L/C). “-” denotes the result is not meaningful. Uncal. represents the uncalibrated model. The best results are highlighted in black.

+CLC	+NLC	GCN				GAT
+CLC	+NLC	Cora	Citeseer	Pubmed	CoraFull	Cora	Citeseer	Pubmed	CoraFull
TS		4.88 $\pm$ 0.55	6.41 $\pm$ 0.87	5.41 $\pm$ 0.38	10.13 $\pm$ 0.61	7.17 $\pm$ 0.98	9.16 $\pm$ 0.87	6.56 $\pm$ 0.46	11.01 $\pm$ 0.51
$\checkmark$		3.72 $\pm$ 0.53	4.92 $\pm$ 0.79	4.01 $\pm$ 0.47	7.09 $\pm$ 0.52	3.64 $\pm$ 0.63	5.98 $\pm$ 0.62	3.99 $\pm$ 0.53	6.89 $\pm$ 0.83
	$\checkmark$	4.08 $\pm$ 0.62	3.63 $\pm$ 0.63	3.88 $\pm$ 0.44	9.23 $\pm$ 0.55	4.18 $\pm$ 0.72	4.96 $\pm$ 0.66	4.13 $\pm$ 0.42	7.12 $\pm$ 0.45
$\checkmark$	$\checkmark$	3.35 $\pm$ 0.65	3.43 $\pm$ 0.58	3.78 $\pm$ 0.53	6.96 $\pm$ 0.48	3.52 $\pm$ 0.74	4.37 $\pm$ 0.83	3.78 $\pm$ 0.84	6.41 $\pm$ 0.63

Table 3: ECE (%) of each component with L/C=20. The best results are highlighted in black.

ECE	Cora	Citeseer	Pubmed	Chameleon	Squirrel
GraphSAGE	9.89 $\pm$ 0.78	10.08 $\pm$ 0.88	7.38 $\pm$ 0.65	13.27 $\pm$ 0.81	11.14 $\pm$ 0.48
SCAR-GraphSAGE	4.88 $\pm$ 0.71	4.32 $\pm$ 0.73	5.86 $\pm$ 0.46	6.74 $\pm$ 0.86	7.17 $\pm$ 0.51
FAGCN	15.94 $\pm$ 0.17	20.56 $\pm$ 4.5	5.83 $\pm$ 0.63	15.83 $\pm$ 0.63	10.36 $\pm$ 0.24
SCAR-FAGCN	4.57 $\pm$ 0.60	4.92 $\pm$ 1.4	3.74 $\pm$ 0.71	6.96 $\pm$ 0.84	7.85 $\pm$ 0.43
GCNII	35.51 $\pm$ 0.56	35.30 $\pm$ 1.11	5.72 $\pm$ 1.77	18.13 $\pm$ 0.26	17.79 $\pm$ 0.61
SCAR-GCNII	9.81 $\pm$ 0.48	8.74 $\pm$ 0.51	3.98 $\pm$ 0.88	7.03 $\pm$ 0.35	8.93 $\pm$ 0.37

Table 4: ECE (%) with 20 bins on different models for deferent base model ( i.e., GCNII and FAGCN). The best results are highlighted in black.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Datasets	Nodes	Edges	Train	Valid	Test Nodes	Features	Classes
Cora	2,708	5,429	140/280/420	500	1000	1,433	7
Citeseer	3,327	4,732	120/124/360	500	1,000	3,703	6
Pubmed	19,717	44,338	60/120/360	500	1,000	500	3
CoraFull	19,793	126,842	1400/2800/4200	500	1000	8,710	70

Appendix

Table 5: The statistics of the datasets

Datasets	L/C	Lr	Weight decay	Hidden units	Dropout	Weight decay of $\mathbf{W}^{(K)}$	$\alpha$	$\beta$
Cora	20	0.015	5e-4	64	0.6	1e-4	2e-4	2e-3
	40	0.015	5e-4	64	0.6	1e-4	1e-4	1e-3
	60	0.015	5e-4	64	0.6	1e-4	1e-4	1e-3
Citeseer	20	0.01	5e-4	64	0.5	4.5e-4	5e-5	5e-3
	40	0.01	5e-4	64	0.5	4.5e-4	5e-5	5e-3
	60	0.01	5e-4	64	0.5	4e-4	5e-4	2e-3

Appendix

Table 6: Settings for the proposed SCAR based on GCN.

Datasets	L/C	Lr	Weight decay	Hidden units	Dropout	Weight decay of $\mathbf{W}^{(K)}$	$\alpha$	$\beta$
Cora	20	0.01	5e-4	8	0.5	2e-5	5e-6	4e-4
	40	0.01	5e-4	8	0.5	2e-5	5e-6	4e-4
	60	0.01	5e-4	8	0.5	2e-5	5e-6	4e-4
Citeseer	20	0.01	5e-4	8	0.6	3e-4	5e-4	5e-3
	40	0.01	5e-4	8	0.6	3e-4	5e-4	5e-3
	60	0.01	5e-4	8	0.6	3e-4	5e-4	5e-3

Appendix

Table 7: Settings for the proposed SCAR based on GAT.

Datasets	L/C	GCN			GAT
Datasets	L/C	Uncal.	AU-LS	SCAR	Uncal.	AU-LS	SCAR
Cora	20	81.53 $\pm$ 0.32	79.58 $\pm$ 0.14	82.01 $\pm$ 0.21	82.56 $\pm$ 0.31	80.53 $\pm$ 1.05	83.45 $\pm$ 0.29
	40	83.04 $\pm$ 0.24	82.90 $\pm$ 0.69	82.70 $\pm$ 0.16	83.56 $\pm$ 0.25	82.28 $\pm$ 0.99	83.35 $\pm$ 0.42
	60	84.35 $\pm$ 0.35	84.03 $\pm$ 0.80	84.49 $\pm$ 0.37	85.06 $\pm$ 0.31	84.55 $\pm$ 0.61	85.07 $\pm$ 0.53
Citeseer	20	71.71 $\pm$ 0.32	65.65 $\pm$ 2.91	71.61 $\pm$ 0.45	71.47 $\pm$ 0.88	68.57 $\pm$ 1.01	71.51 $\pm$ 0.60
	40	72.18 $\pm$ 0.32	70.71 $\pm$ 1.62	72.43 $\pm$ 0.41	72.31 $\pm$ 0.56	71.55 $\pm$ 0.47	72.37 $\pm$ 0.31

Appendix

Table 8: Accuracy of node classification (%), considering various numbers of labels per class (L/C). Uncal. represents the uncalibrated model. The best results are highlighted in black.

Datasets	Chameleon	Squirrel	Arxiv-year
GCN	13.82 $\pm$ 0.75	11.67 $\pm$ 0.18	13.34 $\pm$ 0.72
SCAR-GCN	7.30 $\pm$ 1.48	7.25 $\pm$ 0.35	6.03 $\pm$ 0.38
GAT	14.11 $\pm$ 0.23	11.13 $\pm$ 0.83	12.33 $\pm$ 0.87
SCAR-GAT	7.93 $\pm$ 0.13	8.12 $\pm$ 0.53	5.94 $\pm$ 0.75

Appendix

Table 9: ECE (%) with 20 bins on different models for heterophilic datasets, where the Arxiv-year dataset is also included as a representative large-scale graph. The best results are highlighted in black.

Method	GCN				GAT
Method	Cora	Citeseer	Pubmed	CoraFull	Cora	Citeseer	Pubmed	CoraFull
Only+ $\alpha$	3.81 $\pm$ 0.61	3.91 $\pm$ 0.64	3.94 $\pm$ 0.51	7.21 $\pm$ 0.52	3.94 $\pm$ 0.71	4.88 $\pm$ 0.78	4.08 $\pm$ 0.72	6.73 $\pm$ 0.57
+ $\alpha$ + $\beta$	3.35 $\pm$ 0.65	3.43 $\pm$ 0.58	3.81 $\pm$ 0.53	6.96 $\pm$ 0.48	3.52 $\pm$ 0.74	4.37 $\pm$ 0.83	3.78 $\pm$ 0.84	6.41 $\pm$ 0.63

Appendix

Table 10: ECE (%) of each component on all datasets with L/C=20. The best results are highlighted in black.

Method	Cora		Citeseer
Method	ID	OOD	ID	OOD
Uncal.	13.47 $\pm$ 0.63	7.15 $\pm$ 0.32	12.48 $\pm$ 0.71	10.01 $\pm$ 1.38
AU-LS	4.32 $\pm$ 0.23	6.75 $\pm$ 0.41	6.69 $\pm$ 1.76	8.71 $\pm$ 1.54
SCAR	3.35 $\pm$ 0.65	4.73 $\pm$ 0.58	3.43 $\pm$ 0.58	5.30 $\pm$ 0.91

Appendix

Table 11: ECE (%) of all methods on all datasets with L/C = 20, using GCN as the base model. Best results are highlighted in black. “ID” and “OOD” denote in-distribution and out-of-distribution, respectively.

Explore similar work

The Confidence Trap: Calibration Attacks for Graph Neural Networks
Jun 7, 2026Cuong Dang, Jiahao Zhang, Hieu Ta Quang +3Graph Neural Networks Adversarial Training
Department of Electrical and Computer Engineering Virginia Polytechnic Institute and State University Blacksburg, VA, USA · The Pennsylvania State University University Park, PA, USA · Center for AI Research VinUniversity Hanoi, Vietnam +1
When does distribution shift break graph neural networks calibration?
Jul 12, 2026Abderaouf BahiGraph Neural Networks Distribution Shifts
Computer Science and Applied Mathematics Laboratory (LIMA), Faculty of Science and Technology, Chadli Bendjedid University, El Tarf 36000, Algeria
Reference-Tail Trust:Certified Probability Floors for Learned Updates Inside a Deployed Network
Sep 28, 2026Abdolvahab Khalili Sadaghiani, Jose Nunez-YanezLog-Ratio-Based Gaussian Trust Weight Graph Neural Networks

Jun 7, 2026, cs.LGJ/K move · Enter open · S save

The Confidence Trap: Calibration Attacks for Graph Neural Networks

Cuong Dang, Jiahao Zhang, Hieu Ta Quang, Dung Le+2

Department of Electrical and Computer Engineering Virginia Polytechnic Institute and State University Blacksburg, VA, USA · The Pennsylvania State University University Park, PA, USA · Center for AI Research VinUniversity Hanoi, Vietnam+1

Graph Neural Networks Adversarial Training Neural Network

Open paper AskPDF arXiv

While confidence calibration is essential for trustworthy decision-making in safety-critical applications, the robustness of calibrated GNNs to adversarial structural perturbations remains largely unexplored. However, studying calibration attacks on graphs presents unique technical challenges: (1) the discrete nature of graph structures complicates gradient-based optimization, (2) existing underconfidence objectives fail to drive predictions toward uniform distributions, and (3) GNNs are highly sensitive to edge perturbations, often causing unintended label changes that violate attack constraints. To address these challenges, we propose a \textbf{Unified Graph Calibration Attack (UGCA)} framework designed for \textbf{worst-case (white-box) analysis} of GNN calibration robustness. UGCA introduces a KL-divergence loss to encourage uniform predictive distributions, a reranking mechanism to reduce label flipping, a hybrid loss to recover labels when violations occur, and beam search to explore a broader adversarial search space. We further provide theoretical insights linking model generalization, dataset complexity, and calibration vulnerability, showing that models with higher accuracy or trained on datasets with more classes are more susceptible under this threat model. Extensive experiments demonstrate that UGCA substantially increases Expected Calibration Error while preserving classification accuracy. Our code is publicly available at https://github.com/CaptainCuong/Graph-Calibration-Attack.git.

Abstract

Explore similar work

The Confidence Trap: Calibration Attacks for Graph Neural Networks

When does distribution shift break graph neural networks calibration?

Reference-Tail Trust:Certified Probability Floors for Learned Updates Inside a Deployed Network