HoTS: Homophily-Aware Temperature Scaling for Graph Neural Network Calibration
Organizations: UNIST · Pusan National University · UNIST LinqAlpha
Abstract
For graph node classification, calibrated class probabilities are needed when confidence scores, usually the maximum predicted class probability, are used to rank predictions, defer uncertain nodes to human review, or control risk. Existing post-hoc calibrators either apply one global temperature or use graph-aware modules without a principled structural form. We study how local graph structure should enter node-level calibration. Our first results show that a logit-only temperature rule is insufficient when nodes with identical logits but different local homophily require different optimal temperatures. We then analyze a population-concentration contextual stochastic block model with Gaussian features and a one-layer linear GCN. Under equidistant class means, the Bayes posterior over class-template scores is a temperature-scaled softmax whose inverse-temperature is governed by a homophily-dependent signal strength. In the positive-signal homophilic regime, the resulting temperature decreases approximately inversely with normalized local homophily. This law motivates Homophily-aware Temperature Scaling (HoTS), a simple post-hoc calibrator that assigns each node a positive scalar temperature from entropy-based logit concentration and estimated local homophily. HoTS has three temperature parameters, preserves the predicted class, and learns the strength of the structural correction from calibration data. Across 18 node-classification benchmarks, two GNN backbones, and eight calibration baselines, HoTS achieves the best mean Expected Calibration Error (ECE) of 4.79%, the best average rank, and the most reliable confidence ranking in selective classification. Code is available at https://github.com/inu0104/HoTS.
Figures & tables
| Dataset | Uncal | TS | VS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS (ours) |
|---|---|---|---|---|---|---|---|---|---|---|
| Homophilic | ||||||||||
| Cora | {\color[rgb]{1,0,0}2.50\pm 0.72} | |||||||||
| CiteSeer | {\color[rgb]{1,0,0}2.93\pm 0.63} | |||||||||
| PubMed | {\color[rgb]{1,0,0}0.83\pm 0.30} | |||||||||
| CoraFull | {\color[rgb]{1,0,0}2.20\pm 0.63} | |||||||||
| Computers | {\color[rgb]{1,0,0}1.39\pm 0.34} | |||||||||
| Coverage | Uncal | TS | HTS | HoTS (ours) |
|---|---|---|---|---|
| 100% | 67.39 | 67.39 | 67.39 | 67.39 |
| 95% | 68.87 | 68.90 | 68.92 | 68.94 |
| 90% | 70.01 | 70.08 | 70.13 | 70.15 |
| 85% | 71.04 | 71.12 | 71.17 | 71.18 |
| 80% | 71.94 | 72.05 | 72.13 | 72.11 |
| 75% | 72.81 | 72.92 | 72.94 | 72.96 |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | TS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS (ours) |
|---|---|---|---|---|---|---|---|---|
| NLL | 0.97 | 0.94 | 0.94 | 0.97 | 0.98 | 0.95 | 1.02 | 0.92 |
| deg_ECE | 6.50 | 6.32 | 6.13 | 5.81 | 6.45 | 6.16 | 7.16 | 5.90 |
| Dataset | Backbone | Uncal | TS | VS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Homophilic | |||||||||||
| Cora | GCN | {\color[rgb]{1,0,0}2.28\pm 0.76} | |||||||||
| GAT | {\color[rgb]{1,0,0}2.68\pm 0.85} | ||||||||||
| CiteSeer | GCN | {\color[rgb]{1,0,0}2.90\pm 0.44} | |||||||||
| GAT | {\color[rgb]{1,0,0}2.96\pm 0.78} | {\color[rgb]{1,0,0}2.96\pm 0.87} | |||||||||
| PubMed | GCN | {\color[rgb]{1,0,0}0.74\pm 0.24} | |||||||||
| Dataset | Backbone | Uncal | TS | VS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Homophilic | |||||||||||
| Cora | GCN | {\color[rgb]{1,0,0}2.59\pm 0.41} | |||||||||
| GAT | {\color[rgb]{1,0,0}2.09\pm 0.73} | ||||||||||
| CiteSeer | GCN | {\color[rgb]{1,0,0}2.71\pm 0.43} | {\color[rgb]{1,0,0}2.71\pm 0.55} | ||||||||
| GAT | {\color[rgb]{1,0,0}2.46\pm 0.39} | ||||||||||
| PubMed | GCN | {\color[rgb]{1,0,0}0.92\pm 0.18} | |||||||||
| Dataset | Backbone | Uncal | TS | VS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Homophilic | |||||||||||
| Cora | GCN | {\color[rgb]{1,0,0}0.481\pm 0.026} | |||||||||
| GAT | {\color[rgb]{1,0,0}0.504\pm 0.023} | ||||||||||
| CiteSeer | GCN | {\color[rgb]{1,0,0}0.809\pm 0.023} | |||||||||
| GAT | {\color[rgb]{1,0,0}0.790\pm 0.011} | ||||||||||
| PubMed | GCN | {\color[rgb]{1,0,0}0.365\pm 0.005} | |||||||||
| Backbone | TS | VS | ETS | HTS | CaGCN | GATS | GETS | WATS | HoTS |
|---|---|---|---|---|---|---|---|---|---|
| GCNII | 6.82 | 6.08 | 6.29 | 6.23 | 5.79 | 6.35 | 6.79 | 7.74 | 5.84 |
| GPR-GNN | 6.85 | 5.95 | 6.47 | 5.73 | 6.05 | 5.87 | 7.11 | 7.78 | 5.49 |
| GIN | 6.32 | 5.67 | 5.37 | 5.23 | 6.37 | 5.31 | 6.33 | 8.98 | 4.97 |
| GraphSAGE | 6.60 | 6.10 | 5.53 | 5.75 | 6.63 | 5.79 | 6.51 | 9.20 | 5.11 |
| Dataset | Method | 0% | 5% | 10% | 20% | 30% | 50% |
|---|---|---|---|---|---|---|---|
| Cora | HoTS | 2.27 | 2.72 | 2.21 | 2.55 | 2.77 | 3.03 |
| TS | 3.39 | 3.24 | 2.81 | 3.73 | 3.69 | 4.16 | |
| CiteSeer | HoTS | 2.83 | 3.09 | 3.21 | 3.10 | 2.72 | 3.92 |
| TS | 3.09 | 2.95 | 3.09 | 3.11 | 3.13 | 3.79 | |
| CS | HoTS | 0.99 | 0.95 | 0.95 | 1.26 | 1.19 | 1.15 |
| TS | 1.49 | 1.49 | 1.54 | 1.93 | 1.89 | 2.44 |
| Structural statistic | Category | Pooled | Macro |
|---|---|---|---|
| Oracle normalized homophily | Homophily | ||
| Local clustering coefficient | Motifs | ||
| PageRank | Centrality | ||
| Mean incident augmented Forman–Ricci curvature | Curvature |
| TS | Entropy-only | Homophily-only † | HoTS ( ) | HoTS (ours) | |
|---|---|---|---|---|---|
| Mean ECE (%) |
| Dataset | Nodes | Edges | Classes | Features | Homophily |
| Citation (Planetoid) | |||||
| Cora | 2,708 | 13,264 | 7 | 1,433 | 0.871 0.211 |
| CiteSeer | 3,327 | 12,431 | 6 | 3,703 | 0.820 0.245 |
| PubMed | 19,717 | 108,365 | 3 | 500 | 0.865 0.217 |
| Amazon / Coauthor | |||||
| Computers | 13,381 | 491,556 | 10 | 767 | 0.802 0.239 |
| Group | Hidden | Layers | LR | Weight decay | Dropout | Patience |
|---|---|---|---|---|---|---|
| G1 | 16 | 2 | 0.5 | 50 / – | ||
| G2 | 64 | 2 | 0.5 | 50 | ||
| G3 | 256 | 2 | 0.5 | – |
| Method | Method-specific settings |
|---|---|
| TS, ETS, HTS | Scalar parameters only; no architectural hyperparameters. |
| VS | Per-class affine logit transform . |
| CaGCN, GATS | Per-dataset config (auxiliary GCN/attention width and dropout) following the original releases. |
| GETS | Per-dataset config (mixture-of-experts width and number of experts) following the original release. |
| WATS | Per-dataset config (graph wavelet basis size) following the original release. |
| HoTS (ours) | Three scalar temperature parameters , enforced through softplus parameterizations , , on unconstrained underlying parameters . We initialize and , giving effective initial values , , . The stabilizer in the homophily denominator is fixed to . The homophily predictor that produces is trained separately; see Section C.5 . |
| Method | Calibrator form | Preserves | Rate |
| Global methods | |||
| TS | , scalar | Yes | |
| ETS | , mixture | No ∗ | |
| VS | per-class affine | No | |
| Node-wise methods | |||
| HTS | , , | Yes | |
Model Calibration
Model calibration aligns predicted confidence or model outputs with observed outcomes and measurements to support more reliable decisions.
See all 218 papers