Organizations: 1Shenzhen Loop Area Institute · 2Guangzhou University · 3North China Electric Power University · University of Science and Technology of China
Graph-level out-of-distribution (OOD) generalization is critical for deploying graph neural networks in real-world applications where distribution shifts are inevitable. Existing approaches typically require class labels during representation learning to decide which substructures are predictive, limiting their applicability. We propose SCARCE, a label-free representation-learning framework that yields distribution-shift-robust graph representations from an environment partition of the training graphs alone, whether that partition comes from dataset metadata or, when none is available, from clustering a self-supervised embedding. Our method operates in two stages: (1) a variational graph autoencoder is warm-started for graph reconstruction; and (2) an adversarial confusion mechanism forces the encoder to produce representations from which a discriminator cannot distinguish environments, while preserving graph-reconstruction fidelity. We provide theoretical analysis showing that environment confusion contracts environment-discriminative directions in the representation space, which, under regularized downstream classifiers, is equivalent to imposing a stronger directional penalty on environment-associated features. This feature-reweighting mechanism reduces reliance on spurious correlations without erasing information, offering a label-free, continuous relaxation of last-layer reweighting methods. Experiments on a controlled synthetic datasets and the DrugOOD datasets show that SCARCE achieves state-of-the-art OOD generalization among label-free methods, using no class labels during representation learning.
Figures & tables
Figure 1: Overview of the Scarce framework. Given an environment partition of the training graphs (dataset metadata or, when unavailable, clusters of a frozen self-supervised embedding), (1) a VGAE is warm-started for graph reconstruction and (2) an adversarial confusion mechanism forces the encoder to produce representations indistinguishable across environments while preserving reconstruction fidelity.
Figure 2
Figure 3: Controlled contraction of an environment-associated direction. Cells report ΔROOD (10 seeds; negative = improvement). (a) Contraction helps when environment and task directions are separated but hurts at full alignment. (b) The effect vanishes as λ→0 and peaks at intermediate λ , consistent with the induced penalty λ/ε2 .
Figure 4
Method
EC50
IC50
Assay
Scaffold
Size
Assay
Scaffold
Size
Avg.
5% labels
MVGRL
68.9 ± 3.7
52.1 ± 3.7
57.8 ± 2.9
67.1 ± 0.3
57.4 ± 1.7
62.1 ± 0.8
60.9
VGAE ( β=0 )
58.2 ± 2.8
50.0 ± 6.2
52.9 ± 4.1
61.5 ± 2.2
55.7 ± 2.2
56.8 ± 0.7
55.9
GraphCL
65.1 ± 2.0
54.6 ± 9.0
56.3 ± 1.8
67.0 ± 1.0
61.8 ± 1.9
61.5 ± 1.1
61.1
RGCL
69.5 ± 3.1
54.8 ± 7.5
56.5 ± 2.9
65.8 ± 0.1
60.0 ± 1.6
60.5 ± 1.1
61.2
Table 2: DrugOOD under limited labels. Test ROC-AUC (%), mean ± std over three seeds (IC50-size VGAE at 20%: two), for linear probes on frozen representations. Bold : best per column; Avg.: mean over the six splits; shaded row: Scarce .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Domain
α
β
K
Test ROC-AUC (%)
EC50
Assay
0
10
3
69.84
0
10
20
65.70
0.5
10
20
65.63
0
10
5
65.61
0.5
10
3
64.28
1
10
20
63.96
Appendix
Table 3: Top evaluated pseudo-environment configurations at 5% labels. Within each dataset–domain setting, the six displayed (α,β,K) configurations are ordered from highest to lowest test ROC-AUC (%). Untested configurations are not interpolated.
Dataset
Domain
α
β
K
Test ROC-AUC (%)
EC50
Assay
0
10
3
70.38
0
10
10
68.45
0
10
5
68.32
0.5
10
5
67.17
0
0.1
3
66.59
0.5
1
20
66.39
Appendix
Table 4: Top evaluated pseudo-environment configurations at 10% labels. Within each dataset–domain setting, the six displayed (α,β,K) configurations are ordered from highest to lowest test ROC-AUC (%). Untested configurations are not interpolated.
Dataset
Domain
α
β
K
Test ROC-AUC (%)
EC50
Assay
0
10
3
73.07
0.5
10
3
71.55
0
10
5
69.67
0.5
1
20
69.25
0
1
3
68.31
0.5
1
3
68.31
Appendix
Table 5: Top evaluated pseudo-environment configurations at 20% labels. Within each dataset–domain setting, the six displayed (α,β,K) configurations are ordered from highest to lowest test ROC-AUC (%). Untested configurations are not interpolated.
α
β
Test accuracy (%)
0
0.01
65.0
0
0.1
62.8
0
1
66.0
0
10
61.7
0.5
0.01
62.3
0.5
0.1
69.6
Appendix
Table 6: GOOD-Motif concept hyperparameter sensitivity. The evaluated configurations use fixed K=8 and kadv=2 . Test accuracy is the best-epoch linear-probe result.
Figure 5: Training dynamics on EC50-assay. The plot uses the validation-selected (α,β,K)=(0,10,3) . Reconstruction, confusion, and adversary losses use solid, dashed, and dash-dotted lines with triangle, circle, and square markers, respectively. Lines are means over three seeds and shaded regions denote mean ± sample standard deviation.
Figure 6: Training dynamics on EC50-scaffold. The validation-selected (α,β,K) configuration is (1,10,20) . Lines and shaded regions follow the convention of Figure 5 .
Figure 7: Training dynamics on EC50-size. The validation-selected (α,β,K) configuration is (0.5,10,3) . Lines and shaded regions follow the convention of Figure 5 .
Figure 8: Training dynamics on IC50-assay. The validation-selected (α,β,K) configuration is (1,10,20) . Lines and shaded regions follow the convention of Figure 5 .
Figure 9: Training dynamics on IC50-scaffold. The validation-selected (α,β,K) configuration is (0.5,10,3) . The vertical axis is truncated at 5 to show the stable-loss range; values above this limit are placed at the boundary and connected to the next sampled vertex. Lines and shaded regions follow the convention of Figure 5 .
Figure 10: Training dynamics on IC50-size. The validation-selected (α,β,K) configuration is (0.5,10,3) . The vertical axis is logarithmic because reconstruction has a substantially larger transient scale. Lines and shaded regions follow the convention of Figure 5 .
Out-of-distribution (OOD) generalization remains challenging for graph neural networks (GNNs), as graph distributions can vary substantially across time and domains. Supervised and self-supervised graph representation learning are guided by distinct objectives and offer different perspectives on graph representations. In this work, we study whether self-supervised representations (SSL) can provide complementary signals to improve supervised OOD node classification. We develop two backbone-agnostic frameworks that exploit such information at different stages of learning and prediction. Co-Train jointly learns supervised and SSL representations and adaptively integrates them during training, while Dual-Space Retrieval performs non-parametric prediction in the two representation spaces and combines their predictions through confidence-aware fusion at inference time. The supervised and SSL encoders are separately parameterized and need not share the same GNN architecture. We evaluate multiple GNN backbones and two distinct SSL objectives, DGI and GRACE, on four graph benchmarks spanning temporal and cross-domain distribution shifts. Extensive experiments show that Co-Train consistently outperforms strong supervised OOD baselines, while Dual-Space Retrieval achieves competitive performance as a flexible non-parametric alternative. Results across different backbones and SSL objectives, together with representation analyses and ablations, demonstrate that SSL representations provide complementary information to supervised representations and can improve OOD node classification across diverse settings.
Qingying Hao, Zikang Chen, Chuxuan Hu +4
ShanghaiTech University Shanghai, China · University of Illinois Urbana-Champaign Urbana, Illinois, USA · The Pennsylvania State University University Park, Pennsylvania, USA
Out-of-distribution (OoD) generalization occurs when representation learning encounters a distribution shift. This occurs frequently in practice when training and testing data come from different environments. Covariate shift is a type of distribution shift that occurs only in the input data, while the concept distribution stays invariant. We propose RIA - Regularization for Invariance with Adversarial training, a new method for OoD generalization under convariate shift. Motivated by an analogy to Q-learning, it performs an adversarial exploration for counterfactual data environments. These new environments are induced by adversarial label invariant data augmentations that prevent a collapse to an in-distribution trained learner. It works with many existing OoD generalization methods for covariate shift that can be formulated as constrained optimization problems. We develop an alternating gradient descent-ascent algorithm to solve the problem in the context of causally generated graph data, and perform extensive experiments on OoD graph classification for various kinds of synthetic and natural distribution shifts. We demonstrate that our method can achieve high accuracy compared with OoD baselines.
Simon Zhang, Ryan P. DeMilt, Kun Jin +1
Department of Computer Science, Purdue University, West Lafayette, USA · Department of Computer Science and Engineering, The Ohio State University, Columbus Ohio, USA · Department of Industrial and Systems Engineering, The Ohio State University, Columbus Ohio, USA
Graph Neural Networks (GNNs) have demonstrated remarkable success across diverse tasks. However, their generalization capability is often hindered by spurious correlations between node features and labels in the graph. Our analysis reveals that GNNs tend to exploit imperceptible statistical correlations in training data, even when such correlations are unreliable for prediction. To address this challenge, we propose the Spurious Correlation Learning Graph Neural Network (SCL-GNN), a novel framework designed to enhance generalization on both Independent and Identically Distributed (IID) and Out-of-Distribution (OOD) graphs. SCL-GNN incorporates a principled spurious correlation learning mechanism, leveraging the Hilbert-Schmidt Independence Criterion (HSIC) to quantify correlations between node representations and class scores. This enables the model to identify and mitigate irrelevant but influential spurious correlations effectively. Additionally, we introduce an efficient bi-level optimization strategy to jointly optimize modules and GNN parameters, preventing overfitting. Extensive experiments on real-world and synthetic datasets demonstrate that SCL-GNN consistently outperforms state-of-the-art baselines under various distribution shifts, highlighting its robustness and generalization capabilities.
Yuxiang Zhang, Enyan Dai
The Hong Kong University of Science and Technology (Guangzhou)