Adversarial Label Invariant Graph Data Augmentations for Out-of-Distribution Generalization

Simon Zhang; Ryan P. DeMilt; Kun Jin; Cathy H. Xia

cs.LGApr 9, 2026

Adversarial Label Invariant Graph Data Augmentations for Out-of-Distribution Generalization

Authors: Simon Zhang, Ryan P. DeMilt, Kun Jin, Cathy H. Xia

Organizations: Department of Computer Science, Purdue University, West Lafayette, USA · Department of Computer Science and Engineering, The Ohio State University, Columbus Ohio, USA · Department of Industrial and Systems Engineering, The Ohio State University, Columbus Ohio, USA

Abstract

Out-of-distribution (OoD) generalization occurs when representation learning encounters a distribution shift. This occurs frequently in practice when training and testing data come from different environments. Covariate shift is a type of distribution shift that occurs only in the input data, while the concept distribution stays invariant. We propose RIA - Regularization for Invariance with Adversarial training, a new method for OoD generalization under convariate shift. Motivated by an analogy to $Q$ -learning, it performs an adversarial exploration for counterfactual data environments. These new environments are induced by adversarial label invariant data augmentations that prevent a collapse to an in-distribution trained learner. It works with many existing OoD generalization methods for covariate shift that can be formulated as constrained optimization problems. We develop an alternating gradient descent-ascent algorithm to solve the problem in the context of causally generated graph data, and perform extensive experiments on OoD graph classification for various kinds of synthetic and natural distribution shifts. We demonstrate that our method can achieve high accuracy compared with OoD baselines.

Figures & tables

Figure 1: (a): A casual graph for the data generation process. The exogenous variable E is an integer that indexes a data environment. (b) A labeled attributed graph instance with the joining operation for causal/spurious attributes and edges shown. In the figure, the joining operation $J_{X}$ is shown as the node-wise concatenation of causal and spurious attribute tensors. The joining operation $J_{A}$ shown in the figure provides the sum of the adjacency matrices $\mathbf{A}_{C}$ and $\mathbf{A}_{S}$ where the hadamard product $\mathbf{A}_{C}\odot\mathbf{A}_{S}=\mathbf{0}$ . The half grey color on nodes represents the $\mathbf{X_{S}}$ while the half blue color represents the $\mathbf{X}_{C}$ .

Figure 2: Geometric view of the minimax optimization procedure RIA algorithm on $\textsf{Regularized}\_\textsf{Loss}(\theta,w)$ as given in Equation 20 where $w\in W$ indexes the artificial search environments, $\theta$ indexes the learner’s neural weights. The map $\pi_{j,...,k}:j<k$ is a projection of a set into $k-j+1$ independent dimensions.

Dataset (acc)

CMNIST

\uparrow

SST2

\uparrow

Motif

\uparrow

AMotif

\uparrow

Synth

\uparrow

covariate

color

length

basis

size

basis

size

basis+std,

r=1

OOD

RIA-RICE

61.7

\pm

1.6

48.1\pm 0.8

89.4

\pm

0.6

81.9\pm 0.2

92.4\pm 0.2

65.1

\pm

5.9

92.4\pm 0.2

55.3\pm 0.4

79.3

\pm

1.6

36.8

\pm

4.2

67.4

\pm

1.5

33.4

\pm

1.3

48.0

\pm

9.0

58.5

\pm

1.5

RIA-IRM

65.5\pm 2.8

41.6\pm 0.6

89.7\pm 0.6

81.7\pm 0.5

33.7\pm 0.8

33.9\pm 0.7

33.5\pm 0.8

34\pm 2.9

89.6\pm 0.8

40.5\pm 3.8

48.6\pm 0.4

48.6\pm 2

51\pm 0.6

54\pm 0.8

RIA-VREx

79.3\pm 0.7

38.7\pm 0.7

89.8\pm 2

80.2\pm 4

32.2\pm 2.3

34\pm 1.7

33.5\pm 0.5

34\pm 1.0

90.5\pm 4.5

42.4\pm 0.6

90.3\pm 0.9

47\pm 0.87

40\pm 0.8

60\pm 1.9

ERM

77.5

\pm

0.5

28.3

\pm

0.3

89.4

\pm

0.4

81.2

\pm

0.2

92.3

\pm

0.3

68.3

\pm

0.3

92.1

\pm

0.1

51.4

\pm

0.4

80.8

\pm

1.1

33.2

\pm

1.0

67.9

\pm

2.2

33.2

\pm

1.0

53.5

\pm

1.5

53.5

\pm

1.5

Table 1: Accuracy of all baseline approaches as well as RIA-RICE , RIA-IRM , RIA-VREx on all datasets under different covariate shifts. For each covariate shift, the columns labeled ID refer to the in-distribution test accuracies while the columns labeled OOD refer to the out-of-distribution test scores. Red and gray entries are the max and second max test accuracies, respectively, for each column.

Panel of Figure 3 — Figure 3: Illustration of ERM Collapse on the CMNIST (above) and SST2 (below) dataset. Left: Training loss where ERM collapse is happening to traditional constrained optimization OoD generalization methods. Red and Green are RIA on IRM and VRex, respectively. Right: Test OoD loss. The consequences of ERM collapse are prevented.

Explore similar work

Towards OOD Generalization in Dynamic Graphs via Causal Invariant Learning
Mar 2, 2026Xinxun Zhang, Pengfei Jiao, Mengzhou Gao +2OOD Generalization Dynamic Graph Learning
School of Cyberspace, Hangzhou Dianzi University, China · State Key Laboratory of AI Safety, Beijing · School of Computer Science, Wuhan University, China +1
CGRL: Causal-Guided Representation Learning for Node-Level Out-of-Distribution Generalization
Mar 25, 2026Bowen Lu, Lianqiang Yang, Teng Li +1OOD Generalization Graph Neural Networks
Anhui University, Hefei, China · Carnegie Mellon University, Pittsburgh, Commonwealth of Pennsylvania, USA
PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs
Nov 27, 2025Ali Ghasemi, Farooq Ahmad Wani, Maria Sofia Bucarelli +1OOD Generalization Invariant Representation Learning
Sapienza University of Rome · Université Côte d’Azur, CNRS, Inria, I3S

Mar 2, 2026J/K move · Enter open · S save

Towards OOD Generalization in Dynamic Graphs via Causal Invariant Learning

Xinxun Zhang, Pengfei Jiao, Mengzhou Gao, Tianpeng Li+1

School of Cyberspace, Hangzhou Dianzi University, China · State Key Laboratory of AI Safety, Beijing · School of Computer Science, Wuhan University, China+1

OOD Generalization Dynamic Graph Learning

Open paper AskPDF arXiv

Although dynamic graph neural networks (DyGNNs) have demonstrated promising capabilities, most existing methods ignore out-of-distribution (OOD) shifts that commonly exist in dynamic graphs. Dynamic graph OOD generalization is non-trivial due to the following challenges: 1) Identifying invariant and variant patterns amid complex graph evolution, 2) Capturing the intrinsic evolution rationale from these patterns, and 3) Ensuring model generalization across diverse OOD shifts despite limited data distribution observations. Although several attempts have been made to tackle these challenges, none has successfully addressed all three simultaneously, and they face various limitations in complex OOD scenarios. To solve these issues, we propose a Dynamic graph Causal Invariant Learning (DyCIL) model for OOD generalization via exploiting invariant spatio-temporal patterns from a causal view. Specifically, we first develop a dynamic causal subgraph generator to identify causal dynamic subgraphs explicitly. Next, we design a causal-aware spatio-temporal attention module to extract the intrinsic evolution rationale behind invariant patterns. Finally, we further introduce an adaptive environment generator to capture the underlying dynamics of distributional shifts. Extensive experiments on both real-world and synthetic dynamic graph datasets demonstrate the superiority of our model over state-of-the-art baselines in handling OOD shifts.

Abstract

Explore similar work

Towards OOD Generalization in Dynamic Graphs via Causal Invariant Learning

CGRL: Causal-Guided Representation Learning for Node-Level Out-of-Distribution Generalization

PISA: Prioritized Invariant Subgraph Aggregation for Out-of-Distribution Generalization on Graphs