Organizations: Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, Yunnan, China · Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, 650500, Yunnan, China
Graph Representation Learning (GRL) requires node embeddings to preserve local structural coherence while remaining globally discriminative. Existing graph self-supervised methods generally coordinate these competing objectives through method-specific designs, lacking a unified and adaptive mechanism that responds to the evolving representation state. To address this limitation, we propose SelfGRL, a self-organized graph representation learning framework that reformulates representation learning as Helmholtz free-energy minimization on the unit hypersphere. SelfGRL models local neighborhood attraction as a Structural Binding Energy through neighbor-mean alignment, while introducing a Sampling-Free Repulsive Entropy objective to preserve global dispersion and prevent representation collapse without pairwise negative sampling. To dynamically coordinate these competing effects, we develop an endogenous Bayesian Thermostat that adaptively regulates their relative strengths in response to the evolving representation state, allowing the system to approach a metastable semantic equilibrium without requiring manually tuned loss weights. Extensive experiments on node classification across both homophilic and heterophilic benchmark datasets, together with link prediction evaluations, demonstrate that SelfGRL consistently achieves strong performance across diverse graph structures.
Figures & tables
Figure 1. Overview of SelfGRL. Given an input graph G(A,X) , SelfGRL maps node representations onto the unit hypersphere. Within this space, two competing thermodynamic effects govern the learning dynamics: Structural Binding Energy E , implemented by neighbor-mean alignment Lalign , promotes local structural cohesion; Repulsive Field Entropy S , implemented by sampling-free uniformity Lunif , encourages global dispersion and prevents representation collapse. Both effects are unified under the Helmholtz free-energy objective F=E−TS . The Endogenous Bayesian Thermostat uses alignment and uniformity signals to learn scale variables salign and sunif , adaptively regulating the relative strengths of the two components during training.
Model
Cora
CiteSeer
PubMed
Computers
Photo
CS
Physics
GAT
80.32 ± 0.17
71.02 ± 0.28
78.83 ± 0.71
86.93 ± 0.29
92.56 ± 0.35
92.31 ± 0.24
95.47 ± 0.15
GCN
81.54 ± 0.68
70.73 ± 0.65
79.16 ± 0.25
86.51 ± 0.54
92.42 ± 0.22
93.03 ± 0.31
95.65 ± 0.16
DeepWalk
70.72 ± 0.63
51.39 ± 0.41
73.27 ± 0.86
85.68 ± 0.07
89.40 ± 0.11
84.61 ± 0.22
91.77 ± 0.15
Node2Vec
71.08 ± 0.91
47.34 ± 0.84
66.23 ± 0.95
84.41 ± 0.14
89.68 ± 0.19
85.16 ± 0.04
91.23 ± 0.07
DGI
84.46 ± 0.09
73.30 ± 0.81
85.86 ± 0.15
88.87 ± 0.20
92.83 ± 0.50
93.26 ± 0.09
95.65 ± 0.11
GRACE
83.00 ± 0.31
71.80 ± 0.92
82.87 ± 0.21
85.44 ± 0.23
91.79 ± 0.17
92.65 ± 0.08
OOM
Table 1. Node classification accuracy on homophilic graphs. Optimal results are shown in bold and suboptimal results are underlined. OOM indicates Out-Of-Memory on 24GB RTX 3090 GPU.
Model
Actor
Cornell
Texas
Wisconsin
GAT
28.06 ± 1.48
59.46 ± 3.63
61.62 ± 3.78
54.71 ± 6.87
GCN
30.83 ± 0.77
57.03 ± 3.30
60.00 ± 4.80
56.47 ± 6.55
DeepWalk
22.78 ± 0.64
63.35 ± 4.61
60.59 ± 7.56
55.41 ± 5.96
Node2Vec
28.28 ± 1.27
42.94 ± 7.46
41.92 ± 7.76
37.45 ± 7.09
DGI
30.81 ± 1.29
67.57 ± 5.84
68.47 ± 4.59
58.63 ± 3.20
GRACE
31.03 ± 0.74
57.66 ± 8.92
58.56 ± 1.27
51.63 ± 8.06
Table 2. Node classification accuracy on heterophilic graphs.
Model
Cora
CiteSeer
PubMed
Avg.
Node2Vec
90.97 ± 0.64
94.46 ± 0.59
93.14 ± 0.18
92.86
MF
80.29 ± 2.26
75.92 ± 3.25
93.06 ± 0.43
83.09
MLP
95.32 ± 0.37
94.45 ± 0.32
98.34 ± 0.10
96.04
GCN
95.01 ± 0.32
95.89 ± 0.26
98.69 ± 0.06
96.53
GAT
93.90 ± 0.32
96.25 ± 0.20
98.20 ± 0.07
96.12
SAGE
95.63 ± 0.27
97.39 ± 0.15
98.87 ± 0.04
97.30
Table 3. Performance on link prediction (AUC-ROC).
Figure 2. Thermodynamic evolution on Cora and Texas, showing the training trajectories of the normalized losses Lalign and Lunif , together with the effective temperature Teff .
Figure 3. Sensitivity analysis of the fixed effective temperature on Cora, CS, and Texas.
Figure 4. Accuracy changes (percentage points) with Bayesian thermostat integration.
Backbone
Cora
CS
Texas
GCN
84.78 ± 0.49
93.57 ± 0.03
48.95 ± 1.12
GAT
85.10 ± 0.23
93.42 ± 0.08
58.56 ± 0.74
GraphSAGE
84.87 ± 0.44
93.71 ± 0.05
79.19 ± 0.74
Transformer
85.41 ± 0.11
94.05 ± 0.04
83.48 ± 0.42
Table 4. Accuracy (%) with different GNN backbones.
Method
Cora
CS
Texas
BGRL
80.63 ± 0.93
93.30 ± 0.05
56.92 ± 3.06
BGRL-FU
80.48 ± 1.11
93.28 ± 0.02
56.72 ± 3.06
BGRL-BT
80.66 ± 0.76
93.35 ± 0.14
58.05 ± 4.46
Table 5. Node classification accuracy (%) of BGRL variants. FU denotes fixed-weight uniformity, and BT denotes the Bayesian thermostat.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Description
G=(V,E,A,X)
Attributed input graph with node set V , edge set E , adjacency matrix A , and node feature matrix X .
N,F,d
Number of nodes, input feature dimension, and embedding dimension.
fθ(⋅)
Graph encoder parameterized by θ .
H,Z
Intermediate and ℓ2 -normalized embedding matrices.
Sd−1
(d−1) -dimensional unit hypersphere.
Ni,μi
Neighbor set and normalized neighbor-mean structural anchor of node i .
Appendix
Table 6. Summary of key notations.
Dataset
Type
Nodes
Edges
Features
Classes
Cora
Homophilic
2,708
5,278
1,433
7
CiteSeer
3,327
4,522
3,703
6
PubMed
19,717
44,324
500
3
Computers
13,752
245,861
767
10
Photo
7,650
119,081
745
8
CS
18,333
81,894
6,805
15
Appendix
Table 7. Statistics of the benchmark datasets.
Figure 5. t-SNE embeddings of nodes in the Cora dataset.
Figure 6. Node classification accuracy (%) with varying hidden dimensions (a–b) and network depths (c–d).
Graph Self-Supervised Learning (GSSL) has emerged as a powerful paradigm for generating high-quality representations for graph-structured data. While multi-scale graph contrastive learning has received increasing attention, many existing methods still predominantly focus on a single graph abstraction level. To address this limitation, we propose a unified contrastive framework that can target node-level, proximity-level, cluster-level, and graph-level information and integrate them through a linear combination of similarity scores on positive pairs and dissimilarity scores (i.e., similarity scores on negative pairs). Furthermore, current approaches typically assign uniform penalty strengths to all examples, which reduces optimization flexibility and leads to ambiguous convergence status. To overcome this, we introduce a novel parameter-free fine-grained self-weighting mechanism that adaptively assigns weights to individual similarity and dissimilarity scores. The proposed mechanism emphasizes the scores that deviate significantly from their target values. Our approach not only enhances optimization flexibility but also eliminates the computational overhead of hyperparameter tuning in conventional multi-task GSSL methods. Comprehensive experiments on real-world datasets show that our methods consistently outperform state-of-the-art approaches across downstream tasks, including classification, clustering, and link prediction, in both single-level and multi-level scenarios.
Real-world heterogeneous graphs are inherently noisy and usually not in the optimal graph structures for downstream tasks, which often adversely affects the performance of GRL models in downstream tasks. Although Graph Structure Learning (GSL) methods have been proposed to learn graph structures and downstream tasks simultaneously, existing methods are predominantly designed for homogeneous graphs, while GSL for heterogeneous graphs remains largely unexplored. Two challenges arise in this context. Firstly, the quality of the input graph structure has a more profound impact on GNN-based heterogeneous GRL models compared to their homogeneous counterparts. Secondly, most existing homogenous GRL models encounter memory consumption issues when applied directly to heterogeneous graphs. In this paper, we propose a novel Graph Topology learning Enhanced Heterogeneous Graph Representation Learning framework (ToGRL).ToGRL learns high-quality graph structures and representations for downstream tasks by incorporating task-relevant latent topology information. Specifically, a novel GSL module is first proposed to extract downstream task-related topology information from a raw graph structure and project it into topology embeddings. These embeddings are utilized to construct a new graph with smooth graph signals. This two-stage approach to GSL separates the optimization of the adjacency matrix from node representation learning to reduce memory consumption. Following this, a representation learning module takes the new graph as input to learn embeddings for downstream tasks. ToGRL also leverages prompt tuning to better utilize the knowledge embedded in learned representations, thus enhancing adaptability to downstream tasks. Extensive experiments on five real-world datasets show that our ToGRL outperforms state-of-the-art methods by a large margin.
He Zhao, Zhiwei Zeng, Yongwei Wang +1
Nanyang Technological University, Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Singapore · Nanyang Technological University, Singapore
Self-supervised Continual Graph Learning (CGL) aims to successively learn from a graph sequence with different tasks without label supervision - a paradigm that has attracted widespread attention. Most existing self-supervised CGL methods rely on instance-level consistency objectives that enforce stability of individual node (or node-pair) embeddings. Due to optimizing nodes in isolation, these methods fail to maintain global relational structure, causing inter-node correspondences to progressively distort under continual learning. To this end, we propose a novel Structure-Aware Optimal Transport (SAOT) framework that explicitly captures and preserves relational structure within graph representations across sequential tasks. Specifically, SAOT leverages optimal transport theory to capture global inter-node correspondences, thereby facilitating and enhancing graph representation learning. Simultaneously, SAOT incorporates a cross-task knowledge distillation mechanism to preserve the previous structural knowledge. Extensive experiments on four CGL benchmark datasets demonstrate that SAOT outperforms existing self-supervised baselines. In particular, SAOT achieves significant performance gains, improving average accuracy by up to 5% on CoraFull-CL and over 15% on Products-CL compared with state-of-the-art methods in the Class-IL setting.
Yuting Zhang, Yanbei Liu, Zhitao Xiao +3
School of Electronics and Information Engineering, Tiangong University, Tianjin, China · School of Life Sciences, Tiangong University, Tianjin, China · School of Electrical and Infomation Engineering, Tianjin University, Tianjin, China +1