Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs

Tyler Bonnet; Marek Rei

cs.LGSep 26, 2026

Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs

Authors: Tyler Bonnet, Marek Rei

Organizations: Imperial College London London, United Kingdom

Abstract

Learning transferable representations for dynamic text-attributed graphs (DyTAGs) requires models to capture underlying interaction dynamics that persist across domains. However, existing methods tend to overfit to domain-specific structural, temporal, and semantic patterns, limiting edge classification performance under distribution shifts. To expose and address this, we formally establish a leave-one-domain-out (LODO) transfer learning protocol for edge classification on DyTAGs. Under this protocol, we demonstrate that state-of-the-art self-supervised methods for dynamic graph learning perform poorly when transferred to unseen domains. Strikingly, existing methods underperform a structurally and temporally unaware Bag of Events (BoE) model we introduce, which inputs only unordered sequences of node and edge text features. Proceeding from the BoE, we propose Spatio-Temporal Semantic Alignment (STSA), which integrates a spatio-temporal encoder that fuses representations of time deltas and node occurrence frequencies into a unified manifold. STSA is trained with a Contrastive Semantic Forecasting objective, which anchors edge representations to a multi-domain textual latent space initialized by a pretrained language model, providing a robust prior that outperforms BoE and all existing methods we evaluate.

Figures & tables

Figure 1: Overview of the Spatio-Temporal Semantic Alignment (STSA) Framework. (Left) For a query interaction between nodes $u$ and $v$ at time $t$ , historical one-hop neighbors are retrieved. $u$ and $v$ combined with their respective neighbors form node sequences $S_{u}$ and $S_{v}$ . For each event $i$ in these sequences, its node text embedding $\mathbf{n}_{i}$ , edge text embedding $\mathbf{e}_{i}$ , interaction time delta $\mathbf{t}_{i}$ , within-sequence node frequency embedding $\mathbf{w}_{i}$ , and cross-sequence node frequency embedding $\mathbf{c}_{i}$ are extracted, and the edge text embedding of the query nodes is masked. (Center) All five extracted features $\mathbf{n}_{i}$ , $\mathbf{e}_{i}$ , $\mathbf{t}_{i}$ , $\mathbf{w}_{i}$ , $\mathbf{c}_{i}$ are first independently projected via separate linear layers $\mathbf{W}_{n,e,t,w,c}$ . The spatio-temporal encoder (STE) then fuses the projected temporal and structural features $\mathbf{t}_{i}$ , $\mathbf{w}_{i}$ , and $\mathbf{c}_{i}$ into a unified manifold. This fused spatio-temporal representation is concatenated with the projected node and edge text embeddings to form a single feature vector representing each event in $S_{u}$ and $S_{v}$ . (Right) These event sequences, corresponding to input matrices $\mathbf{X}_{u}$ and $\mathbf{X}_{v}$ , are concatenated on the sequence dimension and jointly processed by a multi-layer Transformer encoder. After the output sequence is split back into distinct representations for $S_{u}$ and $S_{v}$ , mean pooling, with a mask to exclude padding tokens, is applied to form the final node representations $\mathbf{z}_{u}$ and $\mathbf{z}_{v}$ . These representations are passed through a linear task head $\mathbf{W}_{\mathrm{CSF}}$ to yield $\mathbf{z}_{pred}$ , which is optimized via Contrastive Semantic Forecasting (CSF) to predict the masked query edge, anchoring the model to the multi-domain textual latent space.

Figure 2: Analysis of Spatio-Temporal Representations. Principal Component Analysis (PCA) of spatio-temporal representations colored by $\log(1+\Delta t)$ across four synthetic graph topologies. Top 3 : Under domain transfer, concatenated time and node frequency features suffer from representational fragmentation and a restricted Effective Rank (eRank). Bottom : STE fuses the temporal and structural features into a unified manifold and substantially improves eRank.

GCAL
Method	Amazon	Enron	GDELT	Google	ICEWS	Stack E	Stack U	Yelp	Avg
Majority	15.76 ±0.00	4.06 ±0.00	0.06 ±0.00	15.93 ±0.00	0.12 ±0.00	42.82 ±0.00	44.65 ±0.00	13.99 ±0.00	17.17 ±0.00
Random	15.26 ±0.04	7.00 ±0.06	0.16 ±0.01	15.02 ±0.06	0.13 ±0.01	46.72 ±0.09	44.81 ±0.10	16.95 ±0.03	18.26 ±0.05
W/O PT	15.76 ±0.00	4.20 ±1.60	0.04 ±0.03	15.93 ±0.00	0.09 ±0.04	42.82 ±0.00	44.65 ±0.00	13.99 ±0.00	17.19 ±0.21
W/ PT	15.76 ±0.00	3.59 ±2.61	0.05 ±0.02	15.93 ±0.00	0.10 ±0.05	42.82 ±0.00	44.65 ±0.00	13.99 ±0.00	17.11 ±0.34
IDOL

Table 1: Dynamic Edge Classification Results. Values report average Macro F1 (%)

\pm

standard deviation over 10 runs. The best performance in each column is shown in bold . Statistical significance of the best-in-column result over the strongest baseline is indicated by

{}^{\ast}p<0.10

{}^{\ast\ast}p<0.05

, and

{}^{\ast\ast\ast}p<0.01

under a two-tailed paired

t

-test for individual datasets and Wilcoxon signed-rank test for the average. In each method block, the superior variant for each dataset is underlined , comparing models with pretraining ( W/ PT ) and without pretraining ( W/O PT ).

Figure 3: Parametric K-Shot Linear Probe Performance. Average Macro F1 across all target datasets under the leave-one-domain-out (LODO) protocol using a frozen pretrained backbone and a Logistic Regression classifier fitted on $K$ shots per class.

Variant	Amazon	Enron	GDELT	Google	ICEWS	Stack E	Stack U	Yelp	Avg
Reference Baseline
BoE	24.61 ±0.86	7.71 ±0.38	0.28 ±0.03	18.11 ±1.16	0.54 ±0.11	54.22 ±1.34	51.61 ±1.27	26.34 ±2.34	22.93 ±0.94
STSA Variants
w/o STE	24.14 ±0.54	8.18 ±0.38	0.31 ±0.02	18.07 ±0.56	0.70 ±0.09	51.76 ±2.69	51.16 ±1.72	28.16 ±0.53	22.81 ±0.82
CSF $\to$ SF	23.79 ±0.47	8.34 ±0.42	0.41 ±0.02	20.23 ±0.72	1.12 ±0.06	54.38 ±2.29	52.71 ±0.78	28.16 ±0.38	23.64 ±0.64
CSF $\to$ InfoNCE	21.72 ±0.62	10.53 ±1.21	0.37 ±0.01	18.47 ±0.79	0.95 ±0.06	54.08 ±1.22	50.88 ±0.94	26.97 ±0.54	22.99 ±0.68

Table 2: Ablation study of STSA finetuning for dynamic edge classification. Values report average Macro F1 (%)

\pm

standard deviation for individual datasets over 10 runs with different random seeds. Best performance in each column is shown in bold .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Overview of the Bag of Events (BoE) Baseline. (Left) For a query interaction between nodes $u$ and $v$ at time $t$ , historical one-hop neighbors are retrieved. $u$ and $v$ combined with their respective neighbors form node sequences $S_{u}$ and $S_{v}$ . For each event $i$ in these sequences, its node text embedding $\mathbf{n}_{i}$ and edge text embedding $\mathbf{e}_{i}$ are extracted, and the edge text embedding of the query nodes is masked. (Center) Both extracted features $\mathbf{n}_{i}$ and $\mathbf{e}_{i}$ are first independently projected via separate linear layers $\mathbf{W}_{n,e}$ . These projected embeddings are then concatenated to form a single feature vector representing each event in $S_{u}$ and $S_{v}$ . (Right) These event sequences, corresponding to input matrices $\mathbf{X}_{u}$ and $\mathbf{X}_{v}$ , are processed independently by a multi-layer Transformer encoder. Mean pooling, with a mask to exclude padding tokens, is applied to form the final node representations $\mathbf{z}_{u}$ and $\mathbf{z}_{v}$ . These representations are optimized via a symmetric InfoNCE loss.

Figure 5: Analysis of Spatio-Temporal Representations. Principal Component Analysis (PCA) of spatio-temporal representations colored by combinations of within- and cross-sequence node frequency counts across four synthetic graph topologies. Top 3 : Under domain transfer, concatenated time and node frequency features suffer from representational fragmentation and a restricted Effective Rank (eRank). Bottom : STE fuses the temporal and structural features into a unified manifold and substantially improves eRank.

Dataset	#Nodes	#Edges	#Classes	Domain
Enron	42,711	797,907	10	Corporate communication
GDELT	6,786	1,339,245	236	Political events
ICEWS1819	31,796	1,100,071	266	Political events
Stack elec	397,702	1,262,225	2	Technical Q&A forum
Stack ubuntu	674,248	1,497,006	2	Technical Q&A forum
Amazon movies	293,566	3,217,324	5	E-commerce reviews

Appendix

Table 3: Summary statistics for the DTGB datasets.

Figure 6: Label distributions of the DTGB datasets.

Figure 7: Impact of Unseen-Class Dilution on Macro-F1. Because Macro F1 weights all classes equally, the unobserved classes in the 500 training edges of the finetuning experiment performed in Section 4 drag down the overall score on GDELT and ICEWS. Isolating evaluation to seen classes removes this dilution artifact, revealing larger absolute gains for STSA on GDELT and ICEWS.

Text-Attributed Graph Learning

Text-attributed graph learning combines node text and graph structure to improve classification, retrieval, anomaly detection, and reasoning.

See all 66 papers

An Analysis of Semantic and Spatio-Temporal Co-evolution for Edge Classification on Dynamic Text-Attributed Graphs
Oct 19, 2026Tyler Bonnet, Marek ReiText-Attributed Graph Learning Dynamic Graph Learning
Imperial College London London, United Kingdom
S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs
May 18, 2026Yuhan Wang, Haopeng Zhang, Yibo Ding +6Cross-Modal Alignment Text-Attributed Graph Learning
Beihang University, Beijing, China · Tianjin University, Tianjin, China
TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning
Jun 30, 2026Lingjie Chen, Yuanchen Bei, Haobo Xu +3Masked Diffusion Models Link Prediction
UIUC · VISA

Oct 19, 2026J/K move · Enter open · S save

An Analysis of Semantic and Spatio-Temporal Co-evolution for Edge Classification on Dynamic Text-Attributed Graphs

Tyler Bonnet, Marek Rei

Imperial College London London, United Kingdom

Text-Attributed Graph Learning Dynamic Graph Learning

Open paper AskPDF arXiv

Dynamic edge classification on text-attributed Web graphs, such as review networks, community forums, and news-derived knowledge graphs, requires high quality, adaptive semantic representations in the context of network evolution. Existing dynamic graph architectures typically initialize text feature vectors with pretrained language models and hold them static during training to preserve their semantic structure, forgoing the potential benefits of allowing them to co-evolve with network topology. In this work, we investigate the effects of enabling such co-evolution with a unified text-graph alignment framework that integrates a lightweight Transformer adapter to update token-level text representations via spatio-temporal gradients from a contrastive pretraining objective. To prevent detrimental semantic distortion, a language model maintenance objective anchors the evolving text embeddings to their original semantic structure. Evaluations on the Dynamic Text-Attributed Graph Benchmark demonstrate that this anchored text-graph co-evolution consistently improves upon the edge classification performance of frozen-text baselines across diverse Web-related domains. Representational analyses reveal that the framework reduces anisotropy, improving the linear separability of text embeddings, and effectively encodes structural and temporal information into the semantic space. A computational efficiency analysis confirms that this beneficial semantic and spatio-temporal co-evolution is achieved without incurring the prohibitive computational overhead typical of end-to-end joint training.

Abstract

Text-Attributed Graph Learning

Explore similar work

An Analysis of Semantic and Spatio-Temporal Co-evolution for Edge Classification on Dynamic Text-Attributed Graphs

S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs

TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning