An Analysis of Semantic and Spatio-Temporal Co-evolution for Edge Classification on Dynamic Text-Attributed Graphs

Tyler Bonnet; Marek Rei

cs.LGOct 19, 2026

An Analysis of Semantic and Spatio-Temporal Co-evolution for Edge Classification on Dynamic Text-Attributed Graphs

Authors: Tyler Bonnet, Marek Rei

Organizations: Imperial College London London, United Kingdom

Abstract

Dynamic edge classification on text-attributed Web graphs, such as review networks, community forums, and news-derived knowledge graphs, requires high quality, adaptive semantic representations in the context of network evolution. Existing dynamic graph architectures typically initialize text feature vectors with pretrained language models and hold them static during training to preserve their semantic structure, forgoing the potential benefits of allowing them to co-evolve with network topology. In this work, we investigate the effects of enabling such co-evolution with a unified text-graph alignment framework that integrates a lightweight Transformer adapter to update token-level text representations via spatio-temporal gradients from a contrastive pretraining objective. To prevent detrimental semantic distortion, a language model maintenance objective anchors the evolving text embeddings to their original semantic structure. Evaluations on the Dynamic Text-Attributed Graph Benchmark demonstrate that this anchored text-graph co-evolution consistently improves upon the edge classification performance of frozen-text baselines across diverse Web-related domains. Representational analyses reveal that the framework reduces anisotropy, improving the linear separability of text embeddings, and effectively encodes structural and temporal information into the semantic space. A computational efficiency analysis confirms that this beneficial semantic and spatio-temporal co-evolution is achieved without incurring the prohibitive computational overhead typical of end-to-end joint training.

Figures & tables

Figure 1. Overview of the Unified Text-Graph Alignment Framework. (Left) For a query interaction between nodes $u$ and $v$ at time $t$ , historical one-hop neighbors are retrieved. $u$ and $v$ combined with their respective neighbors form node sequences $S_{u}$ and $S_{v}$ . For each event in these sequences, its raw node text attribute $n$ , raw edge text attribute $e$ , interaction time delta $t$ , cross-sequence neighbor frequency $c$ , and within-sequence neighbor frequency $w$ are extracted, and the edge text attribute of the query nodes is masked. (Center) $n$ and $e$ are processed by a frozen PLM into raw token-level text embeddings, which are subsequently transformed by the PLM adapter. For each node and edge text sequence, the final hidden state corresponding to the EOS token is extracted to serve as the feature vector representing the full text. To prevent semantic collapse, these adapted text feature vectors are anchored to their corresponding raw text embeddings via the language model maintenance (LMM) objective. The text feature vectors and the bypassed temporal and structural feature vectors ( $t,c,w$ ) are independently projected via separate linear layers $\mathbf{W}_{n,e,t,c,w}$ and concatenated. Role-semantic positional encodings (RSPE) are added to the projected vectors to distinguish source query, destination query, source neighbors, and destination neighbors. Distinct CLS tokens are prepended to form the final augmented input matrices $\mathbf{X}_{u}$ and $\mathbf{X}_{v}$ . (Right) The sequences $\mathbf{X}_{u}$ and $\mathbf{X}_{v}$ are processed independently by the Transformer encoder, using shared weights. The CLS tokens are extracted to serve as the final source and destination representations $\mathbf{z}_{u}$ and $\mathbf{z}_{v}$ , which are optimized via the directional role alignment (DRA) objective. A block diagram of the Unified Text-Graph Alignment framework.

Method	Amazon	Google	Yelp	Enron	Stack U	Stack E	GDELT	ICEWS	Avg
Random	15.27 ±0.04	15.03 ±0.05	16.95 ±0.03	7.01 ±0.04	44.76 ±0.05	46.72 ±0.07	0.16 ±0.01	0.13 ±0.00	18.25
Majority	15.76 ±0.00	15.93 ±0.00	13.99 ±0.00	4.06 ±0.00	44.65 ±0.00	42.82 ±0.00	0.07 ±0.00	0.12 ±0.00	17.17
DyRep	16.09 ±0.61	13.95 ±3.58	15.12 ±1.10	8.75 ±0.52	44.91 ±0.32	43.92 ±2.14	0.13 ±0.02	0.14 ±0.02	17.88
JODIE	17.25 ±1.90	17.22 ±1.35	15.17 ±2.34	6.81 ±1.48	44.90 ±0.83	42.98 ±1.63	0.11 ±0.02	0.15 ±0.02	18.07
DVGMAE	16.10 ±0.45	16.23 ±0.46	18.35 ±5.34	10.18 ±0.78	45.59 ±1.88	42.82 ±0.01	0.15 ±0.02	0.22 ±0.14	18.71
IDOL	15.91 ±0.30	15.93 ±0.00	24.80 ±0.10	8.14 ±0.25	44.65 ±0.00	42.82 ±0.00	0.12 ±0.01	0.50 ±0.18	19.11

Table 1. Finetuning results for dynamic edge classification on DTGB datasets. Values show average Macro F1 scores in percent with subscripts denoting

\pm

standard deviation. Bold indicates best performance per dataset. Relative performance gains of UTGA over the base framework DyGnROLE are shown in green superscript. *,

\dagger

, and

\ddagger

respectively indicate statistically significant improvement at

p<0.1

p<0.05

, and

p<0.01

over the best-in-column baseline for individual datasets using a paired two-tailed

t

-test, and over the best overall baseline for the Avg column using a Wilcoxon signed-rank test.

Method	Stack U	Stack E	Avg
DyGnROLE	15.16 ±5.46	21.75 ±6.22	18.45
GraphMixer	26.11 ±13.22	31.12 ±7.48	28.61
TGAT	24.45 ±1.98	31.60 ±4.20	28.02
UTGA	29.55 ${}^{{\color[rgb]{0,0.6,0}+94.9\%}}_{\pm 0.90}$	33.59 ${}^{{\color[rgb]{0,0.6,0}+54.4\%}}_{\pm 3.25}$	31.57 +71.1%

Table 2. Minority class (“useful”) F1 scores on the Stack Exchange datasets. Values show average F1 scores in percent with subscripts denoting

\pm

standard deviation. Bold indicates best performance per dataset. Relative performance gains of UTGA over the base framework DyGnROLE are shown in green superscript.

Figure 2. Average Macro F1 performance across the DTGB benchmark as the number of training labels scales from 1k to 20k. UTGA maintains a consistent and substantial advantage over the top three performing baselines (TGAT, GraphMixer, and DyGnROLE) at every data scale. A line graph plotting Average Macro F1 percentage against the number of training labels at 1k, 5k, 10k, and 20k intervals for UTGA and the top 3 baseline models.

UTGA Variants	Amazon	Google	Yelp	Enron	Stack U	Stack E	GDELT	ICEWS	Avg
w/o Pretraining	18.57 $\pm$ 0.42	15.93 $\pm$ 0.00	30.24 $\pm$ 1.35	8.89 $\pm$ 0.71	47.64 $\pm$ 3.68	47.60 $\pm$ 5.90	0.32 $\pm$ 0.05	1.77 $\pm$ 0.08	21.37
w/o LMM	23.40 $\pm$ 1.02	24.06 $\pm$ 0.35	31.35 $\pm$ 0.44	9.08 $\pm$ 0.77	54.66 $\pm$ 1.06	56.53 $\pm$ 0.60	0.56 $\pm$ 0.01	2.07 $\pm$ 0.12	25.21
MLP PLM Adapter	21.91 $\pm$ 0.25	21.45 $\pm$ 0.40	34.17 $\pm$ 0.35	19.23 $\pm$ 1.13	53.52 $\pm$ 1.05	55.64 $\pm$ 1.02	0.43 $\pm$ 0.04	1.24 $\pm$ 0.06	25.95
UTGA	24.79 $\pm$ 0.56	25.13 $\pm$ 0.32	36.81 $\pm$ 0.23	18.82 $\pm$ 1.13	55.50 $\pm$ 0.19	56.33 $\pm$ 0.75	0.63 $\pm$ 0.03	2.26 $\pm$ 0.17	27.53

Table 3. Ablation results for dynamic edge classification showing average Macro F1 scores (%). Bold indicates best on dataset.

Figure 3. Relationship between log-transformed average node text length and relative Macro F1 performance gain of UTGA over the base framework. The red dashed line represents the Ordinary Least Squares (OLS) fit with a 95% confidence interval band. Markers display dataset-level means with standard error bars. A scatter plot showing a negative correlation between log node text length on the x-axis and Macro F1 relative gain percentage on the y-axis.

Figure 4. Average pairwise cosine distance for edge text embeddings. The representations of the raw Qwen3-Embedding-0.6B exhibit high levels of anisotropy, whereas the UTGA framework effectively reduces this collapse to expand the embedding space. A grouped bar chart comparing the average cosine distance of edge text embeddings before and after applying the UTGA framework across the DTGB datasets.

Embedding	In-Degree	Out-Degree	PageRank	Clustering	Lifespan
Random	25.09 $\pm$ 0.58	24.09 $\pm$ 0.46	25.31 $\pm$ 0.65	33.04 $\pm$ 0.82	21.29 $\pm$ 0.35
Qwen3-Embedding	23.75 $\pm$ 0.08	23.93 $\pm$ 3.53	27.69 $\pm$ 0.35	32.90 $\pm$ 4.83	22.43 $\pm$ 0.00
UTGA	30.02 $\pm$ 3.91	29.55 $\pm$ 1.88	38.85 $\pm$ 0.68	44.86 $\pm$ 6.05	27.03 $\pm$ 1.54

Table 4. Node structural probing analysis on GDELT comparing the linear separability (Macro F1 %) of node text embeddings on graph structural properties. Bold indicates best performance per task.

Embedding	Amazon	Google	Yelp
Random	15.27 $\pm$ 0.04	15.03 $\pm$ 0.05	16.95 $\pm$ 0.03
Qwen3-Embedding	18.99 $\pm$ 0.13	23.79 $\pm$ 0.05	24.22 $\pm$ 0.23
UTGA	36.71 $\pm$ 0.52	35.69 $\pm$ 0.53	44.28 $\pm$ 0.56

Table 5. Edge probing analysis comparing the linear separability (Macro F1 %) of edge text embeddings on e-commerce datasets. Bold indicates best performance per dataset.

Architecture	Params	Time	Samples/s	Mem
DyGnROLE	1.55M	21.24 $\pm$ 6.21	5980.29	171.45
UTGA w/ LoRA	6.86M	4485.74 $\pm$ 1743.91	28.53	4335.87
UTGA	3.58M	79.76 $\pm$ 14.61	1600.62	485.34

Table 6. Computational efficiency on the Enron dataset. Metrics include trainable parameters (Params), average per-batch execution time in milliseconds (Time), overall throughput (Samples/s), and peak GPU memory allocation in megabytes (Mem) during pretraining.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	#Classes	#Words
Amazon	293,566	3,217,324	5	623,447,701
Enron	42,711	797,907	10	187,576,324
GDELT	6,786	1,339,245	236	7,632,528
Google	111,168	1,380,623	5	74,401,703
ICEWS	31,796	1,100,071	266	27,003,734
Stack E	397,702	1,262,225	2	340,401,237

Appendix

Table 7. Summary statistics of the DTGB datasets.

Figure 5. Label distributions of the DTGB datasets. Eight bar charts showing the frequency of different classes for the DTGB datasets.

Figure 6. Average node and edge text lengths of the DTGB datasets. A two-panel bar chart displaying the average word count for node and edge texts across the DTGB datasets.

Text-Attributed Graph Learning

Text-attributed graph learning combines node text and graph structure to improve classification, retrieval, anomaly detection, and reasoning.

See all 66 papers

Sep 26, 2026

Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs

Learning transferable representations for dynamic text-attributed graphs (DyTAGs) requires models to capture underlying interaction dynamics that persist across domains. However, existing methods tend to overfit to domain-specific structural, temporal, and semantic patterns, limiting edge classification performance under distribution shifts. To expose and address this, we formally establish a leave-one-domain-out (LODO) transfer learning protocol for edge classification on DyTAGs. Under this protocol, we demonstrate that state-of-the-art self-supervised methods for dynamic graph learning perform poorly when transferred to unseen domains. Strikingly, existing methods underperform a structurally and temporally unaware Bag of Events (BoE) model we introduce, which inputs only unordered sequences of node and edge text features. Proceeding from the BoE, we propose Spatio-Temporal Semantic Alignment (STSA), which integrates a spatio-temporal encoder that fuses representations of time deltas and node occurrence frequencies into a unified manifold. STSA is trained with a Contrastive Semantic Forecasting objective, which anchors edge representations to a multi-domain textual latent space initialized by a pretrained language model, providing a robust prior that outperforms BoE and all existing methods we evaluate.

Tyler Bonnet, Marek Rei

Imperial College London London, United Kingdom

Text-Attributed Graph Learning Cross-Domain Transfer Learning Dynamic Graph Learning Contrastive Learning

Ask PDF

Sep 25, 2026

TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs

Dynamic text-attributed graphs (DTAGs), where nodes, edges, and textual attributes evolve over time, are crucial in applications such as social networks, citation graphs, and knowledge graphs. However, existing approaches struggle to jointly model the temporal evolution of graph structures and the semantic richness of textual attributes. While Temporal Graph Neural Networks (TGNNs) capture evolving node relationships, they often lack contextual text reasoning. Conversely, Large Language Models (LLMs) excel in textual understanding but struggle with structured graph reasoning in temporal settings. To bridge this gap, we propose TemporalGraphLLM, a novel framework that can integrate any temporal GNN with an LLM for enhanced reasoning in DTAGs. Our approach fine-tunes LLMs using graph-time-aware instruction tuning and novel temporal GNNs injection to replace dedicated added tokens with graph embeddings. TemporalGraphLLM effectively leverages pretrained TGNNs within an LLM framework to achieve state-of-the-art performance on edge classification, link prediction, and edge-based text generation tasks. Extensive evaluation on real-world dynamic graph datasets demonstrates state-of-the-art performance. Our findings highlight the synergistic potential of LLMs and TGNNs, opening new directions for learning on evolving graphs.

Moran Beladev, Or Eitan, Gilad Katz +1

Department of Information Systems Engineering Ben-Gurion University of the Negev, Israel

Text-Attributed Graph Learning LLMs for Graph Learning Temporal GNNs Temporal Link Prediction Instruction Tuning

Ask PDF

Jul 9, 2026

Drift-Aware Temporal Graph Rewiring (DATGR) for Adaptive Semantic Modeling in Biomedical Text

Biomedical language evolves rapidly as new discoveries emerge, causing traditional text models to lose semantic fidelity over time. Static embeddings and co-occurrence graphs cannot capture such evolution, leading to performance degradation in retrieval and knowledge discovery tasks. This paper introduces a Drift-Aware Temporal Graph Rewiring (DATGR) framework that models concept evolution by dynamically updating co-occurrence edges based on estimated semantic drift. Instead of retraining embeddings for each time slice, DATGR performs lightweight, feedback-driven rewiring using a logistic update rule applied to edge weights. Evaluated on the Biomedical Multi-Relation Corpus (BIOMRC), the method achieved a mean Area Under the Receiver Operating Characteristic (AUROC) improvement of approximately 0.066 absolute difference (0.699 vs. 0.633) over a static baseline. Area Under the Precision-Recall Curve (AUPRC) remained comparable (0.738 vs. 0.744), showing that drift-aware adaptation enhances link-prediction recall without a loss in precision. These results demonstrate that edge-level adaptation effectively captures temporal semantic change in evolving biomedical text while remaining computationally efficient and interpretable.

Bharathwaj Vijayakumar, Sahana K. Varadaraju

Institutional Data and Analytics, Rowan University, NJ, USA · Information Resources & Technology, Rowan University, NJ, USA

Graph Structure Learning Temporal Link Prediction Dynamic Graph Learning Semantic Change Detection

Ask PDF

Abstract

Text-Attributed Graph Learning

Explore similar work

Transfer Learning for Edge Classification on Dynamic Text-Attributed Graphs

TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs

Drift-Aware Temporal Graph Rewiring (DATGR) for Adaptive Semantic Modeling in Biomedical Text