TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs

Moran Beladev; Or Eitan; Gilad Katz; Lior Rokach

cs.LGSep 25, 2026

TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphs

Authors: Moran Beladev, Or Eitan, Gilad Katz, Lior Rokach

Organizations: Department of Information Systems Engineering Ben-Gurion University of the Negev, Israel

Abstract

Dynamic text-attributed graphs (DTAGs), where nodes, edges, and textual attributes evolve over time, are crucial in applications such as social networks, citation graphs, and knowledge graphs. However, existing approaches struggle to jointly model the temporal evolution of graph structures and the semantic richness of textual attributes. While Temporal Graph Neural Networks (TGNNs) capture evolving node relationships, they often lack contextual text reasoning. Conversely, Large Language Models (LLMs) excel in textual understanding but struggle with structured graph reasoning in temporal settings. To bridge this gap, we propose TemporalGraphLLM, a novel framework that can integrate any temporal GNN with an LLM for enhanced reasoning in DTAGs. Our approach fine-tunes LLMs using graph-time-aware instruction tuning and novel temporal GNNs injection to replace dedicated added tokens with graph embeddings. TemporalGraphLLM effectively leverages pretrained TGNNs within an LLM framework to achieve state-of-the-art performance on edge classification, link prediction, and edge-based text generation tasks. Extensive evaluation on real-world dynamic graph datasets demonstrates state-of-the-art performance. Our findings highlight the synergistic potential of LLMs and TGNNs, opening new directions for learning on evolving graphs.

Figures & tables

Figure 1: A Dynamic Text-Attributed Graph (DTAG) example, where nodes and edges evolve over time with textual attributes. Our work aims to enhance predictive capabilities over such a structure.

Figure 2: Overview of the TemporalGraphLLM architecture. (Step #1) A TGNN computes time-aware node embeddings by incorporating temporal neighborhood information. (Step #2) These embeddings are projected into the LLM space and injected into a prompt alongside node/edge text in temporal context. (Step #3) During fine-tuning, TGNN and LLM embeddings are fused for edge prediction using an MLP, and the LLM decoder is trained via next-token prediction.

Dataset	Nodes	Edges	Edge	Timestamps	Domain	Text	Bipartite
			Categories			Attributes	Graph
Enron	42,711	797,907	10	1,006	E-mail	Node & Edge	No
GDELT	6,786	1,339,245	237	2,591	Knowledge graph	Node & Edge	No
Stack elec	397,702	1,262,225	2	5,224	Multi-round dialogue	Node & Edge	Yes
Googlemap CT	111,168	1,380,623	5	55,521	E-commerce	Node & Edge	Yes
Amazon movies	293,566	3,217,324	5	7,287	E-commerce	Node & Edge	Yes

Table 1: Statistics of datasets and comparison with existing datasets.

Method	Graph-Model	LLM	Enron		Googlemap CT		Stack Elec		GDELT		Amazon Movies
			Acc.	F1	Acc.	F1	Acc.	F1	Acc.	F1	Acc.	F1
Graph-based methods	DyGFormer	-	0.7390	0.7177	0.8336	0.7919	0.7983	0.8353	0.7583	0.7529	0.8293	0.8103
	GraphMixer	-	0.7270	0.7083	0.8436	0.8102	0.7833	0.8239	0.7541	0.7652	0.8190	0.8249
	TGAT		0.7225	0.7220	0.8276	0.8081	0.7858	0.8436	0.7539	0.7603	0.8245	0.8267
LLM-based methods	-	Llama3-1B	0.7085	0.7354	0.7055	0.6549	0.7019	0.7352	0.6695	0.6671	0.7277	0.7494
	-	Llama3-3B	0.6792	0.7230	0.7078	0.6450	0.7146	0.7318	0.6470	0.6451	0.7417	0.7468

Table 2: Results for the Link Prediction task.

Method	Graph Models	LLM	Enron			Googlemap CT			Stack Elec			GDELT			Amazon Movies
			F1 (W)	Acc.	F1 (M)	F1 (W)	Acc.	F1 (M)	F1 (W)	Acc.	F1 (M)	F1 (W)	Acc.	F1 (M)	F1 (W)	Acc.	F1 (M)
Graph-based methods	DyGFormer	-	0.4180	0.4288	0.2404	0.5816	0.6684	0.2444	0.6615	0.7556	0.4438	0.0684	0.1140	0.0056	0.5654	0.6711	0.2111
	GraphMixer	-	0.4209	0.4338	0.2324	0.5695	0.6691	0.2202	0.6602	0.7567	0.4453	0.0759	0.1192	0.0067	0.5615	0.6719	0.1827
	TGAT	-	0.3397	0.3840	0.1441	0.5600	0.6695	0.2077	0.6545	0.7579	0.4337	0.0765	0.1128	0.0074	0.5669	0.6720	0.1972
LLM-based methods	-	Llama3-1B	0.5563	0.5496	0.1364	0.5938	0.6357	0.2648	0.6550	0.7298	0.0171	0.0588	0.0746	0.0068	0.5701	0.6438	0.2127
	-	Llama3-3B	0.5625	0.5518	0.4134	0.5953	0.6437	0.2727	0.6523	0.7452	0.4056	0.0655	0.0805	0.0110	0.5776	0.6487	0.2109

Table 3: Edge Classification results across datasets. F1(W) denotes the weighted F1 score, while F1(M) represents the macro-averaged F1 score across categories.

Method	Graph Model	LLM	Enron			Googlemap CT			Stack Elec			GDELT			Amazon Movies
			BERT-F1	SBERT	R1	BERT-F1	SBERT	R1	BERT-F1	SBERT	R1	BERT-F1	SBERT	R1	BERT-F1	SBERT	R1
LLM-based methods	-	Llama3-1B	0.7801	0.2192	0.1046	0.8483	0.3123	0.1044	0.8396	0.2199	0.1416	0.8630	0.2573	0.1604	0.8569	0.3665	0.1449
	-	Llama3-3B	0.8001	0.1783	0.075	0.8159	0.1596	0.0213	0.8048	0.0394	0.0182	0.8621	0.2564	0.1619	0.8464	0.2006	0.0459
	-	Mistral-7B	0.7789	0.2742	0.1448	0.8446	0.3245	0.1069	0.5275	0.1186	0.0622	0.8645	0.2596	0.1793	0.6588	0.1873	0.0354
Ours – Temporal Graph LLM	DyGFormer	Llama3-1B	0.8335	0.3811	0.2318	0.8521	0.3244	0.1087	0.8396	0.2199	0.1420	0.8639	0.2609	0.1629	0.8674	0.3743	0.1528
		Llama3-3B	0.8481	0.4409	0.2860	0.8259	0.1592	0.0211	0.8020	0.0403	0.0186	0.8632	0.2630	0.1622	0.8419	0.1613	0.0230

Table 4: Results for edge text generation task. BERT-F1 represents BERTScore F1, SBERT represents Sentence-BERT embedding similarity, and R1 represents ROUGE-1.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Edge Category Classification Prompt Template

Figure 4: Link Prediction Prompt Template

Figure 5: Edge Text Generation Prompt Template

LLM backbone	LLM-only	TGNN $\times$ LLM	Rel. change
Llama-1B	8h	9h	+12%
Llama-3B	19h	13h	-32%
Mistral-7B	28h	21h	-25%

Appendix

Table 5: End-to-end wall-clock training time on the Google dataset. TGNN integration adds minor overhead for smaller LLMs but reduces total training time for larger backbones, consistent with faster convergence and more stable optimization.

Model	Temporal Neighbor Sampling			Random Neighbor Sampling
Model	F1 (Weighted)	Acc	F1 (Macro)	F1 (Weighted)	Acc	F1 (Macro)
DyGFormer + Llama3-1B	0.5660	0.5587	0.4239	0.5494	0.5497	0.2391
GraphMixer + Llama3-1B	0.5626	0.5574	0.4102	0.5615	0.5467	0.1695
TGAT + Llama3-1B	0.5710	0.5653	0.4239	0.5623	0.5502	0.2528

Appendix

Table 6: Ablation (1): Temporal vs. Random Neighbor Sampling (

k{=}5

) on Enron with Llama3-1B. Temporal sampling consistently improves performance across all TGNN backbones.

Enron
Dataset / Model	$k{=}0$			$k{=}3$			$k{=}5$			$k{=}10$			$k{=}15$
Dataset / Model	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)
DyGFormer + Llama3-1B	0.4491	0.4384	0.2619	0.5360	0.5259	0.3847	0.5660	0.5587	0.4239	0.6133	0.6069	0.4696	0.5707	0.5731	0.4144
GraphMixer + Llama3-1B	0.4349	0.4246	0.2824	0.5366	0.5308	0.3886	0.5626	0.5574	0.4102	0.5937	0.5801	0.4492	0.5777	0.5729	0.4189
TGAT + Llama3-1B	0.4573	0.4499	0.2929	0.5435	0.5364	0.3792	0.5710	0.5653	0.4239	0.5938	0.5859	0.4422	0.5939	0.5860	0.4879
Googlemap CT

Appendix

Table 7: Ablation (2): Effect of the Number of Sampled Neighbors (

k

) on Enron and Googlemap CT with Llama3-1B.

Figure 6: Ablation on the number of sampled temporal neighbors ( $k$ ) for the edge classification task (F1-Weighted metric). Performance improves up to $k{=}10$ and plateaus thereafter across both Enron and Googlemap CT datasets.

Figure 7: Effect of loss weighting $(\lambda_{1},\lambda_{2})$ on F1-Weighted for edge classification. Each color represents a distinct configuration.

Enron
Dataset / Model	$\lambda_{1}{=}1,\lambda_{2}{=}1$			$\lambda_{1}{=}0,\lambda_{2}{=}1$			$\lambda_{1}{=}1,\lambda_{2}{=}0$			$\lambda_{1}{=}1,\lambda_{2}{=}2$			$\lambda_{1}{=}2,\lambda_{2}{=}1$
Dataset / Model	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)	F1(W)	Acc	F1(M)
DyGFormer + Llama3-1B	0.6133	0.6069	0.4696	0.6017	0.5901	0.4461	0.6017	0.5880	0.4583	0.6065	0.5943	0.4728	0.6211	0.6098	0.4799
GraphMixer + Llama3-1B	0.5937	0.5801	0.4492	0.6011	0.5865	0.4132	0.5854	0.5741	0.4331	0.5985	0.5907	0.4430	0.6088	0.5946	0.4510
TGAT + Llama3-1B	0.5938	0.5859	0.4422	0.5976	0.5883	0.4475	0.6032	0.5912	0.4365	0.6072	0.5927	0.4705	0.6156	0.5988	0.4904
Googlemap CT

Appendix

Table 8: Ablation (3): Effect of Loss Weight Configuration on Enron and Googlemap CT with Llama3-1B. Best F1(W) per dataset in bold.

Text-Attributed Graph Learning

Text-attributed graph learning combines node text and graph structure to improve classification, retrieval, anomaly detection, and reasoning.

See all 64 papers

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs
Sep 8, 2026Longfei Ma, Zemin Liu, Fei WuTemporal GNNs Semantic Drift in Language Models
1Zhejiang University
LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning
Feb 16, 2026Wang Xing, Wei Song, Siyu Lin +2Knowledge Distillation Temporal KGs
School of Computer Science and Technology, Xidian University · School of Computing and Artificial Intelligence, Southwest Jiaotong University · School of Information Science and Engineering, Chongqing Jiaotong University
Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs
Oct 20, 2026Jiapu WangTemporal Reasoning in Language Models Temporal KGs
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu, 210094, China

Sep 8, 2026J/K move · Enter open · S save

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs

Longfei Ma, Zemin Liu, Fei Wu

1Zhejiang University

Temporal GNNs Semantic Drift in Language Models Temporal Link Prediction LLMs for Graph Learning

Open paper AskPDF arXiv

Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, existing benchmarks primarily emphasize structural evolution via temporal link prediction (TLP), while support for semantic evolution remains limited. Although temporal node classification (TNC) is sometimes included, it is typically restricted to simplistic binary settings that fail to capture realistic semantic drift. Moreover, commonly used datasets exhibit high link repetition, leading to inflated performance estimates and obscuring true model capability. To address these limitations, we introduce TTGBench, a new benchmark that jointly evaluates structural and semantic evolution. TTGBench comprises six real-world, text-rich datasets characterized by Dual Volatility, enabling rigorous and fair evaluation of existing models. Notably, it is the first benchmark to support both multi-class and multi-label TNC, filling a critical gap in evaluating temporal semantic drift. We conduct a comprehensive evaluation of 17 state-of-the-art methods across Temporal Graph Neural Networks (TGNNs) and Large Language Model (LLM)-based paradigms. The results reveal a clear capability divide between the two paradigms: TGNN-based methods excel at structural prediction but fail at semantic tracking, whereas LLM-based predictors show the opposite trend. Through in-depth analysis, we uncover their fundamental limitations and provide insights for developing more comprehensive temporal graph models.

Abstract

Text-Attributed Graph Learning

Explore similar work

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs

LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning

Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs