Autoregressive transformer training through next-token prediction has driven remarkable advances in language modeling and has become increasingly effective for time-series modeling. Yet its success in temporal graph learning remains limited, and most temporal link-prediction methods are discriminative rather than autoregressive. While temporal graphs can have a natural serialization through chronological ordering edges, they lack an obvious graph-independent tokenization: existing node tokenization often uses a vocabulary that grows with the number of nodes, limiting scalability and cross-graph training. We propose GTGT, Generative Temporal Graph Transformer, a simple yet powerful vocabulary-free framework for self-supervised next-event prediction on temporal graphs. It predicts events for observed destinations, behavioral descriptors for novel destinations, and time gaps before new event. These targets have shared meanings across graphs, allowing one transformer to train across disjoint node sets without node-specific learned parameters. Local node identifiers let attention capture graph structure: Theoretically, we can show that our transformer-based model has the capacity for time-respecting message passing mechanisms. On Temporal Graph Benchmark (TGB), our GTGT not only achieves comparable or better link-prediction performance than established baselines, but also has much lower training cost. These results open the door to temporal graph pretraining for forecasting and other downstream tasks, providing a concrete step toward temporal graph foundation models.
Figures & tables
Figure 1: GTGT predicts the next destination and time gap from event histories. Local identifiers let causal attention match shared endpoints without a learned node vocabulary.
Method
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
EdgeBank tw
0.571
0.141
0.387
0.580
Recency (no parameters)
0.817
0.166
0.833
0.783
Popularity (no parameters)
0.040
0.013
0.603
0.607
TGN
0.611
0.220
0.791
0.637
GraphMixer
0.519
0.249
0.772
0.760
DyGFormer
0.780
0.395
0.843
0.747
Table 1: Link prediction on TGB (test Mean Reciprocal Rank, or MRR, higher is better). Results for all nine baselines are in Appendix G . Per-corpus uses separate models; joint shares one model. SimpleDyG is our implementation following the original paper. Bold marks the highest MRR. GTGT per-corpus: mean ± std over three seeds; joint: seed 0 ; budgets and test sets in Appendix F .
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Neither
+ Time only
+ Identity only
+ Time + Identity
tgbl-wiki
0.494 ± 0.007
0.656 ± 0.014
0.982 ± 0.001
0.991 ± 0.001
tgbl-lastfm
0.506 ± 0.009
0.524 ± 0.009
0.962 ± 0.010
0.968 ± 0.017
tgbl-coin
0.501 ± 0.019
0.542 ± 0.003
0.976 ± 0.004
0.971 ± 0.005
tgbl-flight
0.506 ± 0.006
0.510 ± 0.010
0.986 ± 0.001
0.963 ± 0.007
Appendix
Table 2: Same-node prediction with time and local identifiers. AUC on the content-free task, with each channel enabled or disabled. The Neither column, with no time and no identifier, is the control: the label is a pure structure function, so with both time and identifiers removed, AUC is near chance. ( 0.5 ). Mean ± standard deviation over three seeds.
Probe
States
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
Source co-reference
trained
0.957
0.832
0.705
0.936
untrained
0.914
0.664
0.623
0.787
input embeddings
0.876
0.685
0.561
0.826
Destination co-reference
trained
0.945
0.752
0.886
0.912
untrained
0.931
0.886
0.914
0.946
input embeddings
0.888
0.867
0.907
0.945
Appendix
Table 3: Linear probes on frozen representations of the seed- 0 per-corpus models of Table 1 . Co-reference: AUC on 4,000 balanced token pairs from held-out contexts; neighbour retrieval: recall@ 10 over 1,000 queries. Untrained applies the same probes to a network with the same architecture and random weights; input embeddings applies them to the token embeddings before the first layer.
Method
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
EdgeBank tw
0.571
0.141
0.387
0.580
EdgeBank ∞
0.495
0.114
0.167
0.359
Recency (no parameters)
0.817
0.166
0.833
0.783
Popularity (no parameters)
0.040
0.013
0.603
0.607
TGAT
0.562
0.231
0.713
0.667
TGN
0.611
0.220
0.791
0.637
Appendix
Table 4: Full link-prediction comparison on TGB (test MRR): all nine baseline configurations and our two training settings. Per-corpus uses one model per dataset (mean ± std over three seeds); joint uses one model trained on all four (seed 0 ). SimpleDyG is our implementation following the original paper, on the same backbone. Bold marks the best result in each column. OOT denotes an evaluation that did not complete within the available budget.
Recurrent
Novel
Dataset
Recurrent (%)
Recency
GTGT
Recency
GTGT
tgbl-wiki
89.2
0.916
0.921
0.002
0.013
tgbl-lastfm
87.6
0.189
0.381
0.002
0.003
tgbl-flight
93.5
0.872
0.931
0.278
0.183
tgbl-coin
79.4
0.913
0.913
0.283
0.317
Appendix
Table 5: Link prediction on recurrent and novel queries (test MRR, seed- 0 per-corpus models, same queries and candidates as Table 1 ). A query is recurrent when its true destination is in the source’s recency list at query time ( 256 slots, 1,024 on tgbl-coin ), and novel otherwise. Recency is the no-parameter control of Table 1 , which ranks each candidate by the time of its latest event with the source.
Model
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
Recency (no parameters)
0.817
0.166
0.833
0.783
GTGT per-corpus
0.823
0.333
0.884
0.793
GTGT joint (all four)
0.822
0.213
0.892
0.793
GTGT leave-one-out (zero-shot)
0.805
0.151
0.888
0.766
Appendix
Table 6: Transfer to an unseen graph (test MRR). Leave-one-out trains one model on the other three graphs and evaluates it zero-shot on the held-out graph, which it never sees during training; joint trains on all four; per-corpus trains on the target graph only. Recency is the no-parameter control of Table 1 . Per-corpus: mean over three seeds; seed 0 otherwise; joint and leave-one-out models use 10 k steps and one slot capacity ( 1,024 ).
Corpus
Events
Src
Dst
Recur.
Times
Days
Bip.
Cand.
tgbl-wiki
157K
8.2K
1.0K
0.884
153K
31
yes
1000
tgbl-lastfm
1.3M
980
1.0K
0.880
1.3M
1,587
yes
1000
tgbl-flight
67M
17K
18K
0.966
1.4K
1,399
no
21
tgbl-coin
23M
613K
638K
0.831
1.3M
214
no
21
Appendix
Table 7: Dataset statistics. Src/Dst count distinct source/destination nodes. Recur. is the fraction of events whose pair appeared earlier. Times counts distinct timestamps; Days is the span. Bip. indicates disjoint source/destination node sets. Cand. counts the positive and evaluation negatives. Each dataset has one edge type.
Corpus
Events
Users
Items
Recurrence
Edge types
tgbl-wiki
157,474
8,227
1,000
0.8841
1
tgbl-lastfm
1,293,103
980
1,000
0.8801
1
tgbl-flight
67,169,570
16,706
18,143
0.9656
1
tgbl-coin
22,809,486
612,852
638,486
0.8307
1
Appendix
Table 8: Exact counts for the four datasets. Users and items denote source and destination roles. Recurrence measures previously observed source–destination pairs.
Corpus
P(ans)
Cold
Contextual
tgbl-wiki
0.8920
0.0007
0.1073
tgbl-lastfm
0.8761
0.0000
0.1239
tgbl-flight
0.9364
0.0000
0.0636
tgbl-coin
0.7452
0.0080
0.2468
Appendix
Table 9: Test-set coverage at recency cap 256 . P(ans) is the fraction of destinations in the source’s retained list. Cold destinations have no prior activity; contextual destinations are outside the list but have prior activity. The three categories partition the split.
Figure 2: Mean test MRR versus total training time across four datasets, for the 6 k-step configuration.
Model
tgbl-wiki
tgbl-lastfm
tgbl-flight
DyRep
0.69 M
0.57 M
0.58 M
TGAT
0.65 M
0.52 M
0.53 M
TGN
0.96 M
0.78 M
0.80 M
CAWN
2.24 M
1.06 M
1.11 M
TCL
0.86 M
0.83 M
0.83 M
GraphMixer
0.61 M
0.58 M
0.59 M
Appendix
Table 10: Trainable parameter counts. The vocabulary-free row describes the supplied fixed reference configuration. Other run configurations, including the joint model, can differ. Non-trainable histories and memory buffers are excluded.
Model
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
TGN
6.2 min
40.8 min
13.7 h
2.24 h
GraphMixer
12.1 min
54.1 min
6.89 h
3.65 h
TCL
13.6 min
2.21 h
7.20 h
2.73 h
TGAT
31.4 min
2.57 h
25.5 h
4.99 h
CAWN
1.09 h
4.08 h
OOT
OOT
DyGFormer
15.2 min
1.54 h
5.01 h
2.61 h
Appendix
Table 11: Recorded training-time estimates on an L40S GPU. Evaluation is excluded. Values are retained from the source. OOM indicates an allocation failure; OOT indicates an unfinished evaluation. Bold identifies our row, not the fastest entry.
Method
tgbl-wiki
tgbl-lastfm
tgbl-flight
tgbl-coin
Negatives per positive
999
999
20
20
DyRep
110
109
3,732
669
TGAT
4.8
5.2
230
254
TGN
113
114
3,787
670
CAWN
2.6
2.1
11.9
26.8
TCL
44
45
2,052
2,058
Appendix
Table 12: Inference throughput on an idle NVIDIA L40S. Positives ranked per second, including the stated evaluation negatives. Baseline batch sizes are 32 on tgbl-wiki / tgbl-lastfm ( 16 for CAWN) and 200 on the larger datasets. Slot capacity also differs: the tgbl-coin run uses 1,024 . Compare methods within a dataset; these are implementation-level measurements.
Model
tgbl-wiki
tgbl-lastfm
GTGT (full)
0.823
0.251
− local identifiers
0.823
0.247
− descriptor prediction
0.823
0.244
− recency pointer
0.472
0.092
− slot-bias head
0.819
0.229
Appendix
Table 13: Component ablations (test MRR, seed 0 , 6 k training steps). Each variant removes one component from the per-corpus model. Local identifiers : the identifier attention bias is set to zero. Descriptor prediction : the descriptor head outputs a uniform distribution, so novel candidates are ranked only by pnov and ∣Ct(v)∣ . Recency pointer : the pointer output is removed and every candidate is scored by the descriptor term of Equation 1 . Slot-bias head : the per-slot bias from history statistics is removed. tgbl-lastfm uses a 50 k-query test subset.
Candidates
tgbl-wiki
tgbl-lastfm
21
0.916
0.724
101
0.884
0.543
1,000
0.823
0.334
Appendix
Table 14: Candidate-width sweep. Test MRR of the seed- 0 per-corpus models of Table 1 with 20 , 100 , or all 999 official negatives per positive. Smaller widths keep a random subset of the official negatives.
tgbl-wiki
tgbl-lastfm
tgbl-coin
Method
NLL
Acc.
NLL
Acc.
NLL
Acc.
Random
2.565
7.7
2.565
7.7
2.565
7.7
Majority
1.920
28.4
2.264
28.9
1.836
29.5
Persistence (last gap)
1.878
27.3
1.742
31.4
1.719
39.9
Poisson process (per user)
1.896
28.4
1.920
28.9
1.620
38.7
Per-user empirical distribution
1.760
33.3
1.482
43.6
1.352
47.9
Appendix
Table 15: Next-event time-gap prediction (NLL in nats, lower is better / bucket accuracy %, 13 buckets; 10,000 test queries per dataset, seed 0 ). Baselines and GTGT use the same source history up to tlast and have no access to the query time; baseline weights are chosen on validation queries. Random reports its expected accuracy ( 1/13 ). GTGT variants: without the gap-history head ( tgbl-wiki 6 k, tgbl-lastfm 20 k, tgbl-coin 10 k steps) and with it ( tgbl-wiki and tgbl-lastfm 6 k, tgbl-coin 10 k steps). tgbl-flight is omitted: its timestamps have day resolution, so 99.7% of targets fall in one bucket (1 day–1 week).
Recurrent/novel
Pointer
Dataset
Majority
Model
Accuracy
MRR
tgbl-wiki
0.927
0.940
0.883
0.922
tgbl-lastfm
0.879
0.889
0.299
0.380
tgbl-flight
0.938
0.938
0.209
0.309
tgbl-coin
0.795
0.846
0.449
0.561
Appendix
Table 16: Complete next-event component metrics. All values are test accuracies except pointer MRR. Pointer metrics are conditional on recurrent events. Time-gap results are in Table 15 .
Most temporal graph learning methods reduce future prediction to discriminative inference over past interactions, focusing on edges or node labels for a largely persistent set of nodes. This view breaks down when new nodes and edges appear, since the future graph becomes a new object with changing size, composition, and structure. We address this gap by framing temporal graph prediction as an inverse topology problem. Instead of predicting edges directly, we first predict a multiscale topological descriptor of the future graph and then reconstruct a plausible future snapshot that realizes this descriptor under inductive constraints. This approach makes global structure and node churn explicit prediction targets and produces forecasted graphs that downstream tasks can evaluate without retraining. Across 14 temporal graph datasets, we evaluate our method, TOPOGED, on node, link, and graph property prediction and compare it against state-of-the-art temporal graph models. TOPOGED significantly improves node-level forecasting accuracy over the strongest baseline, with a macro average of 0.49 versus 0.05. It also outperforms baselines in 60% of graph-structure metric evaluations and substantially increases macro-average edge prediction metrics, from near-zero to 0.12. Our results show that topology-guided graph forecasting can predict inductive future snapshots whose structure supports multiple downstream evaluations.
Ronan Buck, Kiarash Shamsi, Tran Gia Bao Ngo +3
University of Central Florida Orlando, FL, USA · University of Manitoba Winnipeg, Canada · Florida State University Tallahassee, FL, USA +1
Temporal graphs are increasingly used to model dynamic systems across diverse domains, including social, financial, and traffic networks. Predicting both what the next event will be and when it will occur in these systems is crucial for understanding and anticipating complex behaviors, but has not been studied much yet, especially in large-scale graphs. To address this gap, we propose a scalable unified mathematical framework that captures varying degrees of complexity across temporal graphs with events occurring on the edges. Our framework is flexible and expressive enough to accommodate a wide range of network structures and temporal dynamics. Building upon this analysis, we introduce our novel scalable approach for jointly predicting the next event and its occurrence time. Empirical evaluations across multiple datasets demonstrate that our method consistently outperforms existing techniques, particularly in scenarios involving irregular event patterns and complex temporal dependencies. These findings highlight the potential of our framework as a robust foundation for future research in temporal event prediction.
Mohammad Ostadmohammadi, Sepehr Kazemi, Hamid R. Rabiee
Department of Computer Engineering, Sharif University of Technology, Azadi Ave., Tehran, Iran · Department of Electrical Engineering, Sharif University of Technology, Azadi Ave., Tehran, Iran
Networks of timestamped interactions arise across social, financial, and biological domains, where forecasting future events requires modeling both evolving topology and temporal ordering. Temporal link prediction methods typically frame the task as binary classification with negative sampling, discarding the sequential and correlated nature of real-world interactions. We introduce STEP (STochastic Event Predictor), a framework that reformulates temporal link prediction as a sequential forecasting problem in continuous time. STEP models event dynamics through discrete temporal motif transitions governed by Poisson processes, maintaining a set of open motif instances that evolve as new interactions arrive. At each step, the framework decides whether to initiate a new temporal motif or extend an existing one, selecting the most probable event via Bayesian scoring of temporal likelihoods and structural priors. STEP also produces compact, temporal motif-based feature vectors that can be concatenated with existing temporal graph neural network outputs, enriching their representations without architectural modifications. Experiments on five real-world datasets demonstrate up to 21% average precision gains over state-of-the-art baselines in classification and 0.99 precision in next k sequential forecasting, with consistently lower runtime than competing motif-aware methods.