RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training
Authors: Yue Dai, Sarah Zhou, Youtao Zhang
Organizations: Department of Computer Science, Illinois Institute of Technology, Chicago, USA · BASIS Independent Silicon Valley, San Jose, CA, USA · Department of Computer Science, University of Pittsburgh, Pittsburgh, USA
Temporal graph neural networks (TGNNs) have shown strong performance on dynamic graph learning tasks by continuously updating node memories from evolving interactions (i.e., events) and leveraging these memories for downstream predictions. As graphs grow rapidly in modern applications, training requires processing an increasing number of events; meanwhile, each event can update node memories used by later events, creating dependencies that constrain parallel execution and limit the benefits of additional computing resources. These costs motivate an upstream question: Are all events, and the dependencies they introduce, necessary for learning? We investigate this question through two roles of temporal events: prediction targets and historical events sampled as inputs. We observe that targets involving stable node memories tend to exhibit lower prediction loss, while filtering historical events associated with stable memories can retain similar training-loss trajectories in several evaluated settings. Together, these observations suggest an opportunity to identify input redundancy through changes in node memories. Building on this opportunity, we propose RAFT, a redundancy-aware input-filtering framework for TGNN training. RAFT tracks memory changes during training and uses them to guide two filters: a root-event filter removes selected prediction targets before training, and a support-event filter excludes selected historical context during sampling. Filtering inputs before their downstream preparation and processing reduces the work they trigger. Across three TGNN models and five datasets, RAFT reduces training inputs by 35.23% on average and up to 97.11%, while reducing recorded training time by up to 52.56% with negligible changes to model precisions.
Figures & tables
Figure 1: Batched memory-based TGNN training. Root events (e.g., e23 and e34 ) are prediction targets, while support events (e.g., e12 , e13 , and e14 ) provide historical inputs. Step 1 samples support events and fetches the required information. Step 2 updates node memories ( sv−→sv+ ) and computes embeddings hv . Step 3 predicts root events (e.g., e~(ti)=f(h2,h3) ), computes the loss, and updates model parameters through backpropagation.
Figure 2: Root-event loss during TGN training on WIKI, REDDIT, and WIKI-TALK. (a) Stable root events have lower mean positive-event loss than unstable events. (b) Their share of training loss is considerably smaller than their share of root-event counts.
Figure 3: Training-loss curves for TGN and APAN on WIKI and REDDIT with and without stable support events. Solid curves show training with all sampled support events; dashed curves show training with support events whose source nodes are stable removed.
Figure 4: Overview of RAFT. Root filtering removes targets with two stable endpoints, while support filtering removes sampled historical inputs with stable source nodes. The retained inputs enter TGNN training, whose loss feeds back to the controller. Each loss-trend condition requires K consecutive batches; otherwise, the controller retains its current state.
Dataset
Model
Input Reduction ( ↓ %)
Average Precision(AP)
AUC
Time Reduction ( ↓ %)
Base
RAFT
Base
RAFT
WIKI
TGN
16.74
0.9945
0.9947 (0.02%↑)
0.9946
0.9948 (0.01%↑)
5.69
APAN
6.55
0.9751
0.9779 (0.28%↑)
0.9787
0.9805 (0.18%↑)
3.91
JODIE
2.02
0.9832
0.9840 (0.09%↑)
0.9845
0.9857 (0.12%↑)
5.77
REDDIT
TGN
67.68
0.9983
0.9980 (0.03%↓)
0.9984
0.9982 (0.02%↓)
33.73
APAN
22.90
0.9966
0.9966 (0.00%↓)
0.9970
0.9970 (0.00%↑)
11.20
Table 1: Comparison between the original models trained under Cascade (Base) and the same models trained with RAFT. Input reduction indicates the percentage reduction in input events. AP and AUC are validation metrics, where higher values indicate better prediction quality. Time reports the reduction (%) in recorded training time relative to the Base models. Parenthesized percentages report relative changes in AP and AUC from Base; arrows indicate increases or decreases.
WIKI
REDDIT
Model
Filters
Input( ↓ %)
AUC
Time( ↓ %)
Input( ↓ %)
AUC
Time( ↓ %)
TGN
Base
0.00
0.9949
0.00
0.00
0.9984
0.00
Root only
15.00
0.9950 (0.01%↑)
8.15
50.42
0.9982 (0.02%↓)
36.90
Support only
11.07
0.9946 (0.02%↓)
4.16
56.02
0.9983 (0.01%↓)
37.69
Both (RAFT)
16.74
0.9947 (0.02%↓)
9.08
67.68
0.9981 (0.03%↓)
48.32
APAN
Base
0.00
0.9819
0.00
0.00
0.9970
0.00
Table 2: Comparison under different filter configurations. Base disables both filters; Root only and Support only enable the corresponding filter; Both enables both. Input and Time report percentage reductions in sampled inputs and training time relative to Base. AUC is validation AUC; parenthesized percentages and arrows show its relative change. Each model–dataset study uses a shared baseline.
τ=0.8
τ=0.9
τ=0.95
Dataset
Model
Input( ↓ %)
Δ AUC (%)
Input( ↓ %)
Δ AUC (%)
Input( ↓ %)
Δ AUC (%)
WIKI
TGN
13.39
+0.07
13.82
+0.04
14.10
+0.05
APAN
6.34
-0.28
6.80
+0.10
6.33
-0.37
REDDIT
TGN
52.98
-0.01
51.62
-0.01
52.78
-0.01
APAN
22.97
+0.02
27.53
+0.00
25.41
+0.00
Table 3: Sensitivity of RAFT to the memory-stability threshold τ , with both filters enabled for 100 epochs. Input reports sampled-input reduction (%); Δ AUC reports the relative change in validation AUC (%). Within each model–dataset study, all thresholds share the same unfiltered baseline. Positive and negative Δ AUC values indicate increased and decreased AUC, respectively.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Nodes
Events
Feature dim.
WIKI ( Kumar et al., 2019 )
9,227
157,474
172
REDDIT ( Kumar et al., 2019 )
11,000
672,447
172
WIKI-TALK ( Paranjape et al., 2017 )
1,140,149
7,833,140
32
SX-FULL ( Paranjape et al., 2017 )
2,601,977
63,497,050
32
GDELT ( Leetaru and Schrodt, 2013 ; Zhou et al., 2022 )
16,682
191,290,882
186
Appendix
Table 4: Dataset statistics and interaction-feature dimensions. SX-FULL denotes STACK-OVERFLOW. Events are timestamped interactions in the original graph.
Model
Setting
WIKI
REDDIT
WIKI-TALK
SX-FULL
GDELT
TGN
Epochs
100
100
30
20
5
APAN
Epochs
100
100
30
20
5
JODIE
Epochs
100
100
30
20
6
Appendix
Table 5: Epoch budgets for the main Cascade comparisons. Base and RAFT use the same epoch budget within each model–dataset comparison.
Mean positive-event loss
Stable-event share (%)
Model
Dataset
Stable
Unstable
Events
Loss
TGN
WIKI
0.0668
0.2767
12.98
3.48
REDDIT
0.0302
0.0939
36.16
15.40
WIKI-TALK
0.0099
0.3793
17.69
0.56
SX-FULL
0.0414
0.5897
63.59
10.92
GDELT
0.1134
1.2094
98.91
89.47
Appendix
Table 6: Root-event motivation results across three models and five datasets. Stable and Unstable report mean positive-event prediction loss. Events and Loss report stable events’ shares of observed root-event occurrences and aggregate positive-event loss, respectively.
Root memory change
Support memory change
Dataset
TGN
APAN
JODIE
TGN
WIKI
0.445
0.703
0.625
0.405
REDDIT
0.287
0.280
0.289
0.158
WIKI-TALK
0.666
0.724
0.679
0.458
SX-FULL
0.402
0.488
0.474
0.019
Appendix
Table 7: Spearman correlations between memory-change scores and positive-event prediction loss. Root scores average source and destination memory changes; support scores average changes over sampled support nodes. Positive values indicate that larger memory changes tend to accompany larger losses.
Figure 5: Training-loss curves with and without stable support events on WIKI-TALK. Solid curves use full sampled history; dashed curves remove support events whose source nodes are stable. The 50- and 100-iteration panels show separate studies.
Timer
Measured operations
Sample
Preparing root and negative targets, including applicable root filtering, and sampling historical neighbors.
Prep
Building computation blocks; gathering features, memories, and messages; transferring required data; and committing updated memories and messages after model execution.
Model
Forward prediction, loss computation, backpropagation, and optimizer updates, including memory-update computation inside the model.
Total
The enclosing interval from batch input processing through model execution and post-update state maintenance.
Lookup
Reading stored node-stability flags for input selection.
Update
Computing memory cosine similarities and updating the stability flags during node-state maintenance.
Appendix
Table 8: Training-stage and stability-maintenance timers. Lookup and Update may lie within enclosing stage timers and are reported separately rather than added to Total.
Model
Dataset
Method
Sample (s)
Prep (s)
Model (s)
Total (s)
TGN
REDDIT
Base
0.441
8.864
1.868
11.175
RAFT
0.410
6.095
1.805
8.308
TGN
SX-FULL
Base
22.154
241.630
55.350
319.146
RAFT
18.016
128.142
42.008
188.176
APAN
REDDIT
Base
0.442
6.611
1.707
8.761
RAFT
0.559
5.669
1.652
7.884
Appendix
Table 9: Mean training-stage times per epoch (seconds) for Base and RAFT. REDDIT profiles average ten epochs; SX-FULL profiles average five. Total is the enclosing batch-processing timer defined in Table 8 .
Model
Dataset
Lookup (s)
Update (s)
Stability cost (% of Base Total)
Prep reduction (%)
TGN
REDDIT
0.0067
0.1621
1.51
31.24
TGN
SX-FULL
0.1929
6.9066
2.22
46.97
APAN
REDDIT
0.0121
0.2564
3.07
14.25
APAN
SX-FULL
0.2102
6.8293
3.10
15.13
Appendix
Table 10: Stability-maintenance cost and preparation-time reduction. Lookup and Update are mean times per epoch in seconds. Stability cost is their sum as a percentage of Base Total; Prep reduction uses Base Prep as its denominator. Percentages use unrounded timer values.
Dataset
Model
Validation AP
Validation AUC
Time reduction (%)
Base
RAFT
Base
RAFT
WIKI
TGN
0.9962
0.9952
0.9962
0.9953
10.85
APAN
0.9862
0.9852
0.9874
0.9870
6.22
JODIE
0.9823
0.9823
0.9837
0.9845
5.33
REDDIT
TGN
0.9984
0.9985
0.9986
0.9986
16.33
APAN
0.9975
0.9975
0.9978
0.9978
8.44
Appendix
Table 11: Comparison between models trained under TGL (Base) and TGL with RAFT. AP and AUC are validation scores. Time reduction is the percentage decrease in recorded training time relative to the corresponding Base configuration.
Temporal graph neural networks model how node relationships, graph structure, and features change over time to support prediction on evolving networks.
Memory-based temporal graph neural networks (TGN) rely on recurrent node memories to capture evolving interaction histories. However, temporal mini-batching introduces a trade-off between computational efficiency and temporal fidelity: when a node appears multiple times within the same batch, later interactions are evaluated using stale pre-batch memory because the intermediate memory updates have not yet been committed. This problem becomes more severe as temporal batch size increases. To overcome this challenge, we propose AnchorRM, an anchor-based residual memory mechanism that reconstructs event-specific intra-batch node states without executing sequential recurrent updates. AnchorRM treats the last committed node memory at the beginning of each batch as a pre-batch fixed anchor and uses causally preceding same-batch interactions to infer the missing state evolution relative to that anchor. The learned residual then calibrates the anchor to reconstruct an event-specific node memory for each interaction. Because each event-specific residual is predicted directly from the fixed anchor and its causal history, reconstructed states do not depend on previously reconstructed states and can therefore be computed in parallel within the batch while preserving temporal causality. Experiments across four temporal graph datasets and three memory-based backbones demonstrate that AnchorRM achieves an average 6.16x training speedup (up to 10.05x) and 28.67x batch scaling (up to 40x). These results demonstrate that AnchorRM substantially improves the scalability and training efficiency of memory-based temporal graph learning while maintaining competitive predictive performance.
Haroon Gharwi, Kai Shu, Yue Dai
Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA
Temporal graph neural networks (TGNNs) achieve superior performance in many dynamic graph learning tasks. These models use node state vectors (i.e., memories) to capture and record graph changes (i.e., events), and then exploit these memories to assist prediction. However, this memory-centric design makes multi-GPU training challenging because predictions for later events often depend on memory updates from earlier events. To maintain model quality, existing multi-GPU TGNN training schemes strictly preserve all event dependencies, which limits parallelism and causes expensive cross-GPU memory synchronization. In this work, we observe that many of these dependencies can be handled more flexibly without harming training quality. Specifically, (a) dependencies in TGNN training have high locality in both time and graph topology, (b) the short-range dependencies missed within a large batch can often be recovered cheaply, and (c) many memory updates are not immediately needed. Based on these observations, we propose DepBloom to accelerate multi-GPU TGNN training with three techniques: (1) temporally bounded spatial partitioning to reduce unnecessary cross-GPU synchronization and improve parallelism; (2) intra-batch memory refinement to recover missing short-range temporal dependencies inside enlarged batches; and (3) critical-path optimization to prioritize urgent memory updates while deferring non-critical work during multi-GPU pipeline execution. Experimental results across representative models and diverse datasets show that DepBloom achieves an average speedup of 2.84× and up to 5.16× over state-of-the-art methods, while maintaining comparable or better model accuracy.
Yang Zhao, Yue Dai, Liang Qiao +8
University of Science and Technology of China, Hefei, Anhui, China · Illinois Institute of Technology, Chicago, Illinois, USA
Temporal Graph Neural Networks (TGNNs) are widely used for learning from dynamic graphs in applications such as recommendation, social network analysis, and traffic forecasting. However, scaling TGNN training to large dynamic graphs remains challenging due to three intertwined bottlenecks: memory I/O, irregular computation, and temporal neighbor sampling. Existing systems often optimize these stages in isolation, leaving substantial performance headroom on the table. We present FAST, a holistic framework that accelerates end-to-end TGNN training by jointly optimizing sampling, memory I/O, and computation. FAST introduces SlimCache, which exploits within-batch compression and cross-batch caching to reduce host-device data movement under limited GPU memory budgets. It further designs thread-efficient graph operators tailored to sparse temporal subgraphs, improving GPU cache locality and reducing the latency of aggregation and edge softmax. In addition, FAST employs a topology-aware sampling strategy that improves CPU cache locality and accelerates temporal neighbor sampling. Extensive experiments on real-world large dynamic graphs show that FAST achieves an average of 2.1x (up to 4.7x) speedup over state-of-the-art systems without sacrificing model accuracy.
Yushu Cai, Qingrui Zhu, Lei Liu +3
Xidian University Guangzhou, Guangdong, China · Hunan University Changsha, Hunan, China