RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training

Yue Dai; Sarah Zhou; Youtao Zhang

cs.LGOct 17, 2026

RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training

Authors: Yue Dai, Sarah Zhou, Youtao Zhang

Organizations: Department of Computer Science, Illinois Institute of Technology, Chicago, USA · BASIS Independent Silicon Valley, San Jose, CA, USA · Department of Computer Science, University of Pittsburgh, Pittsburgh, USA

Abstract

Temporal graph neural networks (TGNNs) have shown strong performance on dynamic graph learning tasks by continuously updating node memories from evolving interactions (i.e., events) and leveraging these memories for downstream predictions. As graphs grow rapidly in modern applications, training requires processing an increasing number of events; meanwhile, each event can update node memories used by later events, creating dependencies that constrain parallel execution and limit the benefits of additional computing resources. These costs motivate an upstream question: Are all events, and the dependencies they introduce, necessary for learning? We investigate this question through two roles of temporal events: prediction targets and historical events sampled as inputs. We observe that targets involving stable node memories tend to exhibit lower prediction loss, while filtering historical events associated with stable memories can retain similar training-loss trajectories in several evaluated settings. Together, these observations suggest an opportunity to identify input redundancy through changes in node memories. Building on this opportunity, we propose RAFT, a redundancy-aware input-filtering framework for TGNN training. RAFT tracks memory changes during training and uses them to guide two filters: a root-event filter removes selected prediction targets before training, and a support-event filter excludes selected historical context during sampling. Filtering inputs before their downstream preparation and processing reduces the work they trigger. Across three TGNN models and five datasets, RAFT reduces training inputs by 35.23% on average and up to 97.11%, while reducing recorded training time by up to 52.56% with negligible changes to model precisions.

Figures & tables

Figure 1: Batched memory-based TGNN training. Root events (e.g.,

e_{23}

and

e_{34}

) are prediction targets, while support events (e.g.,

e_{12}

e_{13}

, and

e_{14}

) provide historical inputs. Step 1 samples support events and fetches the required information. Step 2 updates node memories (

s_{v}^{-}\rightarrow s_{v}^{+}

) and computes embeddings

h_{v}

. Step 3 predicts root events (e.g.,

\tilde{e}(t_{i})=f(h_{2},h_{3})

), computes the loss, and updates model parameters through backpropagation.

Figure 2: Root-event loss during TGN training on WIKI, REDDIT, and WIKI-TALK. (a) Stable root events have lower mean positive-event loss than unstable events. (b) Their share of training loss is considerably smaller than their share of root-event counts.

Figure 3: Training-loss curves for TGN and APAN on WIKI and REDDIT with and without stable support events. Solid curves show training with all sampled support events; dashed curves show training with support events whose source nodes are stable removed.

Figure 4: Overview of RAFT. Root filtering removes targets with two stable endpoints, while support filtering removes sampled historical inputs with stable source nodes. The retained inputs enter TGNN training, whose loss feeds back to the controller. Each loss-trend condition requires $K$ consecutive batches; otherwise, the controller retains its current state.

Dataset	Model	Input Reduction ( $\downarrow$ %)	Average Precision(AP)		AUC		Time Reduction ( $\downarrow$ %)
Dataset	Model	Input Reduction ( $\downarrow$ %)	Base	RAFT	Base	RAFT	Time Reduction ( $\downarrow$ %)
WIKI	TGN	16.74	0.9945	0.9947 $(0.02\%\uparrow)$	0.9946	0.9948 $(0.01\%\uparrow)$	5.69
	APAN	6.55	0.9751	0.9779 $(0.28\%\uparrow)$	0.9787	0.9805 $(0.18\%\uparrow)$	3.91
	JODIE	2.02	0.9832	0.9840 $(0.09\%\uparrow)$	0.9845	0.9857 $(0.12\%\uparrow)$	5.77
REDDIT	TGN	67.68	0.9983	0.9980 $(0.03\%\downarrow)$	0.9984	0.9982 $(0.02\%\downarrow)$	33.73
	APAN	22.90	0.9966	0.9966 $(0.00\%\downarrow)$	0.9970	0.9970 $(0.00\%\uparrow)$	11.20

Table 1: Comparison between the original models trained under Cascade (Base) and the same models trained with RAFT. Input reduction indicates the percentage reduction in input events. AP and AUC are validation metrics, where higher values indicate better prediction quality. Time reports the reduction (%) in recorded training time relative to the Base models. Parenthesized percentages report relative changes in AP and AUC from Base; arrows indicate increases or decreases.

		WIKI			REDDIT
Model	Filters	Input( $\downarrow$ %)	AUC	Time( $\downarrow$ %)	Input( $\downarrow$ %)	AUC	Time( $\downarrow$ %)
TGN	Base	0.00	0.9949	0.00	0.00	0.9984	0.00
	Root only	15.00	0.9950 $(0.01\%\uparrow)$	8.15	50.42	0.9982 $(0.02\%\downarrow)$	36.90
	Support only	11.07	0.9946 $(0.02\%\downarrow)$	4.16	56.02	0.9983 $(0.01\%\downarrow)$	37.69
	Both (RAFT)	16.74	0.9947 $(0.02\%\downarrow)$	9.08	67.68	0.9981 $(0.03\%\downarrow)$	48.32
APAN	Base	0.00	0.9819	0.00	0.00	0.9970	0.00

Table 2: Comparison under different filter configurations. Base disables both filters; Root only and Support only enable the corresponding filter; Both enables both. Input and Time report percentage reductions in sampled inputs and training time relative to Base. AUC is validation AUC; parenthesized percentages and arrows show its relative change. Each model–dataset study uses a shared baseline.

		$\tau=0.8$		$\tau=0.9$		$\tau=0.95$
Dataset	Model	Input( $\downarrow$ %)	$\Delta$ AUC (%)	Input( $\downarrow$ %)	$\Delta$ AUC (%)	Input( $\downarrow$ %)	$\Delta$ AUC (%)
WIKI	TGN	13.39	+0.07	13.82	+0.04	14.10	+0.05
WIKI	APAN	6.34	-0.28	6.80	+0.10	6.33	-0.37
REDDIT	TGN	52.98	-0.01	51.62	-0.01	52.78	-0.01
REDDIT	APAN	22.97	+0.02	27.53	+0.00	25.41	+0.00

Table 3: Sensitivity of RAFT to the memory-stability threshold

\tau

, with both filters enabled for 100 epochs. Input reports sampled-input reduction (%);

\Delta

AUC reports the relative change in validation AUC (%). Within each model–dataset study, all thresholds share the same unfiltered baseline. Positive and negative

\Delta

AUC values indicate increased and decreased AUC, respectively.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Nodes	Events	Feature dim.
WIKI ( Kumar et al., 2019 )	9,227	157,474	172
REDDIT ( Kumar et al., 2019 )	11,000	672,447	172
WIKI-TALK ( Paranjape et al., 2017 )	1,140,149	7,833,140	32
SX-FULL ( Paranjape et al., 2017 )	2,601,977	63,497,050	32
GDELT ( Leetaru and Schrodt, 2013 ; Zhou et al., 2022 )	16,682	191,290,882	186

Appendix

Table 4: Dataset statistics and interaction-feature dimensions. SX-FULL denotes STACK-OVERFLOW. Events are timestamped interactions in the original graph.

Model	Setting	WIKI	REDDIT	WIKI-TALK	SX-FULL	GDELT
TGN	Epochs	100	100	30	20	5
APAN	Epochs	100	100	30	20	5
JODIE	Epochs	100	100	30	20	6

Appendix

Table 5: Epoch budgets for the main Cascade comparisons. Base and RAFT use the same epoch budget within each model–dataset comparison.

		Mean positive-event loss		Stable-event share (%)
Model	Dataset	Stable	Unstable	Events	Loss
TGN	WIKI	0.0668	0.2767	12.98	3.48
	REDDIT	0.0302	0.0939	36.16	15.40
	WIKI-TALK	0.0099	0.3793	17.69	0.56
	SX-FULL	0.0414	0.5897	63.59	10.92
	GDELT	0.1134	1.2094	98.91	89.47

Appendix

Table 6: Root-event motivation results across three models and five datasets. Stable and Unstable report mean positive-event prediction loss. Events and Loss report stable events’ shares of observed root-event occurrences and aggregate positive-event loss, respectively.

	Root memory change			Support memory change
Dataset	TGN	APAN	JODIE	TGN
WIKI	0.445	0.703	0.625	0.405
REDDIT	0.287	0.280	0.289	0.158
WIKI-TALK	0.666	0.724	0.679	0.458
SX-FULL	0.402	0.488	0.474	0.019

Appendix

Table 7: Spearman correlations between memory-change scores and positive-event prediction loss. Root scores average source and destination memory changes; support scores average changes over sampled support nodes. Positive values indicate that larger memory changes tend to accompany larger losses.

Figure 5: Training-loss curves with and without stable support events on WIKI-TALK. Solid curves use full sampled history; dashed curves remove support events whose source nodes are stable. The 50- and 100-iteration panels show separate studies.

Timer	Measured operations
Sample	Preparing root and negative targets, including applicable root filtering, and sampling historical neighbors.
Prep	Building computation blocks; gathering features, memories, and messages; transferring required data; and committing updated memories and messages after model execution.
Model	Forward prediction, loss computation, backpropagation, and optimizer updates, including memory-update computation inside the model.
Total	The enclosing interval from batch input processing through model execution and post-update state maintenance.
Lookup	Reading stored node-stability flags for input selection.
Update	Computing memory cosine similarities and updating the stability flags during node-state maintenance.

Appendix

Table 8: Training-stage and stability-maintenance timers. Lookup and Update may lie within enclosing stage timers and are reported separately rather than added to Total.

Model	Dataset	Method	Sample (s)	Prep (s)	Model (s)	Total (s)
TGN	REDDIT	Base	0.441	8.864	1.868	11.175
		RAFT	0.410	6.095	1.805	8.308
TGN	SX-FULL	Base	22.154	241.630	55.350	319.146
		RAFT	18.016	128.142	42.008	188.176
APAN	REDDIT	Base	0.442	6.611	1.707	8.761
		RAFT	0.559	5.669	1.652	7.884

Appendix

Table 9: Mean training-stage times per epoch (seconds) for Base and RAFT. REDDIT profiles average ten epochs; SX-FULL profiles average five. Total is the enclosing batch-processing timer defined in Table 8 .

Model	Dataset	Lookup (s)	Update (s)	Stability cost (% of Base Total)	Prep reduction (%)
TGN	REDDIT	0.0067	0.1621	1.51	31.24
TGN	SX-FULL	0.1929	6.9066	2.22	46.97
APAN	REDDIT	0.0121	0.2564	3.07	14.25
APAN	SX-FULL	0.2102	6.8293	3.10	15.13

Appendix

Table 10: Stability-maintenance cost and preparation-time reduction. Lookup and Update are mean times per epoch in seconds. Stability cost is their sum as a percentage of Base Total; Prep reduction uses Base Prep as its denominator. Percentages use unrounded timer values.

Dataset	Model	Validation AP		Validation AUC		Time reduction (%)
Dataset	Model	Base	RAFT	Base	RAFT	Time reduction (%)
WIKI	TGN	0.9962	0.9952	0.9962	0.9953	10.85
	APAN	0.9862	0.9852	0.9874	0.9870	6.22
	JODIE	0.9823	0.9823	0.9837	0.9845	5.33
REDDIT	TGN	0.9984	0.9985	0.9986	0.9986	16.33
	APAN	0.9975	0.9975	0.9978	0.9978	8.44

Appendix

Table 11: Comparison between models trained under TGL (Base) and TGL with RAFT. AP and AUC are validation scores. Time reduction is the percentage decrease in recorded training time relative to the corresponding Base configuration.

Temporal GNNs

Temporal graph neural networks model how node relationships, graph structure, and features change over time to support prediction on evolving networks.

See all 82 papers

AnchorRM: Parallel Intra-Batch State Reconstruction with Anchor-Based Residual Memory for Temporal Graph Neural Networks
Oct 20, 2026Haroon Gharwi, Kai Shu, Yue DaiTemporal GNNs Memory-Augmented Neural Networks
Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA
A Dependency-Aware Multi-GPU Training Framework for Temporal Graph Neural Networks
Oct 11, 2026Yang Zhao, Yue Dai, Liang Qiao +8Temporal GNNs Distributed Training
University of Science and Technology of China, Hefei, Anhui, China · Illinois Institute of Technology, Chicago, Illinois, USA
FAST: A Holistic Framework for Optimizing Memory-I/O, Computation, and Sampling in Temporal GNN Training
Jul 6, 2026Yushu Cai, Qingrui Zhu, Lei Liu +3Temporal GNNs Graph Neural Networks
Xidian University Guangzhou, Guangdong, China · Hunan University Changsha, Hunan, China

Oct 20, 2026J/K move · Enter open · S save

AnchorRM: Parallel Intra-Batch State Reconstruction with Anchor-Based Residual Memory for Temporal Graph Neural Networks

Haroon Gharwi, Kai Shu, Yue Dai

Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA

Temporal GNNs Memory-Augmented Neural Networks

Open paper AskPDF arXiv

Memory-based temporal graph neural networks (TGN) rely on recurrent node memories to capture evolving interaction histories. However, temporal mini-batching introduces a trade-off between computational efficiency and temporal fidelity: when a node appears multiple times within the same batch, later interactions are evaluated using stale pre-batch memory because the intermediate memory updates have not yet been committed. This problem becomes more severe as temporal batch size increases. To overcome this challenge, we propose AnchorRM, an anchor-based residual memory mechanism that reconstructs event-specific intra-batch node states without executing sequential recurrent updates. AnchorRM treats the last committed node memory at the beginning of each batch as a pre-batch fixed anchor and uses causally preceding same-batch interactions to infer the missing state evolution relative to that anchor. The learned residual then calibrates the anchor to reconstruct an event-specific node memory for each interaction. Because each event-specific residual is predicted directly from the fixed anchor and its causal history, reconstructed states do not depend on previously reconstructed states and can therefore be computed in parallel within the batch while preserving temporal causality. Experiments across four temporal graph datasets and three memory-based backbones demonstrate that AnchorRM achieves an average 6.16x training speedup (up to 10.05x) and 28.67x batch scaling (up to 40x). These results demonstrate that AnchorRM substantially improves the scalability and training efficiency of memory-based temporal graph learning while maintaining competitive predictive performance.

Abstract

Temporal GNNs

Explore similar work

AnchorRM: Parallel Intra-Batch State Reconstruction with Anchor-Based Residual Memory for Temporal Graph Neural Networks

A Dependency-Aware Multi-GPU Training Framework for Temporal Graph Neural Networks

FAST: A Holistic Framework for Optimizing Memory-I/O, Computation, and Sampling in Temporal GNN Training