AnchorRM: Parallel Intra-Batch State Reconstruction with Anchor-Based Residual Memory for Temporal Graph Neural Networks

Haroon Gharwi; Kai Shu; Yue Dai

cs.LGOct 20, 2026

AnchorRM: Parallel Intra-Batch State Reconstruction with Anchor-Based Residual Memory for Temporal Graph Neural Networks

Authors: Haroon Gharwi, Kai Shu, Yue Dai

Organizations: Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA · Department of Computer Science, Emory University, Atlanta, GA, USA

Abstract

Memory-based temporal graph neural networks (TGN) rely on recurrent node memories to capture evolving interaction histories. However, temporal mini-batching introduces a trade-off between computational efficiency and temporal fidelity: when a node appears multiple times within the same batch, later interactions are evaluated using stale pre-batch memory because the intermediate memory updates have not yet been committed. This problem becomes more severe as temporal batch size increases. To overcome this challenge, we propose AnchorRM, an anchor-based residual memory mechanism that reconstructs event-specific intra-batch node states without executing sequential recurrent updates. AnchorRM treats the last committed node memory at the beginning of each batch as a pre-batch fixed anchor and uses causally preceding same-batch interactions to infer the missing state evolution relative to that anchor. The learned residual then calibrates the anchor to reconstruct an event-specific node memory for each interaction. Because each event-specific residual is predicted directly from the fixed anchor and its causal history, reconstructed states do not depend on previously reconstructed states and can therefore be computed in parallel within the batch while preserving temporal causality. Experiments across four temporal graph datasets and three memory-based backbones demonstrate that AnchorRM achieves an average 6.16x training speedup (up to 10.05x) and 28.67x batch scaling (up to 40x). These results demonstrate that AnchorRM substantially improves the scalability and training efficiency of memory-based temporal graph learning while maintaining competitive predictive performance.

Figures & tables

Figure 1: Test AP across temporal batch sizes on Wikipedia, Reddit, MOOC, and LastFM using TGN-ID, JODIE, and DyRep backbones. AnchorRM generally maintains higher predictive performance as temporal batch sizes increase compared with the original backbones and PRES.

WIKI

MOOC

LASTFM

Model

AP(%)

Batch

Speedup

AP(%)

Batch

Speedup

AP(%)

Batch

Speedup

AP(%)

Batch

Speedup

TGN-ID

98.62

1\times

(200)

1.00\times

(8.84s)

95.81

1\times

(100)

1.00\times

(2.45s)

84.42

1\times

(100)

1.00\times

(7.61s)

73.87

1\times

(100)

1.00\times

(18.09s)

TGN-ID+PRES †

97.79

5\times

(1000)

3.91\times

(2.26s)

95.25

5\times

(500)

3.60\times

(0.68s)

82.89

2\times

(200)

1.71\times

(4.46s)

74.38

1\times

(100)

0.91\times

(19.98s)

TGN-ID+AnchorRM

98.15

20\times

(4000)

5.53\times

(1.60s)

96.55

40\times

(4000)

9.61\times

(0.25s)

84.04

20\times

(2000)

10.05\times

(0.76s)

72.15

40\times

(4000)

9.00\times

(2.01s)

JODIE

96.00

1\times

(200)

1.00\times

(6.96s)

95.10

1\times

(100)

1.00\times

(2.31s)

76.64

1\times

(200)

1.00\times

(3.32s)

71.28

1\times

(100)

1.00\times

(17.41s)

JODIE+PRES †

95.42

10\times

(2000)

6.73\times

(1.03s)

94.77

5\times

(500)

3.81\times

(0.61s)

80.41

1\times

(200)

0.84\times

(3.96s)

74.35

5\times

(500)

4.23\times

(4.12s)

Table 1: Accuracy-tolerant batch scaling and training efficiency (

\epsilon=0.01

). Bold indicates the highest test AP within each backbone and dataset. Numbers in parentheses indicate batch size or seconds per epoch.

Figure 2: Overall accuracy-tolerant batch scaling and training efficiency under a relative validation-AP tolerance of

\epsilon=0.01

. (a) Temporal batch-scaling factor. (b) Training speedup relative to the original backbone. (c) Test AP retention relative to the baseline, where 100% indicates equal predictive performance. Results cover four datasets and three memory-based temporal graph backbones.

Figure 3: New-node test AP improvement relative to the original backbones under a relative validation-AP tolerance of

\epsilon=0.01

Figure 4: Sensitivity of AnchorRM to (a) causal history length

K

, (b) event-token dimension

d_{q}

, and (c) residual hidden dimension

d_{r}

. The top row reports average test AP, while the bottom row reports normalized training time relative to the default configuration. Results are averaged across 12 backbone–dataset combinations at

B=4000

Method	Val AP	Test AP	New AP	Speedup
Residual	0.9766	0.9707	0.9664	$0.998\times$
Direct	0.9702	0.9609	0.9617	$1.000\times$
Direct-Calibrated	0.9701	0.9593	0.9593	$0.993\times$

Table 2: Comparison of state reconstruction strategies on Wikipedia (

B=4000

K=16

d_{q}=16

d_{r}=64

). Speedup is measured relative to direct reconstruction.

Temporal GNNs

Temporal graph neural networks model how node relationships, graph structure, and features change over time to support prediction on evolving networks.

See all 82 papers

RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training
Oct 17, 2026Yue Dai, Sarah Zhou, Youtao ZhangTemporal GNNs
Department of Computer Science, Illinois Institute of Technology, Chicago, USA · BASIS Independent Silicon Valley, San Jose, CA, USA · Department of Computer Science, University of Pittsburgh, Pittsburgh, USA
A Dependency-Aware Multi-GPU Training Framework for Temporal Graph Neural Networks
Oct 11, 2026Yang Zhao, Yue Dai, Liang Qiao +8Temporal GNNs Distributed Training
University of Science and Technology of China, Hefei, Anhui, China · Illinois Institute of Technology, Chicago, Illinois, USA
Never Skip a Batch: Dense Learning of Temporal GNNs via Adaptive Pseudo-Supervision
May 18, 2025Alexander Panyshev, Dmitry Vinichenko, Oleg Travkin +2Temporal GNNs Temporal Link Prediction
Risk AI Research Lab Moscow Independent Research Institute of Artificial Intelligence Moscow, Russia · Risk AI Research Lab Moscow, Russia · Risk AI Research Lab Applied AI Institute Moscow, Russia

Oct 17, 2026J/K move · Enter open · S save

RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training

Yue Dai, Sarah Zhou, Youtao Zhang

Department of Computer Science, Illinois Institute of Technology, Chicago, USA · BASIS Independent Silicon Valley, San Jose, CA, USA · Department of Computer Science, University of Pittsburgh, Pittsburgh, USA

Temporal GNNs

Open paper AskPDF arXiv

Temporal graph neural networks (TGNNs) have shown strong performance on dynamic graph learning tasks by continuously updating node memories from evolving interactions (i.e., events) and leveraging these memories for downstream predictions. As graphs grow rapidly in modern applications, training requires processing an increasing number of events; meanwhile, each event can update node memories used by later events, creating dependencies that constrain parallel execution and limit the benefits of additional computing resources. These costs motivate an upstream question: Are all events, and the dependencies they introduce, necessary for learning? We investigate this question through two roles of temporal events: prediction targets and historical events sampled as inputs. We observe that targets involving stable node memories tend to exhibit lower prediction loss, while filtering historical events associated with stable memories can retain similar training-loss trajectories in several evaluated settings. Together, these observations suggest an opportunity to identify input redundancy through changes in node memories. Building on this opportunity, we propose RAFT, a redundancy-aware input-filtering framework for TGNN training. RAFT tracks memory changes during training and uses them to guide two filters: a root-event filter removes selected prediction targets before training, and a support-event filter excludes selected historical context during sampling. Filtering inputs before their downstream preparation and processing reduces the work they trigger. Across three TGNN models and five datasets, RAFT reduces training inputs by 35.23% on average and up to 97.11%, while reducing recorded training time by up to 52.56% with negligible changes to model precisions.

Abstract

Temporal GNNs

Explore similar work

RAFT: Redundancy-Aware Input Filtering for Temporal Graph Neural Network Training

A Dependency-Aware Multi-GPU Training Framework for Temporal Graph Neural Networks

Never Skip a Batch: Dense Learning of Temporal GNNs via Adaptive Pseudo-Supervision