Modeling user behavior couples two distinct signals: a per-user sequence of events and a persistent user-to-user relation graph. Standard pipelines preserve these native formats by combining them in two stages: a sequence encoder is pretrained, its per-user embeddings are frozen, and a graph model is trained on top to allow for cheap serving and cross-task reuse. We ask a simple question: does training the sequence and graph models end-to-end help, and if so, when? To isolate the impact of the training strategy, we hold a standard sequence-encoder-to-GNN pipeline fixed and vary only the optimization method; on Brightkite, a matched control that starts both pipelines from the same pretrained encoder and changes only whether it is updated attributes a 10.6-point MRR gain to end-to-end updating. Across relationship prediction and fraud detection on five real-world datasets, we find that the benefit of end-to-end training is conditional: it improves clearly over frozen two-stage pipelines on relationship prediction, where the relation graph appears to carry much of the signal, whereas gains shrink where the predictive signal is largely node-local (fraud detection), a regime in which a text-fed relational graph transformer achieves competitive results. Furthermore, recasting the data as a temporal graph yields inconsistent results compared to this native sequence-graph formulation. Ultimately, this work provides a controlled empirical account of when jointly training the two structures helps.
Figures & tables
Figure 1: A simple example of user behavior data in e-commerce. Each user has a sequence of events, while simultaneously also having a static user-to-user relationship such as friendship, family relation, sharing same address, sharing same devices, etc.
Figure 2: Bridge encodes each user’s event history with a sequence module, compresses the sequence representation into user-level node features, propagates information over the static user-relation graph, and optimizes the full pipeline end-to-end for the downstream task.
Dataset
# Users
# Rel.
# Events
Brightkite
58,228
428,156
4,702,710
Gowalla
196,591
1,900,654
6,442,289
A-Movies
139,138
2,807,368
1,932,230
A-Electronics
254,064
657,050
2,281,128
A-Clothing
185,986
18,078
1,573,869
Table 1: Dataset statistics for relationship prediction (left) and Amazon fraud detection (right).
Brightkite
Gowalla
Movies
Electronics
Clothing
Method
MRR
H@5
MRR
H@5
MRR
H@5
MRR
H@5
MRR
H@5
Graph Only (Static Graph)
GCN
67.4±1.9
82.3±1.3
77.8±0.6
89.4±0.3
76.4±0.5
89.6±0.5
48.9±3.9
66.6±3.3
17.3±1.5
23.3±1.9
GAT
65.4±1.6
80.1±1.0
78.1±0.6
89.3±0.3
76.6±0.7
90.8±0.6
44.7±0.7
62.5±0.7
16.2±2.3
18.0±2.1
TFConv
68.5±0.2
83.2±0.4
80.5±0.2
90.9±0.2
82.4±0.1
94.4±0.0
55.8±0.7
71.5±0.4
18.8±0.7
25.2±0.3
Two-Stage (Static Graph + Sequence)
Table 2: Relationship prediction performance across five datasets (higher is better). The main comparison uses methods aligned with the static-relation formulation: graph-only baselines, frozen two-stage sequence+graph baselines, and end-to-end Bridge variants. Best results are bolded and the runner-ups are underlined .
Amazon-Movies
Amazon-Electronics
Amazon-Clothing
Method
Max F1
PR AUC
Max F1
PR AUC
Max F1
PR AUC
Static Graph Methods
GCN
43.4±0.2
30.9±0.2
12.6±0.2
8.7±0.4
5.3±0.4
2.8±0.3
GAT
43.4±0.2
36.2±0.6
12.6±0.1
7.1±0.1
4.9±0.3
2.9±0.4
TFConv
44.1±0.1
39.1±0.5
12.8±0.0
9.4±0.5
4.9±0.2
2.8±0.2
Temporal Graph Methods
Table 3: Fraud detection performance across three Amazon categories (mean ± std over three seeds, %; higher is better). All methods share one canonical 60/20/20 user split and are reported at the validation-PR-AUC-selected checkpoint, with PR-AUC computed as auc(recall,precision) . We compare static graph, temporal graph, sequence/text-only (Sentence-BERT), frozen two-stage (static graph + text), Bridge , and relational deep learning (RelGT) run both without text information and with the same pooled Sentence-BERT review embedding fed to the two-stage baseline.
Training setup
MRR
Hits@1
Hits@5
MLM-pretrained → frozen encoder → GNN
76.8±0.7
68.2±0.8
87.4±0.5
Same checkpoint → encoder fine-tuned with GNN
87.4±0.4
82.9±0.5
92.9±0.2
Random init. → end-to-end ( Bridge )
92.2±0.6
89.9±0.7
94.8±0.5
Table 4: Matched freezing control on Brightkite (mean ± std over three seeds). The first two rows share the same masked-token-pretrained encoder, architecture, data, and downstream settings (batch size 64) and differ only in whether the encoder is updated. The last row is the main-table Bridge -GCN ( Table 2 ), trained from random initialization with its own selected configuration (batch size 128). The Brightkite no-relation ablation ( Table 24 ) uses a third, reduced protocol (batch size 16, 30 epochs), hence its lower MRR.
Cost
Pretrained + frozen
Pretrained + fine-tuned
MLM pretraining (one-time)
25.0 min
25.0 min
Embedding generation (one-time)
≤2.6 min
not required
Downstream training per epoch
29.8±0.1 min
65.9±1.4 min
Peak GPU memory
21.3±0.1 GiB
40.9±0.2 GiB
Table 5: Training cost of the matched freezing control on Brightkite (one NVIDIA L40S; mean ± std over three seeds where applicable). The MLM pretraining is shared; embedding generation includes process start-up.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Formulation
User history
Relationship
E2E
Seq.
Best-fit setting
Temporal graph
• Timestamped edges • Per-node memory state
• Dynamic edge • E.g., A pays B
✓
×
• High-frequency peer-to-peer events • Relations form/dissolve over time
RDL / Rel. graph transformer
• Neighborhood of child rows, e.g., transactions near a user
• Primary/foreign-key link • Fixed by the DB schema
✓
×
• Enterprise ML on relational DBs • Rich, multi-table schemas
• Separate static graph • Gates which sequences interact
✓
✓
• Dense histories + social structure, e.g., relationship prediction • Weaker if node-local ( Section 5.2 )
Appendix
Table 6: How each formulation models a user history and a relationship, and where it fits best. E2E : trained end-to-end on the downstream task. Seq. : the sequence is encoded as a first-class object.
Method
LR
BS
GAT
0.001
4096
GCN
0.001
4096
TFConv
0.001
4096
Appendix
Table 7: Relationship prediction hyperparameters (graph-only static baselines). We report the learning rate (LR) and batch size (BS) used for each static graph encoder.
Method
LR
BS
Bert
Hidden
GAT + S
0.001
4096
1x2
128
GCN + S
0.001
4096
1x2
128
TFConv + S
0.001
4096
1x2
128
Appendix
Table 8: Relationship prediction hyperparameters (two-stage static + sequence baselines). We report the learning rate (LR), batch size (BS), and sequence-encoder configuration: “Bert” denotes # Transformer layers × #attention heads, and “Hidden” is the Transformer hidden size.
Method
LR
BS
DyRep
0.001
4096
TGN
0.001
4096
TCL
0.001
1024
NAT
0.001
1024
GraphMixer
0.001
1024
TPNet
0.001
1024
Appendix
Table 9: Relationship prediction hyperparameters (temporal graph baselines). We report the learning rate (LR) and batch size (BS) used for each temporal graph method.
Method
Dataset
LR
BS
Bert
Hidden
Bridge -GCN
Brightkite
0.0001
128
1x2
128
Gowalla
0.0001
32
1x2
128
Amazon’s
0.0001
32
1x2
128
Bridge -GAT
Brightkite
0.0001
128
1x2
128
Gowalla
0.0001
32
1x2
128
Amazon’s
0.0001
32
1x2
128
Appendix
Table 10: Relationship prediction hyperparameters ( Bridge ). For each dataset and graph backbone, we report the learning rate (LR), batch size (BS), and sequence-encoder configuration. “Bert” denotes #Transformer layers × #attention heads, and “Hidden” is the Transformer hidden size. “–” indicates the configuration is not applicable or not run for that dataset/backbone.
Method
LR
BS
Heads
GAT
0.001
4096
2
GCN
0.001
4096
–
TFConv
0.001
4096
2
Appendix
Table 11: Fraud detection hyperparameters (static graph baselines). We report the learning rate (LR), batch size (BS), and number of attention heads (Heads) used for each static graph method. “–” indicates the method is non-attentional and does not use heads.
Method
LR
BS
Heads
DyRep
0.001
32768
2
TGN
0.001
32768
2
TCL
0.001
32768
2
NAT
0.001
32768
1
GraphMixer
0.001
32768
1
TPNet
0.001
32768
1
Appendix
Table 12: Fraud detection hyperparameters (temporal graph baselines). We report the learning rate (LR), batch size (BS), and attention heads (Heads) used in the temporal encoder for each method.
Method
Dataset
LR
BS
Heads
Clf.
TF-L
TF-H
FFN
Drop.
GAT+GAT
Movies
0.0003
512
1
2
2
4
512
0.2
Electronics
0.0003
512
1
2
2
4
256
0.2
Clothing
0.0003
512
1
2
2
2
256
0.2
GCN+GCN
Movies
0.0003
512
–
2
2
4
512
0.2
Electronics
0.0003
512
–
2
2
4
256
0.2
Clothing
0.0003
512
–
2
2
2
256
0.15
Appendix
Table 13: Fraud detection hyperparameters ( Bridge variants). For each dataset and Bridge backbone, we list the optimizer learning rate (LR) and batch size (BS), the number of graph attention heads (Heads; “–” for non-attentional GCN), the number of classifier MLP layers (Clf-Layers), and Transformer sequence-encoder settings (TF-Layers/Heads, FFN hidden size TF-FFN, and dropout TF-Dropout).
Brightkite
Gowalla
Method
MRR
H@1
H@3
H@5
H@10
MRR
H@1
H@3
H@5
H@10
Graph Only (Static)
GCN
67.4±1.9
55.6±2.4
75.5±1.8
82.3±1.3
89.2±0.7
77.8±0.6
68.6±0.8
84.9±0.4
89.4±0.3
93.5±0.2
GAT
65.4±1.6
53.6±2.0
73.0±1.4
80.1±1.0
87.8±0.6
78.1±0.6
69.2±0.9
84.9±0.4
89.3±0.3
93.3±0.3
TFConv
68.5±0.2
56.9±0.2
76.6±0.4
83.2±0.4
89.8±0.3
80.5±0.2
72.3±0.2
87.0±0.2
90.9±0.2
94.3±0.1
Two-Stage (Static + Sequence)
Appendix
Table 14: Relationship prediction ranking performance on Brightkite and Gowalla (higher is better). We report mean ± std over three seeds. Hits@k is the fraction of test queries for which at least one ground-truth relation appears in the top- k predictions. Best results are bolded and runner-ups are underlined .
Clothing
Electronics
Method
MRR
H@1
H@3
H@5
H@10
MRR
H@1
H@3
H@5
H@10
Graph Only (Static)
GCN
17.3±1.6
9.1±1.7
17.4±2.0
23.4±1.9
34.1±0.7
48.9±3.9
34.9±4.4
56.9±4.2
66.6±3.3
76.5±2.2
GAT
15.9±1.9
9.9±2.0
14.0±2.5
17.9±2.1
27.5±1.5
44.7±0.7
30.5±0.8
51.8±0.8
62.5±0.7
74.6±0.4
TFConv
18.8±0.5
10.4±0.9
19.1±0.2
25.0±0.4
35.6±0.2
55.8±0.7
43.0±1.0
63.8±0.6
71.5±0.4
79.5±0.1
Two-Stage (Static + Sequence)
Appendix
Table 15: Relationship prediction ranking performance on Amazon-Clothing and Amazon-Electronics (higher is better). Mean ± std over three seeds; best bolded, runner-up underlined.
Amazon-Movies
Method
MRR
Hits@1
Hits@3
Hits@5
Hits@10
Graph Only (Static)
GCN
76.4±0.5
65.8±0.6
84.5±0.7
89.6±0.5
94.6±0.2
GAT
76.6±0.7
65.5±0.8
85.5±0.8
90.8±0.6
95.3±0.3
TFConv
82.4±0.1
73.0±0.2
90.5±0.0
94.4±0.0
97.4±0.0
Two-Stage (Static + Sequence)
Appendix
Table 16: Relationship prediction ranking performance on Amazon-Movies (higher is better). Mean ± std over three seeds; best bolded, runner-up underlined.
Seed
Fine-tuned
Frozen
Δ [95% CI]
p
0
88.37
79.00
+9.36[+9.17,+9.56]
<10−16
1
88.46
77.86
+10.60[+10.39,+10.80]
<10−16
2
88.81
79.12
+9.69[+9.50,+9.88]
<10−16
mean ± std
+9.88±0.64
Appendix
Table 17: Paired per-query test on Brightkite for the matched freezing control: MLM-pretrained encoder fine-tuned minus the same encoder frozen. MRR is in points ( ×100 ), re-evaluated on the identical 85,632 test queries; the 95% CI is from 10,000 bootstrap resamples over queries and p from a two-sided Wilcoxon signed-rank test on reciprocal ranks.
Dataset
Overlap rate
MRR
Hits@1
Hits@5
Bridge MRR
Movies
71.6%
61.52
52.2
69.3
88.6 – 90.4
Electronics
43.0%
35.35
26.5
42.3
75.1 – 80.2
Clothing
29.8%
25.11
18.9
28.6
52.3 – 63.7
Appendix
Table 18: Amazon observed-product overlap baseline (training-free) versus Bridge . Metrics in points; Bridge range over its GNN variants in Table 2 .
PR-AUC gap
Max-F1 gap
Seed
Δ [95% CI]
p
Δ [95% CI]
p
0
+2.23[+0.28,+4.23]
0.028
+1.04[−1.41,+4.04]
0.357
1
−0.59[−3.15,+1.91]
0.627
−0.95[−4.45,+2.27]
0.527
2
+0.54[−1.82,+2.86]
0.657
+0.52[−3.24,+4.07]
0.852
mean ± std
+0.72±1.42
+0.20±1.03
Appendix
Table 19: Paired per-user bootstrap on Amazon-Clothing, RelGT + text minus Bridge -GAT ( 10,000 class-stratified resamples over the shared test users, per seed). Gaps are in points ( ×100 ) with 95% CI and two-sided bootstrap p . The gap changes sign across seeds and only seed 0 (PR-AUC) is individually significant; the across-seed gap is within noise, so the two methods are tied on Clothing.
Brightkite
Gowalla
Movies
Electronics
Clothing
Method
MRR
H@5
MRR
H@5
MRR
H@5
MRR
H@5
MRR
H@5
Static Graph Methods
GCN
67.4±1.9
82.3±1.3
77.8±0.6
89.4±0.3
76.4±0.5
89.6±0.5
48.9±3.9
66.6±3.3
17.3±1.5
23.3±1.9
GAT
65.4±1.6
80.1±1.0
78.1±0.6
89.3±0.3
76.6±0.7
90.8±0.6
44.7±0.7
62.5±0.7
16.2±2.3
18.0±2.1
TFConv
68.5±0.2
83.2±0.4
80.5±0.2
90.9±0.2
82.4±0.1
94.4±0.0
55.8±0.7
71.5±0.4
18.8±0.7
25.2±0.3
Temporal Graph Methods After Conversion
Appendix
Table 20: Relationship prediction temporal-graph diagnostic. Static graph methods use the given static user-relation graph directly. Temporal graph methods require converting user histories and static relations into a timestamped edge stream. Higher is better.
Amazon-Movies
Amazon-Electronics
Amazon-Clothing
Method
Max F1
PR AUC
Max F1
PR AUC
Max F1
PR AUC
Static Graph Methods + Text Embedding
GAT
78.3±1.0
85.9±1.5
46.0±1.5
44.6±2.5
37.9±1.5
35.8±2.4
GCN
78.5±1.0
86.0±1.5
46.0±1.3
45.0±2.7
38.0±1.6
35.9±2.3
TFConv
78.8±0.9
86.6±1.2
46.7±0.9
45.1±2.2
37.8±1.9
35.4±1.7
Temporal Graph Methods After Conversion
Appendix
Table 21: Fraud detection temporal-graph diagnostic. Static graph methods use averaged Sentence-BERT review embeddings as node features. Temporal graph methods receive review text and ratings through the converted timestamped edge stream. Higher is better.
Figure 3: Ablations on Brightkite. Bars indicate test MRR (%) and the orange line indicates relative epoch time (right axis) across (i) sampled neighbors, (ii) No-relation ablation, and (iii) sequence module depth. Runtime is normalized within each sweep to its reference setting.
# TF Layers
MRR
Epoch Time (min)
Rel. Epoch Time
1
62.4±0.4
43.8±0.3
0.96
2
62.4±0.4
45.7±0.1
1.00
4
62.5±0.3
49.5±0.1
1.08
Appendix
Table 22: Sequence module depth. Sensitivity to the number of Transformer layers in the sequence encoder. We report test MRR (mean ± std over three seeds) and per-epoch runtime (minutes); Rel. Epoch Time is normalized to the two-layer reference used in this sweep.
# Neighbors
MRR
Time (min)
Rel. Time
2
77.3±0.6
301.4±120.4
0.09
5
84.2±0.4
1009.7±96.1
0.31
10
85.4±0.6
3264.4±770.9
1.00
15
84.4±0.3
4202.2±609.0
1.29
Appendix
Table 23: Effect of neighborhood fanout on the full Brightkite friend-recommendation task. Mean and standard deviation are over three seeds.
Dataset
Variant
MRR
Time (min)
Rel. Time
Brightkite
With relational MP
85.4±0.6
3264.4±770.9
1.00
No relational MP
62.0±0.2
93.9±1.0
0.03
Gowalla
With relational MP
87.9±0.2
–
–
No relational MP
50.8±0.1
–
–
Appendix
Table 24: Effect of relational message passing on friend recommendation. Mean and standard deviation are over three seeds. Brightkite uses the reduced ablation protocol (batch size 16, 30 epochs), so its full-model MRR ( 85.4 ) is below the main-table value ( 92.2 , batch size 128) and the matched control ( 87.4 , batch size 64, MLM initialization; Table 4 ). Gowalla uses the main-table setting; its “with” row is the main-table Bridge -GCN result.
Analytic Computing, KI, University of Stuttgart, Stuttgart, Germany · Bosch Center for Artificial Intelligence, Stuttgart, Germany · WAIS, University of Southampton, United Kingdom
Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · University of Zurich, Zurich, Zurich, Switzerland · University of Michigan, Ann Arbor, Michigan, USA
Steklov Institute of Mathematics Saint Petersburg, Russia · Independent Researcher Belgrade, Serbia · ISP RAS Steklov Institute of Mathematics Saint Petersburg, Russia +4