From Complex Histories to Simple Predictors: Predictive Dimension in Dynamic Graph Learning
Organizations: School of Cyber Science and Engineering, Southeast University. · School of Mathematics, Southeast University. · Frontiers Science Center for Mobile Information Communication and Security.
Abstract
Existing continuous-time graph learning methods rely heavily on complex neural architectures to encode ever-expanding interaction histories. However, the future conditional distribution induced by complex histories can often be characterized by a low-dimensional predictive state manifold, implying that only a few predictive degrees of freedom are essential for future interactions. We therefore argue that continuous-time graph learning should identify low-dimensional states sufficient for future prediction rather than reconstruct the full history. To make this perspective measurable, we define and independently estimate the intrinsic predictive dimension, which quantifies the effective predictive degrees of freedom required by a given task. Specifically, we remove the history-independent marginal from the history-conditioned future distribution, decompose the resulting predictive residual operator, and estimate the minimum effective rank needed to preserve the predictive power of the full history. Experiments across datasets show that model performance saturation is largely determined by the intrinsic predictive dimension. Based on this finding, we propose PreDyG, a lightweight model whose capacity is matched to the intrinsic predictive dimension. PreDyG achieves superior predictive performance with only 0.07% of the parameters of a representative complex baseline.
Figures & tables
| Dataset | Original | Time/order | Destination | Source | Source–destination pair |
|---|---|---|---|---|---|
| (100%) | (100%) | ( ) | ( ) | (100%) | |
| SocialEvo | (100%) | (100%) | (3%) | (3%) | (50%) |
| UCI | (100%) | ( ) | ( ) | ( ) | ( ) |
| Contacts | (100%) | (100%) | ( ) | (1%) | (100%) |
| Method | Wiki | LastFM | Enron | SocialEvo | UCI | Flights | Contacts | |
|---|---|---|---|---|---|---|---|---|
| JODIE | ||||||||
| DyRep | ||||||||
| TGN | ||||||||
| CAWN | ||||||||
| GraphMixer | ||||||||
| DyGFormer |
| Method | tgbl-wiki | tgbl-review | tgbl-coin | tgbl-comment |
|---|---|---|---|---|
| CAWN | 73.04 .60 | 19.30 .10 | – | – |
| DyRep | ||||
| DyGFormer | ||||
| TCL | ||||
| TGAT | 59.94 1.63 | 19.64 .23 | 60.92 .57 | 56.20 2.11 |
| TGN | 68.93 .53 | 37.48 .23 | 58.60 3.70 | 37.90 2.00 |
| Variant | Wiki | Enron | UCI | |
|---|---|---|---|---|
| Full PreDyG | 99.30 | 99.34 | 93.18 | 97.01 |
| w/o NIF | 97.44 | 97.87 | 84.05 | 93.28 |
| w/o modalities | 98.16 | 98.73 | 90.99 | 95.19 |
| w/o fusion | 99.07 | 99.18 | 90.43 | 95.14 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | # Nodes | # Edges | Information | Duration | Time Granularity | ||
| Wikipedia | - | 172 | 9,227 | 157,474 | Social | 1 month | Unix timestamps |
| - | 172 | 10,984 | 672,447 | Social | 1 month | Unix timestamps | |
| LastFM | - | - | 1,980 | 1,293,103 | Interaction | 1 month | Unix timestamps |
| Enron | - | - | 184 | 125,235 | Communication | 3 years | Unix timestamps |
| Social Evo. | - | - | 59 | 90,879 | Proximity | 77 days | Unix timestamps |
| Contact | - | - | 274 | 28,244 | Proximity | 20 days | Unix timestamps |
| NSS | Datasets | JODIE | DyRep | TGAT | TGN | CAWN | EdgeBank | TCL | GraphMixer | DyGFormer | PhyDyG |
|---|---|---|---|---|---|---|---|---|---|---|---|
| rnd | Wikipedia | 96.50 0.18 | 94.86 0.04 | 96.94 0.06 | 98.45 0.05 | 98.76 0.03 | 90.37 0.00 | 96.47 0.14 | 97.25 0.03 | 98.82 0.02 | 99.30 0.01 |
| 98.31 0.12 | 98.22 0.07 | 98.52 0.02 | 98.63 0.05 | 99.11 0.01 | 94.86 0.00 | 97.53 0.02 | 97.31 0.01 | 99.11 0.01 | 99.34 0.01 | ||
| Enron | 84.77 0.28 | 82.38 3.32 | 71.12 0.96 | 86.53 1.14 | 89.56 0.13 | 83.53 0.00 | 79.70 0.72 | 82.25 0.16 | 92.47 0.15 | 93.18 0.19 | |
| UCI | 89.43 1.06 | 65.14 2.33 | 79.63 0.71 | 92.34 1.02 | 95.18 0.05 | 76.20 0.00 | 89.57 1.61 | 93.25 0.54 | 95.76 0.16 | 97.01 0.03 | |
| LastFM | 70.85 2.11 | 71.92 2.20 | 73.42 0.18 | 77.07 3.99 | 86.99 0.07 | 79.29 0.00 | 67.27 2.13 | 75.61 0.22 | 93.00 0.10 | 93.97 0.06 | |
| Social Evo. | 89.89 0.52 | 88.87 0.31 | 93.16 0.20 | 93.57 0.21 | 84.96 0.09 | 74.95 0.00 | 93.13 0.17 | 93.37 0.07 | 94.47 0.01 | 94.68 0.01 |
| NSS | Datasets | JODIE | DyRep | TGAT | TGN | CAWN | EdgeBank | TCL | GraphMixer | DyGFormer | PhyDyG |
|---|---|---|---|---|---|---|---|---|---|---|---|
| rnd | Wikipedia | 96.33 0.05 | 94.37 0.06 | 96.67 0.07 | 98.37 0.07 | 98.54 0.04 | 90.78 0.00 | 95.84 0.15 | 96.92 0.03 | 98.91 0.02 | 99.30 0.01 |
| 98.31 0.06 | 98.17 0.02 | 98.47 0.02 | 98.60 0.07 | 99.01 0.01 | 95.37 0.00 | 97.42 0.02 | 97.17 0.02 | 99.15 0.01 | 99.31 0.01 | ||
| LastFM | 70.49 1.70 | 71.16 1.86 | 71.59 0.17 | 78.47 2.94 | 85.92 0.09 | 83.77 0.00 | 64.06 1.17 | 73.53 0.09 | 93.05 0.10 | 94.10 0.07 | |
| Enron | 87.96 0.51 | 84.89 3.04 | 68.89 1.08 | 88.32 0.95 | 90.45 0.17 | 87.05 0.00 | 75.74 0.73 | 84.38 0.24 | 93.33 0.14 | 93.88 0.02 | |
| Social Evo. | 92.05 0.47 | 90.76 0.22 | 94.76 0.16 | 95.39 0.13 | 87.34 0.10 | 81.60 0.00 | 94.84 0.20 | 95.23 0.04 | 96.30 0.01 | 96.40 0.01 | |
| UCI | 90.44 0.45 | 68.77 2.38 | 78.53 0.71 | 92.03 1.11 | 93.87 0.11 | 77.30 0.00 | 87.82 1.39 | 91.81 0.69 | 94.49 0.29 | 96.22 0.03 |
| NSS | Datasets | JODIE | DyRep | TGAT | TGN | CAWN | TCL | GraphMixer | DyGFormer | PhyDyG |
|---|---|---|---|---|---|---|---|---|---|---|
| rnd | Wikipedia | 94.82 0.23 | 92.43 0.38 | 96.22 0.06 | 97.83 0.07 | 98.24 0.03 | 96.22 0.15 | 96.65 0.02 | 98.59 0.03 | 98.96 0.01 |
| 96.50 0.16 | 96.09 0.11 | 97.09 0.06 | 97.50 0.09 | 98.62 0.01 | 94.09 0.11 | 95.26 0.02 | 98.84 0.02 | 99.00 0.01 | ||
| LastFM | 81.61 3.81 | 83.02 1.44 | 78.63 0.34 | 81.45 4.29 | 89.42 0.06 | 73.53 1.67 | 82.11 0.40 | 94.23 0.06 | 95.18 0.02 | |
| Enron | 80.72 1.42 | 74.55 3.93 | 67.05 1.54 | 77.94 1.05 | 86.35 0.51 | 76.14 0.82 | 75.88 0.46 | 89.76 0.34 | 89.90 0.19 | |
| Social Evo. | 91.96 0.49 | 90.04 0.50 | 91.41 0.13 | 90.77 0.89 | 79.94 0.20 | 91.55 0.08 | 91.86 0.06 | 93.14 0.06 | 93.20 0.08 | |
| UCI | 79.86 1.46 | 57.48 1.85 | 79.54 0.51 | 88.12 2.05 | 92.73 0.07 | 87.36 2.04 | 91.19 0.41 | 94.54 0.15 | 95.75 0.09 |
| NSS | Datasets | JODIE | DyRep | TGAT | TGN | CAWN | TCL | GraphMixer | DyGFormer | PhyDyG |
|---|---|---|---|---|---|---|---|---|---|---|
| rnd | Wikipedia | 94.33 0.28 | 91.49 0.47 | 95.90 0.10 | 97.72 0.03 | 98.03 0.03 | 95.57 0.18 | 96.30 0.06 | 98.48 0.03 | 98.93 0.01 |
| 96.52 0.10 | 96.05 0.11 | 96.98 0.07 | 97.39 0.08 | 98.42 0.02 | 93.80 0.03 | 94.97 0.02 | 98.71 0.01 | 98.93 0.01 | ||
| LastFM | 81.13 3.37 | 82.24 1.48 | 76.99 0.26 | 82.61 3.15 | 87.82 0.15 | 70.84 0.82 | 80.37 0.21 | 94.08 0.07 | 95.12 0.02 | |
| Enron | 81.96 1.37 | 76.34 4.20 | 64.63 1.72 | 78.83 1.13 | 87.02 0.47 | 72.33 0.96 | 76.51 0.71 | 90.69 0.29 | 90.84 0.17 | |
| Social Evo. | 93.70 0.32 | 91.18 0.52 | 93.41 0.17 | 93.43 0.57 | 84.73 0.30 | 93.71 0.17 | 94.09 0.10 | 95.29 0.03 | 95.37 0.05 | |
| UCI | 78.80 0.97 | 58.08 1.79 | 77.64 0.41 | 86.68 2.26 | 90.40 0.13 | 84.49 1.84 | 89.30 0.59 | 92.63 0.10 | 94.40 0.13 |
Dynamic Graph Learning
Dynamic graph learning models how networks and their node features change over time to predict links, events, labels, and system behavior.
See all 124 papers