Organizations: School of Physics and Electronic Engineering, Nanyang Normal University, Wolong District, Nanyang, 473061, Henan, China · School of Integrated Circuits, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China · School of Architecture and Urban Planning, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China
On million-node graphs, the memory wall arrives before the compute wall: a GNN that never fits in VRAM cannot deliver the all-node inference its accuracy promises. The standard remedy, pruning, has always been an accuracy tax - paid once and never refunded. DPR refuses this framing. Its central insight is that the mask that shrinks training can also scaffold recovery: one compact node set bounds both computations of graph learning, serving as the encoder's training graph and, at inference, as the anchor from which a sparse structure-aware decoder rebuilds every pruned node (double-layer sparsification). The order is inverted - prune for efficiency, then reconstruct for accuracy - and the tax is repaid: the deployed 30%-budget model trains in 1,450 MiB of peak memory on one 6 GB GPU and recovers 96.9% of its dense-reconstruction reference, beating a same-GPU rerun of the edge-sparsification baseline DSpar by +1.36 pp at 50.1% lower peak training memory; against the same-protocol full-batch no-prune encoder the deployed point pays a measured 2.64 pp accuracy tax for a 2.8x smaller training footprint. Pruning without reconstruction leaves the encoder at 0.378 within the same 40-epoch window (Fig. 4), and all-node serving runs 46x faster than sampling-based GraphSAGE, giving up approx. 5.4 pp of test accuracy. The framework scales to a 2.45M-node production graph at 6,285 MiB of peak training memory; extended variants (long-range context, external text features, a CAN-bus application) are reported as extensions and do not compete with the deployed method, and the 41.8%-budget M4 point, scored on a coarsened super-node graph with its own split, is reported separately rather than compared with the original-node baselines. DPR thus turns aggressive node pruning from an accuracy tax into a safe training accelerator for VRAM-limited all-node inference.
Figures & tables
Figure 1: System overview. (a) One feature-norm mask C induces both sparsified graphs: the encoder graph Gp=(C,Ep) keeps only core-internal edges, and the reconstruction graph Gr=(V,Er) with Er=Ecore-1hop adds the core-to-1-hop links needed to rebuild deleted nodes. (b) DPR-S joint training: the encoder fθ runs on the core only ( X[C] with Ep ; a 30% -node, 8% -edge core on which a 3-layer GraphSAGE aggregates ≈1/12 of full); the sparse decoder Eϕ reconstructs pruned-node features within the core’s 1-hop from X and Er , and the reconstructed features feed the head; the CE loss over training nodes jointly updates fθ,Eϕ,ψ — “decoupled” refers to a division of functional roles within one pipeline (§ A.16 ). The dashed box in (a) marks the optional extension DPR-S-M4, which feeds an m4v long-range context row to an extended decoder. (c) Streaming inference: HC stays GPU-resident (cached after training), X and Gr stay in host RAM, and each pruned node v is scored by streaming its row xv with its core-1-hop neighbours Nr(v) through the frozen decoder and head to produce y^v ; the encoder is never re-run on the full graph.
Figure 2: The two pipeline stages. Stage 1 (a–b): core selection by feature magnitude. (a) A cr⋅n fraction of nodes with the smallest Euclidean feature norm ∥xv∥ is chosen as the core C (2-D sketch); the dashed circle is the deterministic threshold τ(cr) . (b) On a real feature distribution the histogram of ∥xv∥ separates cleanly into the cr=0.30 core and the pruned set P ; the cut is a single deterministic pass with zero learnable parameters. We retain the learnable hyperbolic variant only as an ablation (§ A.16 ). Stage 2 (c–e): double-layer sparsification, induced by the same core mask C . (c) The original graph G with core C highlighted in blue; (d) the main graph Gp (layer 1) on which the encoder trains, keeping the core-internal edges ( ≈8% of E ); (e) the reconstruction graph Ecore-1hop (layer 2) used by the decoder, which adds the core-to-1-hop edges needed to rebuild deleted nodes; it has 39 – 45% fewer edges than the full reconstruction graph on the products probes (§ 4.2 ). Both layers are driven by the same mask C , so the second sparsification is endogenous to the decoder rather than a post-hoc cut.
Figure 3: DPR-S efficiency at a glance (ogbn-arxiv; raw node features unless stated). (a) Accuracy vs. peak training GPU memory on the same RTX 4050 card: external baselines (gray; same-card rerun protocols, Table A.6 ) cluster in the high-memory region, while DPR-S (teal) holds a comparable accuracy band at 1,450 MiB — 50.1% less peak memory than the strongest same-card pruning baseline DSpar at +1.36 pp accuracy. (b) Analytical attention-matrix memory at arxiv scale (fp32): full-graph GPS attention ( Rampášek et al., 2022 ) ( N×N ) needs ≈459 GB (fp32 attention logits: N2×4 B ≈114.7 GB per head, four heads) and softmax over 50,743 super-nodes still needs ≈137 GB, while our linear attention holds it at ≈87 MB ( ≈5,300× below full attention) — the collapse that keeps coarsened training on a single commodity GPU.
Block (where)
Question
One-line answer
Main result: deployed-DPR-S recovery rate on ogbn-arxiv and ogbn-products; extended variants (SBERT external features, Car-Hacking) flagged per row (§ 4.2 )
RQ1: does reconstruction restore accuracy after aggressive pruning, across backbones and scales?
DPR-S retains 96.9% (arxiv, pure double-layer) and 96.5% (products) of the dense-reference accuracy; the extended SBERT external-feature variant lands within ∼0.6 pp of the leaderboard peak while training on the same 6 GB card (Table A.2 ).
Ablations & analyses (§ 4.3 ; App. A.16 )
RQ2: which component carries the gain?
Joint encoder–decoder training and the sparse decoder carry it; selector sophistication does not (fixed mask ≈ learnable hyperbolic), and DPR-D quantifies the dense-reference ceiling.
RQ3: does it survive production scale, and when does it break?
Training at 2.45M nodes peaks at 6,285 MiB (fits an 8 GB card); full-scale coarsened Car-Hacking training on one commodity GPU; the PPI study delimits backbone dependence.
Table 1: Roadmap of the experiments: block → question → one-line answer. Appendix blocks are marked (App.).
Method
Training nodes
Test acc
Peak VRAM (MiB)
Full-node references (rerun, same card)
GraphSAGE Hamilton et al. (2017) (full-batch, no prune)
100%
0.7156 (s)
4057
SIGN Rossi et al. (2020) ( A5X precomputed features)
100%
0.7075 (s)
1769
GraphSAINT Zeng et al. (2020) (RW sampler)
100%
0.6874 (s)
5175
Cluster-GCN Chiang et al. (2019) (random partition)
100%
0.6462 (s)
2902
DSpar Liu et al. (2023) (degree-based edge sparsification)
100%
0.6756±0.0012
2908
Table 2: Budget–accuracy comparison on ogbn-arxiv: DPR against contemporaneous full-node baselines, each rerun by us on one 6 GB RTX 4050 card, the same dataset and data split (protocol in § 4.1 ; per-row architectures in App. A.9 ). Seeds: the four full-node references marked (s) are single-run point estimates; every remaining row is a multi-seed mean, reported as mean ± std. † Scored on the coarsened super-node graph with its own split; it documents budget scaling only and is not comparable to the original-node rows.
Dataset / task
no-prune
no-prune +Exp
DSpar (same-card)
HSP- alone
DPR-S (final)
DPR-D (dense)
DPR-S-M4 (ext.)
Core protocol (GraphSAGE encoder, raw 128-dim): deployed DPR-S vs. dense reference
ogbn-arxiv (acc)
0.7156
∼0.70
0.6756±0.0012 c
0.378 d
0.6892±0.0020 e
0.7111±0.0053 f 0.7068±0.0022 g
0.6953±0.0019 h
ogbn-products 500k (acc)
0.7157±0.0035
—
—
—
0.6635±0.0122 e (core-1hop)
0.6874±0.0015 f 0.6876±0.0007 g
—
ogbn-products 2.45M (official split)
—
—
—
—
0.7814±0.0003 e
—
—
Extended variants (not part of the deployed method)
ogbn-arxiv external features (acc)
0.7803 a
—
—
—
—
—
0.7744±0.0003 b
Table 3: Main results: the in-framework recovery ladder on ogbn-arxiv, ogbn-products (500k, 2.45M) and Car-Hacking (protocol and definitions in § 4.1 –§ 4.2 ); the external same-card comparison is Table 2 . (s)/(3s)/(5s) = 1/3/5 seeds; — = not defined under the row’s protocol.
Figure 4: Convergence comparison on the full ogbn-arxiv graph (seed 0, 40 epochs): the full DPR pipeline (HSP+Expander: prune + reconstruct) against the pruning-only baseline (HSP-alone: prune, train on the pruned graph, no reconstruction) and a no-prune companion run over the same window. Reconstruction lifts accuracy above the pruning-only baseline at both core ratios ( 0.591 vs. 0.378 at cr=0.3 ; 0.561 vs. 0.454 at cr=0.7 at epoch 40), so the recovery is not a cr=0.3 -only artifact. Pruning alone stays below the no-prune companion ( 0.452 ) at cr=0.3 , but at cr=0.7 it already tracks it rather than collapsing. All curves are still rising at epoch 40; this window shows early-training dynamics rather than a converged ceiling (the fair no-prune ceiling, 0.7156 , is reported in § 4.2 ).
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Method mechanism. (A) Core sparsity at cr=0.30 : keeping 30% nodes retains only 7.8% of edges ( ∣Ep∣/∣E∣=90805/1166243 ; unique undirected edges with both endpoints in the core), and 57.6% of the core nodes are degree-0 within the core-induced subgraph — each bar is plotted with its own denominator ( ∣E∣ , N , core-node count, respectively); the training-cost saving comes from this extreme edge reduction on the encoder side. The core-1hop reconstruction graph Er (cut edges between core and deleted nodes) retains 34.5% of edges, so the combined structural retention Ep∪Er is 42.3% of unique undirected edges, and the per-aggregation message proxy (encoder on Ep+ decoder on Er ) is ≈25% of a full-graph aggregation (Table A.4 ). (B) The expander recovers accuracy after pruning: HSP-alone (no expander) collapses to 0.378 — its classifier scores the full node set, with core nodes keeping their encoded features and pruned nodes zero-filled exactly as in training (no expander, no fallback) — while fixed-mask + Expander reaches 0.7068 and learned HSP + Expander 0.7111 , near the fair no-prune SAGE 0.7156 . (C) The simpler fixed-magnitude mask is no worse than the learnable hyperbolic pruner on both accuracy and edges kept, so we adopt the simpler fixed mask (observation at matched budget). Train/deploy split: the encoder runs only on the core graph Gp=(C,Ep) ; the decoder is trained jointly on the reconstruction graph Er over all nodes and is then frozen. At deployment only the frozen core embedding HC resides in accelerator memory, while Er stays in host RAM; the decoder reconstructs pruned nodes from Er plus streamed per-node raw-feature rows xv (the extended variant additionally streams precomputed m4v rows) – the encoder is not re-run on the reconstruction graph.
Figure A.2: Data Snapshots (supporting the body numbers): ogbn-products (2.45M nodes) under the official OGB sales-ranked split. ogbn-products hosts no per-node timestamp, so the sales ranking is the only ordering signal. (a) The split is a rank-contiguous partition of the ranking axis: train occupies the top-ranked/best-selling window [0,8%) , valid [8%,9.6%) , and test the remaining long-tail [9.6%,100%) — so train and test differ in sales-volume (popularity) distribution by construction. (b) Label drift follows: normalized class priors per window, with DJS(Ptrain∥Ptest)=0.042 nats while DJS(Ptrain∥Pvalid)=0.0003 — the rank-adjacent validation split matches train, and the long-tail test is a distinct label distribution (log-likelihood χ2=5.0×104 , p≈0 ; class-prior concentration: top-5 share 60%→52% , class entropy 3.62→3.98 bits). (c) Mechanism: the top train categories are far more prevalent among the best-selling nodes, while long-tail categories concentrate in the low-ranked test pool — i.e. the popularity/ranking difference drives the category-prior drift. This is a consequence of the official split , not a separate design choice; we therefore keep the same-graph recovery rate (DPR-D), which compares two methods on one graph and is therefore not exposed to the split-induced prior shift, as the primary metric and treat Test accuracy as a long-tail stress test.
Figure A.3: ogbn-arxiv ships a genuine per-node year; its official time split is a real chronological partition (train 1971 – 2017 , valid 2018 , test 2019 – 2020 ). (a) Publication-year distribution per split. (b) The label prior drifts with true time: the share of the top training-era subjects falls and the emerging-subject share rises across years ( DJS(Ptrain∥Ptest)=0.072 nats on labels, larger than the products figure of 0.042 ). This confirms on the one dataset that ships timestamps that a chronological train/test partition induces a measurable label-prior drift; products lacks timestamps, hence its drift is described as a sales-ranking (popularity) proxy rather than a literal time split.
Configuration
DPR-S test-acc
note
ogbn-products, 500k class-stratified random split
0.8485±0.0004 (3s)
no sales-ranked popularity shift
Appendix
Table A.1: Transfer and different-protocol check (supplementary; not directly comparable to Table 3 : different split partition).
−1.8 pp vs same-pruner full ( 0.7068 ), double sparsification
DPR-S cr=0.3, core-1hop recon (final, s, 5s)
0.6892±0.0020
5-seed, double sparsification
Appendix
Table 12
Configuration
best test-acc
no-prune (+Exp full reconstruction)
0.6334
DPR-D (core + core-1hop)
0.6119
→ recovery rate
96.6% (gap ≈ 2.15pp)
Appendix
Table 13
Configuration
best test-acc (mean ± std, 3 seeds)
no-prune (+Exp full reconstruction)
0.7157 ± 0.0035
DPR-D / HSP pruner (core + full reconstruction)
0.6874 ± 0.0015
DPR-D / simple mask pruner (core + full reconstruction)
0.6876 ± 0.0007
DPR-S (sparse decoder, core-1hop)
0.6635 ± 0.0122
→ recovery rate
96.1% ± 0.4% of no-prune (gap ≈ 2.83pp; no measurable selector difference between the two DPR-D pruner rows); DPR-S = 96.5% of DPR-D
Appendix
Table 14
Role
Backbone
Decoder / scope
Dataset (split)
Seeds
GPU
Result
No-prune reference
GraphSAGE 3L, full-batch
— / full graph
arxiv (OGB)
—
4050 6 GB
0.7156 acc
Pruned-no-reconstruction floor
GraphSAGE 3L
none / Gp only
arxiv (OGB)
1
4050 6 GB
0.378 acc
DPR-D (dense reference)
GraphSAGE 3L h=256
dense [Ke,C′] / full recon.
arxiv (OGB)
5
4050 6 GB
0.7111±0.0053 acc
DPR-S (final method)
GraphSAGE 3L h=128
gated-MP expander / core-1hop
arxiv (OGB)
5
4050 6 GB
0.6892±0.0020 acc
DPR-S-M4 (extended variant)
GraphSAGE 3L h=128
AMP cross-attn + M4 / near-full
arxiv (OGB)
5
4050 6 GB
0.6953±0.0019 acc
DPR-S-M4 (raw kr=0.5 budget point)
GraphSAGE 3L h=128
AMP cross-attn + M4 / near-full, raw feats
arxiv coarsened ( 70,744 )
3
4050 6 GB
0.7257±0.003 acc †
Appendix
Table A.2: Configuration–outcome master table: every headline number in this paper bound to its exact configuration, seed count, and hardware. “core-1hop” / “near-full” denote the reconstruction scope Er ; “final” marks the frozen final method (DPR-S). The three DPR-S-M4 (extended-variant) rows never substitute for final-method validation (the DPR-S final-method rows). The dense 2 × 2 cell row is the per-node-pool core-1hop entry of the design-space summary (Table A.8 ); the full-scale final method (DPR-S, official OGB sales-ranked split) is the DPR-S (final), full scale row. Secondary numbers (PPI transfer, ablation variants, DSpar comparison) appear in their own sections with the same configuration detail.
Mask applied
deleted
cover.
fallb.
nbrs
arxiv core ( cr=0.3 )
118,540
74.4%
25.6%
3.39/2
prod-200k core ( cr=0.3 )
140,000
42.4%
57.6%
1.06/0
prod-200k nm-var. ( 30% super)
96,065
44.7%
55.3%
1.07/0
Appendix
Table A.3: Deletion-side coverage under the mask actually applied by each reported experiment (all masks applied to original nodes unless noted). “Covered” = deleted node with ≥ 1 core neighbor (rebuilt by gated message passing from its core neighbours over Er ); “fallback” = no core neighbor (keeps its projected own feature — a direct consequence of graph connectivity, Algorithm 1 ); last column: mean/median core-neighbors among deleted nodes (unique-undirected counting).
Quantity
arxiv
prod-200k
∣E∣ unique undirected
1,157,799
413,798
Ep core–core
89,900 ( 7.8% )
44,101 ( 10.7% )
Ercut core-1hop cut
399,504 ( 34.5% )
147,867 ( 35.7% )
drop–drop (unused)
668,395 ( 57.7% )
221,830 ( 53.6% )
combined Ep∪Ercut
489,404 ( 42.3% )
191,968 ( 46.4% )
msgs / aggregation
579,304 ( 25.0% )
236,069 ( 28.5% )
Appendix
Table A.4: Edge-count audit under the exact main-experiment masks (fixed-magnitude low-norm core, cr=0.3 , original graph). Structural rows: unique undirected edges; the core–deleted cut is written Ercut because the reconstruction set the decoder gathers over is Er=Ep∪Ercut ( Ep⊆Er⊆E ). “msgs / aggregation”: a per-aggregation message proxy — directed messages of one encoder aggregation pass (symmetrized Ep ) plus one decoder gather over the core–deleted cut ( Ercut ), divided by the directed message count 2∣E∣ of one full-graph aggregation; it is a structure-cost proxy, not a layer-weighted full-epoch traffic count ( Lenc2∣Ep∣+Ldec∣Er∣ with layer counts Lenc,Ldec would give the latter).
Reconstruction path
share
full
no-M4
no- Xv
core kept (encoder path)
32.9%
0.6835
0.6835
0.6835
covered (core-1hop cross-attn)
46.8%
0.7518
0.7518
0.7518
fallback (per-node feat. + M4)
20.4%
0.5980
0.5851
0.5305
all deleted (covered+fallback)
67.1%
0.7052
0.7012
0.6847
overall (test)
100%
0.6980
0.6954
0.6843
Appendix
Table A.5: Reconstruction-path stratified accuracy on ogbn-arxiv under the extended variant DPR-S-M4 ( cr=0.3 main mask, test split, 48,603 nodes, seed 0) with eval-time fallback-input ablations. The fallback path is Wdxv+Wmm4v plus one gated core-neighbour message step (extended-variant mechanism, Algorithm 2 ); “no-M4” / “no- Xv ”: the corresponding streamed feature input is zeroed at eval time (weights unchanged, message term retained). Protocol note: the seed-0 checkpoint here belongs to the matched-capacity control protocol of Table A.9 (same recipe, near-full operating point): its eval-time overall score is 0.6980 (train-time model-selection score 0.6966 ), consistent with that table’s 5-seed mean 0.6953±0.0019 (seed 2 reaches 0.6986 ). The core-1hop operating point of the main results is a different reconstruction radius with its own 5-seed mean ( 0.6892±0.0020 ); path shares and stratified accuracies here are computed under this checkpoint’s actual paths.
Method
Backbone
Training graph
test-acc
Peak (MiB)
ms/epoch
Scalable GNN references (backbone)
SIGN (Frasca et al., 2020)
MLP on A5X , undir.
full (precomputed)
0.7075
1769
72.5
GraphSAGE (full-batch)
3-layer SAGE
full graph
0.7156
4057
739.3
GraphSAINT (Zeng et al., 2020)
3-layer GCN
RW node sampler
0.6874
5175
1359
Cluster-GCN (Chiang et al., 2019)
3-layer SAGE
random partition
0.6462
2902
883
Ours (RTX 4050 same card; matched-pipeline VRAM; ms/epoch omitted — DPR full-batch core, not sampler-comparable)
Appendix
Table A.6: External baseline comparison on ogbn-arxiv (single NVIDIA RTX 4050, 6 GB card). All external-baseline rows are single-seed point estimates on this 6 GB card (seed 42, selected as the highest-validation-accuracy run); the DPR rows are 5-seed averages. The single-seed treatment reflects Table A.6 ’s role: an order-of-magnitude accuracy and peak-VRAM reference against the closest external comparators. The controlled multi-seed numbers for our method live in § 4.2 . The plotted frontier summarizes the reported operating points ( +1.97 pp over the strongest rerun baseline for the extended variant DPR-S-M4; the deployed DPR-S holds +1.36 pp with a separately profiled peak VRAM 50.1% below DSpar’s 2908 MiB); a broader multi-seed evaluation of the external baselines remains future work. Peak VRAM is a single-run profiling figure, not a seed-averaged quantity, and is reported only where separately logged on this card (— otherwise: the h=256 dense reference and the near-full extended variant were not separately profiled here; the 1450 MiB entry is the final-method deployment profile quoted in the abstract). Controlled same-protocol numbers are in § 4.2 .
Figure A.4: Budget–accuracy plane on ogbn-arxiv (same RTX 4050 card; the baseline numbers and per-epoch wall-clock price tags are the same-card reruns of Table A.6 ). (a) External baselines (gray; hatched: DSpar) all train on 97 – 103% of the node budget, whereas our points (colored) operate on pruned cores: DPR-S at 30% and the extended DPR-S-M4 at 41.8% of nodes. The shaded region marks the budgets where our external-feature operating point ( 0.7744 ; leaderboard reference 0.7803 , a different, stronger feature protocol) has a node budget at least as large as every rerun baseline and an accuracy at least as high. The raw-feature kr=0.5 point (open marker) is scored on the coarsened super-node graph with its own split and is shown for budget scaling only. (b) Within-framework budget–recovery operating points (seed std as error bars); the kr=0.5 extended point trains on 41.8% of the nodes but is scored on the coarsened super-node graph, so it is not read against the original-node reference 0.7156 .
Method
Sparsity lever
Recovery
Test acc
Train speedup
Inference
Hardware / Protocol
Paper-reported only (proprietary / different-task regimes)
Edges only (all N retained; ϵ=0.82 , ≈26% undir removed)
None
0.6756±0.0012
∼150× †
Tracks full-SAGE baseline
RTX 4050 6 GB ; same GraphSAGE encoder as DPR; peak VRAM 2908 MiB
Appendix
Table A.7: External prune/sparsify/coarsen baselines on ogbn-arxiv. Same-card rerun regime (RTX 4050 6 GB): DSpar (Liu et al., 2023) is now rerun under the exact same 3-layer GraphSAGE hidden-128 encoder protocol used for DPR-D/DPR-S ( ϵ=0.82⇒≈26% undirected-edge removal, matching the paper’s Table 2 value for arxiv). DPR-S-M4 0.6953 is therefore ≈1.97 pp above DSpar at same hardware, backbone, and training budget, but different reduction regimes (node-level coarsening + reconstruction vs. edge-only sparsification on the full node set). Mechanism regime disclaimer (critical for fair reading): DSpar never drops any node ( 100% of N=169,343 always active) and therefore has no pruned-node recovery problem and no decoder; its per-epoch latency is low because it performs only the forward pass over the full-node graph , but it can never train a graph that is too large to instantiate on the card at the full-node level (the bottleneck DPR solves). DPR trains on a 30% core and then reconstructs the remaining 70% at inference via structure-aware diffusion — which is exactly the regime in which node-level OOM is the primary blocker, not SpMM speed. Coarformer is not reproducible (proprietary internal baseline). FIT-GNN partitions the graph into coarsening-derived induced subgraphs with Extra/Cluster Nodes; its public code is available but ships no ready ogbn-arxiv configuration, so it is not a graph-pruning comparison under the same protocol and is listed only for transparency. GraphSAINT / Cluster-GCN / SIGN / full GraphSAGE are already rerun on the exact same card in Table A.6 (full-node methods, no reconstruction).
Figure A.5: Same-card Pareto comparison; every point is a measured number from this work’s runs and logs (no extrapolation); panels are ordered by service target, and every point is identified by name in the legend beneath its panel (the plots themselves carry no labels). (a) Full 2.45 M ogbn-products, original-node evaluation: DPR-D (dense reconstruction) peaks at 16,100 MiB and the no-prune GraphSAGE baseline at 23,079 MiB — within a 6 GB budget no full-scale original-node baseline runs at all (solid and dashed vertical lines: 6 GB and 24 GB card capacities; red wash: beyond the 6 GB card; dashed step: two-point Pareto frontier). (b) the same products scale at super-node granularity, shown in its own facet because the accuracy axis is not comparable with (a): DPR-S-M4 trains on the coarsened super-node set ( 684,233 super-nodes) at 6285±14 MiB peak (3 seeds, batch 1024, h=128 ; wall 526±3 s per seed) — 2.6× leaner than DPR-D, fitting an 8 GB card. (c) ogbn-arxiv, test accuracy vs. peak training VRAM (log scale); the shaded band marks VRAM beyond the 6 GB card edge and the solid step is the Pareto frontier of the plotted points (non-dominated region above–left). Within the same-card rerun regime (GraphSAGE h=128 encoder, filled squares), DPR-S dominates the DSpar rerun (the final core-1hop DPR-S profiles at 1450 MiB against DSpar’s 2908 MiB, i.e. 2.0× lower peak VRAM; the +1.97 pp DPR-S-M4 advantage over DSpar is a different configuration, reported in the text and not plotted on this axis), as do GraphSAINT and Cluster-GCN — while seeing only 30% of the nodes (DSpar and the external references operate on the full node set). The matched dense decoder control (same family and width, 3 seeds) uses 2040 MiB for 0.682 : the sparse decoder is 32% leaner and +0.7 pp more accurate at matched width and family (§ 4.2 ). DPR-D (h256) arxiv peak VRAM was not separately logged and is therefore not plotted; its encoder scale is represented by the GraphSAGE full-batch point.
cr
edges kept
HSP-alone
DPR-D (dense)
Δ (Exp − alone)
0.9
0.686
0.6123 ± 0.0006
0.7955 ± 0.0051
+0.1832
0.7
0.603
0.5991 ± 0.0014
0.7953 ± 0.0052
+0.1962
0.5
0.367
0.5609 ± 0.0015
0.7935 ± 0.0061
+0.2326
0.3
0.176
0.5047 ± 0.0034
0.7923 ± 0.0034
+0.2875
(no-prune)
1.0
—
0.6613 ± 0.0010
—
Appendix
Table 23
Figure A.6: Sensitivity to the core ratio cr on PPI (node classification (multi-label), 5 seeds). (a) Once reconstruction is added, accuracy is approximately flat in cr , whereas the no-reconstruction baseline degrades monotonically as the pruning becomes more aggressive. (b) The gain from reconstruction grows from +0.183 at cr=0.9 to +0.288 at cr=0.3 – the harder the pruning, the larger the recovery. The dashed line is the fraction of edges retained after pruning, showing the reconstruction simultaneously improves accuracy and reduces graph size.
Figure A.7: The pruner is pluggable: on PPI (node classification (multi-label), 5 seeds, matched edge budget) the learnable hyperbolic pruner (HSP) tracks random pruning of the same budget and falls far below the no-prune reference ( 0.661 ), i.e. the pruning geometry carries no recoverable signal by itself – confirming the gain is primarily attributable to the reconstruction stage. This panel is a separate matched-edge-budget control from the node-budget cr -sweep table above; the curves are that control’s own 5 -seed run (its no-prune mean is 0.641 ), while the dashed level marks the manuscript’s PPI no-prune value 0.6613 .
Configuration
Test Acc
50-epoch wall-clock
Dense [Ke,C′] (control)
0.6649
∼4.4 h (extrapolated)
Sparse bipartite attention (default)
0.6649
∼29 min (measured)
Appendix
Table 26
Configuration (decoder variant / key pool)
Scope
Test Acc
Sparse AMP+M4 (extended DPR-S-M4)
near-full
0.6953±0.0019 (5)
Sparse gated-MP (final DPR-S)
core-1hop
0.6892±0.0020 (5)
Sparse h=128 AMP (matched control)
core-1hop
0.689±0.002 (3)
Dense [Ke,C′] (per-node pool)
core-1hop
0.6981±0.001 (3)
Dense [Ke,C′] ( h=128 AMP, matched control)
core-1hop
0.682±0.001 (3)
Dense [Ke,C′] (union pool)
near-full
not measured
Appendix
Table A.8: Decoder-family × reconstruction-scope design-space summary (ogbn-arxiv, cr=0.3 , GraphSAGE h=128 unless noted; seed count in parentheses). This is a design-space listing, not a controlled 2×2 experiment : the dense union-pool (near-full) cell was not measured; the near-full and core-1hop sparse rows use different decoder variants (extended AMP+M4 vs. final gated-MP); the two core-1hop matched control rows (same h=128 AMP family, 3 seeds) are the same-width dense-vs-sparse calibration analyzed in § A.16 . No interaction or accuracy-equivalence claim is made.
Figure A.8: Empirical motivation: the headline recovery rate decomposed along its two design axes (ogbn-arxiv, 5 seeds except where noted). The panel separates decoder type (sparse DPR-S family vs. dense reconstruction reference DPR-D) from reconstruction scope (near-full vs. core-1hop), which a single recovery percentage would otherwise conflate: sparse near-full (DPR-S-M4) 0.6953±0.0019 , 97.8% of the dense reference; sparse core-1hop (the final method, DPR-S) 0.6892±0.0020 , 96.9% ; dense core-1hop 0.6885±0.0013 , 96.8% ; dense near-full 0.7111±0.0053 is the 100% reference (the dotted vertical line marks this dense-reference bound). The DPR-D rows use h=256 and the DPR-S rows h=128 , so this panel is a design-space map, not a width-matched comparison; the same-width control is the matched dense-vs-sparse comparison of § 4.2 ( h=128 : sparse 0.689 vs. dense 0.682 ). Masks follow the run logs: the final DPR-S row and the dense core-1hop control use the fixed-magnitude (simple) mask, while the dense near-full reference uses the HSP mask (Table 3 ), so the 96.9% headline is a system-level retention (§ 4.2 ). The pruned-no-reconstruction floor ( 0.378 ) lies far below the plotted range and is reported in Table 3 .
Figure A.9: Optimization sensitivity. (a) The decoder is the learning-rate-sensitive component of the method: at lr = 0.01 training diverges to a degenerate flat solution ( ≈0.426 ), while the recommended lr = 0.001 reaches 0.6957 . (b) The three decoder mechanisms (P1 core-confidence residual fusion, P2 structure-aware reconstruction of deleted nodes, and a small learnable modulation term) contribute 1.76 , 1.07 and 0.25 percentage points respectively on ogbn-arxiv (single seed), with all three together reaching 0.6957 (single seed, comparable to the 5-seed DPR-S-M4 0.6953±0.0019 ; the ablation is run within the extended DPR-S-M4 decoder family).
Decoder
Params
Test Acc (5 seeds)
AMP (cross-attn + MP)
206,979
0.6953±0.0019
MLP-only (no edges, no attn)
207,289
0.6702±0.0025
Delta (matched)
—
+0.0251
Appendix
Table A.9: Decoder capacity control (ogbn-arxiv main protocol, five paired seeds): the AMP decoder vs. a parameter-matched MLP-only decoder. Both variants live inside the extended DPR-S-M4 family — same core mask, same encoder, and the same graph-derived M4 long-range features — so the comparison isolates the decoder’s structure use, not the final method’s pruning geometry.
Figure A.10: End-to-end efficiency, grouped by measurement type (each panel states its own dataset / hardware / baseline; the three speed-up panels use different denominators and are deliberately separated so no shared y-axis is implied). (a) Peak training memory (ogbn-products; full 2.45 M graph on a > 16 GB cloud GPU, 200k subgraph on RTX 4050): no-prune reaches 23,079 MiB (near a 24 GB card limit), while the dense DPR-D variant drops the peak to 16,100 MiB ( 1.43× leaner); the 200k subgraph improvement is 1.27× . (b1) End-to-end / per-step training speed-up (ogbn-arxiv, RTX 4050, DPR-S core-1hop vs. its own dense reconstruction): 2.70× (with 21% inference speed-up in the same regime); per-step training 1.57× (products 500k, cloud GPU, App. A.6 ). (b2) Reconstruction inference speed-up : decoder step 1.30× (200k probe, RTX 4050). (b3) Component-level speed-up (analytical / probe, not directly comparable to (b1)/(b2)): node-merge 4.45× (lean configuration; measured against the same-hardware single-batch linear SAGE reference of 216 ms, § A.17 ); linear attention 4.50× measured (up to ≈9× vs. an extrapolated dense baseline). (c) Second sparsification layer + controlled framework : the reconstruction graph has 31394 core-1hop edges vs. 56570 for full reconstruction (ogbn-products, 200k subgraph); both edge counts are attention-probe reconstruction topologies (dense [Ke,C′] key pool vs. bipartite Er ), a different denominator from the original-graph structural audit in Table A.4 . Scope : every value here is a training-feasibility or internal decoder/attention efficiency measurement on the coarsened pipeline; all-node scoring latency is measured separately in Table A.10 .
System
Test acc.
Latency (s)
nodes/s
VRAM (MiB)
RSS (MiB)
GraphSAGE full-batch (no sampler)
0.7094
0.176±0.001
965,165
3,698
1,239
GraphSAGE + NeighborLoader ( 15,10,5 )
0.7097
71.8±0.3
2,359
631
1,275
DPR-S, training-time protocol (ref.)
0.6892±0.0020
137.3
1,233
4,376
3,020
DPR-S serving (Alg. 1 )
0.6558±0.0205
1.55±0.01
109,566
4,743
5,131
Appendix
Table A.10: All-node inference benchmark on ogbn-arxiv ( N=169,343 ; RTX 4050 laptop, 6 GB; PyTorch 2.7.1+cu118). Per-pass steady-state timing after warm-up; ± is the per-pass sd over 5 repeated passes (the training-time reference protocol uses a single repeat, hence no ± ). DPR-S rows use the same core-1hop checkpoints as § 4.2 (5 seeds; single run for the SAGE baselines). Er for the serving row is 2.12 M directed edges ( 183% of undirected E ) with a 0.3 s one-time setup; its accuracy includes a one-time BatchNorm re-estimation on the serving core graph (without it: 0.6507±0.0272 ). The training-time protocol row re-selects the core and re-encodes per batch (accuracy parity with the checkpoints by construction) and is shown to quantify the serving-design cost, not as a deployable system.
As graphs scale to billions of nodes and edges, graph Machine Learning workloads are constrained by the cost of multi-hop traversals over exponentially growing neighborhoods. While various system-level and algorithmic optimizations have been proposed to accelerate Graph Neural Network (GNN) pipelines, data management and movement remain the primary bottlenecks at scale. In this paper, we explore whether graph sparsification, a well-established technique that reduces edges to create sparser neighborhoods, can serve as a lightweight pre-processing step to address these bottlenecks while preserving accuracy on node classification tasks. We develop an extensible experimental framework that enables systematic evaluation of how different sparsification methods affect the performance and accuracy of GNN models. We conduct the first comprehensive study of GNN training and inference on sparsified graphs, revealing several key findings. First, sparsification often preserves or even improves predictive performance. As an example, random sparsification raises the accuracy of the GAT model by 6.8% on the PubMed graph. Second, benefits increase with scale, substantially accelerating both training and inference. Our results show that the K-Neighbor sparsifier improves model serving performance on the Products graph by 11.7x with only a 0.7% accuracy drop. Importantly, we find that the computational overhead of sparsification is quickly amortized, making it practical for very large graphs.
Graph Neural Networks (GNNs) excel at relational reasoning but face two persistent challenges: the lack of interpretable attribution for heterogeneous node types, and the computational overhead of message passing over large, noisy graphs. We propose the Hierarchical Attention-based Heterogeneous GNN (HA-HeteroGNN), a framework that addresses both issues through a unied explainability-to-pruning pipeline. A two-tier attention mechanism separates sensor-level and context-level computation across 16 node types and 18 edge types, producing per-node relevance scores via an attention-based GNN Explainer without requiring gradient backpropagation. These relevance scores then serve as a principled pruning criterion: removing nodes identied as consistently uninformative yields a 27% reduction in graph edges while simultaneously improving classication accuracy by 2.46.1% across all model variants, challenging the conventional assumption that pruning necessarily trades accuracy for eciency. Experiments on a 50,000-record synthetic dataset spanning 11 report categories demonstrate 97.5% cross-strategy explanation stability and domain consistent sensor attribution, with training-time reductions of up to 43.9% and real-time inference latency of approximately 5860 ms per sample.
Seungwoo Kum
Korea Electronics Technology Institute (KETI) Seongnam, Republic of Korea
The rapid growth of modern training datasets has significantly increased computational cost, motivating dataset pruning~(DP) methods which retain only a subset of informative samples to reduce training cost. Existing pruning criteria typically rely on either intrinsic signals that assess samples independently or extrinsic signals that promote diversity via pairwise relations. While effective in their own specific regimes, each captures only one aspect of sample utility and lacks robustness across different pruning ratios or data distribution. In this work, we present a unified graph-based DP framework. By modeling the dataset as a weighted graph, where node weights encode intrinsic value and edge weights encode extrinsic value, DP can be cast as a Maximum Weight Clique Problem (MWCP). Although MWCP is NP-hard, its structure admits a principled greedy solution based on sample-wise marginal gains. Under a few mild conditions, we further prove that this unified objective enjoys a formal approximation guarantee, which applies to a broad family of importance metrics and provides practical design guidelines. Extensive experiments show that our method outperforms existing DP methods while substantially reducing training cost, reducing training time by over 40% without sacrificing accuracy on ImageNet-1k with ResNet-50.
Dongyue Wu, Zilin Guo, Xiaoyu Li +4
State Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China · Ant Group, Hangzhou, China