cs.LGOct 25, 2026
SaveDecoupled Prune-and-Reconstruct: Training on a Compact Core with Double-Layer Sparsification
Organizations: School of Physics and Electronic Engineering, Nanyang Normal University, Wolong District, Nanyang, 473061, Henan, China · School of Integrated Circuits, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China · School of Architecture and Urban Planning, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China · School of Mechanical Science and Engineering, Huazhong University of Science and Technology, 1037 Luoyu Road, Hongshan District, Wuhan, 430074, Hubei, China
Abstract
On million-node graphs, the memory wall arrives before the compute wall: a GNN that never fits in VRAM cannot deliver the all-node inference its accuracy promises. The standard remedy, pruning, has always been an accuracy tax - paid once and never refunded. DPR refuses this framing. Its central insight is that the mask that shrinks training can also scaffold recovery: one compact node set bounds both computations of graph learning, serving as the encoder's training graph and, at inference, as the anchor from which a sparse structure-aware decoder rebuilds every pruned node (double-layer sparsification). The order is inverted - prune for efficiency, then reconstruct for accuracy - and the tax is repaid: the deployed 30%-budget model trains in 1,450 MiB of peak memory on one 6 GB GPU and recovers 96.9% of its dense-reconstruction reference, beating a same-GPU rerun of the edge-sparsification baseline DSpar by +1.36 pp at 50.1% lower peak training memory; against the same-protocol full-batch no-prune encoder the deployed point pays a measured 2.64 pp accuracy tax for a 2.8x smaller training footprint. Pruning without reconstruction leaves the encoder at 0.378 within the same 40-epoch window (Fig. 4), and all-node serving runs 46x faster than sampling-based GraphSAGE, giving up approx. 5.4 pp of test accuracy. The framework scales to a 2.45M-node production graph at 6,285 MiB of peak training memory; extended variants (long-range context, external text features, a CAN-bus application) are reported as extensions and do not compete with the deployed method, and the 41.8%-budget M4 point, scored on a coarsened super-node graph with its own split, is reported separately rather than compared with the original-node baselines. DPR thus turns aggressive node pruning from an accuracy tax into a safe training accelerator for VRAM-limited all-node inference.
Figures & tables
Figure 1: System overview. (a) One feature-norm mask induces both sparsified graphs: the encoder graph keeps only core-internal edges, and the reconstruction graph with adds the core-to-1-hop links needed to rebuild deleted nodes. (b) DPR-S joint training: the encoder runs on the core only ( with ; a -node, -edge core on which a 3-layer GraphSAGE aggregates of full); the sparse decoder reconstructs pruned-node features within the core’s 1-hop from and , and the reconstructed features feed the head; the CE loss over training nodes jointly updates — “decoupled” refers to a division of functional roles within one pipeline (§ A.16 ). The dashed box in (a) marks the optional extension DPR-S-M4, which feeds an long-range context row to an extended decoder. (c) Streaming inference: stays GPU-resident (cached after training), and stay in host RAM, and each pruned node is scored by streaming its row with its core-1-hop neighbours through the frozen decoder and head to produce ; the encoder is never re-run on the full graph.
Figure 2: The two pipeline stages. Stage 1 (a–b): core selection by feature magnitude. (a) A fraction of nodes with the smallest Euclidean feature norm is chosen as the core (2-D sketch); the dashed circle is the deterministic threshold . (b) On a real feature distribution the histogram of separates cleanly into the core and the pruned set ; the cut is a single deterministic pass with zero learnable parameters. We retain the learnable hyperbolic variant only as an ablation (§ A.16 ). Stage 2 (c–e): double-layer sparsification, induced by the same core mask . (c) The original graph with core highlighted in blue; (d) the main graph (layer 1) on which the encoder trains, keeping the core-internal edges ( of ); (e) the reconstruction graph (layer 2) used by the decoder, which adds the core-to-1-hop edges needed to rebuild deleted nodes; it has – fewer edges than the full reconstruction graph on the products probes (§ 4.2 ). Both layers are driven by the same mask , so the second sparsification is endogenous to the decoder rather than a post-hoc cut.
Figure 3: DPR-S efficiency at a glance (ogbn-arxiv; raw node features unless stated). (a) Accuracy vs. peak training GPU memory on the same RTX 4050 card: external baselines (gray; same-card rerun protocols, Table A.6 ) cluster in the high-memory region, while DPR-S (teal) holds a comparable accuracy band at MiB — less peak memory than the strongest same-card pruning baseline DSpar at pp accuracy. (b) Analytical attention-matrix memory at arxiv scale (fp32): full-graph GPS attention ( Rampášek et al., 2022 ) ( ) needs GB (fp32 attention logits: B GB per head, four heads) and softmax over super-nodes still needs GB, while our linear attention holds it at MB ( below full attention) — the collapse that keeps coarsened training on a single commodity GPU.
| Block (where) | Question | One-line answer |
| Main result: deployed-DPR-S recovery rate on ogbn-arxiv and ogbn-products; extended variants (SBERT external features, Car-Hacking) flagged per row (§ 4.2 ) | RQ1: does reconstruction restore accuracy after aggressive pruning, across backbones and scales? | DPR-S retains (arxiv, pure double-layer) and (products) of the dense-reference accuracy; the extended SBERT external-feature variant lands within pp of the leaderboard peak while training on the same GB card (Table A.2 ). |
| Ablations & analyses (§ 4.3 ; App. A.16 ) | RQ2: which component carries the gain? | Joint encoder–decoder training and the sparse decoder carry it; selector sophistication does not (fixed mask learnable hyperbolic), and DPR-D quantifies the dense-reference ceiling. |
| Scale, stress & failure modes (App.: efficiency § A.12 ; Car-Hacking § A.15 ; PPI study) | RQ3: does it survive production scale, and when does it break? | Training at 2.45M nodes peaks at MiB (fits an 8 GB card); full-scale coarsened Car-Hacking training on one commodity GPU; the PPI study delimits backbone dependence. |
Table 1: Roadmap of the experiments: block question one-line answer. Appendix blocks are marked (App.).
| Method | Training nodes | Test acc | Peak VRAM (MiB) |
| Full-node references (rerun, same card) | |||
| GraphSAGE Hamilton et al. (2017) (full-batch, no prune) | 100% | 0.7156 (s) | 4057 |
| SIGN Rossi et al. (2020) ( precomputed features) | 100% | 0.7075 (s) | 1769 |
| GraphSAINT Zeng et al. (2020) (RW sampler) | 100% | 0.6874 (s) | 5175 |
| Cluster-GCN Chiang et al. (2019) (random partition) | 100% | 0.6462 (s) | 2902 |
| DSpar Liu et al. (2023) (degree-based edge sparsification) | 100% | 2908 | |
Table 2: Budget–accuracy comparison on ogbn-arxiv: DPR against contemporaneous full-node baselines, each rerun by us on one 6 GB RTX 4050 card, the same dataset and data split (protocol in § 4.1 ; per-row architectures in App. A.9 ). Seeds: the four full-node references marked (s) are single-run point estimates; every remaining row is a multi-seed mean, reported as mean std. Scored on the coarsened super-node graph with its own split; it documents budget scaling only and is not comparable to the original-node rows.
| Dataset / task | no-prune | no-prune +Exp | DSpar (same-card) | HSP- alone | DPR-S (final) | DPR-D (dense) | DPR-S-M4 (ext.) |
| Core protocol (GraphSAGE encoder, raw 128-dim): deployed DPR-S vs. dense reference | |||||||
| ogbn-arxiv (acc) | c | d | e | f g | h | ||
| ogbn-products 500k (acc) | — | — | — | e (core-1hop) | f g | — | |
| ogbn-products 2.45M (official split) | — | — | — | — | e | — | — |
| Extended variants (not part of the deployed method) | |||||||
| ogbn-arxiv external features (acc) | a | — | — | — | — | — | b |
Table 3: Main results: the in-framework recovery ladder on ogbn-arxiv, ogbn-products (500k, 2.45M) and Car-Hacking (protocol and definitions in § 4.1 –§ 4.2 ); the external same-card comparison is Table 2 . (s)/(3s)/(5s) = 1/3/5 seeds; — = not defined under the row’s protocol.
Figure 4: Convergence comparison on the full ogbn-arxiv graph (seed 0, 40 epochs): the full DPR pipeline (HSP+Expander: prune reconstruct) against the pruning-only baseline (HSP-alone: prune, train on the pruned graph, no reconstruction) and a no-prune companion run over the same window. Reconstruction lifts accuracy above the pruning-only baseline at both core ratios ( vs. at ; vs. at at epoch 40), so the recovery is not a -only artifact. Pruning alone stays below the no-prune companion ( ) at , but at it already tracks it rather than collapsing. All curves are still rising at epoch 40; this window shows early-training dynamics rather than a converged ceiling (the fair no-prune ceiling, , is reported in § 4.2 ).
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Method mechanism. (A) Core sparsity at : keeping nodes retains only of edges ( ; unique undirected edges with both endpoints in the core), and of the core nodes are degree-0 within the core-induced subgraph — each bar is plotted with its own denominator ( , , core-node count, respectively); the training-cost saving comes from this extreme edge reduction on the encoder side. The core-1hop reconstruction graph (cut edges between core and deleted nodes) retains of edges, so the combined structural retention is of unique undirected edges, and the per-aggregation message proxy (encoder on decoder on ) is of a full-graph aggregation (Table A.4 ). (B) The expander recovers accuracy after pruning: HSP-alone (no expander) collapses to — its classifier scores the full node set, with core nodes keeping their encoded features and pruned nodes zero-filled exactly as in training (no expander, no fallback) — while fixed-mask + Expander reaches and learned HSP + Expander , near the fair no-prune SAGE . (C) The simpler fixed-magnitude mask is no worse than the learnable hyperbolic pruner on both accuracy and edges kept, so we adopt the simpler fixed mask (observation at matched budget). Train/deploy split: the encoder runs only on the core graph ; the decoder is trained jointly on the reconstruction graph over all nodes and is then frozen. At deployment only the frozen core embedding resides in accelerator memory, while stays in host RAM; the decoder reconstructs pruned nodes from plus streamed per-node raw-feature rows (the extended variant additionally streams precomputed rows) – the encoder is not re-run on the reconstruction graph.
Figure A.2: Data Snapshots (supporting the body numbers): ogbn-products (2.45M nodes) under the official OGB sales-ranked split. ogbn-products hosts no per-node timestamp, so the sales ranking is the only ordering signal. (a) The split is a rank-contiguous partition of the ranking axis: train occupies the top-ranked/best-selling window , valid , and test the remaining long-tail — so train and test differ in sales-volume (popularity) distribution by construction. (b) Label drift follows: normalized class priors per window, with nats while — the rank-adjacent validation split matches train, and the long-tail test is a distinct label distribution (log-likelihood , ; class-prior concentration: top-5 share , class entropy bits). (c) Mechanism: the top train categories are far more prevalent among the best-selling nodes, while long-tail categories concentrate in the low-ranked test pool — i.e. the popularity/ranking difference drives the category-prior drift. This is a consequence of the official split , not a separate design choice; we therefore keep the same-graph recovery rate (DPR-D), which compares two methods on one graph and is therefore not exposed to the split-induced prior shift, as the primary metric and treat Test accuracy as a long-tail stress test.
Figure A.3: ogbn-arxiv ships a genuine per-node year; its official time split is a real chronological partition (train – , valid , test – ). (a) Publication-year distribution per split. (b) The label prior drifts with true time: the share of the top training-era subjects falls and the emerging-subject share rises across years ( nats on labels, larger than the products figure of ). This confirms on the one dataset that ships timestamps that a chronological train/test partition induces a measurable label-prior drift; products lacks timestamps, hence its drift is described as a sales-ranking (popularity) proxy rather than a literal time split.
| Configuration | DPR-S test-acc | note |
| ogbn-products, 500k class-stratified random split | (3s) | no sales-ranked popularity shift |
Table A.1: Transfer and different-protocol check (supplementary; not directly comparable to Table 3 : different split partition).
| Configuration | test-acc | note |
| Fair no-prune SAGE (full-batch, same OGB split) | fair no-prune ceiling | |
| HSP-alone cr=0.3 | 0.378 | 40-epoch endpoint, no recovery |
| DPR-D cr=0.3, full reconstruction (HSP, 5s) | dense reference, full graph | |
| DPR-D cr=0.3, core-1hop recon (final, simple mask, 5s) | pp vs same-pruner full ( ), double sparsification | |
| DPR-S cr=0.3, core-1hop recon (final, s, 5s) | 5-seed, double sparsification |
| Configuration | best test-acc |
| no-prune (+Exp full reconstruction) | 0.6334 |
| DPR-D (core + core-1hop) | 0.6119 |
| recovery rate | 96.6% (gap 2.15pp) |
| Configuration | best test-acc (mean std, 3 seeds) |
| no-prune (+Exp full reconstruction) | 0.7157 0.0035 |
| DPR-D / HSP pruner (core + full reconstruction) | 0.6874 0.0015 |
| DPR-D / simple mask pruner (core + full reconstruction) | 0.6876 0.0007 |
| DPR-S (sparse decoder, core-1hop) | 0.6635 0.0122 |
| recovery rate | 96.1% 0.4% of no-prune (gap 2.83pp; no measurable selector difference between the two DPR-D pruner rows); DPR-S = 96.5% of DPR-D |
| Role | Backbone | Decoder / scope | Dataset (split) | Seeds | GPU | Result |
| No-prune reference | GraphSAGE 3L, full-batch | — / full graph | arxiv (OGB) | — | 4050 6 GB | acc |
| Pruned-no-reconstruction floor | GraphSAGE 3L | none / only | arxiv (OGB) | 1 | 4050 6 GB | acc |
| DPR-D (dense reference) | GraphSAGE 3L | dense / full recon. | arxiv (OGB) | 5 | 4050 6 GB | acc |
| DPR-S (final method) | GraphSAGE 3L | gated-MP expander / core-1hop | arxiv (OGB) | 5 | 4050 6 GB | acc |
| DPR-S-M4 (extended variant) | GraphSAGE 3L | AMP cross-attn M4 / near-full | arxiv (OGB) | 5 | 4050 6 GB | acc |
| DPR-S-M4 (raw budget point) | GraphSAGE 3L | AMP cross-attn M4 / near-full, raw feats | arxiv coarsened ( ) | 3 | 4050 6 GB | acc † |
Table A.2: Configuration–outcome master table: every headline number in this paper bound to its exact configuration, seed count, and hardware. “core-1hop” / “near-full” denote the reconstruction scope ; “final” marks the frozen final method (DPR-S). The three DPR-S-M4 (extended-variant) rows never substitute for final-method validation (the DPR-S final-method rows). The dense 2 2 cell row is the per-node-pool core-1hop entry of the design-space summary (Table A.8 ); the full-scale final method (DPR-S, official OGB sales-ranked split) is the DPR-S (final), full scale row. Secondary numbers (PPI transfer, ablation variants, DSpar comparison) appear in their own sections with the same configuration detail.
| Mask applied | deleted | cover. | fallb. | nbrs |
| arxiv core ( ) | ||||
| prod-200k core ( ) | ||||
| prod-200k nm-var. ( super) |
Table A.3: Deletion-side coverage under the mask actually applied by each reported experiment (all masks applied to original nodes unless noted). “Covered” deleted node with 1 core neighbor (rebuilt by gated message passing from its core neighbours over ); “fallback” no core neighbor (keeps its projected own feature — a direct consequence of graph connectivity, Algorithm 1 ); last column: mean/median core-neighbors among deleted nodes (unique-undirected counting).
| Quantity | arxiv | prod-200k |
| unique undirected | ||
| core–core | ( ) | ( ) |
| core-1hop cut | ( ) | ( ) |
| drop–drop (unused) | ( ) | ( ) |
| combined | ( ) | ( ) |
| msgs / aggregation | ( ) | ( ) |
Table A.4: Edge-count audit under the exact main-experiment masks (fixed-magnitude low-norm core, , original graph). Structural rows: unique undirected edges; the core–deleted cut is written because the reconstruction set the decoder gathers over is ( ). “msgs / aggregation”: a per-aggregation message proxy — directed messages of one encoder aggregation pass (symmetrized ) plus one decoder gather over the core–deleted cut ( ), divided by the directed message count of one full-graph aggregation; it is a structure-cost proxy, not a layer-weighted full-epoch traffic count ( with layer counts would give the latter).
| Reconstruction path | share | full | no-M4 | no- |
| core kept (encoder path) | ||||
| covered (core-1hop cross-attn) | ||||
| fallback (per-node feat. + M4) | ||||
| all deleted (covered+fallback) | ||||
| overall (test) |
Table A.5: Reconstruction-path stratified accuracy on ogbn-arxiv under the extended variant DPR-S-M4 ( main mask, test split, nodes, seed 0) with eval-time fallback-input ablations. The fallback path is plus one gated core-neighbour message step (extended-variant mechanism, Algorithm 2 ); “no-M4” / “no- ”: the corresponding streamed feature input is zeroed at eval time (weights unchanged, message term retained). Protocol note: the seed-0 checkpoint here belongs to the matched-capacity control protocol of Table A.9 (same recipe, near-full operating point): its eval-time overall score is (train-time model-selection score ), consistent with that table’s 5-seed mean (seed 2 reaches ). The core-1hop operating point of the main results is a different reconstruction radius with its own 5-seed mean ( ); path shares and stratified accuracies here are computed under this checkpoint’s actual paths.
| Method | Backbone | Training graph | test-acc | Peak (MiB) | ms/epoch |
| Scalable GNN references (backbone) | |||||
| SIGN (Frasca et al., 2020) | MLP on , undir. | full (precomputed) | 0.7075 | 1769 | 72.5 |
| GraphSAGE (full-batch) | 3-layer SAGE | full graph | 0.7156 | 4057 | 739.3 |
| GraphSAINT (Zeng et al., 2020) | 3-layer GCN | RW node sampler | 0.6874 | 5175 | 1359 |
| Cluster-GCN (Chiang et al., 2019) | 3-layer SAGE | random partition | 0.6462 | 2902 | 883 |
| Ours (RTX 4050 same card; matched-pipeline VRAM; ms/epoch omitted — DPR full-batch core, not sampler-comparable) | |||||
Table A.6: External baseline comparison on ogbn-arxiv (single NVIDIA RTX 4050, 6 GB card). All external-baseline rows are single-seed point estimates on this 6 GB card (seed 42, selected as the highest-validation-accuracy run); the DPR rows are 5-seed averages. The single-seed treatment reflects Table A.6 ’s role: an order-of-magnitude accuracy and peak-VRAM reference against the closest external comparators. The controlled multi-seed numbers for our method live in § 4.2 . The plotted frontier summarizes the reported operating points ( pp over the strongest rerun baseline for the extended variant DPR-S-M4; the deployed DPR-S holds pp with a separately profiled peak VRAM below DSpar’s MiB); a broader multi-seed evaluation of the external baselines remains future work. Peak VRAM is a single-run profiling figure, not a seed-averaged quantity, and is reported only where separately logged on this card (— otherwise: the dense reference and the near-full extended variant were not separately profiled here; the MiB entry is the final-method deployment profile quoted in the abstract). Controlled same-protocol numbers are in § 4.2 .
Figure A.4: Budget–accuracy plane on ogbn-arxiv (same RTX 4050 card; the baseline numbers and per-epoch wall-clock price tags are the same-card reruns of Table A.6 ). (a) External baselines (gray; hatched: DSpar) all train on – of the node budget, whereas our points (colored) operate on pruned cores: DPR-S at and the extended DPR-S-M4 at of nodes. The shaded region marks the budgets where our external-feature operating point ( ; leaderboard reference , a different, stronger feature protocol) has a node budget at least as large as every rerun baseline and an accuracy at least as high. The raw-feature point (open marker) is scored on the coarsened super-node graph with its own split and is shown for budget scaling only. (b) Within-framework budget–recovery operating points (seed std as error bars); the extended point trains on of the nodes but is scored on the coarsened super-node graph, so it is not read against the original-node reference .
| Method | Sparsity lever | Recovery | Test acc | Train speedup | Inference | Hardware / Protocol |
| Paper-reported only (proprietary / different-task regimes) | ||||||
| DSpar (Liu et al., 2023) (A40 paper) | Edges only (all retained; ) | None | (GraphSAGE full) | – avg / max | products % ms (arxiv not shown) | A40 48 GB; -sampling; 10 seeds; full node set |
| Coarformer (proprietary) | Nodes (cluster-pool coarsen) | Cross-granularity attn | — (arxiv not reported) | — | — | Not reproducible (internal baseline) |
| FIT-GNN (coarsening + induced subgraphs) | Coarsening-derived induced subgraphs + Extra/Cluster Nodes | GNN on induced subgraphs | — (arxiv not reported) | — | — | Public code; no ready ogbn-arxiv configuration in its benchmark suite (not same-protocol; see caption) |
| Same-card reruns (RTX 4050 6 GB; GraphSAGE , 200 ep, ogbn-arxiv; DSpar 3 seeds, DPR rows 5 seeds) | ||||||
| DSpar (this work, RTX 4050) | Edges only (all retained; , undir removed) | None | † | Tracks full-SAGE baseline | RTX 4050 6 GB ; same GraphSAGE encoder as DPR; peak VRAM MiB | |
Table A.7: External prune/sparsify/coarsen baselines on ogbn-arxiv. Same-card rerun regime (RTX 4050 6 GB): DSpar (Liu et al., 2023) is now rerun under the exact same 3-layer GraphSAGE hidden-128 encoder protocol used for DPR-D/DPR-S ( undirected-edge removal, matching the paper’s Table 2 value for arxiv). DPR-S-M4 is therefore pp above DSpar at same hardware, backbone, and training budget, but different reduction regimes (node-level coarsening + reconstruction vs. edge-only sparsification on the full node set). Mechanism regime disclaimer (critical for fair reading): DSpar never drops any node ( of always active) and therefore has no pruned-node recovery problem and no decoder; its per-epoch latency is low because it performs only the forward pass over the full-node graph , but it can never train a graph that is too large to instantiate on the card at the full-node level (the bottleneck DPR solves). DPR trains on a 30% core and then reconstructs the remaining 70% at inference via structure-aware diffusion — which is exactly the regime in which node-level OOM is the primary blocker, not SpMM speed. Coarformer is not reproducible (proprietary internal baseline). FIT-GNN partitions the graph into coarsening-derived induced subgraphs with Extra/Cluster Nodes; its public code is available but ships no ready ogbn-arxiv configuration, so it is not a graph-pruning comparison under the same protocol and is listed only for transparency. GraphSAINT / Cluster-GCN / SIGN / full GraphSAGE are already rerun on the exact same card in Table A.6 (full-node methods, no reconstruction).
Figure A.5: Same-card Pareto comparison; every point is a measured number from this work’s runs and logs (no extrapolation); panels are ordered by service target, and every point is identified by name in the legend beneath its panel (the plots themselves carry no labels). (a) Full M ogbn-products, original-node evaluation: DPR-D (dense reconstruction) peaks at MiB and the no-prune GraphSAGE baseline at MiB — within a GB budget no full-scale original-node baseline runs at all (solid and dashed vertical lines: GB and GB card capacities; red wash: beyond the GB card; dashed step: two-point Pareto frontier). (b) the same products scale at super-node granularity, shown in its own facet because the accuracy axis is not comparable with (a): DPR-S-M4 trains on the coarsened super-node set ( super-nodes) at MiB peak (3 seeds, batch 1024, ; wall s per seed) — leaner than DPR-D, fitting an 8 GB card. (c) ogbn-arxiv, test accuracy vs. peak training VRAM (log scale); the shaded band marks VRAM beyond the GB card edge and the solid step is the Pareto frontier of the plotted points (non-dominated region above–left). Within the same-card rerun regime (GraphSAGE encoder, filled squares), DPR-S dominates the DSpar rerun (the final core-1hop DPR-S profiles at MiB against DSpar’s MiB, i.e. lower peak VRAM; the pp DPR-S-M4 advantage over DSpar is a different configuration, reported in the text and not plotted on this axis), as do GraphSAINT and Cluster-GCN — while seeing only of the nodes (DSpar and the external references operate on the full node set). The matched dense decoder control (same family and width, 3 seeds) uses MiB for : the sparse decoder is leaner and pp more accurate at matched width and family (§ 4.2 ). DPR-D (h256) arxiv peak VRAM was not separately logged and is therefore not plotted; its encoder scale is represented by the GraphSAGE full-batch point.
| cr | edges kept | HSP-alone | DPR-D (dense) | (Exp alone) |
| 0.9 | 0.686 | 0.6123 0.0006 | 0.7955 0.0051 | +0.1832 |
| 0.7 | 0.603 | 0.5991 0.0014 | 0.7953 0.0052 | +0.1962 |
| 0.5 | 0.367 | 0.5609 0.0015 | 0.7935 0.0061 | +0.2326 |
| 0.3 | 0.176 | 0.5047 0.0034 | 0.7923 0.0034 | +0.2875 |
| (no-prune) | 1.0 | — | 0.6613 0.0010 | — |
Figure A.6: Sensitivity to the core ratio on PPI (node classification (multi-label), 5 seeds). (a) Once reconstruction is added, accuracy is approximately flat in , whereas the no-reconstruction baseline degrades monotonically as the pruning becomes more aggressive. (b) The gain from reconstruction grows from at to at – the harder the pruning, the larger the recovery. The dashed line is the fraction of edges retained after pruning, showing the reconstruction simultaneously improves accuracy and reduces graph size.
Figure A.7: The pruner is pluggable: on PPI (node classification (multi-label), 5 seeds, matched edge budget) the learnable hyperbolic pruner (HSP) tracks random pruning of the same budget and falls far below the no-prune reference ( ), i.e. the pruning geometry carries no recoverable signal by itself – confirming the gain is primarily attributable to the reconstruction stage. This panel is a separate matched-edge-budget control from the node-budget -sweep table above; the curves are that control’s own -seed run (its no-prune mean is ), while the dashed level marks the manuscript’s PPI no-prune value .
| Configuration | Test Acc | 50-epoch wall-clock |
| Dense (control) | 0.6649 | h (extrapolated) |
| Sparse bipartite attention (default) | 0.6649 | min (measured) |
| Configuration (decoder variant / key pool) | Scope | Test Acc |
| Sparse AMP+M4 (extended DPR-S-M4) | near-full | (5) |
| Sparse gated-MP (final DPR-S) | core-1hop | (5) |
| Sparse AMP (matched control) | core-1hop | (3) |
| Dense (per-node pool) | core-1hop | (3) |
| Dense ( AMP, matched control) | core-1hop | (3) |
| Dense (union pool) | near-full | not measured |
Table A.8: Decoder-family reconstruction-scope design-space summary (ogbn-arxiv, , GraphSAGE unless noted; seed count in parentheses). This is a design-space listing, not a controlled experiment : the dense union-pool (near-full) cell was not measured; the near-full and core-1hop sparse rows use different decoder variants (extended AMP+M4 vs. final gated-MP); the two core-1hop matched control rows (same AMP family, 3 seeds) are the same-width dense-vs-sparse calibration analyzed in § A.16 . No interaction or accuracy-equivalence claim is made.
Figure A.8: Empirical motivation: the headline recovery rate decomposed along its two design axes (ogbn-arxiv, 5 seeds except where noted). The panel separates decoder type (sparse DPR-S family vs. dense reconstruction reference DPR-D) from reconstruction scope (near-full vs. core-1hop), which a single recovery percentage would otherwise conflate: sparse near-full (DPR-S-M4) , of the dense reference; sparse core-1hop (the final method, DPR-S) , ; dense core-1hop , ; dense near-full is the reference (the dotted vertical line marks this dense-reference bound). The DPR-D rows use and the DPR-S rows , so this panel is a design-space map, not a width-matched comparison; the same-width control is the matched dense-vs-sparse comparison of § 4.2 ( : sparse vs. dense ). Masks follow the run logs: the final DPR-S row and the dense core-1hop control use the fixed-magnitude (simple) mask, while the dense near-full reference uses the HSP mask (Table 3 ), so the headline is a system-level retention (§ 4.2 ). The pruned-no-reconstruction floor ( ) lies far below the plotted range and is reported in Table 3 .
Figure A.9: Optimization sensitivity. (a) The decoder is the learning-rate-sensitive component of the method: at lr 0.01 training diverges to a degenerate flat solution ( ), while the recommended lr 0.001 reaches . (b) The three decoder mechanisms (P1 core-confidence residual fusion, P2 structure-aware reconstruction of deleted nodes, and a small learnable modulation term) contribute , and percentage points respectively on ogbn-arxiv (single seed), with all three together reaching (single seed, comparable to the 5-seed DPR-S-M4 ; the ablation is run within the extended DPR-S-M4 decoder family).
| Decoder | Params | Test Acc (5 seeds) |
| AMP (cross-attn + MP) | ||
| MLP-only (no edges, no attn) | ||
| Delta (matched) | — |
Table A.9: Decoder capacity control (ogbn-arxiv main protocol, five paired seeds): the AMP decoder vs. a parameter-matched MLP-only decoder. Both variants live inside the extended DPR-S-M4 family — same core mask, same encoder, and the same graph-derived M4 long-range features — so the comparison isolates the decoder’s structure use, not the final method’s pruning geometry.
Figure A.10: End-to-end efficiency, grouped by measurement type (each panel states its own dataset / hardware / baseline; the three speed-up panels use different denominators and are deliberately separated so no shared y-axis is implied). (a) Peak training memory (ogbn-products; full M graph on a 16 GB cloud GPU, 200k subgraph on RTX 4050): no-prune reaches MiB (near a 24 GB card limit), while the dense DPR-D variant drops the peak to MiB ( leaner); the 200k subgraph improvement is . (b1) End-to-end / per-step training speed-up (ogbn-arxiv, RTX 4050, DPR-S core-1hop vs. its own dense reconstruction): (with inference speed-up in the same regime); per-step training (products 500k, cloud GPU, App. A.6 ). (b2) Reconstruction inference speed-up : decoder step (200k probe, RTX 4050). (b3) Component-level speed-up (analytical / probe, not directly comparable to (b1)/(b2)): node-merge (lean configuration; measured against the same-hardware single-batch linear SAGE reference of ms, § A.17 ); linear attention measured (up to vs. an extrapolated dense baseline). (c) Second sparsification layer + controlled framework : the reconstruction graph has core-1hop edges vs. for full reconstruction (ogbn-products, 200k subgraph); both edge counts are attention-probe reconstruction topologies (dense key pool vs. bipartite ), a different denominator from the original-graph structural audit in Table A.4 . Scope : every value here is a training-feasibility or internal decoder/attention efficiency measurement on the coarsened pipeline; all-node scoring latency is measured separately in Table A.10 .
| System | Test acc. | Latency (s) | nodes/s | VRAM (MiB) | RSS (MiB) |
| GraphSAGE full-batch (no sampler) | 0.7094 | ||||
| GraphSAGE + NeighborLoader ( ) | 0.7097 | ||||
| DPR-S, training-time protocol (ref.) | 137.3 | ||||
| DPR-S serving (Alg. 1 ) |
Table A.10: All-node inference benchmark on ogbn-arxiv ( ; RTX 4050 laptop, GB; PyTorch 2.7.1+cu118). Per-pass steady-state timing after warm-up; is the per-pass sd over 5 repeated passes (the training-time reference protocol uses a single repeat, hence no ). DPR-S rows use the same core-1hop checkpoints as § 4.2 (5 seeds; single run for the SAGE baselines). for the serving row is M directed edges ( of undirected ) with a s one-time setup; its accuracy includes a one-time BatchNorm re-estimation on the serving core graph (without it: ). The training-time protocol row re-selects the core and re-encodes per batch (accuracy parity with the checkpoints by construction) and is shown to quantify the serving-design cost, not as a deployable system.
More in this topic
Graph Neural Networks
Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.
See all 726 papers