GSR-GNN: Training Acceleration and Memory-Saving Framework of Deep GNNs on Circuit Graph

Yuebo Luo; Shiyang Li; Yifei Feng; Vishal Kancharla; Shaoyi Huang; Caiwen Ding

cs.LGMar 28, 2026

GSR-GNN: Training Acceleration and Memory-Saving Framework of Deep GNNs on Circuit Graph

Authors: Yuebo Luo, Shiyang Li, Yifei Feng, Vishal Kancharla, Shaoyi Huang, Caiwen Ding

Organizations: University of Minnesota, Twin Cities Minneapolis, Minnesota, USA · New York University Brooklyn, New York, USA · Stevens Institute of Technology Hoboken, New Jersy, USA

Abstract

Graph Neural Networks (GNNs) show strong promise for circuit analysis, but scaling to modern large-scale circuit graphs is limited by GPU memory and training cost, especially for deep models. We revisit deep GNNs for circuit graphs and show that, when trainable, they significantly outperform shallow architectures, motivating an efficient, domain-specific training framework. We propose Grouped-Sparse-Reversible GNN (GSR-GNN), which enables training GNNs with up to hundreds of layers while reducing both compute and memory overhead. GSR-GNN integrates reversible residual modules with a group-wise sparse nonlinear operator that compresses node embeddings without sacrificing task-relevant information, and employs an optimized execution pipeline to eliminate fragmented activation storage and reduce data movement. On sampled circuit graphs, GSR-GNN achieves up to 87.2% peak memory reduction and over 30 $\times$ training speedup with negligible degradation in correlation-based quality metrics, making deep GNNs practical for large-scale EDA workloads.

Figures & tables

Figure 1. Performance Comparison: RevGNN vs. SAGE vs. GCN on Circuit Graphs 1 and 3 (See Table 1 ), where RevGNN outperforms SAGE and GCN and keeps gaining return when increasing depth. Negative scores in Spearman, Kendall, and

R^{2}

mean failure to find correlations.

Figure 2. Training results of baseline RevGNN on example graph 356-R in Table 1 , where RevGNN pleateaus around 70 layers, where SAGE and GCN are either too low to stay in the range or crashed because of out-of-memory (OOM) error.

Figure 3. GPU memory usage and epoch runtime summary of 100-layer RevGNN with 384 hidden channels on Graph 356-RISCY-a1-c5 in Table 1 .

Figure 4. Example training time of baseline 100-layer RevGNN on graph 7 in Table 1 . Note that the red ’X’ refers to an out-of-memory (OOM) error.

Figure 5. Comparison of Conventional graph dataset (upper) and circuit graph 4 (lower), Table 1 in 50-node graph component pattern and node degree distribution. The circuit graph shows an apparently different connecting pattern and node degree distribution from the conventional graph.

Figure 6. GSR-GNN in-layer forward workflow.

Figure 7. GSR-GNN in-layer backward workflow.

Size	#	Name	Node	Edge	Train/Val/Test	Labels	Range
L	1	5324-F	69.8k	22.5M	48.8/10.5/10.5k	131	[0.0, 1.0]
L	2	4474-F	65.3k	21.8M	52.3/6.5/6.5k	255	[0.0, 1.04]
M	3	876-R	49.6k	12.2M	39.7/5.0/5.0k	60	[0.0, 0.22]
	4	1522-R	49.5k	12.2M	39.6/5.0/5.0k	106	[0.0, 0.36]
	5	1886-R	49.1k	11.7M	39.3/4.9/4.9k	134	[0.0, 0.58]
	6	639-R	49.1k	11.7M	34.3/7.4/7.4k	105	[0.0, 0.48]

Table 1. Circuit Graph Dataset Summary (Ordered by Size)

Figure 8. Correlation performance of GSR-GNN over GCN/SAGE/RevGNN in selected graphs (Graph 1 and Graph 3 in Table 1 ). The results are averaged over hidden dimension=256 and 384. The higher correlation scores mean better performance, non-reversible baselines (SAGE, GCN) are out-of-range due to very low scores or out-of-memory errors.

Figure 9. Speedup of GSR-GNN over Reversible GNNs at varying sparsity levels defined by

K

, the greater the

K

, the more elements preserved per row of activations and lower sparsity.

Method	Forward $\downarrow$	Backward $\downarrow$	cudaMemCpy $\downarrow$	Total/Spdup $\uparrow$
Baseline	5.84s (20.8%)	20.09s (71.6%)	2.13s (7.6%)	28.06s
Ours (K=4)	0.36s (23.7%)	1.16s (76.2%)	0.001s (0.1%)	1.52s (18.4×)
Ours (K=8)	0.37s (24.2%)	1.17s (75.7%)	0.001s (0.1%)	1.54s (18.2×)
Ours (K=16)	0.40s (24.1%)	1.25s (75.9%)	0.001s (0.1%)	1.64s (17.1×)
Ours (K=32)	0.50s (24.1%)	1.59s (75.9%)	0.001s (0.1%)	2.09s (13.4×)
Ours (K=64)	0.76s (24.7%)	2.31s (75.2%)	0.001s (0.1%)	3.07s (9.1×)

Table 2. Runtime Distribution: 100-layer, 384-hidden channels Baseline (RevGNN) vs Ours (Graph 3 in Table 1 )

Panel of Figure 10 — Figure 10. Memory efficiency comparison between RevGNN and GSN-GNN on 2 sample graphs, graph 1 and graph 3 in Table 1 . The memory efficiency, expressed as a percentage, is the ratio of Allocated GPU Memory to Reserved GPU Memory during the training phase of models.

Explore similar work

Graph Computation Meets Circuit Algebra: A Task-Aligned Analysis of Graph Neural Networks for Electronic Design Automation
May 8, 2026Hyunmog KimGraph Neural Networks Graph Representation Learning
Independent Researcher.
SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction
Aug 6, 2026Guofan Yu, Sitian Chen, Zhenheng Tang +2Graph Neural Networks Communication-Efficient Distributed Training
Hong Kong Baptist University · Hong Kong University of Science and Technology · Hong Kong University of Science and Technology (Guangzhou)
On Efficient Scaling of GNNs via IO-Aware Layers Implementations
May 29, 2026Daria Fomina, Daniil Krasylnikov, Alexey Boykov +3Graph Attention Networks GPU Kernel Optimization
1Yandex · 2HSE University · 3ITMO University

May 8, 2026J/K move · Enter open · S save

Graph Computation Meets Circuit Algebra: A Task-Aligned Analysis of Graph Neural Networks for Electronic Design Automation

Hyunmog Kim

Independent Researcher.

Graph Neural Networks Graph Representation Learning Electronic Design Automation

Open paper AskPDF arXiv

EDA problems are graph-structured, but not all graph-structured problems call for the same GNN computation. We argue that successful GNN-for-EDA methods are those whose propagation, aggregation, and supervision align with the native algebra of the target task. Concretely: static timing analysis is a max-plus/min-plus recurrence on a topologically ordered DAG, structurally aligned with asynchronous DAG-GNNs; placement is governed by hypergraph wirelength and density penalties and is exploited by differentiable placers rather than by message-passing GNNs alone; routing congestion is a sparse demand-supply field over a layout grid; switching-activity propagation is a probabilistic recurrence on a directed netlist; IR drop is a linear system on the power-delivery network; and analog symmetry extraction is a discrete constraint-prediction problem on schematic graphs. Through these task-by-task alignments we (i) review the GNN architectural toolkit relevant to circuits, (ii) formalize how circuit graphs differ from generic graphs (directed, heterogeneous, multi-scale, with sequential and clock structure), (iii) characterize where current methods succeed and where the algebra-architecture mismatch limits them, and (iv) identify failure modes--stage leakage, proxy-to-signoff gap, calibration, and design-distribution shift--that we believe are likely to dominate the next phase of work. We position the paper as a GNN-for-EDA, task-aligned analysis rather than a comprehensive AI-for-chip-design survey. Continuous SE(3)-equivariant geometric GNNs are usually mismatched to Manhattan digital layout, and LLM-for-RTL, HLS, and RL/diffusion-based topology generation are outside our scope.

Abstract

Explore similar work

Graph Computation Meets Circuit Algebra: A Task-Aligned Analysis of Graph Neural Networks for Electronic Design Automation

SNI-GNN: SmartNIC-Assisted Full-Graph GNN Training with In-Network Embedding Prediction

On Efficient Scaling of GNNs via IO-Aware Layers Implementations