Generalizing from small graphs to larger ones can require computation beyond the depth of a neural model used during training. We study target tasks that admit a common local iterative computation and whose effective dependency radius grows with graph size. The central idea is to learn a local update rule through GNN pretraining, then repeatedly apply the same learned processor beyond the training depth to extend information reach on larger graphs without retraining. Calling such parameter-shared architecture a looped GNN, we categorize six schemes for looped GNNs to separate the effects of depth extension, fine-tuning, and freeze-and-steer adaptation. Our analysis establishes fixed-depth information limits and sufficient conditions under which accurate iterative reuse overcomes them. To improve beyond unadapted reuse, we introduce Loop-Freeze-Steer (LFS), which freezes the pretrained processor and learns a small controller to steer its repeated application, using much fewer trainable parameters than full fine-tuning. Our finite-sample analysis gives sufficient conditions for adaptation to improve performance on larger graphs. Experiments on synthetic tasks and chip placement evaluate depth extension and adaptation, highlighting LFS's accuracy--trainable-parameter trade-off for size generalization.
Figures & tables
Figure 1: (a) Six schemes by depth and adaptation. (b) LFS steers each frozen processor application with a shared controller; the readout acts after K steps. Blue blocks are trainable.
Figure 2: Risk-minimizing deployment depth as graph size increases. Operator-aligned pretraining tracks the prescribed task horizon T⋆(N) , whereas standard target supervision remains near the pretraining depth. Shading denotes variability across seeds; dotted line: the largest pretraining depth.
Figure 3: Risk ratio of LFS to Loop-Tune across contraction rates ρ and deployment-to-adaptation size ratios, under target-supervised pretraining. Values <1 indicate that LFS is more accurate. Results are averaged over five seeds.
synthetic netlists, cells per design
real
scheme
params
0.4–0.6k
0.8–1k
1.8–2k
2.4–2.8k
MLCAD
Fixed
0
1.51
3.00
6.17
8.50
7.18
Fixed-Tune
5.85M
1.48 ± .00
2.81 ± .00
5.59 ± .00
7.54 ± .01
6.78 ± .00
Loop-Tune
5.85M
1.11 ± .01
1.86 ± .00
3.69 ± .02
5.30 ± .04
6.01 ± .05
LFS
0.17M
1.27 ± .01
2.06 ± .01
3.95 ± .02
5.61 ± .01
6.24 ± .02
Table 1: Chip placement, physical RMSE (mean ± sd over 3 adaptation runs; lower is better; best per column in bold), with trainable parameters. Looped schemes deployed at K=2 ; ‘params’ counts trained parameters; rightmost column: real MLCAD netlists.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: The four task families and their fixed-radius error. Computed from the task definition, before any learning, with RGG d=2 , ρ=0.7 . Left: the prescribed horizon T⋆(N) of each family. Right: the per-node error of the K0 -step truncation, which uses only a K0 -hop neighborhood. For const this error is exactly zero at every size. For the other families it grows with the graph size. The truncation error upper-bounds the fixed-radius floor of Proposition 1 .
Figure 5: Training-free checks of the two main assumptions , on damped PageRank (Appendix H.4 ). (a) The error of the best radius- k prediction decreases geometrically in k ; the guaranteed rate ρ2k (dashed) is an upper bound, and the measured rate ρeff is somewhat faster. (b) The radius needed to reach a fixed accuracy grows linearly in logN ; using the measured rate ρeff predicts the growth rate within about 10% , whereas using the nominal ρ overestimates it by up to 10× on expander graphs. (c) The distance between local neighborhood distributions at size N and at a large reference size decays as N−1/d : the measured exponent is −0.510 against the predicted −0.500 at d=2 (16 seeds); at d=3 the measurement ( −0.52 ) is consistent with the bound but the graphs are too small to reach the asymptotic rate.
Figure 6: Test risk of the four schemes across deployment sizes , one panel per task family (target-supervised pretraining, RGG d=2 , ρ=0.7 ; 5 seeds, ±1 sd; each scheme selects its deployment depth on a held-out split at every size). On the F1 family all four schemes perform alike and no scheme degrades with size. On the other families, the risk of the two unadapted schemes grows with the graph size while the two adapted schemes are 2.6 – 10× more accurate, and LFS gradually overtakes Loop-Tune as the size grows, most clearly on diam .
Figure 7: Risk ratio of LFS to Loop-Tune as a function of deployment size (target-supervised pretraining, RGG d=2 ; 5 seeds, ±1 sd; adaptation on 100 graphs of 350 – 700 nodes). Values below 1 mean LFS is more accurate. Near the adaptation sizes Loop-Tune leads; the curves cross for poly and diam between 2 and 20 times the adaptation size at ρ≥0.7 , log stays within a few percent of parity, and where a small neighborhood suffices ( ρ=0.5 , and const at every ρ ) Loop-Tune leads at every size.
scheme
depth / adapted params
const
log
poly
diam
Fixed
fixed / none
0.10
2.00
3.31
3.71
Fixed-Tune
fixed / all
0.09
0.77
1.36
0.38
Fixed-Steer
fixed / adapter + readout
0.07
0.74
1.06
0.25
Loop
tunable / none
0.18
1.94
3.26
4.01
Loop-Tune
tunable / all
0.14
0.77
1.28
0.39
LFS
tunable / controller + readout
0.15
0.75
1.25
0.36
Appendix
Table 2: All six schemes of Sec. 4 on the constructed families (RGG d=2 , ρ=0.7 ; per-node MSE ×10−3 , mean over 5 seeds, sd ≤0.3 in these units for every entry; deployment at N=2×104 , i.e. 40× the adaptation sizes). Best per column in bold.
scheme
depth / adapted params
log
diam
Fixed
fixed / none
5.92±0.11
47.0±0.4
Fixed-Tune
fixed / all
4.98±0.58
38.3±3.4
Fixed-Steer
fixed / adapter + readout
3.64±0.86
35.8±1.6
Loop
tunable / none
2.37±0.57
14.4±7.9
Loop-Tune
tunable / all
1.04±0.17
3.26±1.17
LFS
tunable / controller + readout
1.12±0.19
3.99±1.57
Appendix
Table 3: The six schemes at slow contraction under operator-aligned pretraining (RGG d=2 , ρ=0.9 ; per-node MSE ×10−3 , mean ± sd over 5 seeds; deployment at N=2×104 ). Best per column in bold.
Figure 8: Damped PageRank ( ρ=0.95 , RGG d=2 ; 5 seeds). (a) The two adapted schemes are 2.8× more accurate than the unadapted ones at N=103 , growing to about 4× at N=2×104 . (b) With the contractive update described in the text, the risk-minimizing deployment depth is ordered by the contraction rate at every size (about 27/20/13/8 for measured rates 0.92/0.80/0.63/0.42 ) and grows with logN wherever the benefit of resolving more of the target exceeds the estimation error.
Figure 9: Loop-Tune and LFS as the size gap (left) and the adaptation budget (right) vary (damped PageRank, 3 seeds). Left: Loop-Tune on a single adaptation size and on a spread of sizes perform alike at every gap up to 40× : on these homogeneous graphs, single-size fine-tuning does not damage cross-size behavior (cf. Figure 10 ). Right: the comparison crosses with the budget: LFS is more accurate with 30 or fewer adaptation graphs (risk ratios 0.56 – 0.90 ), and Loop-Tune leads once the budget reaches 100 graphs.
Figure 10: How far the learned update deviates from the task update , measured as ∥D(Rθ(h))−UN(D(h))∥2 along the model’s own trajectory (damped PageRank, ρ=0.95 , 3 seeds). Left: change in this deviation caused by adaptation, relative to the frozen pretrained processor, matched per seed and size. Right: the deviation itself for all schemes. The frozen processor (Loop) is the best-aligned reference; fine-tuning increases the deviation uniformly across sizes; the layers of the fixed-depth model (Fixed), measured one by one, sit 5 – 10× higher at every size: trained without weight sharing, they never learn to approximate UN .
Figure 11: Deliberately biased adaptation data (adapt on degree-4 graphs at N≈500 , evaluate on degree-8 graphs; target-supervised pretraining, ρ=0.7 ). Left: change in the learned update’s deviation from the task update, relative to the frozen processor ( diam , 3 seeds): matched adaptation data improve the agreement, while biased data push Loop-Tune away from the task update while LFS stays closer to its pretrained behavior. Right: risk ratio of LFS to Loop-Tune across deployment sizes for log and diam (5 seeds; dashed: matched adaptation data, solid: biased): the bias reverses the comparison, by up to a factor of 2.4 .
Figure 12: Outside the contractive class. (a) For shortest-path distances, the error of the best radius- k prediction does not decay geometrically; it drops only once k reaches the graph diameter, of order N , a property of the task itself. (b) Trained on this task, every scheme degrades severely with the graph size and the differences between schemes collapse: the failure is not specific to LFS, and no ordering among the schemes survives.
Looped transformers reuse a single block across iterations, raising a natural question: how far can a block trained at one depth be safely extrapolated to greater depths? We study this question for all-pairs graph connectivity, where harder graphs require longer-range information propagation. We show that under exchangeability and symmetric positive initialization, population training at depth K converges to an update rule governed by a single effective parameter that balances graph propagation against fitting structure unresolved at training depth. This trade-off creates a finite extrapolation window: too few test-time loops provide insufficient propagation, while too many eventually create false connections. We derive explicit endpoints for this window and bound the graph hardness solvable beyond training as a function of training depth and hardness. In particular, a test depth chosen solely from the training distribution can guarantee recovery on substantially harder graphs, with a certified depth gain that grows exponentially as training depth outpaces training hardness. Experiments validate the predicted windows and show how compute budget and initialization affect the optimal training depth.
Huitao Yang, Jianhao Huang, Zixuan Wang +1
University of California, Los Angeles · Princeton University
Graph neural networks (GNNs) can operate on large graphs but become infrastructure-sensitive at the scale of millions of nodes and typically require scalable training techniques for even larger graphs. This raises a central question: when can a model trained on a smaller, scaled-down replica of a graph be deployed on the full-resolution graph without retraining? We introduce a zero-shot transfer protocol in which a GNN is trained on a graph coarse-grained by geometric renormalization (GR), and the resulting weights are transferred directly to the original network. Across synthetic and real-world networks, training on GR scaled-down replicas preserves much of the original-scale predictive performance while significantly reducing training cost. We further find that learned representations and predictive trajectories remain aligned across scales. These findings suggest that structural similarity may be more important than network size in determining GNN transferability, opening a path toward scale-equivariant graph architectures.
Robert Jankowski, Pedro Almagro-Blanco, Marián Boguñá +2
TU Delft · University of Seville · University of Barcelona +2
The performance of deep learning models crucially depends on the settings of hyperparameters like learning rate, initialization scale, and weight decay. Hyperparameter transfer aims to make near-optimal hyperparameter settings consistent across model scale, so that large models can be optimized by proxy tuning their smaller, cheaper-to-optimize counterparts. While transfer principles are well-studied in the context of dense neural networks in language and vision tasks, they remain comparatively under-explored for graph neural networks (GNNs). We develop and validate a transfer parameterization for GNNs trained with SGD, Adam, and AdamW. Through theoretical scaling analyses and controlled experiments, we show that the proposed parameterization yields stable feature updates, learning rate transfer, and improved performance as width and depth increase. For SGD, we identify graph-dependent first-layer correction factors and show that their use can accelerate early training in graphs with sparse bag-of-words inputs. For Adam, we explore how different message passing normalizations affect early- and late-training transfer behavior, illustrating the importance of message passing normalization and advocating for an associated hyperparameter. For AdamW, we adapt a parameterization that allows for the joint transfer of weight decay and learning rate. Together, these results provide a practical recipe for scaling GNNs across a variety of learning tasks and training scenarios.
Gage DeZoort, Boris Hanin
Department of Operations Research and Financial Engineering · School of Engineering and Applied Science · Princeton University +1