Looped GNNs Beyond Fixed Depth: Graph Size Generalization and Parameter-Efficient Adaptation

Zhiyang Wang; Zhishang Luo; Yusu Wang

cs.LGOct 16, 2026

Looped GNNs Beyond Fixed Depth: Graph Size Generalization and Parameter-Efficient Adaptation

Authors: Zhiyang Wang, Zhishang Luo, Yusu Wang

Organizations: Washington University in St. Louis, St. Louis, MO, USA · University of California, San Diego, La Jolla, CA, USA

Abstract

Generalizing from small graphs to larger ones can require computation beyond the depth of a neural model used during training. We study target tasks that admit a common local iterative computation and whose effective dependency radius grows with graph size. The central idea is to learn a local update rule through GNN pretraining, then repeatedly apply the same learned processor beyond the training depth to extend information reach on larger graphs without retraining. Calling such parameter-shared architecture a looped GNN, we categorize six schemes for looped GNNs to separate the effects of depth extension, fine-tuning, and freeze-and-steer adaptation. Our analysis establishes fixed-depth information limits and sufficient conditions under which accurate iterative reuse overcomes them. To improve beyond unadapted reuse, we introduce Loop-Freeze-Steer (LFS), which freezes the pretrained processor and learns a small controller to steer its repeated application, using much fewer trainable parameters than full fine-tuning. Our finite-sample analysis gives sufficient conditions for adaptation to improve performance on larger graphs. Experiments on synthetic tasks and chip placement evaluate depth extension and adaptation, highlighting LFS's accuracy--trainable-parameter trade-off for size generalization.

Figures & tables

Panel of Figure 1 — Figure 1: (a) Six schemes by depth and adaptation. (b) LFS steers each frozen processor application with a shared controller; the readout acts after $K$ steps. Blue blocks are trainable.

Figure 2: Risk-minimizing deployment depth as graph size increases. Operator-aligned pretraining tracks the prescribed task horizon

T^{\star}(N)

, whereas standard target supervision remains near the pretraining depth. Shading denotes variability across seeds; dotted line: the largest pretraining depth.

Figure 3: Risk ratio of LFS to Loop-Tune across contraction rates $\rho$ and deployment-to-adaptation size ratios, under target-supervised pretraining. Values $<1$ indicate that LFS is more accurate. Results are averaged over five seeds.

		synthetic netlists, cells per design				real
scheme	params	0.4–0.6k	0.8–1k	1.8–2k	2.4–2.8k	MLCAD
Fixed	0	1.51	3.00	6.17	8.50	7.18
Fixed-Tune	5.85M	1.48 $\pm$ .00	2.81 $\pm$ .00	5.59 $\pm$ .00	7.54 $\pm$ .01	6.78 $\pm$ .00
Loop-Tune	5.85M	1.11 $\pm$ .01	1.86 $\pm$ .00	3.69 $\pm$ .02	5.30 $\pm$ .04	6.01 $\pm$ .05
LFS	0.17M	1.27 $\pm$ .01	2.06 $\pm$ .01	3.95 $\pm$ .02	5.61 $\pm$ .01	6.24 $\pm$ .02

Table 1: Chip placement, physical RMSE (mean

\pm

sd over 3 adaptation runs; lower is better; best per column in bold), with trainable parameters. Looped schemes deployed at

K{=}2

; ‘params’ counts trained parameters; rightmost column: real MLCAD netlists.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: The four task families and their fixed-radius error. Computed from the task definition, before any learning, with RGG

d{=}2

\rho{=}0.7

. Left: the prescribed horizon

T^{\star}(N)

of each family. Right: the per-node error of the

K_{0}

-step truncation, which uses only a

K_{0}

-hop neighborhood. For const this error is exactly zero at every size. For the other families it grows with the graph size. The truncation error upper-bounds the fixed-radius floor of Proposition 1 .

Panel of Figure 5 — Figure 5: Training-free checks of the two main assumptions , on damped PageRank (Appendix H.4 ). (a) The error of the best radius- $k$ prediction decreases geometrically in $k$ ; the guaranteed rate $\rho^{2k}$ (dashed) is an upper bound, and the measured rate $\rho_{\mathrm{eff}}$ is somewhat faster. (b) The radius needed to reach a fixed accuracy grows linearly in $\log N$ ; using the measured rate $\rho_{\mathrm{eff}}$ predicts the growth rate within about $10\%$ , whereas using the nominal $\rho$ overestimates it by up to $10\times$ on expander graphs. (c) The distance between local neighborhood distributions at size $N$ and at a large reference size decays as $N^{-1/d}$ : the measured exponent is $-0.510$ against the predicted $-0.500$ at $d=2$ (16 seeds); at $d=3$ the measurement ( $-0.52$ ) is consistent with the bound but the graphs are too small to reach the asymptotic rate.

Figure 6: Test risk of the four schemes across deployment sizes , one panel per task family (target-supervised pretraining, RGG

d{=}2

\rho{=}0.7

; 5 seeds,

\pm 1

sd; each scheme selects its deployment depth on a held-out split at every size). On the

\mathcal{F}_{1}

family all four schemes perform alike and no scheme degrades with size. On the other families, the risk of the two unadapted schemes grows with the graph size while the two adapted schemes are

2.6

–

10\times

more accurate, and LFS gradually overtakes Loop-Tune as the size grows, most clearly on diam .

Figure 7: Risk ratio of LFS to Loop-Tune as a function of deployment size (target-supervised pretraining, RGG

d{=}2

; 5 seeds,

\pm 1

sd; adaptation on 100 graphs of

350

–

700

nodes). Values below

1

mean LFS is more accurate. Near the adaptation sizes Loop-Tune leads; the curves cross for poly and diam between

2

and

20

times the adaptation size at

\rho\geq 0.7

, log stays within a few percent of parity, and where a small neighborhood suffices (

\rho{=}0.5

, and const at every

\rho

) Loop-Tune leads at every size.

scheme	depth / adapted params	const	log	poly	diam
Fixed	fixed / none	0.10	2.00	3.31	3.71
Fixed-Tune	fixed / all	0.09	0.77	1.36	0.38
Fixed-Steer	fixed / adapter + readout	0.07	0.74	1.06	0.25
Loop	tunable / none	0.18	1.94	3.26	4.01
Loop-Tune	tunable / all	0.14	0.77	1.28	0.39
LFS	tunable / controller + readout	0.15	0.75	1.25	0.36

Appendix

Table 2: All six schemes of Sec. 4 on the constructed families (RGG

d{=}2

\rho{=}0.7

; per-node MSE

\times 10^{-3}

, mean over 5 seeds, sd

\leq 0.3

in these units for every entry; deployment at

N=2\times 10^{4}

, i.e.

40\times

the adaptation sizes). Best per column in bold.

scheme	depth / adapted params	log	diam
Fixed	fixed / none	$5.92\pm 0.11$	$47.0\pm 0.4$
Fixed-Tune	fixed / all	$4.98\pm 0.58$	$38.3\pm 3.4$
Fixed-Steer	fixed / adapter + readout	$3.64\pm 0.86$	$35.8\pm 1.6$
Loop	tunable / none	$2.37\pm 0.57$	$14.4\pm 7.9$
Loop-Tune	tunable / all	$\mathbf{1.04\pm 0.17}$	$\mathbf{3.26\pm 1.17}$
LFS	tunable / controller + readout	$1.12\pm 0.19$	$3.99\pm 1.57$

Appendix

Table 3: The six schemes at slow contraction under operator-aligned pretraining (RGG

d{=}2

\rho{=}0.9

; per-node MSE

\times 10^{-3}

, mean

\pm

sd over 5 seeds; deployment at

N=2\times 10^{4}

). Best per column in bold.

Panel of Figure 8 — Figure 8: Damped PageRank ( $\rho=0.95$ , RGG $d{=}2$ ; 5 seeds). (a) The two adapted schemes are $2.8\times$ more accurate than the unadapted ones at $N{=}10^{3}$ , growing to about $4\times$ at $N{=}2\times 10^{4}$ . (b) With the contractive update described in the text, the risk-minimizing deployment depth is ordered by the contraction rate at every size (about $27/20/13/8$ for measured rates $0.92/0.80/0.63/0.42$ ) and grows with $\log N$ wherever the benefit of resolving more of the target exceeds the estimation error.

Figure 9: Loop-Tune and LFS as the size gap (left) and the adaptation budget (right) vary (damped PageRank, 3 seeds). Left: Loop-Tune on a single adaptation size and on a spread of sizes perform alike at every gap up to

40\times

: on these homogeneous graphs, single-size fine-tuning does not damage cross-size behavior (cf. Figure 10 ). Right: the comparison crosses with the budget: LFS is more accurate with

30

or fewer adaptation graphs (risk ratios

0.56

–

0.90

), and Loop-Tune leads once the budget reaches

100

graphs.

Figure 10: How far the learned update deviates from the task update , measured as

\lVert D(R_{\theta}(h))-U_{N}(D(h))\rVert^{2}

along the model’s own trajectory (damped PageRank,

\rho=0.95

, 3 seeds). Left: change in this deviation caused by adaptation, relative to the frozen pretrained processor, matched per seed and size. Right: the deviation itself for all schemes. The frozen processor (Loop) is the best-aligned reference; fine-tuning increases the deviation uniformly across sizes; the layers of the fixed-depth model (Fixed), measured one by one, sit

5

–

10\times

higher at every size: trained without weight sharing, they never learn to approximate

U_{N}

Figure 11: Deliberately biased adaptation data (adapt on degree-4 graphs at

N\approx 500

, evaluate on degree-8 graphs; target-supervised pretraining,

\rho=0.7

). Left: change in the learned update’s deviation from the task update, relative to the frozen processor ( diam , 3 seeds): matched adaptation data improve the agreement, while biased data push Loop-Tune away from the task update while LFS stays closer to its pretrained behavior. Right: risk ratio of LFS to Loop-Tune across deployment sizes for log and diam (5 seeds; dashed: matched adaptation data, solid: biased): the bias reverses the comparison, by up to a factor of

2.4

Panel of Figure 12 — Figure 12: Outside the contractive class. (a) For shortest-path distances, the error of the best radius- $k$ prediction does not decay geometrically; it drops only once $k$ reaches the graph diameter, of order $\sqrt{N}$ , a property of the task itself. (b) Trained on this task, every scheme degrades severely with the graph size and the differences between schemes collapse: the failure is not specific to LFS, and no ordering among the schemes survives.

Graph Neural Networks

Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.

See all 735 papers

Learning to Extrapolate in Depth with Looped Transformers
Oct 10, 2026Huitao Yang, Jianhao Huang, Zixuan Wang +1Position Extrapolation Recurrent Transformers
University of California, Los Angeles · Princeton University
Train Small, Deploy Large: Zero-Shot GNN Transfer Through Geometric Renormalization
Jul 30, 2026Robert Jankowski, Pedro Almagro-Blanco, Marián Boguñá +2Zero-Shot Learning Graph Neural Networks
TU Delft · University of Seville · University of Barcelona +2
Hyperparameter Transfer in Graph Neural Networks
Jul 6, 2026Gage DeZoort, Boris HaninGraph Neural Networks Neural Network Optimization
Department of Operations Research and Financial Engineering · School of Engineering and Applied Science · Princeton University +1

Oct 10, 2026J/K move · Enter open · S save

Learning to Extrapolate in Depth with Looped Transformers

Huitao Yang, Jianhao Huang, Zixuan Wang, Baharan Mirzasoleiman

University of California, Los Angeles · Princeton University

Position Extrapolation Recurrent Transformers Graph Transformer

Open paper AskPDF arXiv

Looped transformers reuse a single block across iterations, raising a natural question: how far can a block trained at one depth be safely extrapolated to greater depths? We study this question for all-pairs graph connectivity, where harder graphs require longer-range information propagation. We show that under exchangeability and symmetric positive initialization, population training at depth K converges to an update rule governed by a single effective parameter that balances graph propagation against fitting structure unresolved at training depth. This trade-off creates a finite extrapolation window: too few test-time loops provide insufficient propagation, while too many eventually create false connections. We derive explicit endpoints for this window and bound the graph hardness solvable beyond training as a function of training depth and hardness. In particular, a test depth chosen solely from the training distribution can guarantee recovery on substantially harder graphs, with a certified depth gain that grows exponentially as training depth outpaces training hardness. Experiments validate the predicted windows and show how compute budget and initialization affect the optimal training depth.

Abstract

Graph Neural Networks

Explore similar work

Learning to Extrapolate in Depth with Looped Transformers

Train Small, Deploy Large: Zero-Shot GNN Transfer Through Geometric Renormalization

Hyperparameter Transfer in Graph Neural Networks