Not All Latents Are Equal: Importance-Weighted Latent Graph Generation

Biraaj Rout; Jay Revolinsky; Han Liu; Harry Shomer

cs.LGNov 5, 2026

Not All Latents Are Equal: Importance-Weighted Latent Graph Generation

Authors: Biraaj Rout, Jay Revolinsky, Han Liu, Harry Shomer

Organizations: University of Texas at Arlington · Michigan State University

Abstract

Graph generation has shown tremendous effectiveness in modeling complex scientific tasks ranging from molecule generation to optimized circuit design. Most recent advancements have developed models that learn to denoise noisy graphs using either discrete diffusion or flow matching. However, discrete graph diffusion scales quadratically with the number of nodes, making it difficult to use such models on larger graphs. On the other hand, latent graph generation instead seeks to denoise noisy graph latents, which scale better to more nodes due to their lower dimension. However, in practice, latent graph generation models lag behind their discrete counterparts in performance. We study this problem, finding that it is caused by the unequal distribution of that different latent dimensions have on graph reconstruction. the impact. We observe that the vast majority of impact is contained in only a smaller number of latent dimensions. Since equal emphasis is placed on reconstructing the entire latent in the loss function, existing models are optimized to reconstruct information irrelevant to the downstream task. To tackle this problem, we propose Latent Importance Reweighting ({\bf LIR)} which weights each latent dimensions via it's importance to final graph reconstruction. Importance is calculated as the relative influence of a single latent unit on graph reconstruction, which is represented via the Jacobian of the decoder relative to the latent. Our method is model-agnostic and requires no additional model parameters or hyperparameters. Experiments show that LIR can boost performance and lead to faster training convergence when applied to two different autoencoders on various benchmark datasets. Code: https://github.com/biraaj/GraphGenLatentReweight

Figures & tables

Figure 1: Accuracy plot on the tree dataset Bergmeister et al. (2023) for multiple models. Note that “VGAE” substitutes VGAE Kipf and Welling (2016) for LG-VAE in Siraudin and Morris (2026) . Observe that the latent models lag behind their discrete counterparts.

Figure 2: The overall flow of LIR. (a) We use a pre-trained and frozen autoencoder to extract the latents from a graph. (b) To determine the importance of each latent dimension on reconstruction, we consider it’s effect on the decoder output via the Jacobian, which is then scaled by mean posterior variance so the total sensitivity is represented as $I_{i}$ . (c) We show how the weighting is used in the loss and difference between uniform and weighted Jacobian loss.

Panel of Figure 3 — Figure 3: Cumulative Jacobian plot illustrating the importance of latent dimensions on graph reconstruction. The results are on the planar and tree dataset with 512 nodes and 128 nodes, respectively, for VGAE Kipf and Welling (2016) and LG-VAE Siraudin and Morris (2026) . The plot demonstrates that the majority of the contribution is concentrated within a few latent dimensions, while the remaining latents contribute little to the reconstruction.

Figure 4: Sample and edge accuracy for the experiment described in Section 4.2 on Planar. Observe that only the “useful” latent dimensions (i.e., top 10% of dimensions via Eq. 7 ) are necessary for strong autoencoder performance while the “least useful” dimensions (bottom 90%) aren’t.

Figure 5

Dataset	Type	# Train / Val / Test	Nodes Mean/Max
Planar	synthetic	8,192 / 256 / 256	512 / 512
Tree	synthetic	8,192 / 256 / 256	128 / 128
SBM	synthetic	8,192 / 256 / 256	125.2 / 225
MOSES	molecules	1,584,663 / 176,074 / 176,225	21.6 / 27
GuacaMol	molecules	1,118,642 / 69,926 / 209,655	27.8 / 88

Table 1: Dataset details

	Planar		Tree		SBM
Model	Accuracy $\uparrow$	Ratio $\downarrow$	Accuracy $\uparrow$	Ratio $\downarrow$	Accuracy $\uparrow$	Ratio $\downarrow$
LG-Flow	$\mathbf{73.5{\scriptstyle\pm 0.007}}$	$16.7{\scriptstyle\pm 2.5}$	$29.8{\scriptstyle\pm 0.01}$	$7.7{\scriptstyle\pm 0.5}$	$30.9{\scriptstyle\pm 0.006}$	$\mathbf{6.0{\scriptstyle\pm 0.3}}$
+ LIR	$63.2{\scriptstyle\pm 0.03}$	$\mathbf{5.5{\scriptstyle\pm 0.8}}$	$\mathbf{47.6{\scriptstyle\pm 0.03}}$	$\mathbf{5.6{\scriptstyle\pm 0.5}}$	$\mathbf{36.5{\scriptstyle\pm 0.012}}$	$7.4{\scriptstyle\pm 0.1}$
% Difference	-14%	+67%	+60%	+27%	+18%	-19%
VGAE+Flow	$72.9{\scriptstyle\pm 0.01}$	$3.1{\scriptstyle\pm 0.6}$	$17.2{\scriptstyle\pm 0.008}$	$10.6{\scriptstyle\pm 0.1}$	$19.5{\scriptstyle\pm 0.01}$	$11.9{\scriptstyle\pm 1.6}$
+ LIR	$\mathbf{82.9{\scriptstyle\pm 0.01}}$	$\mathbf{2.8{\scriptstyle\pm 0.4}}$	$\mathbf{23.2{\scriptstyle\pm 0.02}}$	$\mathbf{8.7{\scriptstyle\pm 0.6}}$	$\mathbf{23.2{\scriptstyle\pm 0.01}}$	$\mathbf{3.6{\scriptstyle\pm 0.4}}$

Table 2: Large Graph generation performance on the synthetic datasets: Planar (512 nodes), Tree (128 nodes) and SBM (125 average nodes). We present the results from five sampling runs with different seeds, each generating 1024 graphs, reported as the mean

\pm

standard deviation.

	Guacamol			MOSES
Model	V.U.N. $\uparrow$	FCD $\uparrow$	NSPDK $\downarrow$	Val. $\uparrow$	Unique $\uparrow$	Novelty $\uparrow$	FCD $\downarrow$	NSPDK $\downarrow$
DeFoG	97.9	73.8	-	92.8	99.9	92.1	1.95	-
Digress	85.1	68.0	-	85.7	100	95.0	1.19	-
DisCo	86.5	59.7	-	88.3	100	97.7	1.44	-
Cometh	97.6	72.7	-	90.5	99.9	92.6	1.27	-
LG-Flow	81.1	78.8	0.49	88.2	99.8	93.2	2.49	0.529

Table 3: Large molecule generation performance.

Figure 5: Accuracy of VGAE-Flow on Planar when re-weighting 128 dimensions vs fewer unweighted dimensions.

Panel of Figure 6 — Figure 6: Validation V.U.N. against training steps on Tree. Re-weighting (LIR) reaches it’s best validation score in $1.6\times$ (LG-Flow) and $3.4\times$ (VGAE) fewer steps.

Type	Method	Time (s)	Graphs/s
Discrete	DeFoG	9888.06	0.01
Latent	LGD	10565.06	0.01
	LG-Flow	36.64	2.73
	VGAE+Flow	37.41	2.67
Latent+LIR	LG-Flow (+LIR)	37.05	2.70
Latent+LIR	VGAE+Flow (+LIR)	37.41	2.67

Table 4: Sampling efficiency on Planar (512 nodes). We measure the wall-clock time to generate 100 graphs. All models use 1K sampling steps and the same GPU configuration.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Model	Variant	Min $n$	Max $n$	AE (h)	FM (h)	Total (h)	Batch (FM/AE)	Epoch (s)	Sampled	Sampling (h)
Planar	LG-Flow ( $D{=}24$ )	Baseline	512	512	4.80	48.54	53.34	256 / 144	11.18	5,120	0.486
Planar	LG-Flow ( $D{=}24$ )	+LIR	512	512	4.80	48.03	52.83	256 / 144	11.07	5,120	0.484
Planar	VGAE + flow	Baseline	512	512	5.01	78.34	83.35	256 / 256	18.05	5,120	0.550
Planar	VGAE + flow	+LIR	512	512	5.01	67.96	72.97	256 / 256	20.34	5,120	0.535
Tree	LG-Flow	Baseline	128	128	0.91	31.51	32.42	256 / 256	7.26	5,120	0.299
Tree	LG-Flow	+LIR	128	128	0.91	31.61	32.52	256 / 256	7.28	5,120	0.299

Appendix

Table 5: Wall-clock training and sampling time for every reported condition.

Model	Steps	Validity (%) $\uparrow$	Unique (%) $\uparrow$	Novelty (%) $\uparrow$	VUN (%) $\uparrow$	FCD $\downarrow$	FCD score $\uparrow$	NSPDK $\downarrow$
Baseline	100	77.8	100.0	100.0	77.8	2.04	66.5	0.497
	250	80.3	100.0	100.0	80.2	1.59	72.8	0.498
	500	81.1	100.0	100.0	81.1	1.36	76.2	0.496
	1000 †	81.1	100.0	100.0	81.1	1.19	78.9	0.491
LIR	100	79.8	100.0	100.0	79.8	2.42	61.6	0.491
	250	82.5	100.0	100.0	82.5	1.45	74.8	0.490

Appendix

Table 6: LG-Flow vs LG-Flow+LIR on sampling steps of 100, 250, 500 and 1000 for GuacaMol

Model	Steps	Validity (%) $\uparrow$	Unique (%) $\uparrow$	Novelty (%) $\uparrow$	VUN (%) $\uparrow$	FCD $\downarrow$	NSPDK $\downarrow$
LG-Flow (Baseline)	100	88.8	100.0	93.5	82.9	4.12	0.530
	250	88.9	100.0	92.9	82.5	3.01	0.532
	500	88.1	100.0	92.7	81.6	2.69	0.530
	1000 †	88.3	99.9	93.3	82.2	2.49	0.529
LG-Flow (LIR)	100	86.4	100.0	95.9	82.9	3.65	0.520
	250	87.2	100.0	95.2	83.0	2.85	0.521

Appendix

Table 7: LG-Flow vs LG-Flow+LIR on sampling steps of 100, 250, 500 and 1000 for Moses

Model	Steps	Acc. (%) $\uparrow$	Ratio $\downarrow$	Degree $\downarrow$	Spectre $\downarrow$	Wavelet $\downarrow$	Clustering $\downarrow$	Orbit $\downarrow$
LG-Flow (Baseline)	100	$53.0_{\pm 2.2}$	$83.8_{\pm 6.7}$	$0.0005_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0006_{\pm 0.0000}$	$0.0499_{\pm 0.0040}$	$0.0039_{\pm 0.0003}$
	250	$72.3_{\pm 1.8}$	$38.0_{\pm 3.8}$	$0.0002_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0003_{\pm 0.0000}$	$0.0226_{\pm 0.0023}$	$0.0015_{\pm 0.0002}$
	500	$74.1_{\pm 0.7}$	$23.5_{\pm 3.4}$	$0.0001_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0002_{\pm 0.0000}$	$0.0139_{\pm 0.0021}$	$0.0007_{\pm 0.0001}$
	1000 †	$73.6_{\pm 0.8}$	$16.7_{\pm 2.6}$	$0.0001_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0098_{\pm 0.0015}$	$0.0004_{\pm 0.0000}$
LG-Flow (LIR)	100	$46.0_{\pm 2.0}$	$40.7_{\pm 3.3}$	$0.0002_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0003_{\pm 0.0000}$	$0.0241_{\pm 0.0020}$	$0.0021_{\pm 0.0002}$
	250	$61.9_{\pm 1.5}$	$14.1_{\pm 1.6}$	$0.0001_{\pm 0.0000}$	$0.0001_{\pm 0.0000}$	$0.0002_{\pm 0.0000}$	$0.0083_{\pm 0.0010}$	$0.0005_{\pm 0.0001}$

Appendix

Table 8: LG-Flow and VGAE+Flow Baseline vs LIR on sampling steps of 100, 250, 500 and 1000 for Planar with 512 nodes

Panel of Figure 7 — Figure 7: Validation accuracy against training step on Planar, both backbones. LG-Flow ( $D{=}24$ ) is the one setting in this paper where re-weighting converges to a lower score than the standard objective (58% vs. 65% on this periodic training-time check), consistent with Table 2 ’s documented exception. VGAE ( $D{=}128$ ) shows the opposite: Re-weighting(LIR) reaches the standard objective’s own best score in $4.6\times$ fewer steps and plateaus substantially higher (76% vs. 61%).

Figure 8: Validity against training step on Guacamol, LG-Flow (no VGAE variant was trained for this dataset). Re-weighting(LIR) reaches the standard objective’s own best validity in

1.4\times

fewer steps, converging to a comparable final value (85% vs. 85%).

Figure 9: LG-Flow — Planar (512 nodes): reference test-set graphs (top) vs. generated samples without re-weighting (middle) and with re-weighting (bottom).

Figure 10: VGAE+Flow — Planar (512 nodes): reference test-set graphs (top) vs. generated samples without re-weighting (middle) and with re-weighting (bottom).

Figure 11: LG-Flow — Tree (128 nodes): reference test-set graphs (top) vs. generated samples without re-weighting (middle) and with re-weighting (bottom).

Figure 12: VGAE+Flow — Tree (128 nodes): reference test-set graphs (top) vs. generated samples without re-weighting (middle) and with re-weighting (bottom).

Graph Generation

Graph generation builds models and algorithms that create graphs with desired structures, properties, or distributions.

See all 70 papers

Jun 2, 2026

Scaling Novel Graph Generation via Lightweight Structure-Guided Autoregressive Models

Generating realistic and diverse graphs is a key problem in machine learning, with applications in molecular discovery, circuit design, cybersecurity, and beyond. However, current graph generative models remain limited by scalability and novelty. Diffusion-based methods often require costly full-adjacency operations and long denoising chains, while many autoregressive and hybrid models have at least quadratic complexity. In addition, these models often imitate training graphs rather than generalize beyond them. We propose a lightweight autoregressive framework to address these issues. It uses a structure-guided topological ordering to serialize graphs into regular edge sequences, enabling near log-linear generation, and a two-phase training strategy that combines exploration-oriented augmentation with iterative refinement to reduce overfitting and promote controlled novelty. Experiments on molecular and non-molecular benchmarks show that our approach improves novelty while preserving high validity and uniqueness. The framework also supports both LSTM and Mamba-style causal sequence backbones, with large-memory accelerators enabling longer graph-sequence experiments beyond typical GPU limits.

Alessio Barboni, Massimiliano Lupo Pasini, Bishal Lakha +1

Boise State University Boise, Idaho, USA · Oak Ridge National Laboratory Oak Ridge, Tennessee, USA

Graph Generation Generative Modeling Autoregressive Generation

Ask PDF

Oct 5, 2026

Latent Flow Matching for Molecular Graph Generation

Modern graph generative models typically operate directly in the discrete graph space, explicitly generating node and edge variables, which can become costly as graphs grow. In this paper, we perform generation explicitly on latent representations of entire graphs obtained from a pretrained Variational Autoencoder with high reconstruction fidelity. The generated representations, obtained through flow matching, are then decoded only at the final step. Across molecular benchmarks of increasing size, our approach achieves strong validity and FCD while offering a favorable quality-efficiency trade-off compared with state-of-the-art explicit graph generative models. One of the main advantages of this formulation is that the graph representation only needs to be learned once, after which the same one can be reused across multiple generative objectives without retraining. We demonstrate generation guided by molecular properties and further introduce validity-aware generation though a classifier learned directly in latent space. All code will be made available upon acceptance.

Mathis Goupillon, Roman Bresson, Konstantinos Divriotis +1

Mohamed bin Zayed University of Artificial Intelligence

Flow Matching Molecular Graph Generation Molecular Generation Flow-Based Generative Modeling

Ask PDF

Mar 9, 2026

Local Message-Passing for Discrete Graph Generation

Discrete graph generation has emerged as a powerful paradigm for modeling graph-structured data, yet state of the art models often rely on Graph Transformers or higher order architectures. We revisit this design assumption by introducing GenGNN, a modular message passing backbone for graph generation. GenGNN enables powerful generation by persisting edge fields through latent refinement of coupled node edge graph states, all without requiring global attention. Diffusion models integrating GenGNN achieve over 90 percent validity on standard benchmark datasets, performing within margins of Graph Transformer backbones and achieving up to 2x or even 5x faster inference. Systematic ablations isolate how GenGNN is resilient to oversmoothing during generative denoising, indicating each GenGNN component is necessary for downstream generation quality. Finally, representation-space analysis suggests GenGNN learns functionally similar representations to more theoretically-expressive architectures; even at deeper layers. As such, GenGNN uplifts local message-passing to challenge prevailing assumptions that performant discrete graph generation requires global attention or higher-order representations. Source Code Available Here

Jay Revolinsky, Harry Shomer, Jiliang Tang

Michigan State University East Lansing, Michigan, USA · University of Texas - Arlington Arlington, Texas, USA

Graph Neural Networks Graph Generation Message Passing Neural Networks

Ask PDF

Abstract

Graph Generation

Explore similar work

Scaling Novel Graph Generation via Lightweight Structure-Guided Autoregressive Models

Latent Flow Matching for Molecular Graph Generation

Local Message-Passing for Discrete Graph Generation