Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

Seokwon Yoon; Youngbin Choi; Seunghyuk Cho; Seungbeom Lee; MoonJeong Park; Dongwoo Kim

cs.LGFeb 25, 2026

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

Authors: Seokwon Yoon, Youngbin Choi, Seunghyuk Cho, Seungbeom Lee, MoonJeong Park, Dongwoo Kim

Organizations: Department of Computer Science & Engineering, POSTECH, South Korea · Graduate School of Artificial Intelligence, POSTECH, South Korea

Abstract

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest as real-world applications often involve multiple, conflicting objectives. However, existing approaches require joint training for each combination of objectives, meaning that any change in the objective set necessitates retraining from scratch. We propose a framework that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without fine-tuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex nonlinear operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization, and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule generation tasks demonstrate that our approach achieves performance comparable to baselines.

Figures & tables

	2 Obj.	3 Obj.	4 Obj.	5 Obj.
MOGFN	0.021	0.027	0.042	0.048
HN-GFN	0.017	0.021	0.032	0.035
Ensemble	0.117	0.098	0.113	0.111
Ours	0.003	0.003	0.003	0.003

Table 1:

L_{1}

error on linear scalarization in a 2D grid with a varying number of objectives.

N

Obj. denotes scalarization with

N

objectives, where the specific objectives used are detailed in Table A1 . Results are averaged over 128 evenly spaced preference vectors.

Figure 1: Qualitative result of scalarization on a 2D grid domain. We visualize the density over the grid for the true distribution (left), MOGFN (middle), and ours (right).

Harmonic mean ( $\otimes$ )
	Classifier	Ensemble	Ours
	guidance	Ensemble	Ours
$p_{\text{Shubert}}\otimes p_{\text{Sphere}}$	0.158	0.145	0.136
$p_{\text{Shubert}}\otimes p_{\text{Diagonal}}$	0.142	0.190	0.180
$p_{\text{Circle1}}\otimes p_{\text{Circle2}}$	0.397	0.453	0.229
Contrast ( )

Table 2:

L_{1}

error on logical operators in a 2D grid. We evaluate harmonic mean (

\otimes

) and contrast ( ) operators across different reward pairs. Additional results are provided in Table A2 .

Figure 2: Qualitative result of logical operations on a 2D grid domain. We visualize the density over the grid for the true distribution (left), classifier guidance (middle), and ours (right).

Panel of Figure 3 — Figure 3: Distortion factor $\delta(x)=u_{M}(x)/N_{M}(x)$ vs. unnormalized target density $\mathcal{G}(p_{1}(x),\ldots,p_{k}(x))$ on the 2D grid domain. Red points indicate outliers, defined as states where $\delta(x)$ falls outside $[Q_{1}-1.5\cdot\mathrm{IQR},Q_{3}+1.5\cdot\mathrm{IQR}]$ (gray dotted lines), where $Q_{1}$ , $Q_{3}$ are the 25th/75th percentiles and $\mathrm{IQR}=Q_{3}-Q_{1}$ . The green dashed line marks $1/Z_{M}$ with $Z_{M}=\sum_{x}\mathcal{G}(p_{1}(x),\ldots,p_{k}(x))$ . Additional results are in Fig. A6 .

		Reward $\uparrow$				Diversity $\uparrow$
		MOGFN	HN-GFN	Ours (Model $F$ )	Ours (DB $F$ )	MOGFN	HN-GFN	Ours (Model $F$ )	Ours (DB $F$ )
Fragment	SEH-SA	$0.838_{\pm 0.002}$	$\mathbf{0.839}_{\pm 0.005}$	$0.836_{\pm 0.001}$	$0.835_{\pm 0.003}$	$\mathbf{0.573}_{\pm 0.001}$	$\mathbf{0.573}_{\pm 0.001}$	$0.562_{\pm 0.005}$	$0.554_{\pm 0.006}$
	SEH-QED	$0.764_{\pm 0.006}$	$0.711_{\pm 0.012}$	$0.775_{\pm 0.010}$	$\mathbf{0.777}_{\pm 0.006}$	$\mathbf{0.571}_{\pm 0.007}$	$0.568_{\pm 0.007}$	$0.564_{\pm 0.006}$	$0.565_{\pm 0.013}$
	SA-QED	$0.783_{\pm 0.006}$	$0.790_{\pm 0.009}$	$\mathbf{0.797}_{\pm 0.010}$	$0.790_{\pm 0.008}$	$0.630_{\pm 0.008}$	$\mathbf{0.650}_{\pm 0.006}$	$0.624_{\pm 0.007}$	$0.621_{\pm 0.012}$
	ALL	$0.723_{\pm 0.004}$	$0.670_{\pm 0.001}$	$0.741_{\pm 0.009}$	$\mathbf{0.742}_{\pm 0.010}$	$0.608_{\pm 0.003}$	$0.608_{\pm 0.008}$	$0.608_{\pm 0.009}$	$\mathbf{0.610}_{\pm 0.008}$
QM9	GAP-SA	$0.799_{\pm 0.022}$	$0.742_{\pm 0.035}$	$\mathbf{0.873}_{\pm 0.003}$	$0.868_{\pm 0.003}$	$\mathbf{0.919}_{\pm 0.008}$	$0.912_{\pm 0.003}$	$0.899_{\pm 0.010}$	$0.902_{\pm 0.009}$

Table 3: Results of scalarization on molecule generation tasks. We report the average reward and diversity of the top-10 samples across different objective combinations (mean

\pm

std over 3 seeds). ALL denotes all three objectives (SEH/GAP, SA, and QED).

		Hypervolume $\uparrow$
		MOGFN	HN-GFN	OP-GFN	Ours(Model $F$ )	Ours(DB $F$ )
Fragment	SEH-SA	$0.832_{\pm 0.012}$	$\mathbf{0.842}_{\pm 0.001}$	$0.807_{\pm 0.026}$	$0.834_{\pm 0.003}$	$0.824_{\pm 0.005}$
	SEH-QED	$0.758_{\pm 0.023}$	$0.747_{\pm 0.015}$	$0.581_{\pm 0.029}$	$\mathbf{0.768}_{\pm 0.018}$	$0.754_{\pm 0.026}$
	SA-QED	$0.781_{\pm 0.014}$	$0.790_{\pm 0.017}$	$0.655_{\pm 0.021}$	$\mathbf{0.792}_{\pm 0.001}$	$0.787_{\pm 0.018}$
	ALL	$0.665_{\pm 0.012}$	$0.651_{\pm 0.005}$	$0.476_{\pm 0.038}$	$0.674_{\pm 0.021}$	$\mathbf{0.679}_{\pm 0.023}$
QM9	GAP-SA	$0.976_{\pm 0.051}$	$0.958_{\pm 0.089}$	$0.972_{\pm 0.062}$	$\mathbf{1.000}_{\pm 0.008}$	$0.981_{\pm 0.015}$

Table 4: Hypervolume on molecule generation tasks. We report mean

\pm

std over 3 seeds. ALL denotes all three objectives (SEH/GAP, SA, and QED).

Panel of Figure 4 — Figure 4: Density visualization of $p_{\scriptscriptstyle\mathrm{SEH}}$ , $p_{\scriptscriptstyle\mathrm{QED}}$ , and the composed distribution $p_{M}$ induced by our mixing policy via scalarization on fragment-based molecule generation. Each distribution is estimated from 5,000 samples. We vary the weights as (a) $p_{M}\propto(0.3\cdot R_{\scriptscriptstyle\mathrm{SEH}}+0.7\cdot R_{\scriptscriptstyle\mathrm{QED}})^{32}$ and (b) $p_{M}\propto(0.7\cdot R_{\scriptscriptstyle\mathrm{SEH}}+0.3\cdot R_{\scriptscriptstyle\mathrm{QED}})^{32}$ .

Fragment-based				Atom-based (QM9)
	Classifier	Ours	Ours		Classifier	Ours	Ours
	guidance	(Model $F$ )	(DB $F$ )		guidance	(Model $F$ )	(DB $F$ )
Harmonic mean ( $\otimes$ )				Harmonic mean ( $\otimes$ )
$p_{\scriptscriptstyle\mathrm{SEH}}\otimes p_{\scriptscriptstyle\mathrm{SA}}$	81 $\pm$ 2	87 $\pm$ 3	88 $\pm$ 3	$p_{\scriptscriptstyle\mathrm{GAP}}\otimes p_{\scriptscriptstyle\mathrm{SA}}$	78 $\pm$ 4	87 $\pm$ 1	86 $\pm$ 2
$p_{\scriptscriptstyle\mathrm{SEH}}\otimes p_{\scriptscriptstyle\mathrm{QED}}$	80 $\pm$ 4	87 $\pm$ 2	88 $\pm$ 2	$p_{\scriptscriptstyle\mathrm{GAP}}\otimes p_{\scriptscriptstyle\mathrm{QED}}$	76 $\pm$ 2	79 $\pm$ 10	78 $\pm$ 2
$p_{\scriptscriptstyle\mathrm{SEH}}\otimes p_{\scriptscriptstyle\mathrm{SA}}\otimes p_{\scriptscriptstyle\mathrm{QED}}$	40 $\pm$ 3	65 $\pm$ 2	66 $\pm$ 3	$p_{\scriptscriptstyle\mathrm{GAP}}\otimes p_{\scriptscriptstyle\mathrm{SA}}\otimes p_{\scriptscriptstyle\mathrm{QED}}$	51 $\pm$ 3	61 $\pm$ 8	63 $\pm$ 2

Table 5: Results of logical operators on molecule generation tasks (mean

\pm

std over 3 seeds). We report the percentage of 5,000 samples falling into the target bin : for harmonic mean (

\otimes

), where all composed objectives are high; for contrast ( ), where only the first objective is high.

Panel of Figure 5 — Figure 5: Distributions $p_{M}$ induced by classifier-guidance and our mixing policy for fragment-based molecule generation. Each distribution is estimated from 5,000 samples. Base distributions $p_{\scriptscriptstyle\mathrm{SEH}}$ and $p_{\scriptscriptstyle\mathrm{SA}}$ are shown alongside the composed distribution. (a) Classifier-guidance on $p_{\scriptscriptstyle\mathrm{SEH}}\otimes p_{\scriptscriptstyle\mathrm{SA}}$ . (b) Ours on $p_{\scriptscriptstyle\mathrm{SEH}}\otimes p_{\scriptscriptstyle\mathrm{SA}}$ . (c) Classifier-guidance on $p_{\scriptscriptstyle\mathrm{SA}}\ \hbox to5.83pt{\vbox to5.83pt{\pgfpicture\makeatletter\hbox{\hskip 2.91249pt\lower-2.91249pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 C -2.07 3.75 -3.75 2.07 -3.75 0 C -3.75 -2.07 -2.07 -3.75 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{SEH}}$ . (d) Ours on $p_{\scriptscriptstyle\mathrm{SA}}\ \hbox to5.83pt{\vbox to5.83pt{\pgfpicture\makeatletter\hbox{\hskip 2.91249pt\lower-2.91249pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 C -2.07 3.75 -3.75 2.07 -3.75 0 C -3.75 -2.07 -2.07 -3.75 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{SEH}}$ .

	# Obj.		Fragment-based		Atom-based (QM9)
	# Obj.		Time (ms)	Traj. Len.	Time (ms)	Traj. Len.
Scalarization	2	MOGFN	21.62	26.0	38.41	31.5
		HN-GFN	21.35	26.0	38.90	32.8
		OP-GFN	21.78	26.0	50.51	41.7
		Ours (Model $F$ )	29.89	26.0	54.68	30.2
		Ours (DB $F$ )	32.79	26.0	57.88	30.1

Table 6: Inference cost comparison on molecule generation tasks. We report the average generation time (ms) and trajectory length over 1,000 samples. We use preference vectors

\boldsymbol{\omega}\in\{(0.5,0.5),(0.33,0.33,0.34)\}

for scalarization, and the harmonic mean for logical operators.

	Classifier	Ours	Ours
	guidance	(Model $F$ )	(DB $F$ )
$p_{\scriptscriptstyle\mathrm{GAP}}\otimes p_{\scriptscriptstyle\mathrm{SA}}$	93.8 $\pm$ 2.9	97.0 $\pm$ 2.5	97.8 $\pm$ 1.4
$p_{\scriptscriptstyle\mathrm{GAP}}\otimes p_{\scriptscriptstyle\mathrm{SA}}\otimes p_{\scriptscriptstyle\mathrm{QED}}$	90.4 $\pm$ 6.5	95.6 $\pm$ 2.6	97.2 $\pm$ 1.8
$p_{\scriptscriptstyle\mathrm{SA}}\ \hbox to6.43pt{\vbox to6.43pt{\pgfpicture\makeatletter\hbox{\hskip 3.21388pt\lower-3.21388pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 C -2.3 4.17 -4.17 2.3 -4.17 0 C -4.17 -2.3 -2.3 -4.17 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{GAP}}$	92.3 $\pm$ 10.3	98.2 $\pm$ 0.7	98.3 $\pm$ 0.8
$p_{\scriptscriptstyle\mathrm{SA}}\ \hbox to6.43pt{\vbox to6.43pt{\pgfpicture\makeatletter\hbox{\hskip 3.21388pt\lower-3.21388pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 C -2.3 4.17 -4.17 2.3 -4.17 0 C -4.17 -2.3 -2.3 -4.17 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{GAP}}\ \hbox to6.43pt{\vbox to6.43pt{\pgfpicture\makeatletter\hbox{\hskip 3.21388pt\lower-3.21388pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 4.17 0 C 4.17 2.3 2.3 4.17 0 4.17 C -2.3 4.17 -4.17 2.3 -4.17 0 C -4.17 -2.3 -2.3 -4.17 0 -4.17 C 2.3 -4.17 4.17 -2.3 4.17 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{QED}}$	91.3 $\pm$ 12.2	92.3 $\pm$ 11.0	90.9 $\pm$ 13.9

Table 7: Validity of generated molecules on the atom-based molecule generation task. We report the percentage of valid molecules among 1,000 generated samples, averaged over 3 seeds (mean

\pm

std). Note that the base models achieve validity (

p_{\scriptscriptstyle\mathrm{GAP}}

: 98.5

\pm

0.5 ,

p_{\scriptscriptstyle\mathrm{SA}}

: 90.0

\pm

14.5 ,

p_{\scriptscriptstyle\mathrm{QED}}

: 97.2

\pm

2.6 ).

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 13 — Figure A1 : Visualization of reward functions on the $32\times 32$ grid domain used in our experiments.

	Objectives
Mix2	Shubert, Diagonal
Mix3	Shubert, Diagonal, Currin
Mix4	Shubert, Diagonal, Currin, Sphere
Mix5	Shubert, Diagonal, Currin, Sphere, Branin

Appendix

Table A1 : Objective subsets used for Mix-

k

scalarization ( Fig. A1 (a) ).

Harmonic mean ( $\otimes$ )
	Classifier	Ensemble	Ours
	guidance	Ensemble	Ours
$p_{\text{Shubert}}\otimes p_{\text{Sphere}}$	0.158	0.145	0.136
$p_{\text{Branin}}\otimes p_{\text{Sphere}}$	0.053	0.121	0.110
$p_{\text{Shubert}}\otimes p_{\text{Diagonal}}$	0.142	0.190	0.180
$p_{\text{Circle1}}\otimes p_{\text{Circle2}}$	0.397	0.453	0.229

Appendix

Table A2 : Extended results of Table 2 .

L_{1}

error on logical operators in a 2D grid. We evaluate harmonic mean (

\otimes

) and contrast ( ) operators across different reward pairs.

Figure A2 :

L_{1}

error for the scalarization of

p_{\scriptscriptstyle\mathrm{Shubert}}

and

p_{\scriptscriptstyle\mathrm{Diagonal}}

(

\beta=1

) over five preference vectors

\boldsymbol{\omega}=(\omega_{1},\omega_{2})

. GFN denotes single-objective GFlowNets trained separately for each

\boldsymbol{\omega}

Figure A3 : Density visualization of scalarization under different weight settings. For each setting, we show the ground-truth target distribution and the distribution induced by our method, with corresponding $L_{1}$ errors.

Figure A4 : Density visualization of harmonic mean ( $\otimes$ ) and contrast ( ) compositions on pairs of benchmark reward functions. For each composition, we show the ground-truth target distribution and the distribution induced by our method, with corresponding $L_{1}$ errors.

Figure A5 : Density visualization of harmonic mean ( $\otimes$ ) and contrast ( ) compositions on Circle rewards. For each composition, we show the ground-truth target distribution and the distribution induced by our method, with corresponding $L_{1}$ errors.

Panel of Figure 20 — Figure A6 : Extended results of Fig. 3 . Distortion factor $\delta(x)=u_{M}(x)/N_{M}(x)$ vs. unnormalized target density $\mathcal{G}(p_{1}(x),\ldots,p_{k}(x))$ on the 2D grid domain. Red points indicate outliers, defined as states where $\delta(x)$ falls outside $[Q_{1}-1.5\cdot\mathrm{IQR},Q_{3}+1.5\cdot\mathrm{IQR}]$ (gray dotted lines), where $Q_{1}$ , $Q_{3}$ are the 25th/75th percentiles and $\mathrm{IQR}=Q_{3}-Q_{1}$ . The green dashed line marks $1/Z_{M}$ with $Z_{M}=\sum_{x}\mathcal{G}(p_{1}(x),\ldots,p_{k}(x))$ .

Figure A7 : Qualitative results of scalarization with $\beta=1,2,4$ on a 2D grid domain, where the reward is defined as $(0.5\ R_{\scriptscriptstyle\mathrm{Shubert}}+0.5\ R_{\scriptscriptstyle\mathrm{Diagonal}})^{\beta}$ . Pre-trained base models trained with reward exponent $\beta$ are composed using the reward-sharpened mixing policy Eq. 31 . We visualize the density over the grid of the true distribution (left), Ours (middle), and the distortion factor analysis (right).

		Reward $\uparrow$					Diversity $\uparrow$
		SubTB			TB		SubTB			TB
		Ensemble	Ours (Model $F$ )	Ours (DB $F$ )	Ensemble	Ours (DB $F$ )	Ensemble	Ours (Model $F$ )	Ours (DB $F$ )	Ensemble	Ours (DB $F$ )
Fragment	SEH-SA	$0.818_{\pm 0.006}$	$0.836_{\pm 0.001}$	$0.835_{\pm 0.003}$	$0.836_{\pm 0.007}$	$\mathbf{0.851}_{\pm 0.000}$	$0.548_{\pm 0.011}$	$0.562_{\pm 0.005}$	$0.554_{\pm 0.006}$	$0.537_{\pm 0.022}$	$\mathbf{0.567}_{\pm 0.003}$
	SEH-QED	$0.648_{\pm 0.005}$	$0.775_{\pm 0.010}$	$0.777_{\pm 0.006}$	$0.656_{\pm 0.004}$	$\mathbf{0.787}_{\pm 0.003}$	$0.499_{\pm 0.006}$	$0.564_{\pm 0.006}$	$\mathbf{0.565}_{\pm 0.013}$	$0.492_{\pm 0.014}$	$0.560_{\pm 0.004}$
	SA-QED	$0.606_{\pm 0.003}$	$0.797_{\pm 0.010}$	$0.790_{\pm 0.008}$	$0.607_{\pm 0.005}$	$\mathbf{0.801}_{\pm 0.005}$	$0.587_{\pm 0.007}$	$\mathbf{0.624}_{\pm 0.007}$	$0.621_{\pm 0.012}$	$0.596_{\pm 0.004}$	$0.617_{\pm 0.004}$
	ALL	$0.617_{\pm 0.001}$	$0.741_{\pm 0.009}$	$0.742_{\pm 0.010}$	$0.625_{\pm 0.004}$	$\mathbf{0.743}_{\pm 0.005}$	$0.563_{\pm 0.005}$	$0.608_{\pm 0.009}$	$\mathbf{0.610}_{\pm 0.008}$	$0.553_{\pm 0.011}$	$0.601_{\pm 0.006}$

Appendix

Table A3 : Ensemble baseline vs. our mixing policy on molecule generation tasks. Base GFlowNets are trained with either SubTB or TB. We report the average reward and diversity of the top-10 samples across different objective combinations (mean

\pm

std over 3 seeds). ALL denotes all three objectives (SEH/GAP, SA, and QED).

		IGD+ $\downarrow$
		MOGFN	HN-GFN	OP-GFN	Ours (Model F)	Ours (DB F)
Fragment	SEH-SA	$0.058_{\pm 0.002}$	$\mathbf{0.053}_{\pm 0.001}$	$0.065_{\pm 0.008}$	$0.056_{\pm 0.002}$	$0.060_{\pm 0.003}$
	SEH-QED	$0.075_{\pm 0.009}$	$0.082_{\pm 0.008}$	$0.144_{\pm 0.012}$	$\mathbf{0.074}_{\pm 0.005}$	$0.079_{\pm 0.009}$
	SA-QED	$0.072_{\pm 0.005}$	$\mathbf{0.069}_{\pm 0.005}$	$0.115_{\pm 0.007}$	$0.070_{\pm 0.001}$	$0.071_{\pm 0.006}$
	ALL	$0.111_{\pm 0.004}$	$0.117_{\pm 0.002}$	$0.185_{\pm 0.023}$	$0.110_{\pm 0.009}$	$\mathbf{0.108}_{\pm 0.008}$
QM9	GAP-SA	$0.046_{\pm 0.026}$	$0.036_{\pm 0.021}$	$0.041_{\pm 0.018}$	$\mathbf{0.023}_{\pm 0.001}$	$0.028_{\pm 0.003}$

Appendix

Table A4 : IGD+ (

\downarrow

) on molecule generation tasks. We report mean

\pm

std over 3 seeds. ALL denotes all three objectives (SEH/GAP, SA, and QED).

		MOGFN	HN-GFN	OP-GFN	Ours (Model F)	Ours (DB F)
Fragment	SEH-SA	$9_{\pm 2}$	$7_{\pm 0}$	$8_{\pm 2}$	$9_{\pm 2}$	$\mathbf{11}_{\pm 2}$
	SEH-QED	$16_{\pm 2}$	$11_{\pm 1}$	$11_{\pm 4}$	$14_{\pm 2}$	$\mathbf{18}_{\pm 1}$
	SA-QED	$6_{\pm 1}$	$6_{\pm 1}$	$\mathbf{7}_{\pm 0}$	$6_{\pm 3}$	$6_{\pm 1}$
	ALL	$\mathbf{119}_{\pm 18}$	$76_{\pm 11}$	$68_{\pm 12}$	$113_{\pm 5}$	$99_{\pm 22}$
QM9	GAP-SA	$6_{\pm 3}$	$9_{\pm 2}$	$7_{\pm 1}$	$\mathbf{15}_{\pm 2}$	$13_{\pm 1}$
	GAP-QED	$12_{\pm 1}$	$\mathbf{15}_{\pm 2}$	$9_{\pm 3}$	$13_{\pm 2}$	$10_{\pm 2}$

Appendix

Table A5 : Pareto Count (

\uparrow

) on molecule generation tasks. We report the number of non-dominated solutions (mean

\pm

std over 3 seeds). ALL denotes all three objectives (SEH/GAP, SA, and QED).

Base distributions
Fragment-based									Atom-based (QM9)
SEH	low				high				GAP	low				high
SA	low		high		low		high		SA	low		high		low		high
QED	low	high	low	high	low	high	low	high	QED	low	high	low	high	low	high	low	high
$p_{\scriptscriptstyle\mathrm{SEH}}$	1	0	0	0	56	12	28	3	$p_{\scriptscriptstyle\mathrm{GAP}}$	0	0	1	0	43	11	35	10
$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	74	5	0	0	18	3	$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	3	32	0	0	9	56

Appendix

Table A6 : Results of logical operators with our mixing policy (Model

F

) on molecule generation tasks (mean over 3 seeds). We report the percentage of valid samples falling into each of 8 bins based on reward thresholds: Fragment-based (SEH: 0.5, SA: 0.6, QED: 0.25) and Atom-based QM9 (GAP: 0.85, SA: 0.3, QED: 0.3). Bold indicates the target bin : for harmonic mean (

\otimes

), where all objectives are high; for contrast ( ), where the first objective is high and the rest low.

Base distributions
Fragment-based									Atom-based (QM9)
SEH	low				high				GAP	low				high
SA	low		high		low		high		SA	low		high		low		high
QED	low	high	low	high	low	high	low	high	QED	low	high	low	high	low	high	low	high
$p_{\scriptscriptstyle\mathrm{SEH}}$	1	0	0	0	56	12	28	3	$p_{\scriptscriptstyle\mathrm{GAP}}$	0	0	1	0	43	11	35	10
$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	74	5	0	0	18	3	$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	3	32	0	0	9	56

Appendix

Table A7 : Results of logical operators with our mixing policy (DB

F

\otimes

), where all objectives are high; for contrast ( ), where the first objective is high and the rest low.

Base distributions
Fragment-based									Atom-based (QM9)
SEH	low				high				GAP	low				high
SA	low		high		low		high		SA	low		high		low		high
QED	low	high	low	high	low	high	low	high	QED	low	high	low	high	low	high	low	high
$p_{\scriptscriptstyle\mathrm{SEH}}$	1	0	0	0	56	12	28	3	$p_{\scriptscriptstyle\mathrm{GAP}}$	0	0	1	0	43	11	35	10
$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	74	5	0	0	18	3	$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	3	32	0	0	9	56

Appendix

Table A8 : Results of logical operators with classifier-guided baseline on molecule generation tasks (mean over 3 seeds). We report the percentage of valid samples falling into each of 8 bins based on reward thresholds: Fragment-based (SEH: 0.5, SA: 0.6, QED: 0.25) and Atom-based QM9 (GAP: 0.85, SA: 0.3, QED: 0.3). Bold indicates the target bin : for harmonic mean (

\otimes

), where all objectives are high; for contrast ( ), where the first objective is high and the rest low.

GAP	low				high
SA	low		high		low		high
QED	low	high	low	high	low	high	low	high
$p_{\scriptscriptstyle\mathrm{GAP}}$	0	0	1	0	43	11	35	10
$p_{\scriptscriptstyle\mathrm{SA}}$	0	0	3	32	0	0	9	56
$p_{\scriptscriptstyle\mathrm{QED}}$	0	1	0	1	2	54	2	40
$p_{\scriptscriptstyle\mathrm{SA}}\ \hbox to5.83pt{\vbox to5.83pt{\pgfpicture\makeatletter\hbox{\hskip 2.91249pt\lower-2.91249pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} { {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{}{{{{}{}{}{}}} {{}{}{}{}}}{} {} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 Z}{stroke:none} \lx@inpgf@ignorespaces {}{{}}{}{{{}}{\lx@inpgf@ignorespaces}{}{\lx@inpgf@ignorespaces}{}{}{}{}{}}{}\lxSVG@stroke\lxSVG@drawpath@unclipped{M 0 0 M 3.75 0 C 3.75 2.07 2.07 3.75 0 3.75 C -2.07 3.75 -3.75 2.07 -3.75 0 C -3.75 -2.07 -2.07 -3.75 0 -3.75 C 2.07 -3.75 3.75 -2.07 3.75 0 Z M 0 0}{fill:none} \lx@inpgf@ignorespaces ~ } \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\ p_{\scriptscriptstyle\mathrm{QED}}$	0	0	4	30	0	0	13	53

Appendix

Table A9 : Challenging cases of the contrast operator on atom-based molecule generation (QM9). We report the percentage of samples falling into each of 8 bins based on reward thresholds (GAP: 0.85, SA: 0.3, QED: 0.3). Bold indicates the target bin : for contrast ( ), where only the first objective is high.

Panel of Figure 29 — Figure A8 : Reward distributions of base models in QM9, illustrating positive correlations between objectives. We observe that (a) high QED samples tend to have high GAP, and (b) high SA samples tend to have high QED.

Generative Flow Networks

Generative Flow Networks learn to sample diverse objects with probabilities proportional to their rewards.

See all 56 papers

Oct 12, 2026

Mixture of GFlowNets

Generative Flow Networks (GFlowNets) learn a policy that samples compositional objects in proportion to an unnormalized reward

R(x)

. This makes them useful wherever many diverse, high-reward candidates are needed and oracle queries are expensive. In practice, a single GFlowNet often fails to cover all modes of a complex distribution, and the obvious ways to scale compute saturate: larger batches and data-parallel gradient averaging draw every trajectory from one, possibly biased, policy, so extra rollouts rediscover known modes instead of finding new ones. We explore whether this mode-bias problem (mode collapse in the extreme) can be addressed with a population of asynchronously trained policies which divide the state space between them. Mixture of GFlowNets (MoG) replaces the single forward policy with a community of

K

independent GFlowNets coupled through one jointly trained soft classifier, which assigns each member its own region of the state space through reward shaping. Mixing the members under their learned partition functions provably recovers the target

p(x)\propto R(x)

. MoG exchange (MoGX) also lets the classifier route sampled trajectories: a trajectory the classifier assigns to another member is sent to that member and trained on there, with no importance sampling. We evaluate both methods by keeping the environment-interaction budget fixed while scaling the community size, turning parallel compute into mode coverage without extra oracle calls. On HyperGrid, across six reward landscapes and communities of up to

K{=}64

, MoG and MoGX keep finding new modes as

K

grows, while data-parallel training saturates. On a genomic ideotype-design task, MoGX covers

39%

of the

1024

optima at

K{=}64

, against

31\%

for independent agents and

20\%

for data-parallel training. MoG also has a lower per-iteration synchronization cost than data-parallel training.

Omar G. Younis, Joseph D. Viviano, Chirayu Anant Haryan +3

Mila – Québec AI Institute · Intel · Université de Montréal

Generative Flow Networks Mode Collapse

Ask PDF

Feb 19, 2026

Avoid What You Know: Divergent Trajectory Balance for GFlowNets

Generative Flow Networks (GFlowNets) are a flexible family of amortized samplers trained to generate discrete and compositional objects with probability proportional to a reward function. However, learning efficiency is constrained by the model's ability to rapidly explore diverse high-probability regions during training. To mitigate this issue, recent works have focused on incentivizing the exploration of unvisited and valuable states via curiosity-driven search and self-supervised random network distillation, which tend to waste samples on already well-approximated regions of the state space. In this context, we propose Adaptive Complementary Exploration (ACE), a principled algorithm for the effective exploration of novel and high-probability regions when learning GFlowNets. To achieve this, ACE introduces an exploration GFlowNet explicitly trained to search for high-reward states in regions underexplored by the canonical GFlowNet, which learns to sample from the target distribution. Through extensive experiments, we show that ACE significantly improves upon prior work in terms of approximation accuracy to the target distribution and discovery rate of diverse high-reward states.

Pedro Dall'Antonia, Tiago da Silva, Daniel Csillag +2

School of Applied Mathmatics, Getulio Vargas Foundation · MBZUAI

Generative Flow Networks Adaptive Sampling RL Exploration

Ask PDF

Jan 28, 2026

Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networks

Generative Flow Networks (GFlowNets; GFNs) are a class of generative models that learn to sample compositional objects proportional to their a priori unknown value, their reward. We focus on the case where the reward has a specified, actionable structure, namely that it is submodular. We show submodularity can be harnessed to retrieve upper bounds on the reward of compositional objects that have not yet been observed. We provide in-depth analyses of the probability of such bounds occurring, as well as how many unobserved compositional objects can be covered by a bound. Following the Optimism in the Face of Uncertainty principle, we then introduce SUBo-GFN , which uses the submodular upper bounds to train a GFN. We show that SUBo-GFN generates orders of magnitude more learning signal than classical GFNs for the same number of queries to the reward function. We demonstrate the effectiveness of SUBo-GFN in terms of distribution matching and high-quality candidate generation on graph submodular tasks.

Alexandre Larouche, Audrey Durand

D´epartement d’informatique et de g´enie logiciel, Universit´e Laval, Qu´ebec, Canada · Mila - Quebec AI Institute, Montr´eal, Canada · Canada CIFAR AI Chair

Generative Flow Networks Graph Generation

Ask PDF

Abstract

Generative Flow Networks

Explore similar work

Mixture of GFlowNets

Avoid What You Know: Divergent Trajectory Balance for GFlowNets

Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networks