Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

Juan Guevara; Riccardo De Santi; Andreas Krause; Raul Astudillo

cs.LGNov 3, 2026

Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

Authors: Juan Guevara, Riccardo De Santi, Andreas Krause, Raul Astudillo

Organizations: Mohamed Bin Zayed University of Artificial Intelligence · ETH Zurich · ETH AI Center

Abstract

Scientific discovery requires plausible designs spanning useful trade-offs among competing properties: a molecule with strong binding affinity, for example, may be difficult to synthesize. Pretrained generative models, e.g., flow and diffusion generators, provide a foundation for proposing such designs, but steering them toward high-quality, complementary trade-offs remains challenging. Inference-time search incurs high per-sample costs, while fine-tuning typically relies on fixed per-candidate rewards that do not capture how designs complement one another. We introduce Hypervolume Fine-Tuning (HVFT), which adapts a single pretrained generator to maximize the expected hypervolume of its samples while controlling departure from the pretrained distribution. Expected hypervolume jointly rewards candidate quality and collective trade-off coverage. Nonetheless, we show that it cannot be expressed as the expectation of a fixed per-candidate reward and therefore cannot be optimized directly with standard techniques. A key observation of this work is that its first variation corresponds to the expected marginal hypervolume contribution of a candidate, which yields a tractable adaptive reward via an unbiased estimator based on model samples. HVFT uses this reward to decompose the objective into a sequence of KL-proximal scalar-reward subproblems. We establish concavity, relative smoothness, and convergence guarantees for the resulting distributional procedure. Across therapeutic-peptide and molecular design tasks spanning discrete diffusion and continuous flow-matching models, HVFT achieves state-of-the-art expected hypervolume and trade-off coverage.

Figures & tables

Panel of Figure 1 — Figure 1: HVFT for multi-objective generative fine-tuning. (a) HVFT iteratively constructs a scalar reward from the current distribution $p_{k}$ and fine-tunes the generator, shifting the pushforward distribution $\mathbf{r}_{\#}p_{k}$ toward high-quality, complementary trade-offs. (b) The gray region is the hypervolume dominated relative to $\mathbf{r}^{\mathrm{ref}}$ . Hatching shows the marginal contributions of $x_{a}$ and $x_{b}$ , while the dominated candidate $x_{c}$ contributes zero. HVFT estimates these contributions against background sets from $p_{k}$ to define the next fine-tuning reward.

		Objective-space metrics		Design-space metrics
Task	Method	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
GLAST	Base	$2.78\pm 0.04$	$1.61\pm 0.01$	$\underline{60.35\pm 0.14}$	$\underline{401.50\pm 0.28}$	$0.321\pm 0.007$
	PepTune	$2.79\pm 0.07$	$1.59\pm 0.03$	$39.78\pm 1.05$	$134.95\pm 5.87$	$\mathbf{0.603\pm 0.028}$
	TR2D2	$7.78\pm 1.06$	$6.79\pm 1.41$	$9.08\pm 6.24$	$136.64\pm 74.24$	$0.249\pm 0.031$
	TR2D2 ++	$\underline{8.56\pm 0.86}$	$\underline{7.68\pm 0.77}$	$4.13\pm 1.28$	$89.43\pm 28.69$	$0.233\pm 0.010$
	HVFT	$\mathbf{11.67\pm 0.27}$	$\mathbf{10.56\pm 0.25}$	$\mathbf{64.78\pm 7.34}$	$\mathbf{465.45\pm 15.71}$	$\underline{0.336\pm 0.024}$

Table 1: Objective-space and design-space metrics for three therapeutic peptide design tasks. Bold and underlined values indicate the highest and second-highest means, within each task. HVFT achieves the highest hypervolume across the three tasks, as well as the highest design-space diversity and coverage.

Figure 2: Pareto sets induced by 1,000 samples from TR2D2 ( Tang et al., 2025b ) , a state-of-the-art discrete diffusion fine-tuning algorithm, and HVFT , on the GLAST task. HVFT achieves broader objective-space coverage and higher individual sample quality than TR2D2 .

Figure 3: Best four-point subset by hypervolume, selected from 100 samples of DiffusionBlend and HVFT on the Topology task. HVFT ’s samples achieve better coverage of high-quality trade-offs in objective-space.

		Objective-space metrics		Design-space metrics
Task	Method	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
Topology	FlowMol	$3.116\pm 0.014$	$2.665\pm 0.007$	$\mathbf{266.1\pm 4.9}$	$\mathbf{507.1\pm 6.4}$	$0.705\pm 0.008$
	DiffusionNFT	$3.140\pm 0.004$	$\underline{2.754\pm 0.005}$	$128.8\pm 2.7$	$312.8\pm 8.7$	$\mathbf{0.952\pm 0.009}$
	DiffusionBlend	$\underline{3.337\pm 0.024}$	$2.750\pm 0.031$	$150.3\pm 18.8$	$345.8\pm 30.6$	$0.674\pm 0.132$
	HVFT	$\mathbf{3.416\pm 0.005}$	$\mathbf{3.247\pm 0.004}$	$\underline{154.9\pm 9.0}$	$\underline{351.6\pm 11.1}$	$\underline{0.734\pm 0.007}$
Geometry	FlowMol	$4.279\pm 0.024$	$3.655\pm 0.010$	$\mathbf{91.4\pm 2.3}$	$\mathbf{331.6\pm 7.0}$	$0.483\pm 0.009$

Table 2: Objective- and design-space metrics for the small-molecule design tasks. Bold and underlined values indicate the highest and second-highest means, respectively, within each task. HVFT achieves the highest hypervolume among state-of-the-art methods while preserving diversity and coverage.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Execution time (hours)
TR2-D2	$12.384\pm 1.114$
HVFT	$10.734\pm 0.310$
DiffusionNFT	$6.399\pm 0.069$
HVFT	$7.204\pm 0.082$

Appendix

Table 3: Average execution time in hours. The first block aggregates results across the three peptide tasks. The second block aggregates runs for the Geometry task. The performance of these runs is reported in Section 4 . HVFT ’s execution time remains comparable to its inner solver counterpart.

Property	Binding Affinity	Permeability	Nonfouling	Solubility	Hemolysis
Reference Point	4.5	-10.0	0.0	0.0	0.0

Appendix

Table 4: Reference points used for hypervolume computation across all peptides tasks.

Hyperparameter	Name in Tang et al. (2025b)	Value
Num. of Replicates for Weighted Denoising Cross-Entropy	$R$	16
Buffer Size	$B$	20
Sequence Length	$L_{\mathrm{seq}}$	200
Num. of Children	$M$	50
Num. of Decoding Steps	$N_{\mathrm{steps}}$	128
Num. of MCTS Iterations	$N_{\mathrm{iter}}$	20

Appendix

Table 5: Hyperparameters for TR2-D2 baselines and HVFT inner-loop finetuning.

Property	QED	SA	Dipole Moment
Reference Point	-1.0	-1.0	-1.0

Appendix

Table 6: Reference points used for hypervolume computation for the Topology and Geometry tasks.

Hyperparameter	Name in Zheng et al. (2026)	Value
Guidance Strength	$\beta$	50
KL Regularization Coefficient	Not in paper, but in official implementation	0.5
Learning Rate	–	1e-5

Appendix

Table 7: Additional hyperparameters for DiffusionNFT finetuning.

		Objective-space metrics		Design-space metrics
Task	Model / $n$	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
TfR	Base	$2.93\pm 0.04$	$1.69\pm 0.01$	$60.35\pm 0.14$	$401.50\pm 0.28$	$\mathbf{0.321\pm 0.007}$
	$10$	$11.57\pm 0.16$	$10.15\pm 0.18$	$75.89\pm 9.42$	$\mathbf{485.80\pm 17.27}$	$\underline{0.250\pm 0.021}$
	$50$	$\underline{12.231\pm 0.153}$	$\underline{10.589\pm 0.226}$	$\underline{93.2\pm 3.0}$	$437.9\pm 62.9$	$0.210\pm 0.078$
	$150$	$\mathbf{13.757\pm 1.721}$	$\mathbf{11.107\pm 0.891}$	$\mathbf{107.3\pm 8.6}$	$\underline{454.1\pm 43.6}$	$0.235\pm 0.114$
Topology	FlowMol	$3.116\pm 0.014$	$2.665\pm 0.007$	$\underline{266.1\pm 4.9}$	$\underline{507.1\pm 6.4}$	$0.705\pm 0.008$

Appendix

Table 8: Effect of

n

on objective- and design-space metrics for the TfR and topology tasks, with the pretrained base model as a reference. Values are means

\pm

standard errors across three runs. Boldface and underlining indicate the highest and second-highest means within each task, respectively.

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 524 papers

May 28, 2026

Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design

Adapting generative foundation models, in particular diffusion and flow models, to optimize given reward functions (e.g., binding affinity) while satisfying constraints (e.g., molecular synthesizability) is fundamental for their adoption in real-world scientific discovery applications such as molecular design or protein engineering. While recent works have introduced scalable methods for reward-guided fine-tuning of such models via reinforcement learning and control schemes, it remains an open problem how to algorithmically trade-off reward maximization and constraint satisfaction in a reliable and predictable manner. Motivated by this challenge, we first present a rigorous framework for Constrained Generative Optimization, which brings an optimization viewpoint to the introduced adaptation problem and retrieves the relevant task of constrained generation as a sub-case. Then, we introduce Constrained Flow Optimization (CFO), an algorithm that automatically and provably balances reward maximization and constraint satisfaction by reducing the original problem to sequential fine-tuning via established, scalable methods. We provide convergence guarantees for constrained generative optimization and constrained generation via CFO. Ultimately, we present an experimental evaluation of CFO on both synthetic, yet illustrative, settings, and a molecular design task. Across these evaluations, CFO achieves consistent increases in reward while ensuring high constraint satisfaction, showcasing its practical utility for constrained generative optimization.

Sven Gutjahr, Riccardo De Santi, Luca Schaufelberger +2

Department of Computer Science, ETH Zurich · ETH AI Center · NCCR Catalysis, Switzerland +1

Constrained Generative Modeling Molecular Optimization Generative Modeling Flow-Based Generative Modeling

Ask PDF

Sep 30, 2026

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on such data have emerged as a promising approach, but their samples are not inherently better than the data and must be steered toward the Pareto front. Existing methods guide or condition every sampling step. We instead act on the initial noise and leave the sampling process unchanged. Across Off-MOO-Bench, we observe that the objectives, as functions of the noise, are sensitive to only a few directions. We estimate these directions once per task via a Recursive Feature Machine using function values alone, and a small cache serves every trade-off, so each candidate costs one noise displacement and one ODE solve. We prove that this displacement increases the learned scalarized objective in expectation, and that sweeping trade-offs recovers the flow's attainable front up to proxy and steering errors. With additional guidance, for which we introduce novel data-adaptive and Pareto-aware operators, our method attains the best average hypervolume rank among generative methods on 47 tasks, at comparable or lower sampling cost. Steering alone outranks the best prior generative method at a fraction of its sampling cost.

Yuan Lu, Esha Singh, Yi-An Ma +1

Halıcıoğlu Data Science Institute, UC San Diego · Department of Computer Science & Engineering, UC San Diego

Diffusion Model Guidance Multi-Objective Optimization

Ask PDF

Feb 25, 2026

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest as real-world applications often involve multiple, conflicting objectives. However, existing approaches require joint training for each combination of objectives, meaning that any change in the objective set necessitates retraining from scratch. We propose a framework that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without fine-tuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex nonlinear operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization, and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule generation tasks demonstrate that our approach achieves performance comparable to baselines.

Seokwon Yoon, Youngbin Choi, Seunghyuk Cho +3

Department of Computer Science & Engineering, POSTECH, South Korea · Graduate School of Artificial Intelligence, POSTECH, South Korea

Generative Flow Networks Generative Modeling Multi-Objective Optimization

Ask PDF

Abstract

Generative Modeling

Explore similar work

Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation