Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

Juan Guevara; Riccardo De Santi; Andreas Krause; Raul Astudillo

cs.LGNov 3, 2026

Multi-Objective Generative Optimization via Expected Hypervolume Fine-Tuning

Authors: Juan Guevara, Riccardo De Santi, Andreas Krause, Raul Astudillo

Organizations: Mohamed Bin Zayed University of Artificial Intelligence · ETH Zurich · ETH AI Center

Abstract

Scientific discovery requires plausible designs spanning useful trade-offs among competing properties: a molecule with strong binding affinity, for example, may be difficult to synthesize. Pretrained generative models, e.g., flow and diffusion generators, provide a foundation for proposing such designs, but steering them toward high-quality, complementary trade-offs remains challenging. Inference-time search incurs high per-sample costs, while fine-tuning typically relies on fixed per-candidate rewards that do not capture how designs complement one another. We introduce Hypervolume Fine-Tuning (HVFT), which adapts a single pretrained generator to maximize the expected hypervolume of its samples while controlling departure from the pretrained distribution. Expected hypervolume jointly rewards candidate quality and collective trade-off coverage. Nonetheless, we show that it cannot be expressed as the expectation of a fixed per-candidate reward and therefore cannot be optimized directly with standard techniques. A key observation of this work is that its first variation corresponds to the expected marginal hypervolume contribution of a candidate, which yields a tractable adaptive reward via an unbiased estimator based on model samples. HVFT uses this reward to decompose the objective into a sequence of KL-proximal scalar-reward subproblems. We establish concavity, relative smoothness, and convergence guarantees for the resulting distributional procedure. Across therapeutic-peptide and molecular design tasks spanning discrete diffusion and continuous flow-matching models, HVFT achieves state-of-the-art expected hypervolume and trade-off coverage.

Figures & tables

Panel of Figure 1 — Figure 1: HVFT for multi-objective generative fine-tuning. (a) HVFT iteratively constructs a scalar reward from the current distribution $p_{k}$ and fine-tunes the generator, shifting the pushforward distribution $\mathbf{r}_{\#}p_{k}$ toward high-quality, complementary trade-offs. (b) The gray region is the hypervolume dominated relative to $\mathbf{r}^{\mathrm{ref}}$ . Hatching shows the marginal contributions of $x_{a}$ and $x_{b}$ , while the dominated candidate $x_{c}$ contributes zero. HVFT estimates these contributions against background sets from $p_{k}$ to define the next fine-tuning reward.

		Objective-space metrics		Design-space metrics
Task	Method	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
GLAST	Base	$2.78\pm 0.04$	$1.61\pm 0.01$	$\underline{60.35\pm 0.14}$	$\underline{401.50\pm 0.28}$	$0.321\pm 0.007$
	PepTune	$2.79\pm 0.07$	$1.59\pm 0.03$	$39.78\pm 1.05$	$134.95\pm 5.87$	$\mathbf{0.603\pm 0.028}$
	TR2D2	$7.78\pm 1.06$	$6.79\pm 1.41$	$9.08\pm 6.24$	$136.64\pm 74.24$	$0.249\pm 0.031$
	TR2D2 ++	$\underline{8.56\pm 0.86}$	$\underline{7.68\pm 0.77}$	$4.13\pm 1.28$	$89.43\pm 28.69$	$0.233\pm 0.010$
	HVFT	$\mathbf{11.67\pm 0.27}$	$\mathbf{10.56\pm 0.25}$	$\mathbf{64.78\pm 7.34}$	$\mathbf{465.45\pm 15.71}$	$\underline{0.336\pm 0.024}$

Table 1: Objective-space and design-space metrics for three therapeutic peptide design tasks. Bold and underlined values indicate the highest and second-highest means, within each task. HVFT achieves the highest hypervolume across the three tasks, as well as the highest design-space diversity and coverage.

Figure 2: Pareto sets induced by 1,000 samples from TR2D2 ( Tang et al., 2025b ) , a state-of-the-art discrete diffusion fine-tuning algorithm, and HVFT , on the GLAST task. HVFT achieves broader objective-space coverage and higher individual sample quality than TR2D2 .

Figure 3: Best four-point subset by hypervolume, selected from 100 samples of DiffusionBlend and HVFT on the Topology task. HVFT ’s samples achieve better coverage of high-quality trade-offs in objective-space.

		Objective-space metrics		Design-space metrics
Task	Method	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
Topology	FlowMol	$3.116\pm 0.014$	$2.665\pm 0.007$	$\mathbf{266.1\pm 4.9}$	$\mathbf{507.1\pm 6.4}$	$0.705\pm 0.008$
	DiffusionNFT	$3.140\pm 0.004$	$\underline{2.754\pm 0.005}$	$128.8\pm 2.7$	$312.8\pm 8.7$	$\mathbf{0.952\pm 0.009}$
	DiffusionBlend	$\underline{3.337\pm 0.024}$	$2.750\pm 0.031$	$150.3\pm 18.8$	$345.8\pm 30.6$	$0.674\pm 0.132$
	HVFT	$\mathbf{3.416\pm 0.005}$	$\mathbf{3.247\pm 0.004}$	$\underline{154.9\pm 9.0}$	$\underline{351.6\pm 11.1}$	$\underline{0.734\pm 0.007}$
Geometry	FlowMol	$4.279\pm 0.024$	$3.655\pm 0.010$	$\mathbf{91.4\pm 2.3}$	$\mathbf{331.6\pm 7.0}$	$0.483\pm 0.009$

Table 2: Objective- and design-space metrics for the small-molecule design tasks. Bold and underlined values indicate the highest and second-highest means, respectively, within each task. HVFT achieves the highest hypervolume among state-of-the-art methods while preserving diversity and coverage.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Execution time (hours)
TR2-D2	$12.384\pm 1.114$
HVFT	$10.734\pm 0.310$
DiffusionNFT	$6.399\pm 0.069$
HVFT	$7.204\pm 0.082$

Appendix

Table 3: Average execution time in hours. The first block aggregates results across the three peptide tasks. The second block aggregates runs for the Geometry task. The performance of these runs is reported in Section 4 . HVFT ’s execution time remains comparable to its inner solver counterpart.

Property	Binding Affinity	Permeability	Nonfouling	Solubility	Hemolysis
Reference Point	4.5	-10.0	0.0	0.0	0.0

Appendix

Table 4: Reference points used for hypervolume computation across all peptides tasks.

Hyperparameter	Name in Tang et al. (2025b)	Value
Num. of Replicates for Weighted Denoising Cross-Entropy	$R$	16
Buffer Size	$B$	20
Sequence Length	$L_{\mathrm{seq}}$	200
Num. of Children	$M$	50
Num. of Decoding Steps	$N_{\mathrm{steps}}$	128
Num. of MCTS Iterations	$N_{\mathrm{iter}}$	20

Appendix

Table 5: Hyperparameters for TR2-D2 baselines and HVFT inner-loop finetuning.

Property	QED	SA	Dipole Moment
Reference Point	-1.0	-1.0	-1.0

Appendix

Table 6: Reference points used for hypervolume computation for the Topology and Geometry tasks.

Hyperparameter	Name in Zheng et al. (2026)	Value
Guidance Strength	$\beta$	50
KL Regularization Coefficient	Not in paper, but in official implementation	0.5
Learning Rate	–	1e-5

Appendix

Table 7: Additional hyperparameters for DiffusionNFT finetuning.

		Objective-space metrics		Design-space metrics
Task	Model / $n$	100-Hypervolume	10-Hypervolume	Diversity	Coverage	Valid fraction
TfR	Base	$2.93\pm 0.04$	$1.69\pm 0.01$	$60.35\pm 0.14$	$401.50\pm 0.28$	$\mathbf{0.321\pm 0.007}$
	$10$	$11.57\pm 0.16$	$10.15\pm 0.18$	$75.89\pm 9.42$	$\mathbf{485.80\pm 17.27}$	$\underline{0.250\pm 0.021}$
	$50$	$\underline{12.231\pm 0.153}$	$\underline{10.589\pm 0.226}$	$\underline{93.2\pm 3.0}$	$437.9\pm 62.9$	$0.210\pm 0.078$
	$150$	$\mathbf{13.757\pm 1.721}$	$\mathbf{11.107\pm 0.891}$	$\mathbf{107.3\pm 8.6}$	$\underline{454.1\pm 43.6}$	$0.235\pm 0.114$
Topology	FlowMol	$3.116\pm 0.014$	$2.665\pm 0.007$	$\underline{266.1\pm 4.9}$	$\underline{507.1\pm 6.4}$	$0.705\pm 0.008$

Appendix

Table 8: Effect of

n

on objective- and design-space metrics for the TfR and topology tasks, with the pretrained base model as a reference. Values are means

\pm

standard errors across three runs. Boldface and underlining indicate the highest and second-highest means within each task, respectively.

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 528 papers

Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design
May 28, 2026Sven Gutjahr, Riccardo De Santi, Luca Schaufelberger +2Constrained Generative Modeling Molecular Optimization
Department of Computer Science, ETH Zurich · ETH AI Center · NCCR Catalysis, Switzerland +1
Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models
Sep 30, 2026Yuan Lu, Esha Singh, Yi-An Ma +1Diffusion Model Guidance Multi-Objective Optimization
Halıcıoğlu Data Science Institute, UC San Diego · Department of Computer Science & Engineering, UC San Diego
Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation
Feb 25, 2026Seokwon Yoon, Youngbin Choi, Seunghyuk Cho +3Generative Flow Networks Generative Modeling
Department of Computer Science & Engineering, POSTECH, South Korea · Graduate School of Artificial Intelligence, POSTECH, South Korea

May 28, 2026J/K move · Enter open · S save

Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design

Sven Gutjahr, Riccardo De Santi, Luca Schaufelberger, Kjell Jorner+1

Department of Computer Science, ETH Zurich · ETH AI Center · NCCR Catalysis, Switzerland+1

Constrained Generative Modeling Molecular Optimization Generative Modeling Flow-Based Generative Modeling

Open paper AskPDF arXiv

Adapting generative foundation models, in particular diffusion and flow models, to optimize given reward functions (e.g., binding affinity) while satisfying constraints (e.g., molecular synthesizability) is fundamental for their adoption in real-world scientific discovery applications such as molecular design or protein engineering. While recent works have introduced scalable methods for reward-guided fine-tuning of such models via reinforcement learning and control schemes, it remains an open problem how to algorithmically trade-off reward maximization and constraint satisfaction in a reliable and predictable manner. Motivated by this challenge, we first present a rigorous framework for Constrained Generative Optimization, which brings an optimization viewpoint to the introduced adaptation problem and retrieves the relevant task of constrained generation as a sub-case. Then, we introduce Constrained Flow Optimization (CFO), an algorithm that automatically and provably balances reward maximization and constraint satisfaction by reducing the original problem to sequential fine-tuning via established, scalable methods. We provide convergence guarantees for constrained generative optimization and constrained generation via CFO. Ultimately, we present an experimental evaluation of CFO on both synthetic, yet illustrative, settings, and a molecular design task. Across these evaluations, CFO achieves consistent increases in reward while ensuring high constraint satisfaction, showcasing its practical utility for constrained generative optimization.

Abstract

Generative Modeling

Explore similar work

Constrained Flow Optimization via Sequential Fine Tuning for Molecular Design

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation