An Interactive Evolutionary Multiobjective Optimization Algorithm Based on Interpretable Active Preference Learning

Thomas Quadt; Boris Čule; Roy Lindelauf; Herman Monsuur; Mark Voskuijl

cs.NEOct 26, 2026

An Interactive Evolutionary Multiobjective Optimization Algorithm Based on Interpretable Active Preference Learning

Authors: Thomas Quadt, Boris Čule, Roy Lindelauf, Herman Monsuur, Mark Voskuijl

Organizations: Department of Intelligent Systems, Tilburg University, Tilburg, NL · Data Science Center of Excellence, Faculty of Military Sciences, Breda, NL

Abstract

Many decision-making tasks are inherently multi-objective. Interactive evolutionary multi-objective optimization algorithms (iEMOAs) are particularly well-suited to help the decision-maker find their most preferred solution. However, many of these algorithms employ black-box preference models. For high-stakes decision-making tasks, like healthcare and defense, a high level of transparency is needed, for instance to support accountability. We propose an interpretable iEMOA which uses active learning to efficiently learn a tree-based preference model directly from pairwise comparisons. The survival operator of NSGA-II and NSGA-III is extended with the learned preference model to guide the algorithm towards the region of interest. An extensive experimental study compares the proposed method to a state-of-the-art black-box model and a state-of-the-art interpretable model. Results indicate that our proposed model outperforms the alternatives in terms of finding the most preferred solution at the cost of a slightly worse average final population. Finally, a case study on a real-life engineering problem illustrates how inspecting the learned model reveals which objectives drive the decision-maker's preferences.

Figures & tables

Algorithm	Year	EA Type	Preference Model	Interpretable	Preference Information	Question Selection
IEM [ 26 ]	2003	Dominance	Linear Weighted Sum UF	High	PCs	Balanced
PI-NSGA-II-UF [ 27 ]	2010	Dominance	Polynomial UF	Medium	Ranking	Diverse Set
BC-EMOA [ 28 ]	2010	Dominance	Support Vector Machine UF	Low	Ranking/PCs	Greedy
FGKKMW* [ 29 ]	2010	Dominance	Quasi-concave UF	Low	Polarized Solution	Random
iTDEA [ 30 ]	2010	Dominance	Preference Region UF	Medium	Polarized Solution	Greedy
iMOEA/D [ 31 ]	2011	Decomposition	Preference Region UF	Medium	Polarized Solution	Random

TABLE I: Summary of Interactive Evolutionary Multi-objective Algorithms utilizing indirect preference judgments. (UF = Utility Function, PCs = Pairwise Comparisons *-FGKKMW are the initials of the author and PBO-R-NSGA-III is not an official name but a descriptive label reflecting the approach).

Fig. 1: Preferential Bayesian Optimization process of learning a decision tree as preference model. Taken from [ 22 ] .

Fig. 2: Decision tree learned from pairwise comparisons $\{\mathbf{x}_{mid}\succ\mathbf{x}_{left},\mathbf{x}_{mid}\succ\mathbf{x}_{right},\mathbf{x}_{left}\succ\mathbf{x}_{right}\}$ , where $\mathbf{x}_{left}\approx(0.2,0.55)$ , $\mathbf{x}_{mid}\approx(0.40,0.37)$ , $\mathbf{x}_{right}\approx(0.8,0.11)$ and its resulting partitioning and ranking of the objective space of ZDT1.

Fig. 3: High-level workflow of DT-PBO-NSGA.

Fig. 4: Survival operator of DT-PBO-NSGA. Figure adapted from [ 18 ] .

Fig. 5: DT-PBO-NSGA’s population after each optimization stage. The black arrows indicate which pairwise comparison is asked. Specifically the point at the end of the arrow is preferred over the point at the start of the arrow.

Test Problem	$dv$	$G$
ZDT1-4	$30$	$1000$
ZDT6	$10$	$1000$
DTLZ1	$M+4$	$500+100(M-2)$
DTLZ2	$M+9$	$250+50(M-2)$
DTLZ3	$M+9$	$1250+250(M-2)$
DTLZ4	$M+9$	$300+100(M-2)$

TABLE II: Problem settings of each test problem used in the experimental evaluation.

dv

is the number of decision variables,

k

is the number of position-related variables and

l

is the number of distance-related variables.

G

is the number of generations before the first preference elicitation stage, and

M

is the number of objectives.

Competitor	$D$	Average utility	Best-Point utility
		(W/T/L)	(W/T/L)
DT-EMOA	2	21/4/0	25/0/0
	3	16/46/18	71/8/1
	4	13/32/35	70/9/1
	5	14/25/41	72/6/2
PBO-R-NSGA-III	2	7/14/4	5/20/0

TABLE III: Pairwise significance testing of DT-PBO-NSGA against competing methods for average utility and best-point utility. W/T/L denotes the number of cases where DT-PBO-NSGA significantly outperforms the competitor, shows no significant difference, or is significantly outperformed, respectively.

Fig. 6: Scatter plot of each method’s average and best-point utility relative to NSGA. Results are aggregated across runs and dimensions by taking the median.

Fig. 7: Box plot of the Diagonal of the bounding box spanning the final population. A higher value indicates a larger final region of interest and vice versa.

	DT-PBO-NSGA		PBO-R-NSGA-III		DT-EMOA
Utility function	Avg. rank	Best-point rank	Avg. rank	Best-point rank	Avg. rank	Best-point rank
Chebyshev	2.17	1.43	1.96	2.09	1.87	2.47
Stewart case 1	2.32	1.32	1.81	2.06	1.87	2.62
Stewart case 2	2.42	1.45	1.91	1.94	1.68	2.60
Stewart case 3	2.26	1.17	1.36	1.91	2.38	2.92
Stewart case 4	2.28	1.21	1.51	1.91	2.21	2.89

TABLE IV: Average ranks of the methods based on average utility and best-point utility across utility functions.

Fig. 8: 3D scatter plot of the final population identified by DT-PBO-NSGA, the true Pareto front and the decision-maker’s most preferred solution for the vehicle crashworthiness design problem.

Fig. 9: DT-PBO-NSGA’s learned preference model. The decision rules show the partitioning created by the tree and each leaf’s utility prediction (higher is better) is represented by a Gaussian with a mean and standard deviation. The best leaf contains more information: namely the number of individuals of the final population falling in that leaf and the region spanned by that population, including bar charts representing its size and location w.r.t. the Pareto front.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 10: Line plots showing each method’s robustness against noise. Results average all problems and utility functions. The error bars denote one standard deviation.

Fig. 11: Heat maps showing DT-PBO-NSGA’s performance in both best-point utility and average utility when varying both $\lambda$ and noise $\sigma$ .

Problem	$M$	DT-PBO-NSGA			PBO-R-NSGA-III			DT-EMOA
Problem	$M$	Median $\downarrow$	Std	Rank $\downarrow$	Median $\downarrow$	Std	Rank $\downarrow$	Median $\downarrow$	Std	Rank $\downarrow$
ZDT1	2	0.1396	0.1693	1.6760	0.1379	0.1648	1.5800	0.3178	0.2330	2.7440
ZDT2	2	0.2450	0.1799	1.8120	0.2398	0.1633	1.8600	0.3408	0.1923	2.3280
ZDT3	2	0.1657	0.1946	1.5280	0.1973	0.1723	2.1560	0.2465	0.2036	2.3160
ZDT4	2	0.1405	0.1733	1.7520	0.1460	0.1693	1.4840	0.3067	0.2365	2.7640
ZDT6	2	0.2120	0.1835	1.4920	0.2748	0.1936	1.9560	0.3019	0.1897	2.5520

Appendix

TABLE V: Average utility comparison on ZDT, DTLZ, and WFG benchmark problems. Lower median utility is better. For each row the best utility and rank are made bold.

Problem	$M$	DT-PBO-NSGA			PBO-R-NSGA-III			DT-EMOA
Problem	$M$	Median $\downarrow$	Std	Rank $\downarrow$	Median $\downarrow$	Std	Rank $\downarrow$	Median $\downarrow$	Std	Rank $\downarrow$
ZDT1	2	0.1228	0.1668	1.6040	0.1180	0.1642	1.4800	0.2673	0.2279	2.9160
ZDT2	2	0.2274	0.1785	1.6840	0.2319	0.1644	1.8400	0.3020	0.1951	2.4760
ZDT3	2	0.1493	0.1939	1.6120	0.1507	0.1801	1.6640	0.2218	0.2021	2.7240
ZDT4	2	0.1318	0.1711	1.6200	0.1307	0.1689	1.5520	0.2640	0.2368	2.8280
ZDT6	2	0.2044	0.1856	1.5360	0.2594	0.1901	1.7320	0.2931	0.1850	2.7320

Appendix

TABLE VI: Best-point utility comparison on ZDT, DTLZ, and WFG benchmark problems. Lower median utility is better. For each row the best utility and rank are made bold.

Panel of Figure 18 — Fig. 12: Convergence plots for 3- and 5-dimensional DTLZ1, 2, and 7. Subfigure (a) shows the median average utility of the final population and subfigure (b) shows the median best-point utility of the final population. Results are averaged over 50 trials. Confidence intervals show one standard deviation.

Fig. 13: Final population plot for 3-dimensional DTLZ1, 2, and 7.

Fig. 14: Final population plot for 5-dimensional DTLZ1, 2, and 7. For DTLZ1 trial 2, DT-EMOA’s final population converges towards

f5=185

. For legibility the axis limit is set to

2

Fig. 15: Utility values along ZDT1’s Pareto front for 5 randomly generated DMs per utility function type.

Multi-Objective Evolutionary Optimization

Multi-Objective Evolutionary Optimization uses population-based search to find diverse Pareto-optimal solutions when objectives conflict.

See all 85 papers

Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds
Jun 27, 2025Edward Chen, Sang T. Truong, Natalie Dullerud +2Active Learning Multi-Criteria Decision Analysis
Department of Computer Science, Stanford University, Stanford, California, USA
QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization
May 5, 2026Florian A. D. BurnatSurrogate-Assisted Optimization Multi-Objective Evolutionary Optimization
University of Warwick Warwick Business School Coventry, England, UK
ParetoChat: An LLM-Based Interface for Interactive Multi-Objective Optimization
Oct 16, 2026Gokhan CeyhanMulti-Objective Optimization Multi-Criteria Decision Analysis
Just Eat Takeaway.com

Jun 27, 2025J/K move · Enter open · S save

Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds

Edward Chen, Sang T. Truong, Natalie Dullerud, Sanmi Koyejo+1

Department of Computer Science, Stanford University, Stanford, California, USA

Active Learning Multi-Criteria Decision Analysis Multi-Objective Optimization Preference Learning

Open paper AskPDF arXiv

High-stakes decision-making involves navigating multiple competing objectives with expensive evaluations. For instance, in brachytherapy, clinicians must balance maximizing tumor coverage (e.g., an aspirational target or soft bound of >95% coverage) against strict organ dose limits (e.g., a non-negotiable hard bound of <601cGy to the bladder). Selecting Pareto-optimal solutions that match implicit preferences is challenging, as exhaustive Pareto frontier exploration is computationally and cognitively prohibitive, necessitating interactive frameworks to guide users. While decision-makers (DMs) often possess domain knowledge to narrow the search via such soft-hard bounds, current methods often lack systematic approaches to iteratively refine these multi-faceted preference structures. Furthermore, DMs often require confidence that they have not overlooked superior alternatives, a paramount necessity in high-stakes scenarios. We present Active-MoSH, an interactive local-global framework designed for this process. Its local component integrates probabilistic preference learning with an active sampling strategy to adaptively refine Pareto subsets while minimizing cognitive burden. To bolster decision confidence, Active-MoSH's global component, C-MoSH, leverages multi-objective sensitivity analysis to identify potentially overlooked, high-value points beyond immediate feedback. We demonstrate Active-MoSH's performance benefits through diverse synthetic and real-world applications. A high-stakes case study with real cervical cancer brachytherapy treatment plans and an image selection user study further validate our hypotheses regarding the framework's ability to improve convergence, enhance DM confidence, and provide expressive preference articulation.

Abstract

Multi-Objective Evolutionary Optimization

Explore similar work

Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds

QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization

ParetoChat: An LLM-Based Interface for Interactive Multi-Objective Optimization