An Interactive Evolutionary Multiobjective Optimization Algorithm Based on Interpretable Active Preference Learning
Authors: Thomas Quadt, Boris Čule, Roy Lindelauf, Herman Monsuur, Mark Voskuijl
Organizations: Department of Intelligent Systems, Tilburg University, Tilburg, NL · Data Science Center of Excellence, Faculty of Military Sciences, Breda, NL
Many decision-making tasks are inherently multi-objective. Interactive evolutionary multi-objective optimization algorithms (iEMOAs) are particularly well-suited to help the decision-maker find their most preferred solution. However, many of these algorithms employ black-box preference models. For high-stakes decision-making tasks, like healthcare and defense, a high level of transparency is needed, for instance to support accountability. We propose an interpretable iEMOA which uses active learning to efficiently learn a tree-based preference model directly from pairwise comparisons. The survival operator of NSGA-II and NSGA-III is extended with the learned preference model to guide the algorithm towards the region of interest. An extensive experimental study compares the proposed method to a state-of-the-art black-box model and a state-of-the-art interpretable model. Results indicate that our proposed model outperforms the alternatives in terms of finding the most preferred solution at the cost of a slightly worse average final population. Finally, a case study on a real-life engineering problem illustrates how inspecting the learned model reveals which objectives drive the decision-maker's preferences.
Figures & tables
Algorithm
Year
EA Type
Preference Model
Interpretable
Preference Information
Question Selection
IEM [ 26 ]
2003
Dominance
Linear Weighted Sum UF
High
PCs
Balanced
PI-NSGA-II-UF [ 27 ]
2010
Dominance
Polynomial UF
Medium
Ranking
Diverse Set
BC-EMOA [ 28 ]
2010
Dominance
Support Vector Machine UF
Low
Ranking/PCs
Greedy
FGKKMW* [ 29 ]
2010
Dominance
Quasi-concave UF
Low
Polarized Solution
Random
iTDEA [ 30 ]
2010
Dominance
Preference Region UF
Medium
Polarized Solution
Greedy
iMOEA/D [ 31 ]
2011
Decomposition
Preference Region UF
Medium
Polarized Solution
Random
TABLE I: Summary of Interactive Evolutionary Multi-objective Algorithms utilizing indirect preference judgments. (UF = Utility Function, PCs = Pairwise Comparisons *-FGKKMW are the initials of the author and PBO-R-NSGA-III is not an official name but a descriptive label reflecting the approach).
Fig. 1: Preferential Bayesian Optimization process of learning a decision tree as preference model. Taken from [ 22 ] .
Fig. 2: Decision tree learned from pairwise comparisons {xmid≻xleft,xmid≻xright,xleft≻xright} , where xleft≈(0.2,0.55) , xmid≈(0.40,0.37) , xright≈(0.8,0.11) and its resulting partitioning and ranking of the objective space of ZDT1.
Fig. 3: High-level workflow of DT-PBO-NSGA.
Fig. 4: Survival operator of DT-PBO-NSGA. Figure adapted from [ 18 ] .
Fig. 5: DT-PBO-NSGA’s population after each optimization stage. The black arrows indicate which pairwise comparison is asked. Specifically the point at the end of the arrow is preferred over the point at the start of the arrow.
Test Problem
dv
k
l
G
ZDT1-4
30
1000
ZDT6
10
1000
DTLZ1
M+4
500+100(M−2)
DTLZ2
M+9
250+50(M−2)
DTLZ3
M+9
1250+250(M−2)
DTLZ4
M+9
300+100(M−2)
TABLE II: Problem settings of each test problem used in the experimental evaluation. dv is the number of decision variables, k is the number of position-related variables and l is the number of distance-related variables. G is the number of generations before the first preference elicitation stage, and M is the number of objectives.
Competitor
D
Average utility
Best-Point utility
(W/T/L)
(W/T/L)
DT-EMOA
2
21/4/0
25/0/0
3
16/46/18
71/8/1
4
13/32/35
70/9/1
5
14/25/41
72/6/2
PBO-R-NSGA-III
2
7/14/4
5/20/0
TABLE III: Pairwise significance testing of DT-PBO-NSGA against competing methods for average utility and best-point utility. W/T/L denotes the number of cases where DT-PBO-NSGA significantly outperforms the competitor, shows no significant difference, or is significantly outperformed, respectively.
Fig. 6: Scatter plot of each method’s average and best-point utility relative to NSGA. Results are aggregated across runs and dimensions by taking the median.
Fig. 7: Box plot of the Diagonal of the bounding box spanning the final population. A higher value indicates a larger final region of interest and vice versa.
DT-PBO-NSGA
PBO-R-NSGA-III
DT-EMOA
Utility function
Avg. rank
Best-point rank
Avg. rank
Best-point rank
Avg. rank
Best-point rank
Chebyshev
2.17
1.43
1.96
2.09
1.87
2.47
Stewart case 1
2.32
1.32
1.81
2.06
1.87
2.62
Stewart case 2
2.42
1.45
1.91
1.94
1.68
2.60
Stewart case 3
2.26
1.17
1.36
1.91
2.38
2.92
Stewart case 4
2.28
1.21
1.51
1.91
2.21
2.89
TABLE IV: Average ranks of the methods based on average utility and best-point utility across utility functions.
Fig. 8: 3D scatter plot of the final population identified by DT-PBO-NSGA, the true Pareto front and the decision-maker’s most preferred solution for the vehicle crashworthiness design problem.
Fig. 9: DT-PBO-NSGA’s learned preference model. The decision rules show the partitioning created by the tree and each leaf’s utility prediction (higher is better) is represented by a Gaussian with a mean and standard deviation. The best leaf contains more information: namely the number of individuals of the final population falling in that leaf and the region spanned by that population, including bar charts representing its size and location w.r.t. the Pareto front.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 10: Line plots showing each method’s robustness against noise. Results average all problems and utility functions. The error bars denote one standard deviation.
Fig. 11: Heat maps showing DT-PBO-NSGA’s performance in both best-point utility and average utility when varying both λ and noise σ .
Problem
M
DT-PBO-NSGA
PBO-R-NSGA-III
DT-EMOA
Median ↓
Std
Rank ↓
Median ↓
Std
Rank ↓
Median ↓
Std
Rank ↓
ZDT1
2
0.1396
0.1693
1.6760
0.1379
0.1648
1.5800
0.3178
0.2330
2.7440
ZDT2
2
0.2450
0.1799
1.8120
0.2398
0.1633
1.8600
0.3408
0.1923
2.3280
ZDT3
2
0.1657
0.1946
1.5280
0.1973
0.1723
2.1560
0.2465
0.2036
2.3160
ZDT4
2
0.1405
0.1733
1.7520
0.1460
0.1693
1.4840
0.3067
0.2365
2.7640
ZDT6
2
0.2120
0.1835
1.4920
0.2748
0.1936
1.9560
0.3019
0.1897
2.5520
Appendix
TABLE V: Average utility comparison on ZDT, DTLZ, and WFG benchmark problems. Lower median utility is better. For each row the best utility and rank are made bold.
Problem
M
DT-PBO-NSGA
PBO-R-NSGA-III
DT-EMOA
Median ↓
Std
Rank ↓
Median ↓
Std
Rank ↓
Median ↓
Std
Rank ↓
ZDT1
2
0.1228
0.1668
1.6040
0.1180
0.1642
1.4800
0.2673
0.2279
2.9160
ZDT2
2
0.2274
0.1785
1.6840
0.2319
0.1644
1.8400
0.3020
0.1951
2.4760
ZDT3
2
0.1493
0.1939
1.6120
0.1507
0.1801
1.6640
0.2218
0.2021
2.7240
ZDT4
2
0.1318
0.1711
1.6200
0.1307
0.1689
1.5520
0.2640
0.2368
2.8280
ZDT6
2
0.2044
0.1856
1.5360
0.2594
0.1901
1.7320
0.2931
0.1850
2.7320
Appendix
TABLE VI: Best-point utility comparison on ZDT, DTLZ, and WFG benchmark problems. Lower median utility is better. For each row the best utility and rank are made bold.
Fig. 12: Convergence plots for 3- and 5-dimensional DTLZ1, 2, and 7. Subfigure (a) shows the median average utility of the final population and subfigure (b) shows the median best-point utility of the final population. Results are averaged over 50 trials. Confidence intervals show one standard deviation.
Fig. 13: Final population plot for 3-dimensional DTLZ1, 2, and 7.
Fig. 14: Final population plot for 5-dimensional DTLZ1, 2, and 7. For DTLZ1 trial 2, DT-EMOA’s final population converges towards f5=185 . For legibility the axis limit is set to 2 .
Fig. 15: Utility values along ZDT1’s Pareto front for 5 randomly generated DMs per utility function type.
High-stakes decision-making involves navigating multiple competing objectives with expensive evaluations. For instance, in brachytherapy, clinicians must balance maximizing tumor coverage (e.g., an aspirational target or soft bound of >95% coverage) against strict organ dose limits (e.g., a non-negotiable hard bound of <601cGy to the bladder). Selecting Pareto-optimal solutions that match implicit preferences is challenging, as exhaustive Pareto frontier exploration is computationally and cognitively prohibitive, necessitating interactive frameworks to guide users. While decision-makers (DMs) often possess domain knowledge to narrow the search via such soft-hard bounds, current methods often lack systematic approaches to iteratively refine these multi-faceted preference structures. Furthermore, DMs often require confidence that they have not overlooked superior alternatives, a paramount necessity in high-stakes scenarios. We present Active-MoSH, an interactive local-global framework designed for this process. Its local component integrates probabilistic preference learning with an active sampling strategy to adaptively refine Pareto subsets while minimizing cognitive burden. To bolster decision confidence, Active-MoSH's global component, C-MoSH, leverages multi-objective sensitivity analysis to identify potentially overlooked, high-value points beyond immediate feedback. We demonstrate Active-MoSH's performance benefits through diverse synthetic and real-world applications. A high-stakes case study with real cervical cancer brachytherapy treatment plans and an image selection user study further validate our hypotheses regarding the framework's ability to improve convergence, enhance DM confidence, and provide expressive preference articulation.
Edward Chen, Sang T. Truong, Natalie Dullerud +2
Department of Computer Science, Stanford University, Stanford, California, USA
Interactive multi-objective optimization systems face a budget allocation dilemma: one can spend resources on expensive objective evaluations or on eliciting decision-maker preferences that identify the relevant region of the Pareto set. Moreover, preference elicitation itself spans modalities with different information content and cognitive burden, ranging from cheap, noisy pairwise preference statements (PS) to richer but costlier indifference adjustments (IA). We study cost-aware optimization under an unknown scalarization and introduce QUIVER (Query-Informed Value Estimation for Regret), a surrogate-assisted evolutionary multi-objective optimizer that adaptively chooses between objective evaluations and heterogeneous preference queries. At each step, QUIVER selects the next action by maximizing the expected decision-quality improvement per unit total cost. Across DTLZ and WFG benchmarks under synthetic decision-maker models, QUIVER achieves the lowest final utility regret on challenging WFG problems (utility regret of 2.14 on WFG4, 2.82 on WFG9: a 25% improvement over baselines), outperforming all single-modality baselines. We analyze how the optimal mix of PS and IA adapts to problem difficulty: on easy problems (DTLZ2), QUIVER selects 80% PS queries; on hard problems (WFG9), it shifts to 35% IA queries. This adaptive modality selection demonstrates cost-aware preference learning in action.
Florian A. D. Burnat
University of Warwick Warwick Business School Coventry, England, UK
Multi-objective optimization models are often deployed in settings where decision makers (DMs) must balance some conflicting criteria. In practice, however, stakeholders must translate their qualitative preferences between different criteria into numerical weights, priorities, or reference points required by optimization methods, which can be difficult to specify and interpret. This paper proposes ParetoChat, an LLM (large language model)-based expert-agent interface for interactive multi-objective optimization. The agent translates natural-language user input into structured scalarization parameters, invokes an optimization system, presents candidate solutions, and iteratively elicits feedback. We describe a multi-agent architecture that separates input processing, multi-criteria decision-making (MCDM) expertise, solution presentation, and feedback processing. We then present a simulation study using an LLM-based DM agent and benchmark ParetoChat against interactive methods implemented in DESDEO, including reference-point and NIMBUS methods. The computational experiments on multi-objective binary knapsack and set-partitioning instances show that ParetoChat can guide the search efficiently toward solutions that match the DM's preferences. An application to historical courier-order assignment instances illustrates how users can explore objective extremes and intermediate trade-offs through natural-language requests. The results indicate that LLM agents may provide a practical interaction layer between business stakeholders and optimization tools.