In-Context Preference-Conditioned Pareto Set Learning from Limited Offline Data

Yiming Yao; Liang Zhao; Qingfu Zhang

cs.AIOct 15, 2026

In-Context Preference-Conditioned Pareto Set Learning from Limited Offline Data

Authors: Yiming Yao, Liang Zhao, Qingfu Zhang

Organizations: City University of Hong Kong

Abstract

Multi-objective optimization involves diverse preferences over conflicting objectives. Rather than returning a finite set of trade-off solutions, preference-conditioned Pareto set learning seeks to map a specified preference directly to a corresponding Pareto optimal solution. Yet in many real-world scenarios, optimization must proceed offline without further objective evaluations, leaving only a limited static dataset that sparsely covers the decision and objective spaces. Existing methods either optimize separately for each preference or train a problem-specific Pareto solution model, requiring repeated search across preferences or repeated training across problems. We introduce in-context preference-conditioned Pareto set learning (ICPSL), a framework that amortizes both preference-wise solution search and problem-specific model construction. ICPSL pretrains a foundation-style hypernetwork on a large and diverse set of synthetic prior data. Given the offline observations of an unseen problem, the pretrained hypernetwork learns to generate a problem-specific, preference-conditioned Pareto solution model in a single forward pass. The generated model can then be reused to predict solutions for arbitrary preferences without problem-specific training or preference-wise search. Experiments on synthetic and real-world problems show that ICPSL produces high-quality Pareto solutions while providing substantial speedups in problem-specific solution-model construction.

Figures & tables

Figure 1: Comparison of offline preference-conditioned Pareto set learning paradigms. (a) The implicit approach performs preference-wise search over a problem-specific instance model. (b) The explicit approach trains a problem-specific MLP as the Pareto solution model through objective- or regression-based learning. (c) ICPSL uses a frozen foundation-style hypernetwork to generate an problem-specific MLP from offline observations in a single forward pass, which maps arbitrary preferences to solutions without problem-specific training or preference-wise search.

Algorithm	LHS	Sobol	Uniform	Overall	Wins	Top-3 cases
ParetoFlow	4.50	4.50	4.20	4.40	5/30	9/30
PSLR	3.70	3.40	3.80	3.63	2/30	18/30
PSLO	5.10	5.20	5.80	5.37	2/30	5/30
DOMOO	4.30	3.20	5.10	4.20	5/30	10/30
ICPSL-MAP	5.00	4.80	4.20	4.67	1/30	7/30
ICPSL-CAM	2.70	3.70	2.60	3.00	0/30	23/30

Table 1: Average ranks, wins, and top-3 counts across all benchmark instances with LHS, Sobol, and uniform context distributions.

Figure 2: Runtime speedups of ICPSL over the baselines for problem-specific Pareto solution-model construction and preference-conditioned solution generation.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Implementation source	Code repository
ParetoFlow	Official ParetoFlow implementation	https://github.com/mila-iqia/ParetoFlow
PSLR	Implementation in PyTorch	https://github.com/pytorch/pytorch
PSLO	Official PSL-MOBO implementation	https://github.com/Xi-L/PSL-MOBO
DOMOO	Official DOMOO implementation	https://github.com/YaolinWen/DOMOO

Appendix

Table 2: Code sources used to implement the baseline methods.

Component	Configuration	Setting
Synthetic data	Decision dimension	$d\sim\mathcal{U}\{1,\ldots,4\}$
	Objectives dimension	$m\sim\mathcal{U}\{2,3\}$
	Context size	$N_{c}\sim\mathcal{U}\{8,\ldots,64\}$
	Query preferences	$Q=128$
	Sobol latent points	$K=128$
	Kernel family	RBF, Matérn- $3/2$ , or Matérn- $5/2$

Appendix

Table 3: Key configurations of the reported ICPSL model.

Benchmark	Category	Objective construction or design task	$d$	$m$
VLMOP1	Classical synthetic	VLMOP test function	$1$	$2$
VLMOP2	Classical synthetic	VLMOP test function	$3$	$2$
VLMOP3	Classical synthetic	VLMOP test function	$2$	$3$
AR	Composite synthetic	Ackley and Rosenbrock	$2$	$2$
DR	Composite synthetic	Dixon–Price and Rastrigin	$2$	$2$
ARA	Composite synthetic	Ackley and Rastrigin	$2$	$2$

Appendix

Table 4: Benchmark configurations for main experiments.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	1.195e+00(1.2e-03)	1.412e-01(2.6e-02)	9.452e-01(1.9e-01)	6.669e-01(1.2e-01)	1.084e+00(5.7e-02)	5.181e-01(1.2e-01)	1.854e-01(2.8e-02)	1.190e+00(1.0e-02)	1.179e+00(2.0e-02)	1.316e+00(1.1e-02)
PSLR	1.194e+00(2.6e-03)	2.015e-01(4.3e-02)	1.134e+00(1.5e-01)	6.608e-01(1.3e-01)	1.117e+00(5.5e-02)	5.560e-01(1.7e-01)	2.925e-01(2.9e-02)	1.182e+00(7.7e-03)	1.155e+00(1.9e-02)	1.302e+00(1.4e-02)
PSLO	1.199e+00(1.1e-06)	1.040e-01(4.8e-02)	9.881e-01(2.1e-01)	7.005e-01(1.3e-01)	1.090e+00(5.3e-02)	5.328e-01(1.9e-01)	2.624e-01(5.7e-02)	8.673e-01(3.1e-01)	9.071e-01(2.5e-01)	1.160e+00(2.4e-01)
DOMOO	1.199e+00(9.8e-06)	1.571e-01(6.7e-02)	9.928e-01(2.3e-01)	6.813e-01(1.4e-01)	9.841e-01(1.3e-01)	4.631e-01(1.2e-01)	2.803e-01(3.9e-02)	1.072e+00(2.0e-01)	1.161e+00(5.5e-02)	1.317e+00(2.0e-02)
ICPSL-MAP	1.197e+00(1.5e-03)	1.185e-01(3.4e-02)	9.683e-01(2.0e-01)	6.461e-01(1.3e-01)	1.073e+00(6.4e-02)	5.426e-01(1.3e-01)	2.306e-01(4.6e-02)	1.175e+00(1.2e-02)	1.166e+00(2.2e-02)	1.286e+00(2.0e-02)
ICPSL-CAM	1.197e+00(1.2e-03)	3.835e-01(5.0e-02)	1.054e+00(1.7e-01)	7.706e-01(1.2e-01)	1.135e+00(5.3e-02)	6.371e-01(1.3e-01)	3.012e-01(2.3e-02)	1.176e+00(1.1e-02)	1.162e+00(1.5e-02)	1.281e+00(1.9e-02)

Appendix

Table 5: Mean hypervolume (

\uparrow

) with 95% confidence intervals using LHS context over 10 independent runs.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	1.192e+00(3.2e-03)	1.672e-01(4.7e-02)	9.878e-01(1.9e-01)	5.787e-01(1.3e-01)	1.089e+00(4.0e-02)	4.686e-01(1.0e-01)	1.859e-01(2.2e-02)	1.196e+00(5.5e-03)	1.197e+00(7.4e-03)	1.318e+00(5.1e-03)
PSLR	1.197e+00(1.6e-03)	3.334e-01(4.8e-02)	1.196e+00(1.3e-01)	5.584e-01(1.2e-01)	1.117e+00(4.2e-02)	5.889e-01(1.3e-01)	3.116e-01(2.7e-02)	1.187e+00(3.9e-03)	1.115e+00(2.4e-02)	1.310e+00(1.2e-02)
PSLO	1.199e+00(8.3e-07)	1.771e-01(1.1e-01)	1.014e+00(1.9e-01)	5.500e-01(1.5e-01)	1.038e+00(6.4e-02)	5.425e-01(1.3e-01)	2.608e-01(4.5e-02)	1.005e+00(2.1e-01)	7.207e-01(2.7e-01)	1.054e+00(2.8e-01)
DOMOO	1.199e+00(8.7e-06)	3.293e-01(8.3e-02)	1.186e+00(8.6e-02)	6.968e-01(8.5e-02)	8.080e-01(2.3e-01)	5.148e-01(1.0e-01)	2.869e-01(2.8e-02)	1.205e+00(4.2e-03)	1.132e+00(6.9e-02)	1.327e+00(3.5e-03)
ICPSL-MAP	1.198e+00(2.4e-03)	1.496e-01(4.7e-02)	8.693e-01(2.2e-01)	6.083e-01(1.5e-01)	1.122e+00(3.0e-02)	5.377e-01(1.2e-01)	2.222e-01(4.4e-02)	1.178e+00(6.3e-03)	1.130e+00(2.5e-02)	1.304e+00(8.7e-03)
ICPSL-CAM	1.199e+00(3.6e-04)	4.087e-01(3.0e-02)	9.261e-01(2.0e-01)	7.639e-01(1.0e-01)	1.087e+00(6.1e-02)	5.710e-01(9.4e-02)	3.116e-01(2.4e-02)	1.183e+00(5.8e-03)	1.126e+00(1.8e-02)	1.301e+00(9.2e-03)

Appendix

Table 6: Mean hypervolume (

\uparrow

) with 95% confidence intervals using Sobol context over 10 independent runs.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	1.185e+00(9.5e-03)	1.679e-01(4.3e-02)	9.592e-01(1.9e-01)	6.662e-01(1.3e-01)	1.003e+00(5.5e-02)	6.411e-01(1.4e-01)	2.439e-01(2.8e-02)	1.190e+00(1.1e-02)	1.169e+00(4.3e-02)	1.312e+00(1.0e-02)
PSLR	1.196e+00(2.9e-03)	3.301e-01(3.7e-02)	1.043e+00(1.7e-01)	6.447e-01(1.4e-01)	1.081e+00(5.3e-02)	6.002e-01(1.5e-01)	2.794e-01(3.1e-02)	1.184e+00(8.6e-03)	1.089e+00(5.5e-02)	1.297e+00(1.6e-02)
PSLO	1.196e+00(6.0e-03)	6.835e-02(2.3e-02)	1.037e+00(1.7e-01)	7.567e-01(1.2e-01)	9.210e-01(9.2e-02)	5.642e-01(1.3e-01)	2.773e-01(3.2e-02)	8.950e-01(2.5e-01)	7.684e-01(2.7e-01)	1.045e+00(3.1e-01)
DOMOO	1.199e+00(3.3e-05)	2.162e-01(9.0e-02)	1.011e+00(1.9e-01)	5.823e-01(1.3e-01)	9.521e-01(1.7e-01)	4.719e-01(1.3e-01)	2.037e-01(4.6e-02)	1.114e+00(1.7e-01)	1.015e+00(2.0e-01)	1.330e+00(8.1e-04)
ICPSL-MAP	1.175e+00(3.4e-02)	1.107e-01(4.1e-02)	1.060e+00(1.6e-01)	7.417e-01(1.4e-01)	1.085e+00(3.7e-02)	6.496e-01(1.5e-01)	2.504e-01(3.3e-02)	1.161e+00(1.3e-02)	1.104e+00(5.3e-02)	1.289e+00(2.4e-02)
ICPSL-CAM	1.198e+00(1.2e-03)	3.182e-01(6.9e-02)	1.084e+00(1.5e-01)	7.952e-01(1.3e-01)	1.087e+00(2.8e-02)	6.646e-01(1.3e-01)	2.940e-01(2.3e-02)	1.172e+00(1.2e-02)	1.099e+00(5.1e-02)	1.288e+00(2.1e-02)

Appendix

Table 7: Mean hypervolume (

\uparrow

) with 95% confidence intervals using uniform context over 10 independent runs.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	7.60e+00/8.89e-01	7.25e+00/8.84e-01	7.24e+00/9.33e-01	8.47e+00/9.33e-01	7.19e+00/8.92e-01	7.14e+00/8.84e-01	8.63e+00/1.05e+00	7.19e+00/8.90e-01	7.15e+00/8.85e-01	7.25e+00/9.42e-01
PSLR	2.54e+00/8.53e-04	2.64e+00/5.57e-04	2.45e+00/6.17e-04	2.63e+00/8.67e-04	2.48e+00/1.23e-03	2.47e+00/1.36e-03	2.44e+00/1.28e-03	2.49e+00/8.12e-04	2.35e+00/1.25e-03	2.24e+00/1.84e-02
PSLO	3.07e+00/4.62e-04	3.36e+00/7.12e-04	3.12e+00/4.81e-04	4.58e+00/1.06e-03	4.83e+00/8.63e-04	4.62e+00/7.92e-04	5.20e+00/1.08e-03	4.95e+00/6.29e-04	4.19e+00/4.99e-04	5.23e+00/9.04e-04
DOMOO	6.53e+00/1.98e-04	5.63e+00/1.90e-04	7.89e+00/1.83e-04	7.13e+00/2.04e-04	8.69e+00/1.94e-04	8.83e+00/2.18e-04	1.39e+01/2.22e-04	9.63e+00/2.11e-04	9.22e+00/1.82e-04	1.30e+01/1.90e-04
ICPSL-MAP	4.91e-02 /1.17e-03	3.48e-03 /1.84e-03	3.51e-03 /1.42e-03	3.21e-03 /1.38e-03	3.53e-03 /1.41e-03	6.42e-02 /1.38e-03	3.47e-03 /1.42e-03	4.75e-02 /2.25e-03	3.60e-03 /2.81e-03	3.48e-03 /2.30e-03
ICPSL-CAM	4.91e-02 /6.74e-04	3.48e-03 /6.62e-04	3.51e-03 /6.66e-04	3.21e-03 /6.29e-04	3.53e-03 /6.55e-04	6.42e-02 /6.77e-04	3.47e-03 /6.55e-04	4.75e-02 /6.55e-04	3.60e-03 /6.53e-04	3.48e-03 /1.62e-03

Appendix

Table 8: Mean training / inference time in seconds (

\downarrow

) using LHS context over 10 independent runs.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	7.75e+00/9.20e-01	8.40e+00/9.00e-01	7.28e+00/9.39e-01	7.30e+00/8.97e-01	7.16e+00/8.86e-01	7.27e+00/8.98e-01	7.77e+00/1.02e+00	7.29e+00/9.17e-01	8.33e+00/9.01e-01	8.36e+00/9.68e-01
PSLR	2.49e+00/1.32e-03	2.73e+00/4.17e-04	2.43e+00/6.01e-04	2.57e+00/9.40e-04	2.38e+00/9.86e-04	2.31e+00/1.32e-03	2.57e+00/1.09e-03	2.12e+00/9.25e-04	2.41e+00/8.07e-04	2.30e+00/9.56e-03
PSLO	3.24e+00/9.60e-04	3.51e+00/2.68e-04	3.73e+00/8.59e-04	3.93e+00/5.54e-04	4.91e+00/8.45e-04	4.48e+00/5.25e-04	5.59e+00/5.07e-04	5.23e+00/4.74e-04	5.24e+00/5.66e-04	5.56e+00/6.58e-04
DOMOO	6.91e+00/2.04e-04	5.80e+00/2.01e-04	8.99e+00/2.00e-04	9.00e+00/2.29e-04	1.01e+01/2.05e-04	8.11e+00/2.21e-04	1.36e+01/1.83e-04	9.15e+00/3.54e-04	9.85e+00/1.99e-04	1.30e+01/1.85e-04
ICPSL-MAP	3.24e-03 /1.22e-03	3.62e-03 /1.85e-03	3.25e-03 /1.38e-03	3.32e-02 /1.43e-03	4.66e-02 /1.48e-03	3.51e-03 /1.43e-03	4.81e-02 /1.43e-03	3.59e-03 /2.25e-03	3.52e-03 /1.44e-03	3.49e-03 /2.22e-03
ICPSL-CAM	3.24e-03 /6.60e-04	3.62e-03 /2.79e-03	3.25e-03 /6.39e-04	3.32e-02 /6.71e-04	4.66e-02 /6.31e-04	3.51e-03 /6.81e-04	4.81e-02 /6.67e-04	3.59e-03 /6.58e-04	3.52e-03 /6.36e-04	3.49e-03 /6.77e-04

Appendix

Table 9: Mean training / inference time in seconds (

\downarrow

) using Sobol context over 10 independent runs.

Algorithm	VLMOP1	VLMOP2	VLMOP3	AR	DR	ARA	ABC	RE23	RE24	RE32
ParetoFlow	7.71e+00/9.27e-01	8.57e+00/9.36e-01	8.40e+00/1.01e+00	7.23e+00/9.14e-01	7.32e+00/9.24e-01	7.24e+00/9.10e-01	8.63e+00/1.01e+00	7.16e+00/8.82e-01	7.22e+00/8.81e-01	8.51e+00/9.53e-01
PSLR	2.57e+00/7.19e-04	2.75e+00/7.95e-04	2.44e+00/1.07e-02	2.59e+00/1.41e-03	2.53e+00/1.21e-03	2.43e+00/1.20e-03	2.55e+00/1.24e-03	2.50e+00/1.24e-03	2.37e+00/9.88e-04	2.33e+00/1.28e-03
PSLO	2.97e+00/7.03e-04	2.74e+00/4.93e-04	3.27e+00/8.91e-04	4.80e+00/7.41e-04	4.48e+00/8.14e-04	4.93e+00/4.09e-04	4.49e+00/4.96e-04	4.50e+00/7.83e-04	4.42e+00/7.95e-04	5.15e+00/5.95e-04
DOMOO	5.99e+00/2.56e-04	5.89e+00/2.02e-04	1.06e+01/1.96e-04	6.97e+00/1.85e-04	9.57e+00/1.93e-04	9.29e+00/1.79e-04	1.37e+01/1.83e-04	9.16e+00/2.02e-04	9.28e+00/1.80e-04	1.41e+01/2.16e-04
ICPSL-MAP	4.78e-02 /1.12e-03	3.65e-03 /1.78e-03	3.63e-03 /1.40e-03	3.31e-03 /1.44e-03	3.51e-03 /1.42e-03	4.81e-02 /1.47e-03	3.23e-03 /1.42e-03	4.77e-02 /3.52e-03	3.59e-03 /1.38e-03	3.52e-03 /2.26e-03
ICPSL-CAM	4.78e-02 /6.60e-04	3.65e-03 /6.85e-04	3.63e-03 /6.43e-04	3.31e-03 /6.71e-04	3.51e-03 /6.19e-04	4.81e-02 /6.37e-04	3.23e-03 /6.57e-04	4.77e-02 /6.17e-04	3.59e-03 /6.42e-04	3.52e-03 /6.62e-04

Appendix

Table 10: Mean training / inference time in seconds (

\downarrow

) using uniform context over 10 independent runs.

Figure 3: Evolution of hypervolume (higher is better) for ICPSL-MAP, ICPSL-CAM, and ICPSL-MM throughout pretraining under LHS, Sobol, and uniform context distributions.

Figure 4: Hypervolume (higher is better) of ICPSL-MAP, ICPSL-CAM, and ICPSL-MM with varying context sizes under LHS, Sobol, and uniform context distributions.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	2.399e-01(2.5e-02)	8.306e-01(4.3e-02)	1.687e-01(1.3e-02)	6.356e-01(1.1e-01)
PSLR	2.852e-01(4.9e-02)	9.703e-01(6.8e-02)	2.367e-01(4.0e-02)	4.873e-01(8.3e-02)
PSLO	2.397e-01(3.0e-02)	6.403e-01(1.0e-01)	1.441e-01(3.1e-02)	3.958e-01(5.5e-03)
DOMOO	2.193e-01(6.2e-02)	7.125e-01(1.3e-01)	1.312e-01(2.3e-02)	3.922e-01(6.0e-02)
ICPSL-MAP	3.065e-01(3.8e-02)	9.214e-01(3.7e-02)	1.736e-01(3.3e-02)	4.538e-01(4.7e-02)
ICPSL-CAM	4.945e-01(7.9e-02)	9.875e-01(4.2e-02)	4.593e-01(9.7e-02)	4.032e-01(3.9e-04)

Appendix

Table 11: Mean hypervolume (

\uparrow

) with 95% confidence intervals on OOD-dimensional benchmarks using LHS context over 10 independent runs.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	2.489e-01(2.2e-02)	8.671e-01(3.9e-02)	1.871e-01(2.3e-02)	6.481e-01(8.8e-02)
PSLR	3.081e-01(3.2e-02)	9.528e-01(5.0e-02)	2.803e-01(5.1e-02)	4.199e-01(3.1e-02)
PSLO	2.404e-01(5.6e-02)	7.160e-01(1.4e-01)	1.798e-01(3.1e-02)	3.535e-01(6.9e-02)
DOMOO	2.360e-01(5.2e-02)	5.169e-01(1.6e-01)	1.257e-01(1.6e-02)	3.537e-01(6.9e-02)
ICPSL-MAP	3.455e-01(6.4e-02)	9.219e-01(4.9e-02)	2.484e-01(5.4e-02)	4.349e-01(5.9e-02)
ICPSL-CAM	5.483e-01(9.0e-02)	9.883e-01(5.2e-02)	4.659e-01(5.1e-02)	4.031e-01(4.6e-04)

Appendix

Table 12: Mean hypervolume (

\uparrow

) with 95% confidence intervals on OOD-dimensional benchmarks using Sobol context over 10 independent runs.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	2.969e-01(4.8e-02)	8.684e-01(3.1e-02)	2.104e-01(4.2e-02)	6.231e-01(1.0e-01)
PSLR	3.537e-01(7.5e-02)	9.302e-01(6.1e-02)	2.809e-01(5.6e-02)	4.888e-01(8.9e-02)
PSLO	2.365e-01(4.0e-02)	6.416e-01(1.1e-01)	1.562e-01(3.5e-02)	4.017e-01(1.6e-02)
DOMOO	1.441e-01(3.5e-02)	5.239e-01(1.7e-01)	1.310e-01(4.5e-02)	3.711e-01(7.6e-02)
ICPSL-MAP	3.364e-01(5.9e-02)	9.235e-01(5.3e-02)	2.185e-01(6.0e-02)	4.292e-01(2.8e-02)
ICPSL-CAM	5.001e-01(5.1e-02)	9.746e-01(4.9e-02)	4.214e-01(5.7e-02)	4.031e-01(5.3e-04)

Appendix

Table 13: Mean hypervolume (

\uparrow

) with 95% confidence intervals on OOD-dimensional benchmarks using uniform context over 10 independent runs.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	9.71e+00/5.56e+00	9.30e+00/5.27e+00	8.94e+00/4.64e+00	7.32e+00/9.88e-01
PSLR	4.77e-01/1.99e-04	5.65e-01/1.71e-04	5.44e-01/1.81e-04	6.26e-01/1.93e-04
PSLO	1.58e+00/1.76e-04	1.99e+00/1.78e-04	2.02e+00/1.87e-04	2.63e+00/1.74e-04
DOMOO	3.76e+00/1.63e-04	4.58e+00/1.61e-04	4.91e+00/1.73e-04	9.99e+00/1.64e-04
ICPSL-MAP	3.28e-03 /2.91e-03	3.22e-02 /5.67e-03	3.73e-03 /2.78e-03	4.67e-02 /2.18e-03
ICPSL-CAM	3.28e-03 /6.29e-04	3.22e-02 /6.27e-04	3.73e-03 /6.68e-04	4.67e-02 /6.63e-04

Appendix

Table 14: Mean training / inference time in seconds (

\downarrow

) on OOD-dimensional benchmarks using LHS context over 10 independent runs.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	8.39e+00/3.00e+00	9.26e+00/4.92e+00	9.10e+00/5.37e+00	9.50e+00/5.40e+00
PSLR	5.40e-01/1.72e-04	5.32e-01/1.84e-04	5.51e-01/1.78e-04	5.09e-01/1.92e-04
PSLO	1.78e+00/2.24e-04	1.87e+00/1.76e-04	1.83e+00/1.85e-04	2.56e+00/2.67e-04
DOMOO	4.83e+00/1.96e-04	4.68e+00/1.65e-04	4.79e+00/1.63e-04	1.06e+01/1.71e-04
ICPSL-MAP	3.77e-03 /6.09e-03	3.29e-03 /2.84e-03	6.22e-03 /2.93e-03	3.61e-03 /2.22e-03
ICPSL-CAM	3.77e-03 /5.78e-03	3.29e-03 /6.56e-04	6.22e-03 /3.19e-03	3.61e-03 /6.45e-04

Appendix

Table 15: Mean training / inference time in seconds (

\downarrow

) on OOD-dimensional benchmarks using Sobol context over 10 independent runs.

Algorithm	AR (5d2m)	DR (5d2m)	ARA (5d2m)	DTLZ4 (4d4m)
ParetoFlow	9.29e+00/3.10e+00	9.16e+00/5.27e+00	9.46e+00/5.46e+00	9.70e+00/5.37e+00
PSLR	4.59e-01/1.75e-04	5.30e-01/1.82e-04	5.38e-01/1.84e-04	5.16e-01/1.84e-04
PSLO	2.01e+00/1.96e-04	1.97e+00/1.76e-04	1.88e+00/1.79e-04	2.54e+00/1.77e-04
DOMOO	4.63e+00/1.65e-04	4.68e+00/1.89e-04	4.95e+00/1.59e-04	9.55e+00/1.82e-04
ICPSL-MAP	3.17e-03 /2.95e-03	3.08e-03 /2.86e-03	3.33e-02 /2.84e-03	3.53e-03 /2.24e-03
ICPSL-CAM	3.17e-03 /6.40e-04	3.08e-03 /6.72e-04	3.33e-02 /6.38e-04	3.53e-03 /6.39e-04

Appendix

Table 16: Mean training / inference time in seconds (

\downarrow

) on OOD-dimensional benchmarks using uniform context over 10 independent runs.

Multi-Objective Optimization

Multi-objective optimization studies how to find and choose solutions that balance competing goals, constraints, and preferences.

See all 256 papers

A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets
May 9, 2026Akihiro Kubo, Kosuke Nakanishi, Shin IshiiMulti-Objective Reinforcement Learning Policy Optimization
International Research Center for Neurointelligence, The University of Tokyo, Tokyo, Japan · ATR Neural Information Analysis Laboratories, Kyoto, Japan · Department of Information Science, Kyoto University, Kyoto, Japan
AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization
Jul 30, 2026Shengda Gu, Kai Li, Xinyi Ke +3Automated Algorithm Discovery Pairwise Preference Evaluation
C2DL, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent AI Lab +1
Interactive Pareto navigation for deep multi-task learning
Jun 17, 2026Augustina C. Amakor, Konstantin Sonntag, Sebastian PeitzMulti-Task Learning Pareto Optimization
Department of Computer Science, TU Dortmund, Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence

May 9, 2026J/K move · Enter open · S save

A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets

Akihiro Kubo, Kosuke Nakanishi, Shin Ishii

International Research Center for Neurointelligence, The University of Tokyo, Tokyo, Japan · ATR Neural Information Analysis Laboratories, Kyoto, Japan · Department of Information Science, Kyoto University, Kyoto, Japan

Multi-Objective Reinforcement Learning Policy Optimization Policy Learning Multi-Objective Optimization

Open paper AskPDF arXiv

Preference-conditioned multi-objective reinforcement learning aims to learn a single policy that captures trade-offs across preferences, but under nonlinear scalarization the uniqueness and continuity of the preference-to-solution correspondence remain unclear. We study this problem in tabular multi-objective Markov decision processes (MDPs) using smooth Tchebycheff scalarization as a monotone utility. Under mild interior conditions on the preference set, we prove that each preference induces a unique Pareto-optimal return vector and that this vector depends Lipschitz-continuously on the preference, providing a principled foundation for preference sweeping toward dense Pareto-front coverage. To compute these targets, we formulate the problem over occupancy measures and derive Concave Mirror Descent Policy Iteration (CMDPI), which achieves an

O(1/k)

objective-suboptimality rate. We further show that each update is equivalent to solving a Kullback-Leibler-regularized MDP with the previous policy as reference, yielding a policy-iteration interpretation and finite-iterate policy continuity across preferences. We instantiate the update as a deep actor-critic algorithm preserving previous-policy regularization. On eight MO-Gymnasium tasks, it achieves the best average hypervolume rank among recent baselines and strong expected-utility performance. Continuous-control experiments indicate gains beyond the discrete-action setting.

Abstract

Multi-Objective Optimization

Explore similar work

A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets

AutoPref: Automatic Discovery of Task-Specific Preference Objectives for Neural Combinatorial Optimization

Interactive Pareto navigation for deep multi-task learning