cs.LGNov 13, 2026

ParetoM3^3: Learning on the Pareto Set under Preference Guidance via Min-Max-Min Optimization

Authors: Pei Tang, Songtao Lu

Organizations: Department of Computer Science and Engineering, Shun Hing Institute of Advanced Engineering, The Chinese University of Hong Kong, Hong Kong SAR

Abstract

Learning across multiple inherently conflicting objectives requires selecting desirable trade-offs from the Pareto set. In this work, we study preference-guided multi-objective optimization, where a predefined preference function is minimized over the weak Pareto set of multiple nonconvex objectives. Leveraging a merit function that vanishes on the weak Pareto set, we propose a penalty-based min-max-min formulation, termed the M3^3 problem, which unifies preference optimization and the enforcement of weak Pareto optimality in a single objective. We further develop ParetoM3^3, a single-loop first-order algorithm for computing stationary solutions of the resulting M3^3 problem. Under a local Kurdyka-Lojasiewicz assumption, we prove that ParetoM3^3 finds an εε-stationary point of the M3^3 problem that is also approximately weakly Pareto optimal for the original multi-objective problem when the penalty parameter is sufficiently large, with explicit non-asymptotic convergence rates. We also show that this stationarity notion recovers existing optimality criteria, including approximate preference stationarity and approximate Karush-Kuhn-Tucker conditions, under the same assumptions used in prior works. Experiments on a synthetic benchmark, image classification, regression, and math reasoning demonstrate that ParetoM3^3 reliably attains preferred weak Pareto optimal solutions while achieving strong practical performance. Code is available at https://github.com/tpgoodboy/ParetoM3.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets

    May 9, 2026Akihiro Kubo, Kosuke Nakanishi, Shin IshiiMulti-Objective Reinforcement LearningPolicy Optimization

  2. SURFACE: Preference Distribution Matching on Pareto Fronts via Scalarization Steering

    Nov 12, 2026Liuyuan Jiang, Prasanna Sattigeri, Lisha ChenMulti-Objective OptimizationPareto Optimization