Specializing Meta Multi-Objectivization as Baseline for Software Engineering Optimization
Authors: Chengjie Chuan, Pengzhou Chen, Tao Chen
Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, China, 610056. · IDEAS Lab, School of Computer Science, University of Birmingham, UK, B15 2TT.
Given the discrete nature of many software engineering (SE) optimization problems, trapping at local optima remains a crucial issue in the optimizer designs. Meta Multi-Objectivization (MMO) is an optimization model for overcoming local optima by introducing an extra, auxiliary objective. Yet, MMO is originally designed only for configuration tuning, leaving its suitability for other SE problems in doubt, particularly regarding the choice of the auxiliary objective. In this paper, we study the possibility and resolution for specializing MMO as a general baseline across a range of SE optimization problems, making it a fundamental counterpart for future SE optimizers that seek to mitigate the issue of local optima. We do so by providing both theoretical remit and empirical evidence: we analytically show how the choice of the auxiliary objective is crucial while demonstrating the effectiveness of MMO under nine SE optimization problems comprising 212 cases when the right one can be chosen, i.e., MMO is able to outperform the other state-of-the-art and general-purpose optimizers in 79% and 94% of the cases, respectively. Drawing on the theoretical understanding and empirical findings, we then propose L2RAC, a tool that automatically recommends the best auxiliary objective choice for a given SE problem/case, hence helping to specialize MMO therein. Through extensive experiments, we reveal that, in general, L2RAC can more accurately rank the choice of auxiliary objective for MMO under a case than other guesses by up to 60%, saving as much as 99% of the cost that would otherwise be required to conduct trial-and-error when using MMO.
Figures & tables
Fig. 1: Two SE optimization problems showing Pearson correlations between the target and the auxiliary objective (i.e., other properties of the software artifact), % improvement of MMO over PMO where a negative value means PMO is better.
Fig. 2: The original and MMO space of two different cases that show the implication of auxiliary objective
Fig. 4: Overview of L2RAC .
Problem
L2RAC
Top1
Top2
Rank
Mean ± Std
Rank
Mean ± Std
Rank
Mean ± Std
Next Release Problem (NRP)
1
0.924±0.209
2
0.765±0.147
3
0.739±0.101
Software Configuration Tuning (SCT)
1
0.693±0.253
2
0.570±0.255
2
0.548±0.226
Software Effort Estimation (SEE)
1
0.762±0.220
1
0.762±0.200
2
0.590±0.227
Software Defect Prediction (SDP)
1
0.811±0.294
1
0.811±0.304
2
0.775±0.172
Software Product Lines Testing (SPLT)
1
0.750±0.216
2
0.626±0.214
3
0.456±0.191
TABLE XII: Comparing L2RAC with the best top 2 auxiliary-objective choices for all cases/SE problems from our empirical study. The format follows Table VI .
Multi-objective Bayesian optimization (MOBO) maximizes M expensive black-box objectives with the goal of approximating the Pareto set, whose objective vectors cannot be improved in one component without sacrificing another. A common quality measure is hypervolume, the volume in objective space dominated by these vectors relative to a reference point. Expected hypervolume improvement (EHVI) directly targets this metric and is one-step Bayes-optimal, but its standard computation scales exponentially with M, making it prohibitively expensive for many-objective problems. We introduce random max-margin multi-objective Bayesian optimization (RaMMOBO), whose acquisition function lower-bounds EHVI while requiring only linear cost in M to evaluate. RaMMOBO measures the volume of a max-margin box contained in a candidate's hypervolume-improvement region and averages over random reweightings of the objective space, yielding a utility that is strictly Pareto-monotone on hypervolume-improving outcomes. It naturally handles noisy observations and different probabilistic surrogates and supports efficient gradient-based optimization. Across synthetic and real-world benchmarks, RaMMOBO matches or outperforms state-of-the-art methods, with largest gains at higher M, with acquisition optimization up to two orders of magnitude faster than qLogNEHVI. Our implementation is available at https://anonymous.4open.science/r/rammobo.
Thomas Wedenig, Paul Plantosar, Robert Peharz
Institute of Machine Learning and Neural Computation, TU Graz, Austria · Graz Center for Machine Learning (GraML), Austria
Existing Meta-Black-Box Optimization (MetaBBO) methods focus on how to search when controlling optimizers, but largely overlook where to search. We propose MetaSG-SAEA, a bi-level MetaBBO framework for expensive constrained multi-objective optimization problems (ECMOPs), in which a meta-policy provides search guidance to the low-level Surrogate-Assisted Evolutionary Algorithm (SAEA). To achieve this, we introduce Max-Min Constraint-Calibrated Inequality (MM-CCI), a compact, problem-agnostic region abstraction that maps heterogeneous constraint evaluations to an ordered scalar level; we further provide a theoretical analysis of its fundamental properties. Building on this region abstraction, we adopt diffusion-based population initialization to translate the meta-policy's region-level guidance into solution-level priors for the SAEA. To make MetaSG-SAEA scalable, we construct an attention-based state representation across varying problem dimensions, population sizes, and numbers of objectives and constraints. Experimental results demonstrate that MetaSG-SAEA outperforms state-of-the-art baselines across diverse benchmarks and exhibits the ability to generalize across problem distributions.
Yukun Du, Haiyue Yu, Jiang Jiang +5
National University of Defense Technology, Changsha, Hunan, China.
Agent skills are increasingly used to configure coding agents for software engineering (SE) tasks, yet current practice treats them as static, hand-crafted assets, or evolved on pass rate alone. This is insufficient: a skill can improve task success while substantially raising token cost, or introducing misleading guidance. We argue that SE agent skill bundles can be treated as multi-objective search objects and present SkillMOO, a framework that evolves skill bundles through LLM-proposed edits and NSGA-II Pareto selection on pass rate and inference cost. Evaluated across all 16 SkillsBench SE tasks, SkillMOO achieves the top pass rate rank on 11 of 12 non-zero-pass tasks while achieving cost reductions of up to 31.7% over static bundles, with pass rate gains up to 21 percentage points. Analysis of 38 skill edits shows that pruning and substitution dominate successful operations, offering actionable principles for skill bundle design. Thereby, the current practice of deploying skills without cost-aware validation leaves better skill configurations unexplored, motivating a new class of cost-aware, search-based skill engineering.
Jingzhi Gong, Ruizhen Gu, Zhiwei Fei +7
King’s College London, London, United Kingdom · Queen’s University Belfast, Belfast, United Kingdom · Nanjing University, Nanjing, China +3