Multi-Objective Optimization

Overview

The field studies problems where improving one objective—such as accuracy, cost, safety, energy use, or fairness—can worsen another, and represents the alternatives as a Pareto frontier. Common approaches include evolutionary algorithms such as NSGA-II, Bayesian optimization for expensive evaluations, gradient-based methods for learning systems, and reinforcement-learning or generative methods for sequential and design tasks. Researchers also develop preference elicitation and constraint-handling methods so people can select useful trade-offs without always computing an entire frontier. A central tension is balancing broad, reliable coverage of feasible trade-offs against evaluation cost, while scalarization, objective scaling, and changing preferences can all affect which solutions are found.

Latest papers

287 papers23 papers in the last four weeks, down 42% on the four weeks before. 0.2% of all new papers.

Aug 19, 2026

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

EDA flow parameter tuning is critical for quality-of-results~(QoR), yet the parameter space is large, tightly coupled, and full evaluations are prohibitively expensive. Prior LLM-assisted tuners mainly use the LLM as an external proposer with transient working context; we instead present StateTune, which reformulates LLM-assisted EDA tuning as a closed-loop, state-carrying process. Its optimizer state is a typed, evidence-gated persistent optimization memory that is updated by every evaluation and shared between candidate generation and budget allocation. On top of this optimizer state, an expected hypervolume improvement (EHVI)-guided, runtime-aware promotion policy ranks quick-stage candidates by expected Pareto frontier gain per unit of runtime cost. Evaluated on a Cadence industrial flow across six benchmark blocks (two technology nodes ×\times three designs), against five baselines including LLM+retrieval-augmented generation (RAG) and preference-based Bayesian optimization (BO) tuners, StateTune achieves the strongest final hypervolume on all six benchmark blocks, showing a stable improvement in frontier quality across the full matrix; it also matches or surpasses the strongest baselines on worst negative slack (WNS), area, and power across the same set. Ablation shows persistent memory is the largest contributor: removing it costs 58.5% of the hypervolume. Dedicated analyses of evidence-gating sensitivity, memory poisoning, cross-design transfer, and three-seed reproducibility (CV,<<,7% on five of six blocks) further validate the memory design.
Aug 19, 2026

Multi-Objective Optimization Under Uncertainty of Part Quality in Fused Filament Fabrication

This work presents a data-driven methodology for multi-objective optimization under uncertainty of process parameters in the fused filament fabrication (FFF) process. The proposed approach optimizes the process parameters with the objectives of minimizing the geometric inaccuracy and maximizing the filament bond quality of the manufactured part. First, experiments are conducted to collect data pertaining to the part quality. Then, Bayesian neural network (BNN) models are constructed to predict the geometric inaccuracy and bond quality as functions of the process parameters. The BNN model captures the model uncertainty caused by the lack of knowledge about model parameters (neuron weights) and the input variability due to the intrinsic randomness in the input parameters. Using the stochastic predictions from these models, different robustness-based design optimization formulations are investigated, wherein process parameters such as nozzle temperature, nozzle speed, and layer thickness are optimized under uncertainty for different multi-objective scenarios. Epistemic uncertainty in the prediction model and the aleatory uncertainty in the input is considered in the optimization. Finally, Pareto surfaces are constructed to estimate the tradeoffs between the objectives. Both the BNN models and the effectiveness of the proposed optimization methodology are validated using the actual manufacturing of the parts.
Aug 17, 2026

GoalEvolve: From Handcrafted Algorithm Priors to Goal-Driven Evolution of Physical Design Algorithms

Physical design algorithms operate within tightly coupled, multi-stage optimization flows, where stage-local gains may vanish or induce downstream degradation. Existing program-evolution frameworks often rely on stage-local objectives or undifferentiated multi-metric feedback, which neither guarantee better final results nor identify which unmet requirement should guide the next iteration. We present GoalEvolve, a goal-driven framework that makes physical design algorithm evolution accountable for the final quality of results (QoR) of the complete flow. Given a multi-objective QoR target region, GoalEvolve converts unmet requirements into normalized target gaps, identifies the dominant bottleneck, and uses stage-resolved checkpoint evidence to locate the responsible stage. An LLM-based Teacher then narrows the search to a relevant algorithmic decision and source region, while parallel Student agents implement and validate hypotheses through full-flow evaluation. Local effects, optimization debt, and downstream retention are retained as mechanism evidence for subsequent evolution. Across eight ASAP7 designs, GoalEvolve improves post-route TNS by 30.67% on average and reduces leakage and dynamic power by 21.18% and 9.42% versus default OpenROAD. Relative to commercial-tool goals, it closes 62.20% of the normalized power gap on power-dominant designs, surpasses the TNS goals on both timing-dominant designs, and closes 32.48% of the equal-weight timing-power gap on joint designs. Across all three designs evaluated against Codex goal mode under matched budgets, GoalEvolve further improves TNS by 26.46% while reducing leakage and dynamic power by 12.38% and 0.76%, respectively.
Aug 14, 2026

Multi-Objective Bayesian Optimization for Model Merging

Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge parameters is nevertheless difficult because downstream evaluations are expensive, gradients are unavailable, and source capabilities can conflict. We formulate merge-parameter selection as a black-box multi-objective optimization problem and introduce MOBO-Merge, a merge-operator agnostic framework that uses multi-objective Bayesian optimization to approximate the Pareto front under a limited evaluation budget. We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings using Linear, SLERP, TIES, and block-wise merge operators. On held-out benchmark partitions, MOBO-Merge obtains higher mean hypervolume than random search in 11 of 12 reported comparisons. The gain is small for one-dimensional Linear interpolation but substantially larger for several TIES, block-wise, and three-objective searches. No merge operator is uniformly best: TIES leads in three of four family-setting combinations, whereas Block-Linear 4x is strongest for the Llama three-model merge. These results show that multi-objective Bayesian optimization is valuable as a search layer for expressive merge parameterizations.
Aug 12, 2026

MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning

Multi-objective optimization (MOO) has demonstrated significant success in multi-task learning by mitigating task conflicts through gradient manipulation. However, most existing methods flatten model parameters into vectors and perform gradient manipulation under Euclidean geometry, thereby overlooking the matrix structure prevalent in modern architectures such as Transformers. In this paper, we show that gradient manipulation in Euclidean space does not generally yield the steepest descent direction under matrix geometry, potentially limiting optimization efficiency. Drawing from the theory of steepest descent for matrix-valued parameters, we propose MOON (Multi-Objective OrthoNormalized Updates), which performs gradient manipulation under spectral--nuclear norm geometry and uses the orthonormalized manipulated gradient for parameter updates. Theoretically, for smooth non-convex objectives, we establish convergence of the averaged Pareto-stationarity measure at rates of O(T−1/2)\mathcal{O}(T^{-1/2}) in the deterministic setting and O(T−1/4)\mathcal{O}(T^{-1/4}) under stochastic gradients. Empirical results across various benchmarks show that MOON consistently improves both optimization efficiency and final multi-task performance. Our code is available at https://github.com/KunlinLyu/MOON.
Aug 12, 2026

High-dimensional Multi-objective Bayesian Optimization with Learned Variable Interactions

Multi-objective Bayesian optimization (MOBO) is effective in identifying the Pareto fronts for expensive black-box problems. However, most current MOBO approaches are limited to low-dimensional decision space due to its exponential sampling complexity. This paper presents decision variable interaction analysis-based MOBO, ViaMOBO, a generic framework for expensive multi-objective problems with high-dimensional decision space. The key idea of ViaMOBO is that it utilizes a variable interaction analysis model to determine whether the decision space can be completely or partially divided, and then performs local Bayesian optimization in the divided decision subspaces. Through the variable analysis model, it can be derived whether the objectives in black-box problems are separable, partially separable, or non-separable based on the potential independent or interdependent relationships among decision variables without any strong assumptions. We compare ViaMOBO with the state-of-the-art MOBO methods on both synthetic and real-world benchmarks. The experimental results demonstrate that ViaMOBO outperforms other related MOBO baselines in approximating the Pareto front of high-dimensional expensive multi-objective problems.
Aug 11, 2026

Exploring new directions in enhancing the ACTS parameter optimization suite

Track seeding strongly affects both the quality and computational cost of charged-particle reconstruction, yet its many configuration parameters are commonly tuned through expert intuition and repeated trial and error. ACTS reduces this burden with an Optuna Tree-structured Parzen Estimator auto-tuner, but expensive evaluations, a restricted search space, and a scalarized objective can limit evaluation efficiency, exclude promising configurations, and obscure performance trade-offs. We investigate whether Bayesian optimization can address these limitations using ACTS with the Open Data Detector (ODD). Under identical search ranges and a common 100-trial budget, we compare Expected Improvement and Upper Confidence Bound with TPE and random search on the existing eight-parameter problem, extend the best-performing Bayesian method to fifteen parameters, and apply Expected Hypervolume Improvement to optimize efficiency, fake rate, duplicate rate, and runtime without fixed scalar weights. Candidate configurations are evaluated through the full ACTS reconstruction chain and validated on disjoint held-out events. The Bayesian acquisition methods identify strong configurations earlier than TPE, and their advantage persists in held-out validation. Expanding the search further improves performance, while multi-objective optimization reveals competitive non-dominated solutions spanning distinct trade-offs. These results indicate that Bayesian optimization can strengthen ACTS auto-tuning through efficient evaluation, broader parameter searches, and post-hoc expert selection among non-dominated alternatives.
Aug 10, 2026

OGG-FR: Orthogonal Gradient Gaming and Frequency Rectification for Unmanned Aerial Vehicle Infrared Image Super-Resolution

Unmanned aerial vehicle (UAV) infrared image super-resolution aims to recover weak thermal structures for deployment on resource-constrained platforms; lightweight models are therefore preferred, but multi-loss training can be unstable. A common strategy combines pixel-domain and frequency-domain objectives; however, low contrast, limited high-frequency content, and sensor-specific noise often make their gradients weakly aligned or conflicting. To address this optimization ambiguity, we propose Orthogonal Gradient Gaming and Frequency Rectification (OGG-FR), a plug-and-play optimization framework that decomposes the frequency gradient into a redundant parallel component and an orthogonal innovation component relative to the pixel gradient. In the conflict regime, OGG-FR computes a safe base gradient using the Multiple Gradient Descent Algorithm (MGDA) and adds a variance-rectified orthogonal innovation; in the compatible regime, it discards redundant parallel information and injects the orthogonal innovation according to a confidence score estimated from the high-frequency residual. Experimental results on the UAV thermal benchmark show broad gains under BI and BD degradations at ×4\times 4 and ×8\times 8 scales, while gradient analyses support the effectiveness of the proposed conflict-aware update rule.
Aug 7, 2026

Human-Centered Explainable AI for TinyML Edge Devices: A Pareto-Based Selection Framework with LLM-Guided Design

Edge Artificial Intelligence (Edge AI) enables the deployment of AI models directly on local edge devices, while such deployments are subject to strict resource constraints, particularly in clinical applications requiring local and timely inference. In such contexts, explainable artificial intelligence (XAI) can serve as a human-AI interface intended to support healthcare professionals' and patients' understanding of model predictions and informed decision-making. To fulfill this role, XAI method selection for TinyML deployments can be formulated as a human-centered multi-objective design problem that jointly considers qualitative stakeholder preferences, explanation quality, and proxy-based deployment cost. We propose a framework that integrates a large language model (LLM)-guided design interface that maps qualitative stakeholder preferences to candidate XAI methods, followed by deterministic feasibility filtering and Pareto-based optimization. The framework exposes trade-offs among explanation fidelity, stability, and proxy-based deployment cost while characterizing their implications for explanation quality and estimated deployment feasibility. A proof-of-concept evaluation on a skin lesion classification task illustrates how the framework systematically compares candidate XAI methods and identifies Pareto-efficient trade-offs. The present evaluation covers the computational selection stages, while physical MCU deployment and empirical human-expert validation remain outside the scope of this study.
Aug 6, 2026

A Special Point Skeleton Reconstruction Algorithm for Dynamic Multiobjective Optimization

To address the issue that existing dynamic multi-objective optimization algorithms mainly rely on individual migration or independent special point sampling after environmental changes, while failing to fully exploit the structural relationships among representative solutions, a Special Point Skeleton Reconstruction based Dynamic Multi-Objective Evolutionary Algorithm (SPSR-DMOEA) is proposed. First, the centroid, knee points, and extreme points are extracted from the Pareto optimal solution set of the current environment, and their positions in the new environment are adaptively predicted according to their movement velocities across consecutive environments. Subsequently, in the decision space, the centroid is connected with other anchor points, and a minimum spanning tree is constructed among the non-centroid anchor points, thereby establishing a prediction skeleton capable of describing the overall population structure. According to the lengths of the skeleton edges, the number of individuals allocated to each edge is determined proportionally. Candidate solutions are uniformly generated along each edge, and random orthogonal perturbations are introduced to expand the search region around the skeleton. Experimental results on the DF dynamic multi-objective benchmark suite demonstrate the effectiveness of the proposed method in dynamic tracking capability.
Aug 5, 2026

Active Learning Guided Design Space Refinement for Scalable Multi-Objective Bayesian Optimization in Materials Discovery

Advanced materials discovery increasingly relies on machine learning and Bayesian optimization to explore large discrete design spaces under limited evaluation budgets. However, conventional Bayesian optimization (BO) can become inefficient as candidate spaces grow, often evaluating low-value regions before reaching informative areas. We propose an active-learning (AL)-guided adaptive search-space refinement framework combined with multi-objective BO to accelerate materials optimization while preserving Pareto-relevant regions. We evaluate the approach on CH4/N2 separation in covalent-organic frameworks and pressure-vessel design with material-direction stress components and thickness objectives. Results show that the AL-guided refinement reduces the candidate space by approximately half while preserving more than 99 percent of the original hypervolume. The reduced-space strategy improves early convergence and cumulative Pareto-front discovery from the BO, demonstrating efficient large-scale materials optimization across constrained autonomous materials discovery settings.
Aug 5, 2026

Approximate Multi-Objective Search Under Rulebooks

Robotic planning often involves multiple objectives with complex priority relationships, such as safety, efficiency, and regulatory compliance. Rulebooks formalize these relationships, allowing partial ordering of objectives that generalizes both Pareto and lexicographic dominance. Computing the full set of rulebook-optimal solutions, however, is computationally expensive. To address this challenge, we introduce the concept of epsilon-rule-dominance, a principled notion of approximate dominance under rulebooks, and propose RApex, a best-first search algorithm that efficiently computes a compact set of epsilon-approximate rulebook-optimal solutions. RApex leverages dimensionality reduction, a technique used to speed up existing multi-objective search algorithms, while respecting rule hierarchies by maintaining separate closed sets and performing dominance checks over truncated and residual rule sets. We provide a formal analysis of RA*pex, proving that every rulebook-optimal solution is epsilon-rule-dominated (a generalization of approximate dominance we introduce) by at least one solution in the returned set. Empirical results demonstrate that our approach achieves computation times over two orders of magnitude faster than existing methods.
Aug 5, 2026

Cost-Aware Multi-Objective Bandits: Theory and Application to Budgeted LLM Configuration Evaluation

Large language model (LLM) configuration evaluation is challenging due to limited evaluation budgets, varying costs, and multiple competing objectives. In this paper, we formulate LLM configuration evaluation as a cost-aware multi-objective bandit problem, where each configuration evaluation incurs a configuration-dependent cost and yields a noisy vector-valued outcome. Under this framework, we study two fundamental problems: online configuration selection and Pareto configuration identification. For online configuration selection, we propose a hypervolume-based UCB algorithm that optimizes an optimistic hypervolume-per-cost index. We establish a budgeted regret bound of order O(∑i≠i⋆log⁡BΔi)O\bigl(\sum_{i\ne i^\star}\frac{\log B}{Δ_i}\bigr), where BB is the evaluation budget, i⋆i^\star is the optimal configuration in terms of hypervolume efficiency, and ΔiΔ_i is the corresponding efficiency gap of configuration ii. This bound retains the logarithmic budget dependence of classical single-objective budgeted bandits. For fixed-budget Pareto identification, we develop a cost-aware empirical gap elimination algorithm and prove that its error probability is of order O(exp⁡(−BHμ,c))O\bigl(\exp(-\frac{B}{H_{μ,c}})\bigr), where Hμ,cH_{μ,c} is a cost-aware Pareto identification complexity depending on configuration costs and Pareto classification gaps. This error probability decays exponentially with the evaluation budget and recovers the standard Pareto set identification guarantee when all configuration costs are identical. Experiments on LLM configuration evaluation tasks demonstrate that the proposed framework enables efficient online decision-making and accurate cost-aware Pareto identification under limited budgets.
Aug 4, 2026

RAG-Stack: Co-Optimizing RAG Serving Performance and Quality

Retrieval-augmented generation (RAG), which augments large language model (LLM) generation with information retrieved from databases, has become a widely used approach for knowledge-intensive applications. Modern RAG systems, however, expose many configuration choices, such as retrieval indexes, model selections, and how models invoke retrieval. Each configuration yields a different trade-off between answer quality and serving performance, making it challenging to choose the optimal setting for a specific application deployment. We present RAG-Stack, a framework for efficiently discovering quality-performance Pareto frontiers across diverse RAG applications and serving systems. RAG-Stack consists of RAG-PE, an iterative design-space exploration algorithm that selects the next RAG configuration to evaluate; RAG-IR, a workload abstraction for diverse RAG algorithms; and RAG-CM, a performance model that predicts the optimal deployment and serving performance on the given hardware. Together, these components allow RAG-Stack to search the joint algorithm-system configuration space without deploying every candidate and to transfer an existing Pareto frontier to a new serving system. Given the same number of optimization iterations across diverse datasets, the Pareto frontiers found by RAG-Stack cover 52.5% to 153.2% more of the normalized quality-performance space than those found by state-of-the-art configuration-search methods evaluated over the same RAG design space.
Aug 4, 2026

Impacts of Single-objective Landscapes on Multi-objective Optimization

This work revealed a relationship between a multi-objective optimization problem and single-objective optimization problems that exist in the multi-objective problem. This work focused on combinatorial problems and investigated the relations between the local optima networks of the single-objective problems and the Pareto optima network of the multi-objective problem. Each of their networks has a graph structure. We divided the entire network into subgraphs. Each subgraph was called a component and characterized by overlapping relations between the single-objective local optima networks and the multi-objective Pareto optima network. Results on multi-objective landscape problems showed that most Pareto optimal solutions were reachable from the single-objective local optimal solutions. This tendency was emphasized by increasing the number of objectives and the objective correlation. The number of co-variables impacted the number of cross-link relations between the single-objective local optima networks and the multi-objective Pareto optima network. The results suggested that searching for single-objective problems is a clue to multi-objective optimization.
Aug 2, 2026

Agentic Stage-One Stellarator Optimization: Autonomous Multi-Objective Search for Finite-Beta Equilibria

Stage-one stellarator design searches a high-dimensional family of three-dimensional plasma boundaries and fixed-boundary MHD equilibria for configurations that jointly meet requirements on confinement, field-line topology, force balance, stability proxies, and geometry. These specifications do not provide a general constructive map to a validated finite-beta equilibrium. High-quality targets are commonly developed through iterative numerical optimization whose outcome depends on the initial configuration, active Fourier resolution, objective priorities, and local solver budget. Coordinating this process is computationally costly and expert-intensive, limiting both design throughput and the production of consistently evaluated data. We present a proof of concept for agentic stage-one optimization. A bounded language-model agent diagnoses the current equilibrium and selects the next local optimization experiment, while deterministic DESC execution owns prescribed profiles and flux, symmetry, metric evaluation, solver validity, and acceptance. On a common-budget subset from an expanding finite-beta campaign, the number of gate-valid configurations increases from five inputs to nineteen outputs; median Boozer QS RMS decreases from 2.39×10−42.39\times10^{-4} to 1.07×10−41.07\times10^{-4}, and median maximum principal curvature decreases from 62.5662.56 to 33.00 m−133.00\,\mathrm{m}^{-1}. A complementary long route achieves a 9.10×9.10\times QS reduction while repairing magnetic-well and curvature defects. The system also records every attempted local action as transition evidence, yielding 734 structured parent--action--outcome records in the reported experiments. These results show that agentic outer-loop control can sustain finite-beta, multi-objective search and turn repeated optimization into a scalable source of improved equilibria and reusable decision data.
Aug 1, 2026

SDDMO-Bench: A Benchmark Suite for Streaming Data-Driven Dynamic Multi-Objective Optimization

Streaming data-driven dynamic multi-objective optimization requires algorithms to track time-varying Pareto fronts using only sequential observations under concept drift. However, systematic evaluation remains difficult because real-world problems usually lack ground-truth optima, drift annotations, and controllable conditions, while existing benchmarks provide limited support for standardized comparison. This paper proposes SDDMO-Bench, a benchmark suite that transforms classical dynamic multi-objective test problems into streaming environments by combining intrinsic objective-mapping evolution, controllable distributional drift, and sequential data revelation. By combining five representative time-dependent base functions with six distributional drift patterns, SDDMO-Bench constructs 30 scenarios with diverse levels of non-stationarity, problem complexity, sample-distribution variation, and Pareto-front evolution. Experiments with representative evolutionary algorithms demonstrate that SDDMO-Bench provides challenging and discriminative test scenarios, offering a standardized, controllable, and reproducible benchmark for evaluating adaptability, robustness, and Pareto-front tracking in streaming data-driven dynamic multi-objective optimization.
Jul 28, 2026

Top-kk Pareto Bandits: Hypervolume Regret for Multi-Objective Slate Selection

We consider a stochastic multi-objective bandit problem where, at each round, the agent selects a slate of kk arms and observes their dd-dimensional reward vectors under semi-bandit feedback. We do not aim at identifying a single optimal arm; instead, we consider the problem of maintaining a small set of actions that jointly approximate the Pareto frontier. We formalize this objective through the dominated hypervolume induced by the selected subset of arms, and define an αα-approximate hypervolume regret with respect to the best size-kk subset achievable in hindsight, where α=1−1/eα= 1 - 1/e reflects the approximation guarantee of greedy maximization for monotone submodular functions. To address this problem, we introduce THV-UCB, an optimistic algorithm that selects arms greedily based on optimistic estimates of their marginal hypervolume contributions. We establish a gap-free regret bound O~(dnkT)\tilde{O}(d\sqrt{nkT}) that holds on every instance, together with a gap-dependent bound O~(nk2.5/Δmin⁡)\tilde{O}(nk^{2.5}/Δ_{\min}) that becomes polylogarithmic in TT once the arms are sufficiently well separated. Our results provide theoretical support for using small subsets to approximate Pareto fronts in various multi-objective applications.
Jul 22, 2026

Cardinality-Decomposed Loss: Matching Training Objectives to Relation Structure in Heterogeneous Recommendation Graphs

Graph Neural Networks trained on heterogenous bipartite graphs form a common basis in recommendation systems. These graphs often express relations that vary in cardinality, for example, user-item preferences are one-to-many and user-attribute features are one-to-one. Traditionally, a unique loss function is applied for all of the network components which is often Bayesian Personalized Ranking (BPR). While BPR works well for the recommendation task, we find that it causes attribute embeddings to collapse to near-random geometry -- a silent failure that leaves standard ranking metrics largely unaffected and therefore invisible to conventional evaluation. This in turn pollutes user node embeddings, which are shaped by both edge types simultaneously, hurting downstream tasks like personalization, segmentation, etc. Here we propose a Cardinality-Decomposed Loss (CDL) that combines both Cross Entropy (CE) and BPR to enable the model to collectively optimize for relations across cardinalities. We confirm this CE-BPR conflict by showing the two losses compete in the shared encoder's parameter space. We evaluate CDL on five datasets spanning two structural configurations -- one-to-one attributes on user nodes (MovieLens-1M, Last.fm-360K, PayPal Audience Factory, BookCrossing) and on item nodes (Yelp) -- and find that CDL consistently improves discriminability in attribute embeddings. We also show that ranking (NDCG) improves when attributes carry meaningful preference signal, but conflicts with it when the correlation is weak. We use a lambda parameter to navigate this trade-off, and a lambda-sweep reveals that dataset behavior is governed by two graph properties -- semantic alignment and topology leakage. Semantic alignment measures whether the attribute predicts preferences, while topology leakage measures whether the graph's connectivity already encodes it.
Jul 22, 2026

AlphaRoute: Large Language Models as Semantic Optimizers for Multi-Objective Routing

Very Large Scale Integration (VLSI) global routing is an NP-hard combinatorial optimization problem requiring signal net assignment across capacity-constrained 3D grids while minimizing congestion, wirelength, and via transitions. Because traditional heuristics rely on static penalty schedules that fail on complex congestion topologies, we present AlphaRoute: a multi-objective adaptive search framework reformulating rip-up and reroute (R&R) into a dynamic optimization system. We introduce SHAP-based overflow decomposition to isolate per-net congestion, driving targeted subgraph extraction via 3D Dijkstra maze routing and an adaptive PathFinder policy. Crucially, AlphaRoute employs Large Language Models (LLMs) as semantic policy optimizers. Bounded by a deterministic knowledge graph, the LLMs interpret congestion metrics to dynamically adjust penalty parameters. Evaluated on ISPD 2025 benchmarks, AlphaRoute reduces overflow by 98.6% on MEMPOOL. On the constrained ARIANE design, we achieve an overflow of 146,109 (a 29.8x reduction in overflow over the state of the art), yielding a penalized score of S_orig = 0.0538 versus the State-of-the-art (SOTA) 1.780. These results demonstrate that superior algorithmic search geometry can overcome the latency of interpreted Python implementations.
Jul 19, 2026

The Optimization Trilemma: Efficiency, Comfort and Fairness in Decentralized Multi-agent Coordination

The problem of fair multi-agent coordination in decentralized settings is one of the most pressing challenges for building efficient collaborative systems. Resource allocation is based on optimized collective arrangements accounting for agents' needs. Such coordination should not only be computationally efficient but also account for fairness, i.e., equitable redistribution of costs incurred by all agents. Recent literature has proposed several algorithms that efficiently determine optimal plan combinations balancing system-wide efficiency and individual discomfort of agents in a centralized setting. However, these works do not address equitable resource optimization in fully decentralized scenarios, specifically, the optimized redistribution of discomfort among coordinating agents so that none experiences a discomfort level that could lead to loss of incentive or polarization that can disrupt planned operations. In this work, we study the problem of optimizing three objectives: (i) system-wide efficiency, (ii) individuals' comfort and (iii) fairness (i.e., balancing of incurred discomfort costs) in decentralized multi-agent coordination. We design a novel model to optimize those three orthogonal objectives, without any substantial increase in communication and computational overhead. Through experiments on two real-world datasets, we validate the model and demonstrate that it can achieve fairer optimization outcomes, while satisfying agents' preferences and system goals.
Jul 18, 2026

Decision Variable Analysis-Guided Differentiated Fuzzy Search for Large-Scale Multi-Objective Optimization

Large-scale multi-objective optimization problems (LSMOPs) are challenging due to their high-dimensional decision spaces. Fuzzy search is an effective technique for improving search efficiency, while decision variable analysis can reveal the distinct roles of variables in promoting convergence and maintaining diversity. However, existing fuzzy search methods generally employ a uniform search granularity for all variables, overlooking the heterogeneous search requirements implied by variable roles. To address this limitation, this paper proposes a Decision variable analysis-guided Differentiated Fuzzy Search method, termed DDFS. The proposed method establishes an explicit mapping between decision-variable roles and fuzzy search granularities. Decision variable analysis is employed to identify variable roles and search sensitivities, enabling different variable groups to adopt differentiated fuzzy search behaviors during offspring generation. Furthermore, a Dual-Indicator Stage Transition Mechanism is developed to dynamically adjust fuzzy-updating intensity throughout the evolutionary process, balancing early-stage search-space compression and late-stage convergence refinement. Extensive experiments on the LSMOP and UF benchmark suites with up to 1000 decision variables show that DDFS generally achieves competitive performance against several representative large-scale multi-objective evolutionary algorithms. The results suggest that explicitly incorporating decision-variable roles into fuzzy search can help improve optimization performance in high-dimensional decision spaces.
Jul 16, 2026

Regularity-Aware Stochastic MGDA with Adaptive Per-Iterate Conflict-Avoidant Update

Multi-objective learning (MOL) aims to optimize multiple objectives simultaneously. The multi-gradient descent algorithm (MGDA) constructs a common descent or conflict-avoidant (CA) direction by solving a gradient-aggregation subproblem. In stochastic settings, the vanilla stochastic MGDA method, SMG, replaces full-batch gradients with mini-batch estimates, causing bias in the resulting CA direction. Existing analyses control this error using continuity of the CA-direction mapping, but do not distinguish how its sensitivity varies with the geometry of the MGDA subproblem. Motivated by this limitation, we show that the MGDA direction map is Lipschitz continuous under reduced nondegeneracy, whereas its general 1/21/2-Hölder exponent on bounded sets is sharp. This geometry motivates the Multi-objective Regularity-aware (MoRe) method, which retains adaptive MGDA aggregation in the regular regime and switches to a fixed-weight linear scalarization (LS) near degeneracy. The fallback branch skips the MGDA subproblem solve, which can reduce aggregation cost on fallback iterations while avoiding a potentially noise-sensitive update. Theoretically, we establish a per-iterate CA guarantee, and a O~(T−1)\widetilde{\mathcal O}(T^{-1}) convergence rate, improving upon the previously established O~(T−1/4)\widetilde{\mathcal O}(T^{-1/4}) SMG guarantee, where O~\widetilde{\mathcal O} hides logarithmic factors. Experiments demonstrate favorable multi-task performance and convergence behavior consistent with the theoretical rate.
Jul 15, 2026

Constraint-Driven Model Optimization: An Industry Framework for Selecting Compression and Acceleration Techniques in Modern Machine Learning Systems

The rapid deployment of machine learning systems across cloud, edge, and enterprise environments has brought model optimization to the forefront of systems-engineering. Despite a rich literature spanning quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference-time optimization, practitioners are often left navigating these techniques through heuristics rather than principled methodology. We argue that optimization should be formulated as a constraint-driven, multi-objective engineering decision and introduce a unified framework that characterizes any production deployment along five interacting constraint dimensions: data availability, latency budget, memory budget, accuracy tolerance, and retraining budget. Building on this taxonomy, we synthesize empirical gains reported across the research literature and map them to operational constraints rather than algorithmic categories. To ensure practical relevance, we selected these techniques by reviewing recent literature for methods that report measurable improvements against critical deployment bottlenecks. We propose a prescriptive decision framework and provide optimization pipelines for four representative industrial scenarios to illustrate it in practice. To the best of our knowledge, this work provides one of the first structured attempts to formalize model optimization as a constraint-aware, multi-objective engineering process, synthesizing quantitative evidence from the research literature.
Jul 15, 2026

The impact of objective interactions on the performance of massive objective optimization algorithms

Many-objective optimization has been a field of interest over the past two decades and several evolutionary optimization algorithms have been introduced to tackle these problems; yet two fundamental questions remain underexplored: (i) What happens when the number of objectives grows beyond the typical many-objective regime of about fifteen and becomes massive? (ii) How do problem characteristics, such as the nature of interactions between objectives, influence algorithmic performance? To answer these questions we employ a diagnostic benchmark suite that allows control over problem characteristics and can be scaled to extremely high objective counts. Using this framework we evaluate several state-of-the-art evolutionary algorithms including NSGA-II, NSGA-III, MOEA/D and lexicase selection across a range of dimensionalities and diagnostic problem landscapes. Our experiments reveal that problem characteristics significantly affect algorithm performance. In particular, the nature of interactions between objectives appears important. These results highlight the importance of understanding these properties before selecting an algorithm for a specific problem. We also show that lexicase selection, an algorithm originally designed for genetic programming, compares favorably with state-of-the-art many-objective optimization algorithms while avoiding the dependence on predefined reference directions.
Jul 14, 2026

Hybrid multi-objective evolutionary algorithms for service placement in the computing continuum: a comparative study with genetic traceability

This paper addresses multi-objective service placement in computing continuum environments through a collaborative hybrid island-model MOEA. The key innovation is not the design of a new general hybrid algorithm, but the systematic application and analysis of heterogeneous hybridization for this specific optimization domain through two independent experimental campaigns: a first one with four state-of-the-art MOEAs (NSGA-II, NSGA-III, U-NSGA-III, and SMS-EMOA), and a second one with a complementary hybrid configuration based on NSGA-II, MOEA/TS, and MOCPO, both co-evolving and periodically exchanging solutions. These designs enable complementary search behaviors across islands and are naturally aligned with the distributed edge-fog-cloud architecture of the computing continuum, facilitating scalable parallel execution. To evaluate the approach, we define two research hypotheses: (i) whether hybrid cooperation yields significant performance gains over standalone algorithms, and (ii) whether all constituent algorithms contribute equally to the final outcomes. We combine standard Pareto-front quality indicators (GD, IGD, HV, S, and STE) with a traceability-oriented analysis based on genetic load, which quantifies the contribution of each island to the evolved solutions. Across 30 independent runs, the hybrid method outperforms most of the standalone baselines, and statistical tests confirm significant improvements. Results also show non-uniform contributions among islands, providing interpretable evidence of effective hybrid cooperation.
Jul 14, 2026

A new dual-population constrained multi-objective evolutionary optimization algorithm with repair constraint handling for structural optimization

Structural optimization problems often involve a large number of decision variables and highly non-convex feasible regions, making convergence to the true Pareto front extremely challenging. Even when convergence is achievable, it typically requires thousands of function evaluations, resulting in significant computational cost. This highlights the need for efficient and robust optimization algorithms for real-world engineering applications. In this study, we introduce a novel constrained multi-objective evolutionary algorithm, termed DPCME. The algorithm employs two interacting populations that exchange information, enabling effective global exploration and reducing the risk of convergence to local optima. To further enhance performance, a recent repair-based constraint-handling technique is incorporated, and alternative repair approaches are proposed and systematically evaluated. The proposed algorithm is tested on three engineering problems: the 72-bar truss, the 120-bar truss, and a chemical tanker structure, each involving hundreds of nonlinear failure constraints. Its performance is evaluated against state-of-the-art constrained multi-objective optimization algorithms from the latest PlatEMO package. A total of 43 algorithms are initially tested, from which the 12 best-performing methods are selected for detailed comparison. The results demonstrate that DPCME achieves superior or competitive convergence and diversity across all test cases, and that the inclusion of repair-based constraint handling further improves its performance.
Jul 13, 2026

Representing the Non-dominated Set of Multi-objective Network Problems by Supported Non-dominated Points

In multi-objective combinatorial optimization, unsupported non-dominated points typically outnumber supported points and are often significantly more challenging to compute. Recent studies show that extreme supported non-dominated points provide high-quality representations of the non-dominated set for certain binary problems. We demonstrate that this observation does not generalize to capacitated network optimization problems: representation quality decreases with increasing arc capacities, whereas supported non-dominated points consistently provide high-quality representations with respect to several quality indicators. However, supported point sets may still be too large in practical applications, where only a small, fixed number of alternatives is typically desired. Selecting fixed-size representations from the non-dominated set requires its computationally expensive generation and thus diminishes the computational advantages that representations are intended to provide. We therefore suggest the (extreme) supported points as alternative candidate sets in subset selection problems. Our numerical results show that restricting the candidate set to supported non-dominated points yields fixed-size representations of nearly the same quality as those selected from the complete non-dominated set. Overall, supported non-dominated points serve both as high-quality representations and as reasonable candidate sets for subset selection.
Jul 11, 2026

Is Model Instability just Noise to be Tolerated or a Property that can be Managed?

In software analytics, rerunning the same analysis twice often yields different models and conclusions. This reduces trust in the model and limits its use. We find that model instability is a major problem. Across 127 multi-objective SE optimization problems (12,700 test cases), repeated runs of a state-of-the-art optimizer agree on only 13.7% of test cases, even under improved settings. We argue that this instability is not merely noise to tolerate, but a property that can be measured and managed. By adjusting how labels are spent, how complex the models become, and how splits are scored, we obtain models that agree 4.8 times as often as the default configuration. The standard deviation of optimization error falls by 22% on average (mean std 17.4 to 13.6), while recommendation quality improves rather than degrades. In terms of quality, the refined settings are statistically top-ranked on 119 of 127 datasets, compared to 74 for the defaults. We then test causal and data-locality interventions and find that they help only partially, suggesting a residual stability floor. Our evidence suggests there are fundamental limits to stability set by the data itself (noise, scarce labels, proxy objectives, and the many near-equivalent models a dataset admits). We conclude that instability should be treated as a standard evaluation axis in SE optimization, which should be routinely measured, reported alongside performance, and used to calibrate trust in any single run. The methods in this paper provide a baseline against which future efforts to reduce SBSE instability can be judged. To support open science, we offer the following reproduction package: https://tinyurl.com/Model-Instability
Jul 10, 2026

EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDi

In large-scale ride-hailing, hold control is a critical mechanism for improving passenger-driver experience. By selectively delaying certain driver-order pairs, the system waits for better opportunities, reduces cancellations, and mitigates wasted driver effort. However, existing industrial hold strategies often rely on heuristic thresholding over multiple predictive models, which can be brittle under non-stationary traffic and hard to optimize for multi-objective experience signals. We propose EXHOLD, a deployable two-stage framework decoupling experience-aware pair assessment from hold-time execution. In Stage I, we learn a decision model assigning each driver-order pair to discrete, interpretable experience tiers by optimizing a unified objective that aggregates satisfaction signals across the matching funnel. In Stage II, we solve for a monotone hold-time schedule via constrained optimization over empirical quantiles. This explicitly enforces service guardrails bounding the unnecessary holding of promising matches while maximizing overall experience improvement. We evaluate EXHOLD through randomized A/B experiments in DiDi's production system in Brazil. Results show consistent gains in marketplace efficiency and experience: EXHOLD increases trip completion and driver income, significantly reduces passenger cancellations, and improves funnel efficiency. Ablations and behavioral analyses confirm both stages are essential and that the policy makes calibrated decisions under spatiotemporal heterogeneity. EXHOLD is currently deployed, serving production traffic in Brazil.