Multi-Objective Bayesian Optimization for Model Merging

Unified language models are increasingly expected to combine heterogeneous capabilities, such as mathematics, code, instruction following, and controllable thinking behavior, within a single set of parameters. A common solution is sequential post-training on multiple objectives, but this entangles all objectives along one optimization trajectory and makes the final model highly sensitive to training order, data ratios, schedules, and stopping criteria. Weight-space merging offers a modular alternative, but naive merging of single-objective experts often fails: domain capabilities degrade sharply, or think/non-think modes collapse into one dominant behavior. We attribute both failures to incompatible weight-space geometry: experts trained on single objectives drift to distant regions of parameter space, placing their interpolations outside any shared low-loss basin. We propose Mixture-Trained Merging (MTM), which trains each branch on an objective-biased data mixture rather than a single objective, exposing it to cross-objective interactions and making branches compatible at merge time. MTM uses merged-model evaluations as a low-cost signal for selecting branch mixtures, avoiding expensive data-mixture ablations. The procedure is iterative: each round promotes the base model using globally selected merge coefficients and refines each branch mixture using domain-preferred coefficients under constraints that preserve other objectives. To scale beyond simplex grid search, MTM uses qNEHVI-based multi-objective Bayesian optimization. Across code, mathematics, instruction following, and think/non-think control, MTM outperforms naive merging and preserves behavioral separation where single-objective merging collapses, suggesting that effective unified models require branches trained to be mergeable.

LLM services need to offer a family of models spanning different capability--cost trade-offs to accommodate diverse user preferences. Model merging offers a practical way to construct such a model family by combining a reasoning-enhanced model with an instruction-tuned model. Compared with model-level merging, layer-wise merging offers finer control over the capability--cost trade-off by assigning different merge weights to individual layers. However, it introduces two practical challenges: the layer-wise search space is high-dimensional, and existing methods rarely exploit informative signals from the source models; moreover, the highly variable runtime of LLM evaluations makes synchronous batch optimization inefficient by leaving GPU resources idle while waiting for slow evaluations. To address these challenges, we propose Asynchronous Prior-Guided Bayesian Model Merging (AP-BMM), which formulates layer-wise merging as a multi-objective optimization problem to approximate a Pareto set of merged LLMs, yielding a family of Pareto-optimal merged models with diverse capability--cost trade-offs. AP-BMM leverages parameter and activation discrepancies between the source models to guide the early layer-wise search, employs asynchronous pending-aware Bayesian optimization to maximize GPU utilization through asynchronous evaluations and to select high-quality candidates via pending awareness, and applies lightweight ranking over an oversized candidate pool to improve Pareto-front coverage. Under fixed evaluation budgets, AP-BMM achieves higher hypervolume (HV) and broader Pareto-front coverage than synchronous layer-wise and representative model-level merging methods, while reducing wall-clock time through higher GPU utilization.

Model merging combines the parameters of multiple neural networks into a single model without additional training. As fine-tuned large language models (LLMs) proliferate, merging offers a computationally efficient alternative to ensembles and full retraining, enabling practitioners to compose specialized capabilities at minimal cost. This survey examines model merging in the LLM era through the FUSE taxonomy, organized along Foundations, Unification Strategies, Scenarios, and Ecosystem. We first establish the theoretical underpinnings of merging, including loss landscape geometry and mode connectivity, then systematically review the algorithmic space spanning weight averaging, task vector arithmetic, sparsification-enhanced methods, mixture-of-experts architectures, and evolutionary optimization. We further examine downstream applications across multi-task learning, safety alignment, domain specialization, and federated learning, and survey the supporting ecosystem of tools and evaluation benchmarks. Finally, we identify key open challenges and future directions, aiming to equip researchers and practitioners with a structured foundation for advancing model merging.