Multi-Objective Bayesian Optimization for Model Merging

Utkarsh Agarwal; Vamshi Bonagiri; Raul Astudillo; Monojit Choudhury

cs.LGAug 14, 2026

Multi-Objective Bayesian Optimization for Model Merging

Authors: Utkarsh Agarwal, Vamshi Bonagiri, Raul Astudillo, Monojit Choudhury

Organizations: Division of Computing and Mathematical Sciences Mohamed bin Zayed University of Artificial Intelligence

Abstract

Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge parameters is nevertheless difficult because downstream evaluations are expensive, gradients are unavailable, and source capabilities can conflict. We formulate merge-parameter selection as a black-box multi-objective optimization problem and introduce MOBO-Merge, a merge-operator agnostic framework that uses multi-objective Bayesian optimization to approximate the Pareto front under a limited evaluation budget. We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings using Linear, SLERP, TIES, and block-wise merge operators. On held-out benchmark partitions, MOBO-Merge obtains higher mean hypervolume than random search in 11 of 12 reported comparisons. The gain is small for one-dimensional Linear interpolation but substantially larger for several TIES, block-wise, and three-objective searches. No merge operator is uniformly best: TIES leads in three of four family-setting combinations, whereas Block-Linear 4x is strongest for the Llama three-model merge. These results show that multi-objective Bayesian optimization is valuable as a search layer for expressive merge parameterizations.

Figures & tables

Figure 1: Overview of MOBO-Merge. Model merging is cast as a multi-objective optimization problem. (Left) Source models specialized for different capabilities (e.g., instruction following

\theta_{1}

and mathematical reasoning

\theta_{2}

) to be merged via a parameterized merge operator. (Middle) Probabilistic surrogate models of the mapping from merge parameters to objective scores and are used to construct an acquisition function that selects the next merge configuration to evaluate. (Right) Iterating this process yields an approximation of the Pareto front, exposing optimal trade-offs between competing capabilities.

Panel of Figure 2 — Figure 2: Validation hypervolume in the two-model AB setting. Running dominated hypervolume for Linear, TIES, and Block-Linear 4x on instruction following and mathematical reasoning. Solid curves show MOBO-Merge with NEHVI and dotted curves show random search; curves show the mean across ten seeds, and shaded regions indicate the mean $\pm$ one standard error. The Linear gap is small, while MOBO-Merge produces clearer gains for the higher-dimensional TIES and Block-Linear search spaces, particularly for Llama-3.1-8B.

Panel of Figure 3 — Figure 3: Validation hypervolume in the three-model ABC setting. Running dominated hypervolume for Linear, TIES, and Block-Linear 4x on instruction following, mathematical reasoning, and code generation. Solid curves show MOBO-Merge with NEHVI and dotted curves show random search; curves show the mean across ten seeds, and shaded regions indicate the mean $\pm$ one standard error. The advantage of guided search is larger than in the AB setting, most notably for Llama-3.1-8B, where Block-Linear 4x attains the highest final hypervolume.

		Linear		TIES		Block-Linear 4x
		MOBO-Merge	Random	MOBO-Merge	Random	MOBO-Merge	Random
4B	AB	0.0659	0.0664	0.0847	0.0739	0.07368	0.0677
4B	ABC	0.0131	0.0129	0.0184	0.0145	0.0161	0.0144
8B	AB	0.1036	0.1017	0.1145	0.0764	0.1057	0.0879
8B	ABC	0.0430	0.0367	0.0429	0.0055	0.0634	0.0289

Table 1: Mean hypervolume on the held-out partition. For each run, the validation Pareto set is re-evaluated on the complementary partition and its held-out hypervolume is computed. Cells average over 10 completed runs. Reference points differ by family and objective set, so values should be compared only within rows, not across rows. Bold denotes the best value in each row.

Panel of Figure 4 — Figure 4: Representative validation Pareto fronts for instruction following and mathematical reasoning. Solid lines denote MOBO-Merge, dashed lines denote random search, and black squares mark the two source models. The Qwen front contains merges that improve both annotated source scores simultaneously, whereas the Llama front retains a clear capability trade-off.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Acquisition-function ablation on Qwen3-4B AB. Validation hypervolume over cumulative evaluations for NEHVI, qNParEGO, and random search across six merge operators. Curves show the mean across ten seeds, and shaded regions indicate the mean $\pm$ one standard error. Neither acquisition function uniformly dominates.

Language Model Merging

Language model merging combines capabilities from multiple models or adapters by composing their weights, updates, or outputs.

See all 109 papers

Mixture-Trained Merging for Unified Multi-Objective Models
Oct 1, 2026SeongHyeon Kim, Chaeyun Jang, Seungyoo Lee +4Multi-Task Learning Language Model Merging
KAIST · Kakao
AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging
Dec 10, 2025Kesheng Chen, Yamin Hu, Zhenqian Zhu +2Bayesian Optimization Language Model Merging
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, China.
Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions
Mar 10, 2026Mingyang Song, Mao ZhengLanguage Model Merging Model Merging
Large Language Model Department, Tencent, China

Oct 1, 2026J/K move · Enter open · S save

Mixture-Trained Merging for Unified Multi-Objective Models

SeongHyeon Kim, Chaeyun Jang, Seungyoo Lee, Jiyeon Ham+3

KAIST · Kakao

Multi-Task Learning Language Model Merging Model Merging

Open paper AskPDF arXiv

Unified language models are increasingly expected to combine heterogeneous capabilities, such as mathematics, code, instruction following, and controllable thinking behavior, within a single set of parameters. A common solution is sequential post-training on multiple objectives, but this entangles all objectives along one optimization trajectory and makes the final model highly sensitive to training order, data ratios, schedules, and stopping criteria. Weight-space merging offers a modular alternative, but naive merging of single-objective experts often fails: domain capabilities degrade sharply, or think/non-think modes collapse into one dominant behavior. We attribute both failures to incompatible weight-space geometry: experts trained on single objectives drift to distant regions of parameter space, placing their interpolations outside any shared low-loss basin. We propose Mixture-Trained Merging (MTM), which trains each branch on an objective-biased data mixture rather than a single objective, exposing it to cross-objective interactions and making branches compatible at merge time. MTM uses merged-model evaluations as a low-cost signal for selecting branch mixtures, avoiding expensive data-mixture ablations. The procedure is iterative: each round promotes the base model using globally selected merge coefficients and refines each branch mixture using domain-preferred coefficients under constraints that preserve other objectives. To scale beyond simplex grid search, MTM uses qNEHVI-based multi-objective Bayesian optimization. Across code, mathematics, instruction following, and think/non-think control, MTM outperforms naive merging and preserves behavioral separation where single-objective merging collapses, suggesting that effective unified models require branches trained to be mergeable.

Abstract

Language Model Merging

Explore similar work

Mixture-Trained Merging for Unified Multi-Objective Models

AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging

Model Merging in the Era of Large Language Models: Methods, Applications, and Future Directions