Design and Tuning of Multi-Agent Systems Using Bayesian Evolutionary Optimization

Dominik Jehle; Tom Zehle

cs.LGOct 21, 2026

Design and Tuning of Multi-Agent Systems Using Bayesian Evolutionary Optimization

Authors: Dominik Jehle, Tom Zehle

Organizations: University of Freiburg Freiburg im Breisgau, Germany

Abstract

Designing effective multi-agent systems (MAS) remains a challenging and resource-intensive task. We propose a Bayesian Evolutionary Optimization framework for automatically discovering task-specific MAS topologies. The approach combines evolutionary operators to generate diverse workflow graphs with Bayesian optimization to focus cost intensive evaluations on promising candidates. In contrast to LLM-based candidate selection, our approach uses a graph-aware Gaussian Process and Expected Improvement to provide an explicit selection criterion during evolutionary search. We evaluate the method on BigCodeBench for code generation and on a tabular preprocessing benchmark using TabPFN. On BigCodeBench, evolved architectures achieve the highest overall performance among the evaluated agent systems. On temporal tabular datasets, optimized MAS preprocessing improves over the default TabPFN model on several tasks, while remaining competitive on IID datasets. Overall, the results demonstrate the potential of Bayesian selection for guiding evolutionary MAS search under expensive evaluation budgets.

Figures & tables

Figure 1: Overview of the proposed optimization pipeline: Candidate MAS topologies are generated through evolutionary expansion, selected via Bayesian optimization, instantiated with agent types and prompts, and evaluated iteratively.

Figure 4: Performance on BigCodeBench aggregated over all four evaluation seeds. The figure shows the results for all four seeds together with the mean performance and its standard deviation.

Architecture	Tasks Solved	Percentage
Evolved	295	38.1%
Planner-Executor-Validator	290	37.4%
Single Agent	265	34.2%

Table 1: Comparison of multi-agent architectures on BigCodeBench across 775 unique tasks.

Table 5

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset Name	Number of Rows
airfoil_self_noise	1,503
concrete_compressive_strength	1,030
healthcare_insurance_expenses	1,338
miami_housing	13,776
superconductivity	21,263
cirrhosis_patient_survival_prediction	161

Appendix

Table 4: Overview of the IID datasets from BeyondArena [ 23 ] , including the number of rows.

Dataset Name	Number of Rows
garments_worker_productivity	1,197
coffee_rating_prediction	2,369
mercedes_benz_greener_manufacturing	4,204
sberbank_housing_market_forecasting	27,195
california_house_prices_2020	41,528

Appendix

Table 5: Overview of the temporal datasets from BeyondArena [ 23 ] , including the number of rows.

Dataset	Evolved	Single Agent	Plan.-Ex.-Val.	Raw TabPFN
california_house_prices_2020	0.940543	0.940592	0.940009	0.941049
coffee_rating_prediction	0.111621	-0.252738	0.005208	0.203051
garments_worker_productivity	0.444023	0.453376	-0.404151	0.382659
mercedes_benz_greener_manu.	0.580626	0.580993	0.580140	0.578253
sberbank_housing_market_fore.	0.641153	0.640405	0.634187	0.572113

Appendix

Table 6: Results on the temporal datasets.

dataset	Evolved	Single Agent	Plan.-Ex.-Val.	Raw TabPFN
airfoil_self_noise	0.975258	0.975962	0.956852	0.971863
blood_tests_drink_prediction	0.129454	0.118084	0.118628	0.118950
body_density_prediction	0.692294	0.692378	0.676094	0.688951
cirrhosis_patient_survival_pred.	0.263664	0.259219	0.274007	0.318891
concrete_compressive_strength	0.951657	0.951404	0.943976	0.951536
fiat_500	0.853831	0.855042	0.846765	0.855032

Appendix

Table 7: Results on the IID datasets.

Table 10

Figure 5: Relative $R^{2}$ difference between the evolved architecture and the raw TabPFN baseline on the 5 temporal regression datasets.

Figure 6: BigCodeBench solve-set overlap across the three considered architectures: Evolved, Planner-Executor-Validator, and Single Agent. Each bar represents the number of tasks belonging to a specific solve-set combination, while the connected markers indicate which architectures successfully solved the corresponding tasks. The largest group consists of 438 tasks that were not solved by any of the three architectures. A further 212 tasks were solved by all three architectures. Among the pairwise overlaps, 41 tasks were solved by both Evolved and Planner-Executor-Validator but not by Single Agent, while 24 tasks were solved by both Evolved and Single Agent and another 24 tasks were solved by both Planner-Executor-Validator and Single Agent. Considering tasks solved exclusively by a single architecture, Evolved solved 18 tasks, Planner-Executor-Validator solved 13 tasks, and Single Agent solved 5 tasks.

Parameter	Value
Dataset / task	BigCodeBench (code generation)
Population size	5
Generations	10
Crossover / mutation prob.	0.5 / 0.5
Candidates per step	50
Search space (nodes / edges)	2–6 / $\leq$ 9 (acyclic)

Appendix

Table 10: Evolutionary search configuration for BigCodeBench.

Parameter	Value
Dataset / task	TabularData (SQL feature engineering)
Population size	5
Generations	10
Crossover / mutation prob.	0.5 / 0.5
Candidates per step	50
Search space (nodes / edges)	2–6 / $\leq$ 9 (acyclic)

Appendix

Table 11: Evolutionary search configuration for the TabularData run.

Multi-Agent System Optimization

Multi-Agent System Optimization studies how to improve agent teams’ coordination, performance, resource use, and reliability.

See all 161 papers

OptiMAS: Automatically Optimize Multi-Agent System
Aug 22, 2026Yuxin Cheng, Chang Liu, Hanxin Yu +9Multi-Agent System Optimization Multi-Agent LLM Systems
The University of Hong Kong · Huawei Foundation Model Department, Huawei Technologies Ltd. · City University of Hong Kong +2
TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems
May 10, 2026Chen Xu, Yicheng Hu, Ruizi Wang +4Multi-Agent LLM Systems Multi-Agent System Optimization
1Carnegie Mellon University · University of Science and Technology of China · 3National University of Singapore +1
MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networks
Mar 3, 2026Zhi Hong, Qian Zhang, Jiahang Sun +5Multi-Agent LLM Systems Multi-Agent System Optimization
The Chinese University of Hong Kong, Shenzhen · South China University of Technology · Ritsumeikan University +1

Aug 22, 2026J/K move · Enter open · S save

OptiMAS: Automatically Optimize Multi-Agent System

Yuxin Cheng, Chang Liu, Hanxin Yu, Haochen Tan+8

The University of Hong Kong · Huawei Foundation Model Department, Huawei Technologies Ltd. · City University of Hong Kong+2

Multi-Agent System Optimization Multi-Agent LLM Systems Self-Evolving Agents

Open paper AskPDF arXiv

Automated evolution of Multi-Agent Systems (MAS) holds significant potential for reducing the manual effort required to design and optimize LLM-based agent architectures. However, extant search-based paradigms face a fundamental trade-off, where an expanded optimization scope exacerbates evolutionary instability, while discrete branch-and-discard search isolates insights across lineages. To address these limitations, we propose a continuous, data-driven optimization paradigm built upon a unified ReAct-based infrastructure that reconciles a broad optimization scope with operational stability. Under this paradigm, we present OptiMAS, a task-agnostic agentic optimizer that leverages textual interaction trajectories and task feedback as loss signals for end-to-end MAS evolution. Equipped with a novel dual-track memory mechanism, OptiMAS sustains performance improvement over extended optimization horizons. Evaluation on four heterogeneous agentic benchmarks with three varying scale and accessibility LLM backbones, demonstrates that OptiMAS consistently achieves competitive or superior accuracy relative to both domain-specialized hand-crafted systems and existing evolutionary methods. Our work establishes a practical milestone toward robust, automated MAS evolution.

Abstract

Multi-Agent System Optimization

Explore similar work

OptiMAS: Automatically Optimize Multi-Agent System

TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems

MASPOB: Bandit-Based Prompt Optimization for Multi-Agent Systems with Graph Neural Networks