CEPO: A General Framework for Improving Prompt Optimization in Multi-Agent Systems

Jongha Kim; Seohyun Lee; Jonathan Minho Chung; Jinsung Yoon; Hyunwoo J. Kim

cs.AIOct 26, 2026

CEPO: A General Framework for Improving Prompt Optimization in Multi-Agent Systems

Authors: Jongha Kim, Seohyun Lee, Jonathan Minho Chung, Jinsung Yoon, Hyunwoo J. Kim

Organizations: Korea University · KAIST · Google Cloud AI Research

Abstract

Multi-agent systems (MAS) require both effective coordination and specialized task execution. Existing prompt optimization approaches typically couple these functions within agent-specific prompts. We introduce Coordinator-Executor Prompt Optimization (CEPO), a general framework for improving prompt optimization in MAS. CEPO separates system-level coordination from agent-level execution through a shared coordinator prompt and agent-specific executor prompts, and guides their updates according to their respective roles. This structure allows shared coordination rules to be refined consistently across agents while keeping execution instructions separately adjustable. CEPO provides the prompt structure and optimization guidance, while existing prompt optimizers perform the underlying search. Extensive experiments spanning 11 benchmarks across four domains demonstrate that applying CEPO to four existing prompt optimizers consistently improves their performance. Ablations and qualitative analyses support the proposed design, while controlled studies on a document-understanding MAS further demonstrate improved optimization budget efficiency and performance scaling with the number of agents.

Figures & tables

Figure 1: CEPO separates coordination and execution for role-aligned prompt optimization. (a) Prior work vs. CEPO (left). Existing methods couple coordination and execution roles within agent-specific prompts. CEPO represents these functions as distinct optimizable parameters through a shared coordinator prompt $p_{C}$ and agent-specific executor prompts $p_{E}$ , and guides their updates according to their respective roles using an existing prompt optimizer. (b) Component-wise comparison (right). Accuracy on MDocAgent ( Han et al., 2025a ) + MMLongBench-Doc ( Ma et al., 2024 ) . Vanilla optimizer improves baseline ( $54.1\to 57.9$ ), prompt decomposition yields a further gain ( $58.6$ ), and combining decomposition with role-aligned optimization (CEPO) performs the best ( $61.0$ ).

Coordinator optimizer meta-instruction (

\Phi_{C}

)

Executor optimizer meta-instruction (

\Phi_{E}

)

You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL five agents. Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace.

\cdots

Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look.

Your task is to optimize only the target agent’s executor prompt : the how-to instructions that tell this specific agent how to perform

\cdots

Infer the execution behavior that would have prevented it and convert it into reusable executor-level instructions. The final prompt should be concrete and operational

\cdots

provide enough task-specific guidance to avoid recurring failure patterns. Include procedural details when they clarify how the agent should select evidence.

Table 1: Meta-instructions for role-aligned guidance. The coordinator meta-instruction

\Phi_{C}

directs the optimizer toward team-level coordination signals , while the executor meta-instruction

\Phi_{E}

directs it toward agent-level execution signals . Full prompts reported in Appendix C.2 and C.3 .

Figure 2: Overview of CEPO. CEPO decomposes prompts within a multi-agent system into two layers with distinct roles. The coordinator prompt $p_{C}$ (top), shared across all agents, captures coordination signals (team composition, role assignment, and shared norms for reasoning). Each executor prompt $p_{E}^{(i)}$ (bottom), local to agent $A_{i}$ , captures execution signals (evidence anchoring, task scoping, and procedure execution). The effective prompt for $A_{i}$ is the concatenation $p_{C}\oplus p_{E}^{(i)}$ . Two layers are sequentially optimized with role-aligned guidance under CEPO: the coordinator is optimized using meta-instruction $\Phi_{C}$ with executors fixed (Phase 1), then executors are optimized using $\Phi_{E}$ with the coordinator fixed (Phase 2). CEPO applies these design principles to existing prompt optimizers $\mathcal{O}$ , consistently improving those methods.

Method	$T_{C}$	$T_{E}$	# API Calls		Acc.
Method	$T_{C}$	$T_{E}$	Opt.	LLM	Acc.
PE2	0	3	10	5,714	57.9
PE2	0	6	22	11,945	57.4
+ CEPO	1	1	2	3,120	60.1
+ CEPO	3	3	20	11,458	61.0

Table 4: Component ablation results.

Figure 3: Performance under varying numbers of agents. Annotations indicate CEPO’s gain over the baseline.

Figure 4: Qualitative comparison of optimized prompts. Through optimization, the coordinator prompt evolves from a generic description into team-level orchestration rules. The executor prompt of GENERAL agent evolves from vague instructions into a concrete agent-level execution procedure.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Full test set accuracy	Split test set accuracy	$\Delta$
MMLongBench-Doc	54.2	54.1	-0.1
LongDocURL	73.1	74.9	1.8
PaperTab	47.6	46.3	-1.3
PaperText	71.1	72.1	1.0
FetaTab	77.2	78.8	1.6

Appendix

Table S1: Test set accuracy on full set and split set.

\Delta

: Split test set accuracy - full test set accuracy.

Dataset	Train		Validation		Test		Average # Q / Doc
Dataset	# Docs	# Q	# Docs	# Q	# Docs	# Q	Average # Q / Doc
FetaTab	87	104	87	100	697	812	1.17
LongDocURL	40	209	40	226	316	1,890	5.87
MMLongBench-Doc	13	123	13	103	108	847	8.01
PaperTab	31	36	31	37	245	320	1.28
PaperText	109	292	109	300	869	2,212	2.58

Appendix

Table S2: Data split statistics. Each split is constructed at the PDF/document level. # Q denotes the number of questions.

#Agents	Configuration	Baseline	PE2	PE2 + CEPO	$\Delta$
1	GENERAL	54.3	–	–
2	GENERAL + SUM	53.0	55.5	56.9	+ 3.9
3	Average	53.6	53.2	57.7	+ 4.1
	GENERAL + TEXT + SUM	52.2	47.5	55.5	+3.3
	GENERAL + IMAGE + SUM	55.0	58.8	59.9	+4.9
4	Average	54.2	57.5	58.7	+ 4.5

Appendix

Table S3: Performance variation across agent configurations. For each agent count, the bold row reports the headline result (single configuration when only one is valid, otherwise the average across valid configurations). Indented rows below show the per-configuration breakdown for multi-configuration cases.

\Delta

denotes the improvement of PE2 + CEPO over the baseline. “–”: for 1-agent configuration, we did not apply PE2 or CEPO to compare performance on multi-agent configurations. Averaged results are also visually plotted in Fig. 3 .

Method	$T_{C}$	$T_{E}$	# API Calls		Acc.
Method	$T_{C}$	$T_{E}$	Opt.	LLM	Acc.
PE2	0	3	10	5,714	57.9
PE2	0	6	22	11,945	57.4
+ CEPO	1	1	2	3,120	60.1
	2	2	12	7,288	60.2
	3	3	20	11,458	61.0

Appendix

Table S4: Results by optimization budget. Opt: # optimizer API calls, LLM: # MDocAgent internal API calls.

Method	Throughput (sec/sample)	Output tokens (tok/sample)	Acc.
Baseline	15.92	847.7	54.1
PE2	13.35	595.8	57.9
+ CEPO	15.61	824.7	61.0

Appendix

Table S5: Inference cost by optimization method. Throughput (sec/sample): average wall-clock time per sample. Output tokens: average number of output tokens per sample.

Schedule	Coordinator-first	Executor-first
Sequential	61.0	60.7
Alternating	60.1	58.2

Appendix

Table S6: Analysis on the optimization schedule. Two design axes are varied: schedule type (sequential vs. alternating) and which role is optimized first (coordinator vs. executor). All schedules use the same total budget of six iterations.

$\|\mathcal{I}\|$	Target executor set $\mathcal{I}$	PE2 + CEPO	PE2
1	GENERAL (ours)	61.0	57.9
	CRITIC	58.0	57.9
	TEXT	59.1	55.3
	IMAGE	58.0	54.2
	SUM	55.8	57.5
2	GENERAL $\to$ TEXT	59.7	56.1

Appendix

Table S7: Ablation on the target executor set

\mathcal{I}

Method	# Tokens	Accuracy
Baseline	226	54.1
PE2 (short)	631	57.9
PE2 (long)	1410	54.0
PE2 + CEPO	1408	61.0

Appendix

Table S8: Prompt length controlled comparison. # Tokens: total tokens of optimized prompts.

Coordinator Prompt $p_{C}^{\star}$	LongDocURL	PaperTab	PaperText	FetaTab
Baseline (no coordinator)	74.9	46.3	72.1	78.8
Cross-dataset ( $p_{C}^{\star}$ from MMLongBench-Doc)	74.8	49.7	73.4	79.4
In-domain ( $p_{C}^{\star}$ from target dataset)	75.4	53.7	76.4	80.8

Appendix

Table S9: Coordinator prompt transferability.

p_{C}^{\star}

optimized on MMLongBench-Doc transfers to four other benchmarks with negligible loss compared to dataset-specific optimization.

Evaluator Prompt

Question: {question} Predicted Answer: {answer} Ground Truth Answer: {gt} Please evaluate if the predicted answer is correct compared to the ground truth. Score the answer on: Binary correctness (0-1): 1 if the answer is correct, 0 if it is incorrect Return only a string with these scores in a dictionary and can be parsed by json.loads, e.g. {{"binary_correctness": 1}}

Appendix

Table S10: Evaluator LLM prompt.

Actor Optimizer Prompt

Optimization target: ONE shared multi agent system prompt used by GENERAL, CRITICAL, TEXT, IMAGE, and SUM. ### Agent System Prompt Optimization Objective You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL five agents (GENERAL, CRITICAL, TEXT, IMAGE, SUM). Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace: uncertainty preservation, modality discipline, evidence handling, handoff, conflict surfacing, and final synthesis. Do not primarily optimize output format, templates, XML wrappers, field names, or rigid reporting schemas. It is acceptable to add short role-dependent guidance only when it defines division of labor across the system, not detailed role-specific heuristics. Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look. ### Brevity as an Optimizer-Only Constraint Brevity is a constraint on how you edit and select the new shared system prompt. It is not, by itself, a rule that should be inserted into the produced prompt. Do not add self-referential brevity instructions such as ‘‘be concise,’’ ‘‘keep outputs minimal,’’ ‘‘use a short format,’’ or similar rules unless failure analysis shows that runtime verbosity is itself causing a shared pipeline failure. Do not mention token budgets, prompt length, minimality, or concision inside the produced shared system prompt unless that is directly justified by the failure pattern. Instead, satisfy the brevity constraint by deleting, merging, or rewriting redundant shared rules while preserving or improving coordination quality. When two candidate prompts are likely to behave similarly, prefer the shorter one. When a useful change can be made either by revising an existing rule or by adding a new one, prefer revising the existing rule. Hard limit: the optimized shared system prompt (‘new_prompt‘) must remain under 600 tokens. ### Multimodal Failure Case Format When failure examples include retrieved images, each case may attach the images once near the top of that case using aliases such as ‘case1_image1‘, ‘case1_image2‘, etc. Treat those aliases as the actual images retrieved for that failure case. Later agent input traces may reference the same aliases in ‘referenced_images‘; do not expect repeated image blocks inside every agent input. Use the case-local aliases to reason about what GENERAL, CRITICAL, and IMAGE could see, while keeping the prompt edit focused on shared multi-agent behavior. ### Multimodal Optimizer Attachments The optimizer request includes 20 actual image(s). They are attached to this request in the same order as the image mapping shown inside the failure examples. When analyzing a case, use only the images mapped to that example and only when the target agent input shows those images were runtime-visible. ### Step 1 (Analyze end-to-end failures) For each example, inspect the full multi-agent trace and classify the final-answer failure. Decide whether the optimized target prompt could have changed the trace toward the gold answer. Separate target-actionable failures from downstream-only, other-agent-only, and evidence-missing cases. Current prompt: {CURRENT PROMPT} Failure examples: {FAIL EXAMPLES INCLUDED} ### Step 2 (Produce optimized prompt) Return ONLY a JSON object with the keys: {"reasoning": "...", "new_prompt": "...", "new_history": "..."}

Appendix

Table S11: Meta-instruction for optimizing the coordinator prompt on MDocAgent.

Pipeline Optimizer Prompt

Optimization target: one shared DocAgent pipeline instruction. It is composed into the actual system message with the original DocAgent system prompt. ### Agent System Prompt Optimization Objective You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL three agents (actor, reviewer, reflector). Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace: evidence handling, handoff, conflict surfacing, and final synthesis. Encourage each agent to act as evidence support for the final synthesis. Non-final agents should preserve information that may help the final answerer, including verified evidence, partial support, and conflicts. Optimize the shared prompt toward better evidence preservation. The resulting prompt should help the final synthesizer recover the strongest answer supported by the available context. Do not primarily optimize output format, templates, XML wrappers, field names, or rigid reporting schemas. It is acceptable to add short role-dependent guidance only when it defines division of labor across the system, not detailed role-specific heuristics. Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look. ### Brevity as an Optimizer-Only Constraint Brevity is a constraint on how you edit and select the new shared system prompt. It is not, by itself, a rule that should be inserted into the produced prompt. Do not add self-referential brevity instructions such as ‘‘be concise,’’ ‘‘keep outputs minimal,’’ ‘‘use a short format,’’ or similar rules unless failure analysis shows that runtime verbosity is itself causing a shared pipeline failure. Do not mention token budgets, prompt length, minimality, or concision inside the produced shared system prompt unless that is directly justified by the failure pattern. Instead, satisfy the brevity constraint by deleting, merging, or rewriting redundant shared rules while preserving or improving coordination quality. When two candidate prompts are likely to behave similarly, prefer the shorter one. When a useful change can be made either by revising an existing rule or by adding a new one, prefer revising the existing rule. Hard limit: the optimized shared system prompt (‘new_prompt‘) must remain under 600 tokens. ### Step 1 (Analyze end-to-end failures) For each example, inspect the actor/reviewer/reflection trace and classify the final-answer failure. Decide whether shared pipeline behavior could have changed the trace toward the gold answer. Separate prompt-actionable failures from evidence-missing or judge-only issues. The optimized prompt must preserve these template placeholders: {agent_role} and {agent_specific_prompt}. Current prompt: {current_prompt} Failure examples: {failure_examples} ### Step 2 (Produce optimized prompt) {history_section} Return ONLY a JSON object with keys ‘‘reasoning’’, ‘‘new_prompt’’, and ‘‘new_history’’.

Appendix

Table S12: Meta-instruction for optimizing the coordinator prompt on DocAgent.

Pipeline Optimizer Prompt

Target agent: {TARGET AGENT} You are optimizing one target agent prompt in a multi-agent system. Your task is to optimize only the target agent’s executor prompt: the how-to instructions that tell this specific agent how to perform its assigned subtask under the given context. Analyze the failure examples using only the target-agent input and target-agent output shown in each case. Do not rely on hidden upstream reasoning, downstream corrections, missing pages, unavailable images, or information that was not visible to the target agent. Infer the execution behavior that would have prevented each failure, then generalize those behaviors into reusable executor-level instructions. Target optimized prompt granularity: - The final prompt should be concrete and operational, not merely a generic multimodal QA prompt. - It should provide enough task-specific guidance for the target agent to avoid recurring failure patterns. - Merge similar behaviors into broader rules when possible, but keep the resulting rules specific enough to guide execution. - Include procedural details when they clarify how the agent should select evidence, reason over multimodal inputs, and produce its answer. - Avoid overfitting to individual examples or encoding narrow case-specific fixes. - Avoid making the prompt unnecessarily long, but do not omit useful execution guidance just to keep it minimal. Prompt organization: - Write the optimized prompt as an executor-facing how-to guide. - Prefer a compact structure with: 1. a brief statement of the target agent’s task; 2. a ‘‘Core behavior’’ section describing the main execution rules the agent should follow; 3. a ‘‘Procedure’’ section describing the order in which the agent should apply those rules. - The ‘‘Core behavior’’ section should contain stable behavioral principles derived from the failures. Keep the structure focused on how this target agent should execute its own task. The optimized prompt should stay at the executor level. It should describe how the target agent should perform its own multimodal document QA task under the given context. Do not redefine the overall multi-agent system, assign roles to other agents, change global routing, or add unsupported capabilities. ### Step 1 (Analyze target-agent failures) For each example, inspect only the target agent input/output shown in the failure case. Decide what prompt behavior would have made that target agent answer correctly from the same input. Do not rely on hidden upstream or downstream agent traces. Current prompt: {CURRENT PROMPT} Failure examples: {FAIL EXAMPLES INCLUDED} ### Step 2 (Produce optimized prompt) Return ONLY a JSON object with the keys: {"reasoning": "...", "new_prompt": "...", "new_history": "..."}

Appendix

Table S13: Meta-insturction for optimizing the executor prompt on MDocAgent.

MDocAgent Coordinator Prompt

Current prompt: You are one agent in a multi-agent document question answering system. There are five agents: - GENERAL: can broadly inspect both text and image evidence - CRITICAL: can broadly inspect both text and image evidence - TEXT: can only inspect text evidence extracted from the document image, including OCR text - IMAGE: can only inspect the relevant document image - SUM: final judge and answer composer; can only view the discussion history from GENERAL, TEXT, and IMAGE You are the {agent_role} agent. {agent_specific_prompt}

Appendix

Table S14: Initial coordinator prompt

p^{(0)}_{C}

for MDocAgent.

DocAgent Coordinator Prompt

You are part of DocAgent, a document question answering pipeline for long multimodal documents. The pipeline has three prompt-driven stages: - ACTOR: studies the document outline, uses search, section, page-image, image, and table-image tools to gather evidence, and produces a concise answer inside <final_result>. - REVIEWER: checks the actor answer against document evidence using only necessary tools, then either confirms or corrects the final answer inside <final_result>. - REFLECTION: when the reviewer changes the actor answer, updates a concise reusable guideline that may help future questions. You are the {agent_role}. Follow the core system instruction below while preserving the shared pipeline behavior: {agent_specific_prompt}

Appendix

Table S15: Initial coordinator prompt

p^{(0)}_{C}

for DocAgent.

	Coordinator prompt	Executor prompt
WHAT-TO	You are one agent in a multi-agent document question answering system. … - Use only the evidence available to your role ; do not claim unseen text, images, pages, or prior discussion. - Separate direct evidence from inference; unsupported values, phrases, identifiers, or conclusions should not be presented as found. - Preserve uncertainty ; if evidence is missing, ambiguous, or insufficient, prefer not answerable over guessing. - Keep scope local to the question; do not replace local evidence with broader document-level statistics or unrelated pages. - Do not link items across pages, sections, tables, or map regions unless the evidence explicitly connects them. … - If your role cannot fully answer, provide grounded observations and identify what remains unresolved for other agents or final synthesis. …	Answer the user’s question using only document content that is visibly established in the provided text and images. … Your answer must preserve the exact target of the question: the requested entity type, group, field, location relation, count, title, ratio, or other attribute. Favor the most specific evidence that directly matches the ask. Do not substitute a nearby heading, broader list, related item, parent category, child category, or different semantic type just because it appears relevant. … Reflect the document faithfully: when the answer depends on labels, captions, map placement, section membership, figure association, or category wording, use only what is directly shown and clearly linked . … If the exact requested content is not visibly established, or if the visible evidence only supports a related but different answer, say so rather than guessing .
HOW-TO	You are one agent in a multi-agent document question answering system. … Before answering, check all of the following : 1) Do I have the right evidence source for this question? 2) Is the referenced page, region, table, chart, figure, row, item, label, or field actually visible or explicitly quoted? 3) Does the evidence support the exact answer requested , rather than a nearby, broader, parent-category, child-category, or implied one? 4) Am I using the correct unit, denominator, item identity , counting rule, and formatting rule? … Do not assume the full document is available; retrieved items may be partial, cropped, non-consecutive , or from different parts of the document. If the question targets a specific page, region, figure, row, item, or file, verify that the target content is visibly present and clearly linked . For exact-string tasks, copy carefully and preserve unclear punctuation, separators, spacing, decimals, or digit grouping. For visual-region questions, anchor your reasoning to the specified region; if it is cropped, occluded, or absent, say so. If multiple plausible answers exist, keep them as candidates and state what evidence remains unresolved. For maps, first identify the relevant visible map page or region before naming a place or sight. For tables, identify the table title , row, and column headers before extracting any cell. For charts, identify title, axes, units , legend, and series before reading values. For counts, ratios, or percentages, define the unit before calculating and verify the numerator and denominator. … If text and image disagree, state the conflict rather than forcing a single answer. Use exactly: Not answerable from the provided evidence when the required support is missing or unresolved. …	You can inspect both OCR/text and visible document images. Answer only from evidence actually visible to you. … 1. Ground every answer in the most local relevant evidence: page, slide, figure, table, header, footer, caption, panel, or image region. 2. Treat OCR text and the image as joint evidence ; use page numbers, labels, count cues, identifiers, row labels, or years from either source. 3. Identify the exact target scope before answering: page, last page, row, subject, figure, chapter, year, page range, component, or subset. … use the visible index/table/list when the question asks about all examples. … 7. For exact strings and identifiers, copy exactly ; do not normalize punctuation, spacing, hyphens, abbreviations, or formatting. 8. For table questions, first filter to rows whose key field exactly matches the requested entity; then extract only the requested column(s). 9. For conditional lookups, bind the condition to the exact local anchor. 10. For percentages, ratios, and counts, use only visible operands tied to the same local evidence. 11. For distinct examples or samples, count unique matching items and deduplicate repeated renderings. 12. If the question asks about all or each, verify that all requested members are visible ; otherwise answer Not answerable rather than generalizing. 13. If multiple visible candidates remain, state the ambiguity instead of guessing. … Procedure : first identify the exact target scope; verify that the scope is visible and complete enough; find the nearest governing anchor ; extract using the correct structure. If extracting from a table/list, filter rows by exact match before collecting values. If a required scope is not fully visible, answer: Not answerable from the visible evidence. …

Appendix

Table S16: Coordinator and executor prompts optimization results under different objectives. Rows denote objective types ( what-to vs. how-to ), and columns denote prompt layers (coordinator vs. executor). Blue highlights indicate high-level what-to signals defining desirable behavior, and yellow highlights how-to signals describing procedural details and rules. Green-shaded cells mark the role-aligned configuration used in our method.

Prompt Optimization

Prompt optimization automates the revision or selection of instructions and other prompt components to improve model behavior for specific tasks.

See all 141 papers

May 7, 2026

MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems

Large language model (LLM)-based Multi-agent systems (MAS) have shown promise in tackling complex collaborative tasks, where agents are typically orchestrated via role-specific prompts. While the quality of these prompts is pivotal, jointly optimizing them across interacting agents remains a non-trivial challenge, primarily due to the misalignment between local agent objectives and holistic system goals. To address this, we introduce MASPO, a novel framework designed to automatically and iteratively refine prompts across the entire system. A core innovation of MASPO is its joint evaluation mechanism, which assesses prompts not merely by their local validity, but by their capacity to facilitate downstream success for successor agents. This effectively bridges the gap between local interactions and global outcomes without relying on ground-truth labels. Furthermore, MASPO employs a data-driven evolutionary beam search to efficiently navigate the high-dimensional prompt space. Extensive empirical evaluations across 6 diverse tasks demonstrate that MASPO consistently outperforms state-of-the-art prompt optimization methods, achieving an average accuracy improvement of 2.9. We release our code at https://github.com/wangzx1219/MASPO.

Zhexuan Wang, Xuebo Liu, Li Wang +4

Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China.

Multi-Agent LLM Systems Multi-Agent System Optimization Multi-Agent Collaboration Prompt Optimization

Ask PDF

Jun 22, 2026

MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?

Multi-agent systems (MAS) offer a scalable path forward for agentic AI, comprising multiple LLM-based agents, each assigned a system prompt and a position within a workflow that governs inter-agent coordination and output aggregation. System prompts thus form a critical and accessible optimization surface: they specify agents' roles and behaviors, enabling system-level improvements without model finetuning. Although prompt optimization has shown substantial potential for single LLMs, extending it to MAS poses distinct challenges, notably an exponentially growing search space. It remains unclear whether, when, and by how much prompt optimization improves MAS performance, and how sensitive such gains are to system configuration. In this work, we systematically study system-prompt optimization across a broad range of MAS setups varying in task, workflow, communication protocol, and team size, benchmarking two prompt optimizers that naturally extend state-of-the-art single-agent methods. The results reveal its potential to unlock significant gains while exposing open challenges, characterizing when and how much prompt optimization helps across diverse MAS settings.

Juyang Bai, Laixi Shi

Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.

Multi-Agent LLM Systems Multi-Agent System Optimization AI Agent Benchmarks Prompt Optimization

Ask PDF

Oct 22, 2026

Not Every Agent Needs the Strongest Model: Unified Prompt-Model Optimization for Heterogeneous LLM-based Multi-Agent Systems

Large language model (LLM)-based multi-agent systems (MAS) rely on role-specific prompts and execution models to solve complex tasks collaboratively. However, prompt optimization typically keeps model assignments fixed, leaving differences in role requirements and their associated inference costs outside the search. We present UniMASO, a Unified framework for heterogeneous Multi-Agent System Optimization through the joint adaptation of prompts and model assignments. UniMASO evaluates candidate updates through local, downstream, and global feedback, using cost-aware Pareto search to retain useful quality-cost trade-offs. Topology-aware co-evolution then coordinates these updates according to agent roles and dependencies. Cross-level trajectory memory further supplies historical successes and failures to guide subsequent optimization. Experiments on five benchmarks and five execution topologies show that UniMASO achieves competitive task performance at substantially lower deployment cost than the strongest homogeneous configurations. Under centralized execution, it improves the average score from 69.79 to 70.40 while reducing token-weighted deployment cost by 85.2% relative to DeepSeek-V4-Pro with MASPO. These results support joint prompt-model adaptation as an effective approach to efficient multi-agent collaboration. The code is available at https://github.com/Achilles008/UniMASO_main.

Mingxiu Cai, Haoyang Xian, Xinyu Ma +4

Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen

Multi-Agent LLM Systems Multi-Agent System Optimization

Ask PDF

Abstract

Prompt Optimization

Explore similar work

MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems

MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?

Not Every Agent Needs the Strongest Model: Unified Prompt-Model Optimization for Heterogeneous LLM-based Multi-Agent Systems