cs.AIOct 26, 2026
SaveCEPO: A General Framework for Improving Prompt Optimization in Multi-Agent Systems
Organizations: Korea University · KAIST · Google Cloud AI Research
Abstract
Multi-agent systems (MAS) require both effective coordination and specialized task execution. Existing prompt optimization approaches typically couple these functions within agent-specific prompts. We introduce Coordinator-Executor Prompt Optimization (CEPO), a general framework for improving prompt optimization in MAS. CEPO separates system-level coordination from agent-level execution through a shared coordinator prompt and agent-specific executor prompts, and guides their updates according to their respective roles. This structure allows shared coordination rules to be refined consistently across agents while keeping execution instructions separately adjustable. CEPO provides the prompt structure and optimization guidance, while existing prompt optimizers perform the underlying search. Extensive experiments spanning 11 benchmarks across four domains demonstrate that applying CEPO to four existing prompt optimizers consistently improves their performance. Ablations and qualitative analyses support the proposed design, while controlled studies on a document-understanding MAS further demonstrate improved optimization budget efficiency and performance scaling with the number of agents.
Figures & tables
Figure 1: CEPO separates coordination and execution for role-aligned prompt optimization. (a) Prior work vs. CEPO (left). Existing methods couple coordination and execution roles within agent-specific prompts. CEPO represents these functions as distinct optimizable parameters through a shared coordinator prompt and agent-specific executor prompts , and guides their updates according to their respective roles using an existing prompt optimizer. (b) Component-wise comparison (right). Accuracy on MDocAgent ( Han et al., 2025a ) + MMLongBench-Doc ( Ma et al., 2024 ) . Vanilla optimizer improves baseline ( ), prompt decomposition yields a further gain ( ), and combining decomposition with role-aligned optimization (CEPO) performs the best ( ).
| Coordinator optimizer meta-instruction ( ) | Executor optimizer meta-instruction ( ) |
| You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL five agents. Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace. Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look. | Your task is to optimize only the target agent’s executor prompt : the how-to instructions that tell this specific agent how to perform Infer the execution behavior that would have prevented it and convert it into reusable executor-level instructions. The final prompt should be concrete and operational provide enough task-specific guidance to avoid recurring failure patterns. Include procedural details when they clarify how the agent should select evidence. |
Table 1: Meta-instructions for role-aligned guidance. The coordinator meta-instruction directs the optimizer toward team-level coordination signals , while the executor meta-instruction directs it toward agent-level execution signals . Full prompts reported in Appendix C.2 and C.3 .
Figure 2: Overview of CEPO. CEPO decomposes prompts within a multi-agent system into two layers with distinct roles. The coordinator prompt (top), shared across all agents, captures coordination signals (team composition, role assignment, and shared norms for reasoning). Each executor prompt (bottom), local to agent , captures execution signals (evidence anchoring, task scoping, and procedure execution). The effective prompt for is the concatenation . Two layers are sequentially optimized with role-aligned guidance under CEPO: the coordinator is optimized using meta-instruction with executors fixed (Phase 1), then executors are optimized using with the coordinator fixed (Phase 2). CEPO applies these design principles to existing prompt optimizers , consistently improving those methods.
| Method | # API Calls | Acc. | |||
| Opt. | LLM | ||||
| PE2 | 0 | 3 | 10 | 5,714 | 57.9 |
| 0 | 6 | 22 | 11,945 | 57.4 | |
| + CEPO | 1 | 1 | 2 | 3,120 | 60.1 |
| 3 | 3 | 20 | 11,458 | 61.0 | |
Table 4: Component ablation results.
Figure 3: Performance under varying numbers of agents. Annotations indicate CEPO’s gain over the baseline.
Figure 4: Qualitative comparison of optimized prompts. Through optimization, the coordinator prompt evolves from a generic description into team-level orchestration rules. The executor prompt of GENERAL agent evolves from vague instructions into a concrete agent-level execution procedure.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Full test set accuracy | Split test set accuracy | |
| MMLongBench-Doc | 54.2 | 54.1 | -0.1 |
| LongDocURL | 73.1 | 74.9 | 1.8 |
| PaperTab | 47.6 | 46.3 | -1.3 |
| PaperText | 71.1 | 72.1 | 1.0 |
| FetaTab | 77.2 | 78.8 | 1.6 |
Table S1: Test set accuracy on full set and split set. : Split test set accuracy - full test set accuracy.
| Dataset | Train | Validation | Test | Average # Q / Doc | |||
| # Docs | # Q | # Docs | # Q | # Docs | # Q | ||
| FetaTab | 87 | 104 | 87 | 100 | 697 | 812 | 1.17 |
| LongDocURL | 40 | 209 | 40 | 226 | 316 | 1,890 | 5.87 |
| MMLongBench-Doc | 13 | 123 | 13 | 103 | 108 | 847 | 8.01 |
| PaperTab | 31 | 36 | 31 | 37 | 245 | 320 | 1.28 |
| PaperText | 109 | 292 | 109 | 300 | 869 | 2,212 | 2.58 |
Table S2: Data split statistics. Each split is constructed at the PDF/document level. # Q denotes the number of questions.
| #Agents | Configuration | Baseline | PE2 | PE2 + CEPO | |
| 1 | GENERAL | 54.3 | – | – | |
| 2 | GENERAL + SUM | 53.0 | 55.5 | 56.9 | + 3.9 |
| 3 | Average | 53.6 | 53.2 | 57.7 | + 4.1 |
| GENERAL + TEXT + SUM | 52.2 | 47.5 | 55.5 | +3.3 | |
| GENERAL + IMAGE + SUM | 55.0 | 58.8 | 59.9 | +4.9 | |
| 4 | Average | 54.2 | 57.5 | 58.7 | + 4.5 |
Table S3: Performance variation across agent configurations. For each agent count, the bold row reports the headline result (single configuration when only one is valid, otherwise the average across valid configurations). Indented rows below show the per-configuration breakdown for multi-configuration cases. denotes the improvement of PE2 + CEPO over the baseline. “–”: for 1-agent configuration, we did not apply PE2 or CEPO to compare performance on multi-agent configurations. Averaged results are also visually plotted in Fig. 3 .
| Method | # API Calls | Acc. | |||
| Opt. | LLM | ||||
| PE2 | 0 | 3 | 10 | 5,714 | 57.9 |
| 0 | 6 | 22 | 11,945 | 57.4 | |
| + CEPO | 1 | 1 | 2 | 3,120 | 60.1 |
| 2 | 2 | 12 | 7,288 | 60.2 | |
| 3 | 3 | 20 | 11,458 | 61.0 | |
Table S4: Results by optimization budget. Opt: # optimizer API calls, LLM: # MDocAgent internal API calls.
| Method | Throughput (sec/sample) | Output tokens (tok/sample) | Acc. |
| Baseline | 15.92 | 847.7 | 54.1 |
| PE2 | 13.35 | 595.8 | 57.9 |
| + CEPO | 15.61 | 824.7 | 61.0 |
Table S5: Inference cost by optimization method. Throughput (sec/sample): average wall-clock time per sample. Output tokens: average number of output tokens per sample.
| Schedule | Coordinator-first | Executor-first |
| Sequential | 61.0 | 60.7 |
| Alternating | 60.1 | 58.2 |
Table S6: Analysis on the optimization schedule. Two design axes are varied: schedule type (sequential vs. alternating) and which role is optimized first (coordinator vs. executor). All schedules use the same total budget of six iterations.
| Target executor set | PE2 + CEPO | PE2 | |
| 1 | GENERAL (ours) | 61.0 | 57.9 |
| CRITIC | 58.0 | 57.9 | |
| TEXT | 59.1 | 55.3 | |
| IMAGE | 58.0 | 54.2 | |
| SUM | 55.8 | 57.5 | |
| 2 | GENERAL TEXT | 59.7 | 56.1 |
Table S7: Ablation on the target executor set .
| Method | # Tokens | Accuracy |
| Baseline | 226 | 54.1 |
| PE2 (short) | 631 | 57.9 |
| PE2 (long) | 1410 | 54.0 |
| PE2 + CEPO | 1408 | 61.0 |
Table S8: Prompt length controlled comparison. # Tokens: total tokens of optimized prompts.
| Coordinator Prompt | LongDocURL | PaperTab | PaperText | FetaTab |
| Baseline (no coordinator) | 74.9 | 46.3 | 72.1 | 78.8 |
| Cross-dataset ( from MMLongBench-Doc) | 74.8 | 49.7 | 73.4 | 79.4 |
| In-domain ( from target dataset) | 75.4 | 53.7 | 76.4 | 80.8 |
Table S9: Coordinator prompt transferability. optimized on MMLongBench-Doc transfers to four other benchmarks with negligible loss compared to dataset-specific optimization.
| Evaluator Prompt | Question: {question} Predicted Answer: {answer} Ground Truth Answer: {gt} Please evaluate if the predicted answer is correct compared to the ground truth. Score the answer on: Binary correctness (0-1): 1 if the answer is correct, 0 if it is incorrect Return only a string with these scores in a dictionary and can be parsed by json.loads, e.g. {{"binary_correctness": 1}} |
Table S10: Evaluator LLM prompt.
| Actor Optimizer Prompt | Optimization target: ONE shared multi agent system prompt used by GENERAL, CRITICAL, TEXT, IMAGE, and SUM. ### Agent System Prompt Optimization Objective You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL five agents (GENERAL, CRITICAL, TEXT, IMAGE, SUM). Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace: uncertainty preservation, modality discipline, evidence handling, handoff, conflict surfacing, and final synthesis. Do not primarily optimize output format, templates, XML wrappers, field names, or rigid reporting schemas. It is acceptable to add short role-dependent guidance only when it defines division of labor across the system, not detailed role-specific heuristics. Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look. ### Brevity as an Optimizer-Only Constraint Brevity is a constraint on how you edit and select the new shared system prompt. It is not, by itself, a rule that should be inserted into the produced prompt. Do not add self-referential brevity instructions such as ‘‘be concise,’’ ‘‘keep outputs minimal,’’ ‘‘use a short format,’’ or similar rules unless failure analysis shows that runtime verbosity is itself causing a shared pipeline failure. Do not mention token budgets, prompt length, minimality, or concision inside the produced shared system prompt unless that is directly justified by the failure pattern. Instead, satisfy the brevity constraint by deleting, merging, or rewriting redundant shared rules while preserving or improving coordination quality. When two candidate prompts are likely to behave similarly, prefer the shorter one. When a useful change can be made either by revising an existing rule or by adding a new one, prefer revising the existing rule. Hard limit: the optimized shared system prompt (‘new_prompt‘) must remain under 600 tokens. ### Multimodal Failure Case Format When failure examples include retrieved images, each case may attach the images once near the top of that case using aliases such as ‘case1_image1‘, ‘case1_image2‘, etc. Treat those aliases as the actual images retrieved for that failure case. Later agent input traces may reference the same aliases in ‘referenced_images‘; do not expect repeated image blocks inside every agent input. Use the case-local aliases to reason about what GENERAL, CRITICAL, and IMAGE could see, while keeping the prompt edit focused on shared multi-agent behavior. ### Multimodal Optimizer Attachments The optimizer request includes 20 actual image(s). They are attached to this request in the same order as the image mapping shown inside the failure examples. When analyzing a case, use only the images mapped to that example and only when the target agent input shows those images were runtime-visible. ### Step 1 (Analyze end-to-end failures) For each example, inspect the full multi-agent trace and classify the final-answer failure. Decide whether the optimized target prompt could have changed the trace toward the gold answer. Separate target-actionable failures from downstream-only, other-agent-only, and evidence-missing cases. Current prompt: {CURRENT PROMPT} Failure examples: {FAIL EXAMPLES INCLUDED} ### Step 2 (Produce optimized prompt) Return ONLY a JSON object with the keys: {"reasoning": "...", "new_prompt": "...", "new_history": "..."} |
Table S11: Meta-instruction for optimizing the coordinator prompt on MDocAgent.
| Pipeline Optimizer Prompt | Optimization target: one shared DocAgent pipeline instruction. It is composed into the actual system message with the original DocAgent system prompt. ### Agent System Prompt Optimization Objective You are optimizing one shared multi-agent system prompt for a multi-agent document QA pipeline. This shared prompt is prepended to ALL three agents (actor, reviewer, reflector). Each agent also receives its own agent-specific prompt with role-specific instructions. The goal is to improve system-level behavior across the trace: evidence handling, handoff, conflict surfacing, and final synthesis. Encourage each agent to act as evidence support for the final synthesis. Non-final agents should preserve information that may help the final answerer, including verified evidence, partial support, and conflicts. Optimize the shared prompt toward better evidence preservation. The resulting prompt should help the final synthesizer recover the strongest answer supported by the available context. Do not primarily optimize output format, templates, XML wrappers, field names, or rigid reporting schemas. It is acceptable to add short role-dependent guidance only when it defines division of labor across the system, not detailed role-specific heuristics. Prefer compact shared rules that help the pipeline make better decisions over formatting instructions that only change how answers look. ### Brevity as an Optimizer-Only Constraint Brevity is a constraint on how you edit and select the new shared system prompt. It is not, by itself, a rule that should be inserted into the produced prompt. Do not add self-referential brevity instructions such as ‘‘be concise,’’ ‘‘keep outputs minimal,’’ ‘‘use a short format,’’ or similar rules unless failure analysis shows that runtime verbosity is itself causing a shared pipeline failure. Do not mention token budgets, prompt length, minimality, or concision inside the produced shared system prompt unless that is directly justified by the failure pattern. Instead, satisfy the brevity constraint by deleting, merging, or rewriting redundant shared rules while preserving or improving coordination quality. When two candidate prompts are likely to behave similarly, prefer the shorter one. When a useful change can be made either by revising an existing rule or by adding a new one, prefer revising the existing rule. Hard limit: the optimized shared system prompt (‘new_prompt‘) must remain under 600 tokens. ### Step 1 (Analyze end-to-end failures) For each example, inspect the actor/reviewer/reflection trace and classify the final-answer failure. Decide whether shared pipeline behavior could have changed the trace toward the gold answer. Separate prompt-actionable failures from evidence-missing or judge-only issues. The optimized prompt must preserve these template placeholders: {agent_role} and {agent_specific_prompt}. Current prompt: {current_prompt} Failure examples: {failure_examples} ### Step 2 (Produce optimized prompt) {history_section} Return ONLY a JSON object with keys ‘‘reasoning’’, ‘‘new_prompt’’, and ‘‘new_history’’. |
Table S12: Meta-instruction for optimizing the coordinator prompt on DocAgent.
| Pipeline Optimizer Prompt | Target agent: {TARGET AGENT} You are optimizing one target agent prompt in a multi-agent system. Your task is to optimize only the target agent’s executor prompt: the how-to instructions that tell this specific agent how to perform its assigned subtask under the given context. Analyze the failure examples using only the target-agent input and target-agent output shown in each case. Do not rely on hidden upstream reasoning, downstream corrections, missing pages, unavailable images, or information that was not visible to the target agent. Infer the execution behavior that would have prevented each failure, then generalize those behaviors into reusable executor-level instructions. Target optimized prompt granularity: - The final prompt should be concrete and operational, not merely a generic multimodal QA prompt. - It should provide enough task-specific guidance for the target agent to avoid recurring failure patterns. - Merge similar behaviors into broader rules when possible, but keep the resulting rules specific enough to guide execution. - Include procedural details when they clarify how the agent should select evidence, reason over multimodal inputs, and produce its answer. - Avoid overfitting to individual examples or encoding narrow case-specific fixes. - Avoid making the prompt unnecessarily long, but do not omit useful execution guidance just to keep it minimal. Prompt organization: - Write the optimized prompt as an executor-facing how-to guide. - Prefer a compact structure with: 1. a brief statement of the target agent’s task; 2. a ‘‘Core behavior’’ section describing the main execution rules the agent should follow; 3. a ‘‘Procedure’’ section describing the order in which the agent should apply those rules. - The ‘‘Core behavior’’ section should contain stable behavioral principles derived from the failures. Keep the structure focused on how this target agent should execute its own task. The optimized prompt should stay at the executor level. It should describe how the target agent should perform its own multimodal document QA task under the given context. Do not redefine the overall multi-agent system, assign roles to other agents, change global routing, or add unsupported capabilities. ### Step 1 (Analyze target-agent failures) For each example, inspect only the target agent input/output shown in the failure case. Decide what prompt behavior would have made that target agent answer correctly from the same input. Do not rely on hidden upstream or downstream agent traces. Current prompt: {CURRENT PROMPT} Failure examples: {FAIL EXAMPLES INCLUDED} ### Step 2 (Produce optimized prompt) Return ONLY a JSON object with the keys: {"reasoning": "...", "new_prompt": "...", "new_history": "..."} |
Table S13: Meta-insturction for optimizing the executor prompt on MDocAgent.
| MDocAgent Coordinator Prompt | Current prompt: You are one agent in a multi-agent document question answering system. There are five agents: - GENERAL: can broadly inspect both text and image evidence - CRITICAL: can broadly inspect both text and image evidence - TEXT: can only inspect text evidence extracted from the document image, including OCR text - IMAGE: can only inspect the relevant document image - SUM: final judge and answer composer; can only view the discussion history from GENERAL, TEXT, and IMAGE You are the {agent_role} agent. {agent_specific_prompt} |
Table S14: Initial coordinator prompt for MDocAgent.
| DocAgent Coordinator Prompt | You are part of DocAgent, a document question answering pipeline for long multimodal documents. The pipeline has three prompt-driven stages: - ACTOR: studies the document outline, uses search, section, page-image, image, and table-image tools to gather evidence, and produces a concise answer inside <final_result>. - REVIEWER: checks the actor answer against document evidence using only necessary tools, then either confirms or corrects the final answer inside <final_result>. - REFLECTION: when the reviewer changes the actor answer, updates a concise reusable guideline that may help future questions. You are the {agent_role}. Follow the core system instruction below while preserving the shared pipeline behavior: {agent_specific_prompt} |
Table S15: Initial coordinator prompt for DocAgent.
| Coordinator prompt | Executor prompt | |
| WHAT-TO | You are one agent in a multi-agent document question answering system. … - Use only the evidence available to your role ; do not claim unseen text, images, pages, or prior discussion. - Separate direct evidence from inference; unsupported values, phrases, identifiers, or conclusions should not be presented as found. - Preserve uncertainty ; if evidence is missing, ambiguous, or insufficient, prefer not answerable over guessing. - Keep scope local to the question; do not replace local evidence with broader document-level statistics or unrelated pages. - Do not link items across pages, sections, tables, or map regions unless the evidence explicitly connects them. … - If your role cannot fully answer, provide grounded observations and identify what remains unresolved for other agents or final synthesis. … | Answer the user’s question using only document content that is visibly established in the provided text and images. … Your answer must preserve the exact target of the question: the requested entity type, group, field, location relation, count, title, ratio, or other attribute. Favor the most specific evidence that directly matches the ask. Do not substitute a nearby heading, broader list, related item, parent category, child category, or different semantic type just because it appears relevant. … Reflect the document faithfully: when the answer depends on labels, captions, map placement, section membership, figure association, or category wording, use only what is directly shown and clearly linked . … If the exact requested content is not visibly established, or if the visible evidence only supports a related but different answer, say so rather than guessing . |
| HOW-TO | You are one agent in a multi-agent document question answering system. … Before answering, check all of the following : 1) Do I have the right evidence source for this question? 2) Is the referenced page, region, table, chart, figure, row, item, label, or field actually visible or explicitly quoted? 3) Does the evidence support the exact answer requested , rather than a nearby, broader, parent-category, child-category, or implied one? 4) Am I using the correct unit, denominator, item identity , counting rule, and formatting rule? … Do not assume the full document is available; retrieved items may be partial, cropped, non-consecutive , or from different parts of the document. If the question targets a specific page, region, figure, row, item, or file, verify that the target content is visibly present and clearly linked . For exact-string tasks, copy carefully and preserve unclear punctuation, separators, spacing, decimals, or digit grouping. For visual-region questions, anchor your reasoning to the specified region; if it is cropped, occluded, or absent, say so. If multiple plausible answers exist, keep them as candidates and state what evidence remains unresolved. For maps, first identify the relevant visible map page or region before naming a place or sight. For tables, identify the table title , row, and column headers before extracting any cell. For charts, identify title, axes, units , legend, and series before reading values. For counts, ratios, or percentages, define the unit before calculating and verify the numerator and denominator. … If text and image disagree, state the conflict rather than forcing a single answer. Use exactly: Not answerable from the provided evidence when the required support is missing or unresolved. … | You can inspect both OCR/text and visible document images. Answer only from evidence actually visible to you. … 1. Ground every answer in the most local relevant evidence: page, slide, figure, table, header, footer, caption, panel, or image region. 2. Treat OCR text and the image as joint evidence ; use page numbers, labels, count cues, identifiers, row labels, or years from either source. 3. Identify the exact target scope before answering: page, last page, row, subject, figure, chapter, year, page range, component, or subset. … use the visible index/table/list when the question asks about all examples. … 7. For exact strings and identifiers, copy exactly ; do not normalize punctuation, spacing, hyphens, abbreviations, or formatting. 8. For table questions, first filter to rows whose key field exactly matches the requested entity; then extract only the requested column(s). 9. For conditional lookups, bind the condition to the exact local anchor. 10. For percentages, ratios, and counts, use only visible operands tied to the same local evidence. 11. For distinct examples or samples, count unique matching items and deduplicate repeated renderings. 12. If the question asks about all or each, verify that all requested members are visible ; otherwise answer Not answerable rather than generalizing. 13. If multiple visible candidates remain, state the ambiguity instead of guessing. … Procedure : first identify the exact target scope; verify that the scope is visible and complete enough; find the nearest governing anchor ; extract using the correct structure. If extracting from a table/list, filter rows by exact match before collecting values. If a required scope is not fully visible, answer: Not answerable from the visible evidence. … |
Table S16: Coordinator and executor prompts optimization results under different objectives. Rows denote objective types ( what-to vs. how-to ), and columns denote prompt layers (coordinator vs. executor). Blue highlights indicate high-level what-to signals defining desirable behavior, and yellow highlights how-to signals describing procedural details and rules. Green-shaded cells mark the role-aligned configuration used in our method.
More in this topic
Prompt Optimization
Prompt optimization automates the revision or selection of instructions and other prompt components to improve model behavior for specific tasks.
See all 141 papers