IF-MAP: Influence-Function-Inspired Agent Prioritization in Multi-Agent Systems

Jie Mao; Pingchen Lu; Zikun Qu; Jiahang Sun; Xiangyi Wang; Zhiwei Shang; Zhiyong Wang; Guojun Xiong

cs.AIOct 17, 2026

IF-MAP: Influence-Function-Inspired Agent Prioritization in Multi-Agent Systems

Authors: Jie Mao, Pingchen Lu, Zikun Qu, Jiahang Sun, Xiangyi Wang, Zhiwei Shang, Zhiyong Wang, Guojun Xiong,

Organizations: The Chinese University of Hong Kong, Shenzhen · Harbin Institute of Technology · School of Engineering and Applied Sciences, Harvard University

Abstract

Improving every agent in an LLM-based multi-agent system can be prohibitively costly, making it important to identify which agent should be prioritized to achieve the greatest system-level gain with limited resources. Failure attribution can inform this decision, but an agent's responsibility for failures does not directly reveal the system-level gains achievable by improving it. We therefore introduce IF-MAP, an influence-function-inspired method for identifying agents with high potential for system-level improvement. For each agent, IF-MAP measures end-to-end loss changes under controlled semantic prompt perturbations, fits a regularized local response model in a low-dimensional prompt-embedding subspace, and uses the estimated response norm as a priority score. This black-box procedure requires no access to model parameters or gradients and produces agent priorities that can guide subsequent improvement effort. We evaluate IF-MAP across six multi-agent workflows spanning code generation, question answering, and mathematical reasoning, using the same priorities for prompt optimization and model upgrades. Optimizing the selected agents' prompts achieves a macro-average gain of 3.81 percentage points, compared with 1.85 for uniform agent selection. Across five target models, upgrading the selected agents yields macro-average gains of 7.62-16.42 percentage points, compared with 3.81-8.62 for uniform agent selection. Despite its probing overhead, IF-MAP reduces Qwen3-8B and DeepSeek-V3.2 token use by 74.2% and 35.2%, respectively, relative to optimizing every agent, and reduces target-model tokens by 63.6-64.9% relative to upgrading every agent. These results support the use of IF-MAP to identify promising agents and direct improvement effort toward greater end-to-end performance gains.

Figures & tables

Figure 1: Overview of IF-MAP. Given a fixed multi-agent system \scriptsize1⃝, we perturb one agent’s prompt at a time and measure end-to-end losses on the probing set \scriptsize2⃝. Subspace ridge regression on normalized prompt-embedding displacements and loss responses estimates a local response vector; its norm gives an influence-function-inspired priority score \scriptsize3⃝. We rank agents by this score \scriptsize4⃝ and direct improvement effort toward the top-ranked agent \scriptsize5⃝, e.g., through prompt revision, model upgrades, or fine-tuning. The ranking is illustrative.

Figure 2: Macro-average performance and ranking agreement across six workflows spanning code generation (HumanEval and MBPP), question answering (NQ, HotpotQA and DROP), and mathematical problem solving (MATH). The base model (before the model upgrade) is Qwen3-8B. Higher mean performance and ranking agreement (Kendall

\tau_{b}

) indicate better system performance and stronger agreement between the agent-priority and realized-improvement rankings, respectively.

Algorithm	HumanEval		MBPP		NQ		DROP		HotpotQA		MATH		Macro (6)
	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\overline{\Delta}\uparrow$	$\overline{\tau}_{b}\uparrow$
IF-MAP (ours)	+4.44	0.556	+7.91	0.889	+2.14	0.778	+4.99	1.000	+1.53	0.556	+1.85	0.667	+3.81	0.741
Who&When-AAO	+3.23	-0.222	+3.02	0.222	+2.14	0.333	+4.99	1.000	+1.53	0.333	+1.38	-0.035	+2.72	0.272
A2P	+3.23	-0.556	+3.02	0.000	+2.14	0.333	+4.99	1.000	+1.53	0.333	+1.38	0.004	+2.72	0.186
Adapted locator	+4.55	0.816	+5.27	0.516	-0.11	-0.333	+2.98	0.333	-0.55	-0.778	-0.40	-0.713	+1.96	-0.026
AgentRx	+3.74	-0.161	+0.16	-0.411	+2.14	0.333	+4.99	1.000	+1.53	0.333	+1.05	-0.084	+2.27	0.169

Table 1: Prompt optimization: mean selected gain and mean Kendall

\tau_{b}

for each dataset and their six-dataset macro means.

A. Qwen3-32B / Qwen3-235B-A22B / Qwen3.7-Plus (mean)
Algorithm	HumanEval		MBPP		NQ		DROP		HotpotQA		MATH		Macro (6)
	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\overline{\Delta}\uparrow$	$\overline{\tau}_{b}\uparrow$
IF-MAP (ours)	+17.31	0.778	+13.41	0.889	+17.40	0.778	+10.19	1.000	+8.03	0.630	+8.80	0.578	+12.52	0.775
Who&When-AAO	+14.88	0.494	+7.67	0.222	+17.40	0.333	+10.19	1.000	+8.03	0.556	+3.76	0.035	+10.32	0.440
A2P	+14.88	0.111	+7.67	0.000	+17.40	0.333	+10.19	1.000	+8.03	0.556	+3.76	0.099	+10.32	0.350
Adapted locator	+14.85	0.816	+8.84	0.516	-0.47	-0.333	+8.82	0.333	+0.36	-0.556	+2.22	-0.305	+5.77	0.079

Table 2: Model upgrades: selecting an agent for upgrade. Panel A averages the metrics over Qwen3-32B, Qwen3-235B-A22B, and Qwen3.7-Plus; panels B and C report DeepSeek-V3.2 and GPT-5 nano separately. Each panel includes its uniform-agent reference (the

\dagger

row). Mean gain and mean Kendall are reported for each dataset and in macro average.

Scoring-rule ablation
	Prompt optimization		Model upgrades
Method	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$
IF-MAP	$+3.812$	$0.741$	$+12.394$	$0.778$
w/ I-abs	$+3.276$	$0.752$	$+9.396$	$0.641$
w/ MeanAbs	$+3.576$	$0.774$	$+9.090$	$0.663$
Probe-budget comparison

Table 3: Macro-average ablation results over six datasets. IF-MAP uses G-norm and the full probe pool (at most 10 probes per node). Scoring variants replace only the scoring rule; budget variants retain G-norm. Model upgrades use per-node gains averaged over all five target models. Dataset means are equally weighted.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Workflow	Probing	Optimization	Validation	Test	Nodes
HumanEval	45	43	43	33	3
MBPP	115	113	113	86	4
NQ	268	266	266	266	3
DROP	268	266	266	200	2
HotpotQA	268	266	266	200	3
MATH (reduced execution)	80	80	80	121	5

Appendix

Table 4: Sizes of the probing, optimization, validation, and test sets. The optimization and validation sets are capped within their separate thirds for ordinary workflows. MATH uses the reduced execution sizes for the performance experiments; the full-sample row is used only for the cost scenario in Appendix H .

Workflow	Intervenable nodes
HumanEval	CustomCodeGenerate, ScEnsemble, Test
MBPP	CodeGenerate, ScEnsemble, Test, FixCode
NQ	AnswerGenerate, ScEnsemble, FormatAnswer
DROP	Solve, Format
HotpotQA	AnswerGenerate, ScEnsemble, FormatAnswer
MATH	Programmer, RefineAnswer, DetailedSolution, GenerateSolution, ScEnsemble

Appendix

Table 5: Node inventory for the six task-specific AFlow-derived workflows. A named node is the unit of intervention and can execute repeatedly in a trace.

Mechanism	Intended revision
Downstream interface	Clarify what the next operator needs and how this operator must provide it, without changing the workflow’s interface.
Decomposition control	Make the allocation or ordering of reasoning steps explicit, keeping the original task and output requirements.
Ambiguity resolution	Resolve an underspecified term, priority, or instruction that could admit incompatible interpretations.
Semantic precision	Sharpen the meaning of an existing instruction rather than merely adding emphasis.
Verification	Add or refine a task-relevant internal check while preserving the required externally visible answer.
Edge cases	Clarify boundary conditions or exceptional cases in a task-general way, not by copying development answers.

Appendix

Table 6: Formal mechanisms, grouped by construction family. Auxiliary response measurements support the adaptive construction process.

			Model-upgrade target / average
Workflow	Node	Prompt optimization	32B	3.7-plus	235B	DeepSeek V3.2	GPT-5 nano	MeanGain
HumanEval	CustomCodeGenerate	$+4.242$	$+15.152$	$+27.273$	$+24.242$	$+18.788$	$+24.849$	$+22.061$
	ScEnsemble	$+1.212$	$+0.606$	$+0.000$	$+0.000$	$+0.000$	$+0.606$	$+0.242$
	Test	$+4.849$	$+4.242$	$+9.091$	$+9.091$	$+9.697$	$+8.485$	$+8.121$
MBPP	CodeGenerate	$+3.023$	$+6.744$	$+9.070$	$+7.209$	$+8.605$	$+2.093$	$+6.744$
	ScEnsemble	$+0.000$	$-0.233$	$-0.233$	$-0.465$	$+0.233$	$-0.465$	$-0.233$

Appendix

Table 7: Per-node mean test gains (pp) for prompt optimization and all five model-upgrade targets used as evaluation references, rounded to three decimals. The listed nodes cover all six workflows. MeanGain averages each node’s five model-upgrade means before ranking. Each prompt-optimization and model-upgrade entry averages five intervention runs. No uncertainty is implied by this mean-only table.

Workflow	Target	$G\text{-norm}$ gain (pp)	$G\text{-norm}$ $\tau_{b}$	Workflow	Target	$G\text{-norm}$ gain (pp)	$G\text{-norm}$ $\tau_{b}$
HumanEval	32B	$+11.515\,{\scriptstyle\pm 3.636}$	$0.778\,{\scriptstyle\pm 0.222}$	DROP	32B	$+6.633\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$
	3.7-plus	$+21.212\,{\scriptstyle\pm 6.061}$	$0.778\,{\scriptstyle\pm 0.222}$		3.7-plus	$+12.822\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$
	235B	$+19.192\,{\scriptstyle\pm 5.051}$	$0.778\,{\scriptstyle\pm 0.222}$		235B	$+11.126\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$
	DeepSeek-V3.2	$+15.758\,{\scriptstyle\pm 3.030}$	$0.778\,{\scriptstyle\pm 0.222}$		DeepSeek-V3.2	$+8.969\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$
	GPT-5 nano	$+19.394\,{\scriptstyle\pm 5.455}$	$0.778\,{\scriptstyle\pm 0.222}$		GPT-5 nano	$+6.503\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$
MBPP	32B	$+7.442\,{\scriptstyle\pm 0.000}$	$0.889\,{\scriptstyle\pm 0.111}$	HotpotQA	32B	$+5.739\,{\scriptstyle\pm 0.000}$	$0.778\,{\scriptstyle\pm 0.222}$

Appendix

Table 8: Expanded view of the IF-MAP uncertainty in Table 2 . Every

\pm

\mathrm{SE}_{\rm sel}

across the three prioritization runs against the per-node mean gains computed from five intervention runs. It is not

\mathrm{SE}_{\rm test}

. Zero

\mathrm{SE}_{\rm sel}

can result from identical selections despite nonzero test execution variability. The table covers all five upgrade targets. Target aliases match Table 2 .

Workflow	Upgrade target	Agent ranking (mean gain, pp)
HumanEval	Qwen3-32B	CustomCodeGenerate (+15.152) $>$ Test (+4.242) $>$ ScEnsemble (+0.606)
	Qwen3-235B-A22B	CustomCodeGenerate (+24.242) $>$ Test (+9.091) $>$ ScEnsemble (+0.000)
	Qwen3.7-Plus	CustomCodeGenerate (+27.273) $>$ Test (+9.091) $>$ ScEnsemble (+0.000)
	DeepSeek-V3.2	CustomCodeGenerate (+18.788) $>$ Test (+9.697) $>$ ScEnsemble (+0.000)
	GPT-5 nano	CustomCodeGenerate (+24.849) $>$ Test (+8.485) $>$ ScEnsemble (+0.606)
MBPP	Qwen3-32B	Test (+7.442) $>$ CodeGenerate (+6.744) $>$ FixCode (+0.930) $>$ ScEnsemble (-0.233)

Appendix

Table 9: Agent rankings by mean model-upgrade test gain for HumanEval, MBPP, NQ. Each row ranks nodes from highest to lowest gain for one target model. Parentheses give mean gains in percentage points;

=

denotes a tie. Gains use the references in Table 7 ; rankings are computed before display rounding.

Workflow	Upgrade target	Agent ranking (mean gain, pp)
DROP	Qwen3-32B	Solve (+6.633) $>$ Format (+3.425)
	Qwen3-235B-A22B	Solve (+11.126) $>$ Format (+7.346)
	Qwen3.7-Plus	Solve (+12.822) $>$ Format (+7.448)
	DeepSeek-V3.2	Solve (+8.969) $>$ Format (+4.947)
	GPT-5 nano	Solve (+6.503) $>$ Format (+2.658)
HotpotQA	Qwen3-32B	AnswerGenerate (+5.739) $>$ ScEnsemble (+0.403) $>$ FormatAnswer (+0.359)

Appendix

Table 10: Agent rankings by mean model-upgrade test gain for DROP, HotpotQA, MATH. Each row ranks nodes from highest to lowest gain for one target model. Parentheses give mean gains in percentage points;

=

denotes a tie. Gains use the references in Table 7 ; rankings are computed before display rounding.

Algorithm	HumanEval		MBPP		NQ		DROP		HotpotQA		MATH		Macro (6)
	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\overline{\Delta}\uparrow$	$\overline{\tau}_{b}\uparrow$
IF-MAP (ours)	+4.44 $\pm 0.20$	0.556 $\pm 0.222$	+7.91 $\pm 0.00$	0.889 $\pm 0.111$	+2.14 $\pm 0.00$	0.778 $\pm 0.222$	+4.99 $\pm 0.00$	1.000 $\pm 0.000$	+1.53 $\pm 0.00$	0.556 $\pm 0.222$	+1.85 $\pm 0.15$	0.667 $\pm 0.067$	+3.81 $\pm 0.05$	0.741 $\pm 0.056$
Who&When-AAO	+3.23 $\pm 1.01$	-0.222 $\pm 0.401$	+3.02 $\pm 0.00$	0.222 $\pm 0.111$	+2.14 $\pm 0.00$	0.333 $\pm 0.000$	+4.99 $\pm 0.00$	1.000 $\pm 0.000$	+1.53 $\pm 0.00$	0.333 $\pm 0.000$	+1.38 $\pm 0.00$	-0.035 $\pm 0.070$	+2.72 $\pm 0.17$	0.272 $\pm 0.075$
A2P	+3.23 $\pm 1.01$	-0.556 $\pm 0.222$	+3.02 $\pm 0.00$	0.000 $\pm 0.000$	+2.14 $\pm 0.00$	0.333 $\pm 0.000$	+4.99 $\pm 0.00$	1.000 $\pm 0.000$	+1.53 $\pm 0.00$	0.333 $\pm 0.000$	+1.38 $\pm 0.00$	0.004 $\pm 0.164$	+2.72 $\pm 0.17$	0.186 $\pm 0.028$
Adapted locator	+4.55 $\pm 0.00$	0.816 $\pm 0.000$	+5.27 $\pm 2.64$	0.516 $\pm 0.289$	-0.11 $\pm 0.00$	-0.333 $\pm 0.000$	+2.98 $\pm 2.01$	0.333 $\pm 0.667$	-0.55 $\pm 0.00$	-0.778 $\pm 0.222$	-0.40 $\pm 0.36$	-0.713 $\pm 0.059$	+1.96 $\pm 0.46$	-0.026 $\pm 0.038$
AgentRx	+3.74 $\pm 0.51$	-0.161 $\pm 0.342$	+0.16 $\pm 0.16$	-0.411 $\pm 0.297$	+2.14 $\pm 0.00$	0.333 $\pm 0.000$	+4.99 $\pm 0.00$	1.000 $\pm 0.000$	+1.53 $\pm 0.00$	0.333 $\pm 0.000$	+1.05 $\pm 0.33$	-0.084 $\pm 0.221$	+2.27 $\pm 0.10$	0.169 $\pm 0.063$

Appendix

Table 11: Prompt optimization with standard errors: complete mean

\pm

SE version of Table 1 . Each dataset has separate columns for selected gain and Kendall

\tau_{b}

; the final pair gives six-dataset macro means.

A. Qwen3-32B
Algorithm	HumanEval		MBPP		NQ		DROP		HotpotQA		MATH		Macro (6)
	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\overline{\Delta}\uparrow$	$\overline{\tau}_{b}\uparrow$
IF-MAP (ours)	+11.52 $\pm 3.64$	0.778 $\pm 0.222$	+7.44 $\pm 0.00$	0.889 $\pm 0.111$	+10.63 $\pm 0.00$	0.778 $\pm 0.222$	+6.63 $\pm 0.00$	1.000 $\pm 0.000$	+5.74 $\pm 0.00$	0.778 $\pm 0.222$	+3.76 $\pm 1.19$	0.600 $\pm 0.115$	+7.62 $\pm 0.54$	0.804 $\pm 0.036$
Who&When-AAO	+10.30 $\pm 4.85$	0.494 $\pm 0.417$	+6.74 $\pm 0.00$	0.222 $\pm 0.111$	+10.63 $\pm 0.00$	0.333 $\pm 0.000$	+6.63 $\pm 0.00$	1.000 $\pm 0.000$	+5.74 $\pm 0.00$	1.000 $\pm 0.000$	+1.65 $\pm 0.00$	0.176 $\pm 0.070$	+6.95 $\pm 0.81$	0.538 $\pm 0.074$
A2P	+10.30 $\pm 4.85$	0.111 $\pm 0.222$	+6.74 $\pm 0.00$	0.000 $\pm 0.000$	+10.63 $\pm 0.00$	0.333 $\pm 0.000$	+6.63 $\pm 0.00$	1.000 $\pm 0.000$	+5.74 $\pm 0.00$	1.000 $\pm 0.000$	+1.65 $\pm 0.00$	0.219 $\pm 0.167$	+6.95 $\pm 0.81$	0.444 $\pm 0.027$
Adapted locator	+9.70 $\pm 0.00$	0.816 $\pm 0.000$	+4.88 $\pm 2.56$	0.516 $\pm 0.289$	-0.30 $\pm 0.00$	-0.333 $\pm 0.000$	+5.56 $\pm 1.07$	0.333 $\pm 0.667$	+0.40 $\pm 0.00$	-0.111 $\pm 0.222$	-0.91 $\pm 0.51$	-0.102 $\pm 0.058$	+3.22 $\pm 0.44$	0.187 $\pm 0.040$

Appendix

Table 12: Model upgrades with standard errors: complete mean

\pm

SE version of Table 2 . Each panel uses one target model and its own uniform-agent reference.

D. DeepSeek-V3.2
Algorithm	HumanEval		MBPP		NQ		DROP		HotpotQA		MATH		Macro (6)
	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\Delta\uparrow$	$\tau_{b}\uparrow$	$\overline{\Delta}\uparrow$	$\overline{\tau}_{b}\uparrow$
IF-MAP (ours)	+15.76 $\pm 3.03$	0.778 $\pm 0.222$	+12.33 $\pm 0.00$	0.889 $\pm 0.111$	+23.08 $\pm 0.00$	0.778 $\pm 0.222$	+8.97 $\pm 0.00$	1.000 $\pm 0.000$	+5.18 $\pm 0.00$	0.556 $\pm 0.222$	+9.59 $\pm 2.70$	0.246 $\pm 0.186$	+12.48 $\pm 0.48$	0.708 $\pm 0.069$
Who&When-AAO	+12.53 $\pm 6.26$	0.494 $\pm 0.417$	+8.60 $\pm 0.00$	0.222 $\pm 0.111$	+23.08 $\pm 0.00$	0.333 $\pm 0.000$	+8.97 $\pm 0.00$	1.000 $\pm 0.000$	+5.18 $\pm 0.00$	0.333 $\pm 0.000$	+2.20 $\pm 0.00$	-0.704 $\pm 0.098$	+10.09 $\pm 1.04$	0.280 $\pm 0.081$
A2P	+12.53 $\pm 6.26$	0.111 $\pm 0.222$	+8.60 $\pm 0.00$	0.000 $\pm 0.000$	+23.08 $\pm 0.00$	0.333 $\pm 0.000$	+8.97 $\pm 0.00$	1.000 $\pm 0.000$	+5.18 $\pm 0.00$	0.333 $\pm 0.000$	+2.20 $\pm 0.00$	-0.639 $\pm 0.127$	+10.09 $\pm 1.04$	0.190 $\pm 0.028$
Adapted locator	+14.24 $\pm 0.00$	0.816 $\pm 0.000$	+8.29 $\pm 4.03$	0.516 $\pm 0.289$	+1.70 $\pm 0.00$	-0.333 $\pm 0.000$	+7.63 $\pm 1.34$	0.333 $\pm 0.667$	+1.47 $\pm 0.00$	-0.778 $\pm 0.222$	+3.86 $\pm 0.33$	-0.074 $\pm 0.096$	+6.20 $\pm 0.55$	0.080 $\pm 0.028$

Appendix

Table 13: Model upgrades with standard errors: complete mean

\pm

SE version of Table 2 . Each panel uses one target model and its own uniform-agent reference.

		Prompt optimization		Model upgrades
Workflow	Score	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$
HumanEval	$G\text{-norm}$	$+4.444\,{\scriptstyle\pm 0.202}$	$0.556\,{\scriptstyle\pm 0.222}$	$+17.414\,{\scriptstyle\pm 4.647}$	$0.778\,{\scriptstyle\pm 0.222}$
	$I\text{-abs}$	$+4.849\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$	$+8.121\,{\scriptstyle\pm 0.000}$	$0.333\,{\scriptstyle\pm 0.000}$
	$\mathrm{MeanAbs}$	$+4.849\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$	$+8.121\,{\scriptstyle\pm 0.000}$	$0.333\,{\scriptstyle\pm 0.000}$
MBPP	$G\text{-norm}$	$+7.907\,{\scriptstyle\pm 0.000}$	$0.889\,{\scriptstyle\pm 0.111}$	$+13.955\,{\scriptstyle\pm 0.000}$	$0.889\,{\scriptstyle\pm 0.111}$
	$I\text{-abs}$	$+4.651\,{\scriptstyle\pm 1.628}$	$0.778\,{\scriptstyle\pm 0.111}$	$+9.148\,{\scriptstyle\pm 2.404}$	$0.778\,{\scriptstyle\pm 0.111}$

Appendix

Table 14: Per-workflow scoring-rule ablation. Each cell reports the mean

\pm\mathrm{SE}_{\rm sel}{}

over three prioritization runs. Model-upgrade gains first average each node over all five target models.

		Prompt optimization		Model upgrades
Workflow	Probes	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$	Gain (pp) $\uparrow$	$\tau_{b}\uparrow$
HumanEval	3	$+4.849\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$	$+8.121\,{\scriptstyle\pm 0.000}$	$0.333\,{\scriptstyle\pm 0.000}$
	5	$+4.849\,{\scriptstyle\pm 0.000}$	$1.000\,{\scriptstyle\pm 0.000}$	$+8.121\,{\scriptstyle\pm 0.000}$	$0.333\,{\scriptstyle\pm 0.000}$
	Full	$+4.444\,{\scriptstyle\pm 0.202}$	$0.556\,{\scriptstyle\pm 0.222}$	$+17.414\,{\scriptstyle\pm 4.647}$	$0.778\,{\scriptstyle\pm 0.222}$
MBPP	3	$+5.426\,{\scriptstyle\pm 2.481}$	$0.667\,{\scriptstyle\pm 0.192}$	$+10.587\,{\scriptstyle\pm 3.368}$	$0.667\,{\scriptstyle\pm 0.192}$
	5	$+5.426\,{\scriptstyle\pm 2.481}$	$0.667\,{\scriptstyle\pm 0.192}$	$+10.587\,{\scriptstyle\pm 3.368}$	$0.667\,{\scriptstyle\pm 0.192}$

Appendix

Table 15: Per-workflow perturbation-budget comparison using

G\text{-norm}

. Each cell reports the mean

\pm\mathrm{SE}_{\rm sel}{}

over three prioritization runs. The full setting uses at most ten formal probes per node.

Method	Qwen3-8B requests	Qwen tokens (M)	DeepSeek-V3.2 requests	DeepSeek tokens (M)
IF-MAP	$63{,}088$ $(100.00\%)$	$47.141$ $(100.00\%)$	$662$ $(100.00\%)$	$3.049$ $(100.00\%)$
Who&When-AAO	$35{,}288$ $(55.93\%)$	$26.672$ $(56.58\%)$	$1{,}943$ $(293.50\%)$	$18.341$ $(601.54\%)$
A2P	$35{,}288$ $(55.93\%)$	$26.672$ $(56.58\%)$	$1{,}938$ $(292.75\%)$	$18.645$ $(611.51\%)$
Adapted locator	$35{,}288$ $(55.93\%)$	$26.672$ $(56.58\%)$	$3{,}197$ $(482.93\%)$	$21.138$ $(693.28\%)$
AgentRx	$35{,}288$ $(55.93\%)$	$26.672$ $(56.58\%)$	$8{,}506$ $(1{,}284.89\%)$	$202.576$ $(6{,}644.01\%)$
CHIEF	$35{,}288$ $(55.93\%)$	$26.672$ $(56.58\%)$	$10{,}225$ $(1{,}544.56\%)$	$142.013$ $(4{,}657.69\%)$

Appendix

Table 16: Cost of one agent-prioritization run over all six datasets.

Method	Qwen3-8B requests	Qwen tokens (M)	DeepSeek-V3.2 (TextGrad) requests	DeepSeek (TextGrad) tokens (M)
IF-MAP	$203{,}546.74$ $(100.00\%)$	$352.994$ $(100.00\%)$	$950.00$ $(100.00\%)$	$4.249$ $(100.00\%)$
Full-Agent PromptOpt	$663{,}341.97$ $(325.89\%)$	$1{,}366.978$ $(387.25\%)$	$1{,}662.01$ $(174.95\%)$	$6.559$ $(154.37\%)$

Appendix

Table 17: Prompt-optimization cost over all six datasets. IF-MAP first selects an agent and optimizes only that agent; Full-Agent PromptOpt applies the same custom prompt optimizer to every agent.

Target	Method	Qwen3-8B requests	Qwen3-8B tokens (M)	DeepSeek-V3.2 (selection) requests	DeepSeek-V3.2 (selection) tokens (M)	Target model (upgrade) requests	Target model (upgrade) tokens (M)
Qwen3-32B	IF-MAP	$91{,}748$ $(100.00\%)$	$151.102$ $(100.00\%)$	$794$ $(100.00\%)$	$4.028$ $(100.00\%)$	$1{,}557$ $(100.00\%)$	$1.541$ $(100.00\%)$
Qwen3-32B	FullSwap	$12{,}553$ $(13.68\%)$	$15.710$ $(10.40\%)$	$0$ $(0.00\%)$	$0$ $(0.00\%)$	$4{,}066$ $(261.14\%)$	$4.389$ $(284.82\%)$
Qwen3-235B-A22B	IF-MAP	$91{,}748$ $(100.00\%)$	$151.102$ $(100.00\%)$	$794$ $(100.00\%)$	$4.028$ $(100.00\%)$	$1{,}534$ $(100.00\%)$	$1.601$ $(100.00\%)$
Qwen3-235B-A22B	FullSwap	$12{,}518$ $(13.64\%)$	$15.730$ $(10.41\%)$	$0$ $(0.00\%)$	$0$ $(0.00\%)$	$4{,}085$ $(266.30\%)$	$4.481$ $(279.89\%)$
Qwen3.7-Plus	IF-MAP	$91{,}748$ $(100.00\%)$	$151.102$ $(100.00\%)$	$794$ $(100.00\%)$	$4.028$ $(100.00\%)$	$1{,}524$ $(100.00\%)$	$1.781$ $(100.00\%)$
Qwen3.7-Plus	FullSwap	$12{,}774$ $(13.92\%)$	$16.030$ $(10.61\%)$	$0$ $(0.00\%)$	$0$ $(0.00\%)$	$4{,}134$ $(271.26\%)$	$4.923$ $(276.42\%)$

Appendix

Table 18: Model-upgrade cost for one complete run over all six datasets. Each IF-MAP row includes agent selection, one original test-set baseline, and one post-upgrade test-set evaluation. FullSwap upgrades and evaluates every agent and therefore has no DeepSeek-V3.2 selection-stage cost.

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,001 papers

Sep 8, 2026

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

Large language model (LLM)-based multi-agent systems (MAS) achieve strong performance by employing specialized multiple agents, yet their performance depends on the prompt design of each agent. For MAS prompt optimization, textual gradient methods that guide prompt updates using natural-language feedback have emerged as a leading paradigm. In this paper, we identify limitations in two stages of existing textual gradient approaches: gradient extraction and gradient aggregation. In gradient extraction, previous works select a target prompt without verifying whether modifying it resolves the failure, and derive gradients without agent-level supervision over the corresponding agent's intermediate output. In gradient aggregation, individual gradients are randomly grouped and concatenated, often mixing unrelated failure modes and producing prompts that fail to generalize. To address these limitations, we propose AgentGrad, a prompt optimization framework for multi-agent systems based on sequential intervention and semantic textual gradient abstraction. For each failure, sequential intervention modifies the behavior of one agent at a time to identify the target agent whose modification resolves the failure. The modified output of the target agent then serves as agent-level supervision for extracting a fine-grained gradient. Semantic textual gradient abstraction clusters semantically similar gradients to prevent mixing unrelated failure modes, and abstracts each cluster into a generalized gradient that captures the shared corrective pattern. Experimental results show that AgentGrad achieves state-of-the-art performance across five MAS benchmarks while reducing wall-clock optimization time by

2.5\times

and optimization cost by 21.8% on average compared to the next-best baselines.

Jaewon Chu, Jinwoo Seo, Jaewon Cho +4

Korea University · KAIST · Meta AI +1

Multi-Agent LLM Systems Multi-Agent System Optimization Prompt Optimization

Ask PDF

May 26, 2026

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

As multi-agent systems (MAS) become increasingly complex, identifying the contributions of individual agents is critical for system optimization. However, existing approaches lack a rigorous, unified framework for credit assignment. In this work, we formalize agent attribution as a cooperative game, parameterized by the coalition distribution, removal protocol, and target metric. Using this framework, we show that Leave-One-Out (LOO) identifies bottleneck agents as effectively as combinatorial methods, but at a fraction of the computational cost. We also demonstrate that removal protocols induce distinct games: Agent ablation isolates structural bottlenecks, whereas introspective LLM judges fail to faithfully approximate this behavior. Furthermore, to evaluate the utility of specific agent backbones, we introduce attribution via model replacement. By substituting underlying models of low-contribution agents, we improve task performance by up to 17% while reducing cost by up to 35% across three benchmarks. Finally, we apply our framework to audit a medical MAS, revealing that agent contributions to diagnostic accuracy and ethical behavior are often decoupled. By intervening on counterproductive roles, we observe an increase in ethics alignment while maintaining diagnostic accuracy. Overall, this work provides a principled approach for cost-effective MAS attribution and intervention.

Mingyu Lu, Yushan Huang, Chris Lin +1

Paul G. Allen School of Computer Science & Engineering, University of Washington

Multi-Agent System Optimization Cooperative Game Theory Feature Attribution Multi-Agent LLMs

Ask PDF

May 13, 2026

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution

LLM-based multi-agent systems have demonstrated strong performance across complex real-world tasks, such as software engineering, predictive modeling, and retrieval-augmented generation. Yet automating their configuration remains a structural challenge, as scores are available only at the system level, whereas the parameters governing agent behavior are local. We argue that optimizing these systems is fundamentally a credit-assignment problem. We therefore introduce CANTANTE, a framework that decomposes system-level rewards into per-agent update signals by contrasting rollouts of multiple joint configurations on the same query. We instantiate it for prompt optimization, treating agent prompts as learnable system parameters. We evaluate CANTANTE against GEPA and MIPROv2 on programming (MBPP), mathematical reasoning (GSM8K), and multi-hop question answering (HotpotQA). Across these benchmarks, CANTANTE achieves the best average rank among all evaluated optimizers and consistently outperforms unoptimized prompts. It improves over the strongest baseline by +18.9 percentage points on MBPP and +12.5 percentage points on GSM8K, while incurring a lower inference cost. It remains within one standard deviation of the strongest baseline on HotpotQA. Crucially, our credit correlation analysis confirms that the attributer produces meaningful per-agent signals rather than echoing the global system score.

Tom Zehle

University of Freiburg · ELLIS Institute Tübingen

Multi-Agent LLM Systems Multi-Agent System Optimization Large Language Model-Guided Optimization LLM Agent Workflow Optimization Prompt Optimization Multi-Agent Reinforcement Learning

Ask PDF

Abstract

Multi-Agent LLM Systems

Explore similar work

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution