Beyond Division of Labor: How Roles Shape Feedback and Recovery in Multi-Agent Systems
Organizations: Argonne National Laboratory, United States · University of Houston, United States
Abstract
Roles in multi-agent large language model (LLM) systems allocate decisions as well as work. In a Planner--Executor--Reviewer team, the Reviewer's approval can determine when the team submits an answer and receives corrective feedback. We study both functions by varying role instructions while holding agents, topology, call budget, and ground-truth evaluator constant, using three open-weight models on Omni-MATH and two on SciBench. Removing roles largely eliminates functional specialization without lowering first-pass accuracy. After a failed first answer, however, submission decisions matter: a role-structured Reviewer often withholds approval, so its team submits less often and receives fewer evaluator verdicts. In an exploratory comparison with feedback available to both teams, the role-free team recovers more often across all five model--benchmark settings. Two further analyses support the importance of feedback opportunities. In a post hoc comparison at equal numbers of verdicts, the pooled recovery difference is small (role-structured minus role-free: , 95% interval ). In an exploratory intervention on Omni-MATH problems with gemma-4, keeping the role prompts but forcing submission in every cycle raises recovery from to ; the paired comparison detects no difference from the role-free team. The takeaway is to evaluate roles through recovery after failure as well as specialization. For systems with reliable external feedback, this requires measuring who may submit, how often the team receives feedback, and which agents can act on it.
Figures & tables
| functional collapse | adjudicated no division | ||||
|---|---|---|---|---|---|
| actor | RS | RF - | RF | RS | RF - |
| gemma-4-31B | |||||
| gpt-oss-120b | |||||
| Llama-3.3-70B | |||||
| recovery | recovered only by (paired) | verdict-matched | final accuracy | submissions | RS never | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| actor, domain ( ) | RS | RF | RF - | RS | RF | RS | RF | RS | RF | RS | RF | submits | ||||
| gemma-4, Omni-MATH ( ) | ||||||||||||||||
| gpt-oss, Omni-MATH ( ) | ||||||||||||||||
| Llama-3.3, Omni-MATH ( ) | ||||||||||||||||
| gemma-4, SciBench ( ) | ||||||||||||||||
| gpt-oss, SciBench ( ) | ||||||||||||||||
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| functional | exact byte | adjudicated | |
| actor | collapse | duplication | no division |
| role-structured (RS) | |||
| gemma-4-31B-it | |||
| gpt-oss-120b | |||
| Llama-3.3-70B-Instruct | |||
| role-free, feedback withheld (RF - ) | |||
| Marginal population: first pass paired; recovery on each condition’s own denominator | ||||||
|---|---|---|---|---|---|---|
| first pass | recovery first verdict wrong | |||||
| actor | RS | RF - | difference | RS ( ) | RF - ( ) | difference |
| gemma-4-31B-it | (316) | (316) | ||||
| gpt-oss-120b | (463) | (380) | ||||
| Llama-3.3-70B-Instruct | (276) | (268) | ||||
| Paired population: problems first-pass wrong under both conditions; one shared denominator | ||||||
| first pass | recovery | paired recovery | final | paired final | |
| Omni-MATH | |||||
| role-structured (frozen) | – | – | |||
| role-free, with feedback, run 1 | vs , | vs , | |||
| role-free, with feedback, run 2 | vs , | vs , | |||
| SciBench | |||||
| role-structured (frozen) | – | – | |||
| RS declining cycles | clause | RS never | free-cycle approval | by 2nd verdict | per later verdict | approvals | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| actor, domain | before | FAIL | free | share | submits | RS | RF | RS | RF | RS | RF | RS | RF |
| gemma-4-31B-it, Omni-MATH | |||||||||||||
| gpt-oss-120b, Omni-MATH | |||||||||||||
| Llama-3.3-70B-Instruct, Omni-MATH | |||||||||||||
| gemma-4-31B-it, SciBench | |||||||||||||
| gpt-oss-120b, SciBench | |||||||||||||
| cell | paired | forced ties | RS never submitted | problems left | held-answer recodes (RS / RF) |
|---|---|---|---|---|---|
| gemma-4-31B-it, Omni-MATH | / | ||||
| gpt-oss-120b, Omni-MATH | / | ||||
| Llama-3.3-70B-Instruct, Omni-MATH | / | ||||
| gemma-4-31B-it, SciBench | / | ||||
| gpt-oss-120b, SciBench | / |
| condition | first pass | recovery | final accuracy | verdicts | Reviewer approvals |
|---|---|---|---|---|---|
| role-structured, every cycle submitted | |||||
| role-structured, re-run after the freeze | |||||
| role-structured, frozen snapshot | |||||
| role-free with feedback |
| first pass (paired, all problems) | recovery (paired on first-pass wrong under both) | |||||||||||
| actor | RS | RF - | diff. | exact | RS | RF - | diff. | RS only | RF - only | exact | ||
| gemma-4-31B-it | ||||||||||||
| gpt-oss-120b | ||||||||||||
| Planner authors | locked answer | recovery | final accuracy, SE RS | |||
|---|---|---|---|---|---|---|
| actor | answer (RF - ) | (RF - / RS) | (RS) | attrition | pre-registered | as graded |
| gemma-4-31B-it | (0.986) | (0.845/0.288) | (0.222) | [-0.0387,-0.0063] | [+0.0054,+0.0333] | |
| gpt-oss-120b | (0.916) | (0.208/0.052) | (0.324) | [-0.0219,+0.0112] | [+0.0215,+0.0507] | |
| Llama-3.3-70B-Instruct | (0.958) | (0.806/0.750) | (0.014) | [-0.0200,+0.0550] | [+0.0030,+0.0876] | |
| endpoint | disposition | ||||
| P1 intervention fidelity | failed as written; re-designated. The interface forbids an answer field, not | ||||
| (also the manipulation check) | computation. Re-scoped to answer authorship, which it drives to in all three | ||||
| actors (Table S8 ). Reported as a second dissociation, not as a pass. | |||||
| P2 actor interv. on fidelity | not estimable across the panel: the fidelity clauses are condition-specific, so no | ||||
| common scale exists. Reported as unresolved rather than substituted. | |||||
| P3 intervention accuracy (enforced role-structured), Holm across three: | |||||
| RS | RF - | ||||
| context quartile | recovery | recovery | difference | ||
| gemma-4-31B-it | |||||
| Q1 (least context) | |||||
| Q2 | |||||
| Q3 | |||||
| Q4 (most) | |||||
| organization | team | single agent | difference |
|---|---|---|---|
| gemma-4-31B-it | |||
| role-structured | |||
| , , | |||
| role-free | |||
| , , | |||
| gpt-oss-120b | |||
Multi-Agent Systems
Multi-Agent Systems studies how autonomous agents coordinate, compete, communicate, and learn in shared or interdependent tasks.
See all 709 papers