Bad Apples in the Kitchen: How LLM Teams Detect, Correct, and Calibrate Against Their Misaligned Members
Organizations: Seoul National University, Seoul, Republic of Korea
Abstract
What protects an LLM team when one teammate quietly goes wrong? Most multi-agent LLM work assumes cooperative partners and inspects chat between turns before acting. Real-time embodied cooperation does not allow this pause: chat runs alongside action, so a teammate whose words stay cooperative while behavior diverges can cause damage faster than the team can name what is happening. We study this case by injecting three theory-grounded misaligned agents into a custom four-player Overcooked environment, each stressing a different joint-action layer (speech-action mismatch, role invasion, and aggregate-progress disruption), and decompose robustness into three sequential gates of detection, correction, and calibration through a 3x4x2 design (n=720 episodes). Nearly all trolls are eventually named, but recovery follows only when attribution and situational diagnosis appear in the same utterance; direct source-attribution prompting improves the hardest case only by degrading the troll-free baseline by 37.5%. Robustness, then, depends less on detection alone than on whether team chat combines attribution with diagnosis in a single utterance, suggesting that sensitivity to misalignment should be targeted rather than uniformly raised.
Figures & tables
| Condition | D1 | D1+D2 | D1+D2+D3 |
|---|---|---|---|
| Mean soups completed | |||
| Reneger | 5.9 | 6.4 | 8.7 |
| Misallocator | 6.2 | 6.4 | 6.7 |
| Disruptor | 4.7 | 5.0 | 5.5 |
| % episodes reaching stage (single-utterance criterion) | |||
| Reneger | 90% | 61% | 12% |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Condition | full | mini | nano |
|---|---|---|---|
| Mean soups ( SD) | |||
| None | 6.9 2.4 | 7.4 2.1 | 4.6 2.0 |
| L1 | 5.4 2.8 | 6.0 2.8 | 3.4 1.9 |
| L2 | 5.6 2.6 | 6.3 2.4 | 3.0 1.7 |
| L3 | 4.3 2.3 | 4.7 2.3 | 2.1 1.5 |
| Detection step | |||
| Condition | GPT | Gemini | Qwen |
|---|---|---|---|
| Mean soups ( SD) | |||
| None | 7.4 2.1 | 6.6 2.3 | 5.9 2.2 |
| L1 | 6.0 2.8 | 5.4 2.7 | 4.8 2.5 |
| L2 | 6.3 2.4 | 5.0 2.5 | 4.6 2.4 |
| L3 | 4.7 2.3 | 3.9 2.1 | 3.4 2.0 |
| Detection step | |||
| Condition | P1 | P2 | P3 | P4 |
|---|---|---|---|---|
| Mean soups ( SD) | ||||
| L1 | 5.8 2.1 | 6.3 2.8 | 5.6 2.0 | 5.8 2.2 |
| L2 | 5.5 2.2 | 6.0 2.4 | 6.0 2.5 | 5.5 2.1 |
| L3 | 4.4 1.8 | 4.7 2.3 | 5.2 1.9 | 5.0 1.7 |
| Detection step | ||||
| L1 | 44 | 47 | 50 | 46 |
| Layer rate (%) | Total | |||
|---|---|---|---|---|
| Condition | L1 | L2 | L3 | (%) |
| Baseline | 1.8 | 2.7 | 2.1 | 6.5 |
| Reneger | 29.4 | 4.2 | 3.7 | 30.3 |
| Misallocator | 5.1 | 30.8 | 5.6 | 32.0 |
| Disruptor | 3.4 | 5.9 | 29.6 | 31.2 |
| Dimension | Categories | Cohen’s |
|---|---|---|
| D1 Attribution | troll-attributed | 0.89 |
| D2 Action orient. | prescriptive | 0.74 |
| D3 Locus | other-directed | 0.81 |
| Map | Troll | ||||
|---|---|---|---|---|---|
| cramped | L1 | 130 | .017 | .16 | .44 |
| L2 | 155 | .057 | .51 | .33 | |
| L3 | 53 | .001 | .001 ∗∗∗ | .77 | |
| asymm | L1 | 182 | .154 | 1.00 | .22 |
| L2 | 166 | .089 | .80 | .29 | |
| L3 | 68 | .001 | .002 ∗∗ | .71 |
| Condition | 95% CI | ||||
|---|---|---|---|---|---|
| None | 12 | .001 | .001 ∗∗∗ | .95 | |
| Reneger | 221 | .813 | 1.00 | .05 | |
| Misallocator | 67 | .001 | .003 ∗∗ | .71 | |
| Disruptor | 15 | .001 | .001 ∗∗∗ | .94 |
Multi-Agent LLM Systems
Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.
See all 1,119 papers