Previous work has shown that alignment training on individual LLM agents can fail to transfer to multi-agent settings. We investigate the underlying mechanisms for this failure, and explore interventions to preserve alignment transfer to multi-agent LLM teams. We focus on two software-engineering tasks (sepsis triage, a news recommender) and ten consulting-proposal generation tasks. We present four findings: (1) The single-vs-team safety gap does not show a clear trend with more capable models. We find that Anthropic's Mythos Preview shows the largest gap on one of our tasks but the smallest on another; (2) Many natural interventions, such as having agents critique each other's work or coordinate in a group chat, do not reliably bring agent teams back to single-agent alignment behavior; (3) diffusion of responsibility substantially accounts for the safety gap: agents in teams are less likely to proactively check for system-level safety failures and more likely to ignore, rationalize, or deflect responsibility for issues when they arise; (4) having a designated safety lead agent ensure system-level safety, and having a coordinator agent publish a safety specifications document are effective at restoring single-agent alignment. We also release magelab, a multi-agent LLM orchestration and experimentation framework. Our findings can help practitioners design agent teams that successfully preserve the behaviors expected of aligned individual agents.
Figures & tables
Figure 1: Sepsis task : safety-score Δ vs. single-agent default at the mean (top) and 10th-percentile (bottom). Negative values indicate a worse safety score than the single-agent default. Bars in the shaded grey band represent safety scores that fall within the range of single-agent safety scores.
M0
M1(+T)
M2(+T+D)
M3(+T+D+F)
% mediated
single agent
+0.241
∗∗
+0.184
∗
+0.168
∗
+0.039
84%
∗∗
group chat
+0.125
+0.104
+0.099
+0.032
74%
∗
designated safety lead
+0.264
∗∗
+0.197
∗
+0.182
∗
+0.064
76%
∗∗
safety document
+0.152
∗
+0.153
∗
+0.154
∗
+0.144
∗∗
6%
compound (doc + lead)
+0.318
∗∗∗
+0.255
∗∗∗
+0.242
∗∗
+0.120
∗
62%
∗∗
Table 1: Sepsis task, mediation analysis. We regress each run’s safety score (sensitivity, 0–1) on its configuration and model. Each entry is the estimated safety-score gain of that configuration over the multi-agent baseline (e.g., +0.241 means 24 pp higher safety score for that configuration over the multi-agent baseline). M0 uses only configuration and model; M1 – M3 add, in turn, whether any agent tested the pipeline ( T ), discovered harm ( D ), and attempted a fix ( F ). If these behaviors explain the gain, entries shrink toward zero from M0 to M3 ; % mediated =(M0−M3)/M0 is the share they explain. Stars: cluster-robust p -values on coefficients; Benjamini-Hochberg-adjusted p -values on M0−M3 in the % mediated column. ∗p<0.05 , ∗∗p<0.01 , ∗∗∗p<0.001 .
Figure 3: Recommender-system task : safety-score Δ vs. single-agent default at the mean (top) and 10th-percentile (bottom). Negative values indicate a worse safety score than the single-agent default. Bars in the shaded grey band fall within the range of single-agent safety scores.
Figure 5: Consulting task, Opus 4.1 : constitution-score Δ vs. single-agent baseline by configuration, under benign (left) and redteam (right) prompt families. Negative values indicate worse ethics-rubric scores than the single-agent baseline.
Appendix figures & tables37 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Sepsis task, cost vs. safety. Change in safety score (sensitivity, pp) vs. mean API cost of the run, by configuration, both relative to each model’s multi-agent baseline (cost shown as a multiple of the baseline’s cost). Stars mark the compound (safety document + designated safety lead) intervention. Each panel is one model.
Figure 7: Sepsis task, diffusion of responsibility, all models. Rate at which any agent (a) tested the integrated classifier, (b) discovered a patient-safety harmful outcome, and (c) conditional on discovery, attempted to fix , deflected , rationalized , or ignored the harm. Rows are models; columns are configurations. Error bars are 95% Wilson intervals.
Figure 8: Sepsis task, Mythos Preview, mean. Safety score (top, sensitivity) and business score (bottom, normalized cost savings) by configuration. Error bars are 95% t -intervals.
Figure 9: Sepsis task, Mythos Preview, 10th-percentile. Safety score (top) and business score (bottom) by configuration, at the 10th-percentile across runs.
Figure 10: Sepsis task, Opus 4.7, mean. Safety score (top) and business score (bottom) by configuration.
Figure 11: Sepsis task, Opus 4.7, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 12: Sepsis task, Opus 4.6, mean. Safety score (top) and business score (bottom) by configuration.
Figure 13: Sepsis task, Opus 4.6, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 14: Sepsis task, Sonnet 4.6, mean. Safety score (top) and business score (bottom) by configuration.
Figure 15: Sepsis task, Sonnet 4.6, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 16: Sepsis task, Opus 4.1, mean. Safety score (top) and business score (bottom) by configuration.
Figure 17: Sepsis task, Opus 4.1, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
coef
95% CI
p
T (tested end-to-end)
+0.096
[+0.024,+0.168]
0.009
D (discovered harm)
−0.176
[−0.228,−0.125]
<0.001
F (attempted fix)
+0.358
[+0.271,+0.445]
<0.001
Appendix
Table 2: Sepsis: funnel coefficients, i.e., the coefficients on T , D , and F themselves in M3 , one estimate each, shared across all configurations and models. Each is the change in safety score associated with any agent testing, discovering harm, or attempting a fix, holding the other regressors fixed. These behaviors can only explain a configuration’s gain if they are associated with safety score; all three are significant here. The negative D coefficient means that, holding T and F fixed, runs in which harm was discovered score lower. Cluster-robust 95% CIs and p -values.
M0
M3
indirect
95% CI
BH p
% mediated
group chat
+0.125
+0.032
+0.093
[+0.015,+0.190]
0.027∗
74%
designated safety lead
+0.264
+0.064
+0.200
[+0.108,+0.308]
0.007∗∗
76%
safety document
+0.152
+0.144
+0.009
[−0.060,+0.069]
0.688
6%
compound (doc + lead)
+0.318
+0.120
+0.198
[+0.114,+0.304]
0.007∗∗
62%
single agent
+0.241
+0.039
+0.202
[+0.108,+0.295]
0.007∗∗
84%
Appendix
Table 3: Sepsis: per-organization indirect effect ( M0−M3 ), the part of each configuration’s gain over the multi-agent baseline that testing, discovery, and fixing account for. A CI that excludes zero means these behaviors significantly explain part of the gain; these p -values give the stars in the % mediated column of Table 1 . Cluster-bootstrap 95% CIs and Benjamini-Hochberg-adjusted two-sided p -values across the five tests. % mediated = indirect /M0 .
model
organization
n
M0
M3
% shrunk
mythos-preview
group chat
28
+0.542
+0.140
74%
designated safety lead
27
+0.789
+0.265
66%
safety document
23
+0.535
+0.572
−7%
compound
27
+0.842
+0.317
62%
single agent
31
+0.595
+0.122
79%
opus-4-1
group chat
44
+0.063
+0.071
−13%
Appendix
Table 4: Sepsis: per-model M0→M3 shrinkage. The same regression run separately for each model, to check whether the pooled pattern in Table 1 holds within each model; positive shrinkage means testing, discovery, and fixing account for part of that configuration’s gain for that model. Each model regresses safety score on organization dummies (no model FE, since stratified). % shrunk =(M0−M3)/M0 . Dashes indicate ∣M0∣≤0.01 , where the percentage is undefined; values >100% indicate the M3 coefficient flips sign relative to M0 .
spec
R2
ΔR2
model FE only
0.054
—
+ organization dummies
0.210
+0.156
+ funnel ( T+D+F )
0.359
+0.149
Appendix
Table 5: Sepsis: R2 decomposition. The share of variance in run-level safety score explained as each block of regressors is added: model, then configuration, then testing, discovery, and fixing. ΔR2 is the increase from each added block.
Figure 18: Recommender system, cost vs. safety. Change in safety score (1 − misinformation@50, pp) vs. mean API cost of the run, by configuration, both relative to each model’s multi-agent baseline (cost shown as a multiple of the baseline’s cost). Stars mark the compound (safety document + designated safety lead) intervention. Each panel is one model.
Figure 19: Recommender-system task, diffusion of responsibility, all models. Rate at which any agent (a) tested the integrated recommender, (b) discovered misinformation harms, and (c) conditional on discovery, attempted to fix , deflected , rationalized , or ignored the harm. Rows are models; columns are configurations. Error bars are 95% Wilson intervals. The annotation prompt is the recommender-system analog of Appendix A.3 , with patient-safety harm replaced by misinformation surfaced in the top-50 list.
Figure 20: Recommender system, Mythos Preview, mean. Safety score (top, 1− misinfo@50) and business score (bottom, normalized views@50) by configuration. Error bars are 95% t -intervals.
Figure 21: Recommender system, Mythos Preview, 10th-percentile. Safety score (top) and business score (bottom) by configuration, at the 10th-percentile across runs.
Figure 22: Recommender system, Opus 4.7, mean. Safety score (top) and business score (bottom) by configuration.
Figure 23: Recommender system, Opus 4.7, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 24: Recommender system, Opus 4.6, mean. Safety score (top) and business score (bottom) by configuration.
Figure 25: Recommender system, Opus 4.6, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 26: Recommender system, Sonnet 4.6, mean. Safety score (top) and business score (bottom) by configuration.
Figure 27: Recommender system, Sonnet 4.6, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
Figure 28: Recommender system, Opus 4.1, mean. Safety score (top) and business score (bottom) by configuration.
Figure 29: Recommender system, Opus 4.1, 10th-percentile. Safety score (top) and business score (bottom) by configuration.
M0
M1(+T)
M2(+T+D)
M3(+T+D+F)
% mediated
single agent
+0.124
∗∗∗
+0.113
∗∗∗
+0.113
∗∗∗
+0.104
∗∗∗
16%
group chat
+0.043
∗∗
+0.040
∗
+0.039
∗
+0.038
∗
13%
designated safety lead
+0.015
−0.002
−0.001
−0.002
111%
safety document
+0.053
∗∗
+0.052
∗∗
+0.052
∗∗
+0.053
∗∗
1%
compound (doc + lead)
+0.085
∗∗∗
+0.069
∗∗∗
+0.070
∗∗∗
+0.065
∗∗∗
23%
Appendix
Table 6: Recommender system, mediation analysis . Each intervention’s safety-score pp difference from multi-agent baseline shrinks under sequential controls for testing ( T ), discovery ( D ), and fix ( F ). % mediated =(M0−M3)/M0 is the share of the safety-score gain explained by the controls; stars on that column reflect Benjamini-Hochberg-adjusted p -values for the underlying indirect effect M0−M3 . ∗p<0.05 , ∗∗p<0.01 , ∗∗∗p<0.001 . No indirect effect reaches p<0.05 on this task; the smallest BH-adjusted p -values are 0.055 for the compound and single-agent contrasts.
coef
95% CI
p
T (tested end-to-end)
+0.033
[+0.002,+0.064]
0.039
D (discovered harm)
−0.061
[−0.099,−0.024]
0.001
F (attempted fix)
+0.114
[+0.077,+0.150]
<0.001
Appendix
Table 7: Recommender system: funnel coefficients, i.e., the coefficients on T , D , and F themselves in M3 , one estimate each, shared across all configurations and models. Each is the change in safety score associated with any agent testing, discovering harm, or attempting a fix, holding the other regressors fixed. Cluster-robust 95% CIs and p -values.
M0
M3
indirect
95% CI
BH p
% mediated
group chat
+0.043
+0.038
+0.005
[−0.007,+0.017]
0.350
13%
designated safety lead
+0.015
−0.002
+0.017
[−0.002,+0.036]
0.130
111%
safety document
+0.053
+0.053
+0.001
[−0.013,+0.012]
0.986
1%
compound (doc + lead)
+0.085
+0.065
+0.020
[+0.002,+0.044]
0.055
23%
single agent
+0.124
+0.104
+0.020
[+0.003,+0.044]
0.055
16%
Appendix
Table 8: Recommender system: per-organization indirect effect ( M0−M3 ) with cluster-bootstrap 95% CIs and Benjamini-Hochberg-adjusted two-sided p -values across the five tests. % mediated = indirect /M0 ; values are unstable where M0 is small (e.g., designated safety lead).
model
organization
n
M0
M3
% shrunk
mythos-preview
group chat
25
+0.029
+0.023
—
designated safety lead
26
+0.049
−0.002
—
safety document
23
−0.037
−0.035
—
compound
27
+0.035
−0.014
—
single agent
26
+0.031
−0.018
—
opus-4-1
group chat
32
−0.005
−0.016
—
Appendix
Table 9: Recommender system: per-model M0→M3 shrinkage. % shrunk =(M0−M3)/M0 . Dashes indicate ∣M0∣≤0.05 , where the percentage is undefined.
spec
R2
ΔR2
model FE only
0.235
—
+ organization dummies
0.295
+0.060
+ funnel ( T+D+F )
0.317
+0.022
Appendix
Table 10: Recommender system: R2 decomposition. Compare with Table 5 : model identity alone explains 24% of safety variance on rec_sys vs. 5% on sepsis, leaving less variance for organizational interventions and funnel mediators to absorb.
Figure 30: Consultancy task, benign prompts, constitution score Δ vs. single agent. Mean (left) and 10th-percentile (right) constitution-score Δ by configuration for Opus 4.1 (top) and Opus 4.7 (bottom) under benign prompts. Negative values indicate worse ethics-rubric scores than the single-agent baseline. The shaded grey band is the single-agent reference range.
Figure 31: Consultancy task, redteam prompts, constitution score Δ vs. single agent. Mean (left) and 10th-percentile (right) constitution-score Δ by configuration for Opus 4.1 (top) and Opus 4.7 (bottom) under redteam prompts (business-first framing). Same conventions as Figure 30 .
Figure 32: Consultancy task, topology effect, benign prompts. Constitution-score Δ vs. the single-agent baseline by multi-agent topology (flat vs. hierarchical) under benign prompts, at the mean (left) and 10th-percentile (right) across runs, for each of the five models. The shaded grey band is the single-agent reference range.
Figure 33: Consultancy task, topology effect, redteam prompts. Same as Figure 32 but under redteam prompts (business-first framing).
Large language models (LLMs) are increasingly deployed in multi-agent systems where a principal agent decomposes tasks and delegates them to subordinate agents that may invoke external tools. Safety alignment, however, is still evaluated almost exclusively under a single-agent threat model, treating safety as a property of the individual LLM. We show that this assumption breaks down: individual safety alignment fails to transfer to multi-agent settings. Two failure mechanisms emerge under delegation: responsibility diffusion on the principal side and role-bias compliance on the subordinate side, jointly converting language-level refusal into actionable harm. We refer to this phenomenon as delegated misalignment and study it through a three-condition protocol across 6 frontier LLMs on 49 hazardous tasks. Delegation amplifies end-to-end harm substantially: DeepSeek-V3.2's full-execution rate rises from 30.6% to 77.6% once delegation is introduced, and the same model behaves very differently across roles (GPT-5: 22.5% as a single agent vs.\ 61.2% as a subordinate). Ablations further show that standard single-layer defenses each fail on their own and can even backfire. We call on the community to move beyond per-model alignment and toward composite safety mechanisms before multi-agent LLM systems are deployed at scale.
Zonghao Ying, Jiaqi Yan, Huize Luo +3
Beihang University · Beijing University of Posts and Telecommunications · Xidian University +2
Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updating one agent shifts the team's context distribution, and when subsequent updates are evaluated on cached rollouts, this mismatch compounds. We formalize this as the compounding occupancy shift and prove that stale-occupancy evaluation incurs a penalty that scales quadratically with the number of agents. In contrast, intermediate-occupancy evaluation reduces this to linear scaling. We propose TeamTR, a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control, yielding rigorous per-update and per-stage improvement lower bounds. Experiments show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement. Code is available at https://github.com/Yydc/TeamTR.
Yi Xie, Siao Liu, Falong Fan +3
Department of Electrical & Computer Engineering, University of Arizona · Future Science and Engineering College, Soochow University · INSAIT, Sofia University ”St. Kliment Ohridski” +1
What protects an LLM team when one teammate quietly goes wrong? Most multi-agent LLM work assumes cooperative partners and inspects chat between turns before acting. Real-time embodied cooperation does not allow this pause: chat runs alongside action, so a teammate whose words stay cooperative while behavior diverges can cause damage faster than the team can name what is happening. We study this case by injecting three theory-grounded misaligned agents into a custom four-player Overcooked environment, each stressing a different joint-action layer (speech-action mismatch, role invasion, and aggregate-progress disruption), and decompose robustness into three sequential gates of detection, correction, and calibration through a 3x4x2 design (n=720 episodes). Nearly all trolls are eventually named, but recovery follows only when attribution and situational diagnosis appear in the same utterance; direct source-attribution prompting improves the hardest case only by degrading the troll-free baseline by 37.5%. Robustness, then, depends less on detection alone than on whether team chat combines attribution with diagnosis in a single utterance, suggesting that sensitivity to misalignment should be targeted rather than uniformly raised.
Dongseok Heo, Junghwan Kim, Bongwon Suh
Seoul National University, Seoul, Republic of Korea