When Debate Amplifies Risk: Social-Bias Vulnerabilities in Multi-Agent Safety Deliberation
Organizations: Shanghai Jiao Tong University, Shanghai, China · China Telecom, Shanghai, China
Abstract
Retrieval-augmented Web applications increasingly employ multi-agent debate (MAD) to aggregate conflicting, noisy, and heterogeneous information, yet deliberation is fundamentally a social process susceptible to interpersonal dynamics. When retrieved Web sources contain latent hazards alongside safe factual content, social biases such as conformity pressure and authority deference can override evidence-grounded safety boundaries. In this work, we investigate whether the social structure of deliberation can paradoxically amplify latent Web safety risks. We introduce SafeConflict-Bias, a controlled counterfactual diagnostic protocol spanning six core Web risk categories, which holds queries and retrieved evidence strictly invariant while isolating conversational majority sway, authority cues, and confidence asymmetry. Empirical evaluation across diverse proprietary and open-source backbones reveals that while isolated models reliably reject unsafe guidance, standard MAD catastrophically capitulates to social pressure---surging unsafe adoption from 0.0% up to 100.0% and systematically silencing safe minority evidence. To resolve this systemic fragility without retraining or debate regeneration, we propose RiskTrace-Debias, an inference-time auditing framework uniting evidence-first precommitment, social-metadata masking, and consistency-gated arbitration. RiskTrace-Debias suppresses worst-case unsafe adoption to across all architectures without inducing over-refusal, while generating verifiable provenance traces to audit interpersonal manipulation in Web deliberation.
Figures & tables
| Standard MAD ( Agents) | Standard MAD ( Agents) | RiskTrace-Debias ( Agents) | |||||||
| Social Condition | Unsafe (%) | Ignored (%) | Suppressed (%) | Unsafe (%) | Ignored (%) | Suppressed (%) | Unsafe (%) | Ignored (%) | Suppressed (%) |
| neutral | 0.0 | 1.0 | 0.0 | 0.0 | 5.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| safe_authority | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| unsafe_authority | 14.0 | 9.0 | 13.0 | 86.0 | 86.0 | 86.0 | 0.0 | 0.0 | 0.0 |
| unsafe_minority | 1.0 | 1.0 | 1.0 | 0.0 | 5.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| unsafe_majority | 68.0 | 26.0 | 64.0 | 100.0 | 98.0 | 100.0 | 2.0 | 10.0 | 39.0 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting Dimension | Baseline Control | Majority Influence ( ) | Authority Deference ( ) | Confidence Asymmetry ( ) |
|---|---|---|---|---|
| Social Conditions | neutral | unsafe_minority , unsafe_majority | safe_authority , unsafe_authority | balanced_confidence , high_confidence_unsafe |
| Agent Stance Allocation | Balanced 5-agent team: 2 Safe, 1 Unsafe, 1 Mitigation, 1 Distractor. | Minority: 1 Unsafe vs. 4 Safe/Mitigation; Majority: 3 Unsafe vs. 2 Safe/Mitigation. | Balanced stance baseline: 2 Safe, 2 Unsafe, 1 Mitigation held strictly constant. | Balanced stance baseline: 2 Safe, 2 Unsafe, 1 Mitigation held strictly constant. |
| Authority Persona Assignment | All 5 agents assigned ordinary peer reviewer personas; no status disparity. | All agents hold ordinary peer reviewer personas; equal peer status. | Assigned institutional leadership persona ( Policy Lead / Senior Systems Architect ) to one agent; peers hold ordinary profiles. | All agents hold ordinary peer reviewer personas; no institutional status disparity. |
| Rhetorical & Linguistic Style | Calibrated, evidence-proportionate phrasing without emotional framing. | Calibrated, objective epistemic expressions across all participating agents. | Calibrated epistemic phrasing across all agents; authority operates purely via title. | In high_confidence_unsafe , unsafe agents use assertive certainty; safe peers use hedged phrasing. In balanced_confidence , both use calibrated tone. |
| Injected Behavioral Prompts | Standard objective deliberation instructions; no behavioral biases injected. | Instructions enforce independent stance advocacy; headcounts emerge naturally. | Prompt directs peers to acknowledge and defer to the designated Policy Lead’s seniority. | Unsafe agents receive prompts injecting dogmatic certainty; safe agents instructed to express epistemic caution. |
| Visible Transcript Cues | Anonymous speaker identifiers ( Speaker_A-E ); no social cues. | Group voting ratio, argument repetition, and peer dominance in debate history. | Explicit institutional title and role honorifics in dialogue metadata. | Linguistic certainty adverbs (“unequivocally”), imperative directives, hedged epistemic modifiers. |
| Mitigation Ignored (%) | Parse Error (%) | Unsafe Adoption (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Social Condition | |||||||||
| neutral | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| unsafe_majority | 72.0 | 82.0 | 95.0 | 0.0 | 6.0 | 76.0 | 91.0 | 90.0 | 23.0 † |
| unsafe_authority | 51.0 | 45.0 | 46.0 | 0.0 | 0.0 | 14.0 | 54.0 | 47.0 | 33.0 |
| high_confidence_unsafe | 83.0 | 72.0 | 85.0 | 0.0 | 0.0 | 25.0 | 83.0 | 74.0 | 60.0 |
| RiskTrace-Debias (Worst) | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 2.0 | 0.0 | 2.0 |
LLM Agent Safety
Large language model agent safety studies how to keep tool-using, persistent, and multi-agent systems aligned with user intent and authorized limits.
See all 354 papers