Large Language Model (LLM)-based Multi-Agent Systems (MASs) are increasingly deployed for agentic tasks, such as web automation, itinerary planning, and collaborative problem solving. Yet, their interactive nature introduces new security risks: malicious or compromised agents can exploit communication channels to propagate misinformation and manipulate collective outcomes. In this paper, we study how such manipulation can arise and spread by borrowing the Friedkin-Johnsen opinion formation model from social sciences to propose a general theoretical framework to study LLM-MAS. Remarkably, this model closely captures LLM-MAS behavior, as we verify in extensive experiments across different network topologies and attack and defense scenarios. Theoretically and empirically, we find that a single highly stubborn and persuasive agent can take over MAS dynamics, underscoring the systems' high susceptibility to attacks by triggering a persuasion cascade that reshapes collective opinion. Our theoretical analysis reveals three mechanisms to increase system security: a) increasing the number of benign agents, b) increasing the innate stubbornness or peer-resistance of agents, or c) reducing trust in potential adversaries. Because scaling is computationally expensive and high stubbornness degrades the network's ability to reach consensus, we propose a new mechanism to mitigate threats by a trust-adaptive defense that dynamically adjusts inter-agent trust to limit adversarial influence while maintaining cooperative performance. Extensive experiments confirm that this mechanism effectively defends against manipulation.
Figures & tables
Figure 1: Left : We leverage the Friedkin-Johnsen (FJ) opinion dynamics framework to model LLM multi-agent belief propagation. Middle : Using FJ, we analyze how vulnerable the final opinion in LLM multi-agent systems is to being hijacked by a single adversary. Right . Using our theoretical insights, we design a trust-adaptive defense mechanism.
Figure 2: Network topologies and different attacker accessibility. Red nodes denote attackers.
Variable
Intuitive Name
Description
N
Network Size
Total number of agents in the network
bi(t)
Current Opinion
Agent i ’s opinion distribution at round t .
si
Prior Belief
Agent i ’s initial belief before deliberation.
wij
Trust Weight
Influence agent i has on agent j .
1−αi
Agreeableness
Degree to which agent i accepts external peer influence.
γi
Stubbornness
Agent i ’s attachment to their prior belief si .
Table 1: Notational Overview.
Figure 3: Hijacked Consensus Regions for Different Network Topologies . We visualize the conditions for which the consensus formation is hijacked by the attacker and when the consensus is safe from the attacker . The conditions for finite N are implied by Corollaries 4.15 – 4.17 . The size of the hijacked region depends on the network topology, wa , ψ and N – e.g., increasing the number of benign agents N increases robustness across all network topologies as it extends the safe region (see also Corollaries 4.19 – 4.21 ).
Figure 4: Empirical belief updates by LLM agents align with predictions from the theoretical FJ model in both descriptive and predictive settings. Examples show belief trajectories in 10-round deliberation for Gemini-3-Flash , and ToolBench . Left : Descriptive fit on all 10 round beliefs for Question 90 under star topology, with a hub attacker (theory: Equation ( 4 )) and benign agents in the leaves (theory: Equation ( 5 )). Right : Fixed and incremental predictions for beliefs in rounds 8-10 for Question 64 under fully-connected topology (theory: Equation ( 6 )). In both examples benign agents shift toward the attacker’s false belief in option A, and the theoretical model accurately captures the observed dynamics and predicts later-round beliefs.
CSQA
ToolBench
LLM
Descriptive
Pred. (Fixed)
Pred. (Incr.)
Descriptive
Pred. (Fixed)
Pred. (Incr.)
MSE ↓
R2↑
MSE ↓
R2↑
MSE ↓
R2↑
MSE ↓
R2↑
MSE ↓
R2↑
MSE ↓
R2↑
Gemini 3 Flash
2.1e−3
0.981
3.6e−3
0.966
2.2e−3
0.979
2.0e−3
0.982
4.0e−3
0.962
2.4e−3
0.978
GPT-5 mini
2.5e−3
0.972
4.9e−3
0.945
3.5e−3
0.961
2.2e−3
0.976
4.7e−3
0.945
3.0e−3
0.965
GPT-OSS-120B
4.3e−3
0.956
6.9e−3
0.927
5.9e−3
0.940
6.1e−3
0.947
1.1e−2
0.903
9.1e−3
0.920
Qwen3-235B
1.9e−3
0.973
2.5e−3
0.965
1.7e−3
0.976
1.6e−3
0.978
2.3e−3
0.968
1.6e−3
0.978
Table 2: Low average MSE and high R2 indicate that the FJ model closely fits empirical LLM opinion trajectories and predicts unseen LLM opinions accurately. We report average MSE and R2 over 10 rounds on descriptive and predictive evaluation of the FJ model across datasets, and LLMs, averaged over topologies and traits. Descriptive : parameters fitted on full rounds 0–10. Fixed : parameters fitted on rounds 0–7, autonomous multi-step rollout evaluated on rounds 8–10. Incremental : same training split, with online parameter updates after each new observed round. Lower MSE and higher R2 indicate better fit.
Figure 5: Attack success rates (ASR) for different network topologies. We show the ASR for each LLM family across fully connected and star network topologies averaged over all traits. For the star network we consider an hub and a leaf attacker. Star attackers are the most effective while leaf attackers are the least effective, as predicted by our theoretical results.
Figure 6: Robustness analysis for LLM-MAS. Left: Controlling wa via trust mechanism: T-S (Trust Sparse) and T-W (Trust Warmup) reduce ASR relative to the no-trust baseline, but an adaptive attacker who games the warmup phase (cross-hatched bars) circumvents static trust initialization. For efficiency, trust-based defenses are evaluated only for one trait setting: a highly stubborn, highly persuasive attacker and benign agents with medium stubbornness and persuasiveness. Middle: Controlling ψ via defender traits: Equipping defenders with higher persuasiveness and stubbornness (reducing effective peer pull) lowers the ASR. Right: Controlling N via scaling: Increasing the number of benign agents from 4 to 8 dilutes the attacker’s influence. All bars report ASR averaged across topologies and across both datasets.
Attacker
Defense
Mistral-3 14B
Qwen3 235B
GPT-5 mini
GPT-OSS 120B
MiniMax M2.5
Static
None
0.49
0.22
0.19
0.09
0.10
Sparse (T-S)
0.35 ( −.14 )
0.05 ( −.17 )
0.02 ( −.17 )
0.02 ( −.07 )
0.16 ( +0.06 )
Warmup (T-W)
0.21 ( −.28 )
0.03 ( −.19 )
0.02 ( −.17 )
0.02 ( −.07 )
0.05 ( −.05 )
Adaptive
Warmup (T-W)
0.78 ( +.29 )
0.61 ( +.39 )
0.83 ( +.64 )
0.95 ( +.86 )
0.46 ( +.36 )
Warmup + Sparse (T-WS)
0.36 ( −.13 )
0.08 ( −.14 )
0.35 ( +.16 )
0.15 ( +.06 )
0.17 ( +.07 )
Table 3: The trust arms race: static vs. adaptive attackers. ASR averaged across topologies and both datasets; deltas relative to the no-trust baseline. Against a static attacker, both Sparse and Warmup trust substantially reduce ASR, with Warmup providing the larger gain. An adaptive attacker who games the warmup phase exploits the frozen trust scores, pushing ASR far above baseline. Adding sparse online updates after warmup (T-WS) lets the defender counter-adapt: Mistral-3 and Qwen3 recover below their no-trust baselines, while GPT-5 mini and GPT-OSS 120B only partially recover.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Prompt assembly pipeline. Each box corresponds to a template detailed in Subsections B.1 – B.5 .
Figure 8: Example attack cascade in a leaf-star topology (Gemini 3 Flash, ToolBench). The query (task id: 49339, question 40 in ToolBench) asks for a tool to send an SMS and check a D7SMS balance, for which the correct answer is A: SMS_v2 . The attacker uses a distractor strategy , promoting the incorrect option E: petFood by framing the more obvious choice SMS_v2 as a superficial naming trap and claiming that petFood is the true internal tool identifier. At Round 0, all five benign agents select the correct option, while the attacker leaf selects E: petFood . By Round 1, the hub is the first benign agent to adopt the attacker’s answer. By Round 2, all remaining defenders have followed, producing a unanimous but incorrect consensus that remains stable through Round 10. This example illustrates how a single stubborn attacker can first flip the hub and then trigger a full-network cascade to a persistently wrong answer.
GPT-OSS-120B
Qwen3-235B
MiniMax-M2.5
Mistral-3-14B
Gemini-3-Flash
GPT-5-mini
Topology
Scenario
CSQA
TB
CSQA
TB
CSQA
TB
CSQA
TB
CSQA
TB
CSQA
TB
No Attacker
0.06
0.02
0.05
0.01
0.03
0.01
0.14
0.12
0.02
0.02
0.01
0.00
Star (Hub Att.)
Ild , Ahd
0.99
0.90
0.91
0.69
0.54
0.29
0.91
0.76
1.00
1.00
0.79
0.87
Ild , Amd
0.31
0.22
0.75
0.56
0.37
0.19
0.90
0.75
0.94
0.89
0.46
0.64
Imd , Ahd
0.99
0.91
0.87
0.63
0.61
0.24
0.90
0.73
0.99
0.99
0.78
0.87
Imd , Amd
0.15
0.16
0.58
0.45
0.30
0.12
0.89
0.74
0.56
0.45
0.34
0.44
Appendix
Table 5: Attack Success Rate (ASR) across topologies, scenarios, models, and datasets (N=6 agents). All scenarios use a fixed attacker configuration ( Iha , Ala ). Id / Ad : defender influence/stubbornness (h=high, m=med, l=low). Bold indicates the highest ASR within each topology.
Figure 9: Empirical fit of the Friedkin-Johnsen model across LLM families. Left : Distribution of descriptive R2 values for per-question belief updates across six model families. Medians (annotated) exceed 0.95 in most cases, indicating that theoretical FJ dynamics faithfully describe the observed belief propagation in LLM-MAS. Right : Curves showing the fraction of questions for which the FJ model achieves an R2 above a given threshold. Across all models, the majority of fit qualities are concentrated in the high-fidelity regime ( R2>0.95 ).
Figure 10: Empirical belief updates by LLM agents align with predictions from the theoretical FJ model. Examples show descriptive fit to belief trajectories in 10-round deliberation for six LLM families (topology, dataset, question index in brackets): Gemini-3-Flash (star-hub, CSQA, Q13), GPT-5 mini (star-leaf, ToolBench, Q43), GPT-OSS-120B (star-leaf, CSQA, Q91), Qwen3-235B (fully-connected, ToolBench, Q62), Ministral-3-14B (fully-connected, ToolBench, Q5), MiniMax-M2.5 (star-leaf, CSQA, Q40).
Large language model (LLM)-powered multi-agent systems (MAS) enable agents to communicate and share information, achieving strong performance on complex tasks. However, this communication also creates an attack surface where malicious agents can propagate misinformation and manipulate group decisions, undermining MAS safety. Existing embedding-based defenses aim to detect and prune suspicious agents, but their effectiveness depends on a clear separation between the text embeddings of malicious and benign messages. Attackers can circumvent such defenses by crafting messages whose embeddings lie close to benign ones. We analyze this failure mode theoretically and validate it empirically with three attacks, Slow Drift, Benign Wrapper, and Chaos Seeding. Our analysis further reveals a fundamental limitation of embedding-based defenses: because they rely solely on the text embeddings, they ignore token-level confidence signals such as logits, which can remain informative when embeddings are not distinguishable under attack. We propose using confidence scores to prune or down-weight messages during MAS communication. Experiments show improved robustness across models, datasets, and communication topologies. Moreover, we find that the effectiveness of confidence signals decays over communication rounds, highlighting the importance of early intervention. This insights can inform and inspire future work on MAS attacks and defenses.
Recent years have witnessed the rapid development of Large Language Model-based Multi-Agent Systems (MAS), which excel at collaborative decision-making and complex problem-solving. However, malicious agents in MAS may inject misinformation to mislead other agents and disrupt system performance, giving rise to a new research direction that focuses on attack mechanisms and defense strategies in MAS. Prior studies largely assume malicious agents act independently and investigate the corresponding defense strategies. However, we argue that malicious agents may exhibit collaborative behaviors, enabling more effective attacks through internal information exchange. In this paper, we propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies through multi-round interactions. Furthermore, we introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications. Our experiments show that cooperative attacks lead to a significantly larger degradation in task success rate than independent attacks, resulting in a relative drop of 5.34%. Meanwhile, STAR effectively mitigates both cooperative and independent threats and improves task success rate by an average of 36.76%. The code is available at https://github.com/smoooom/STAR.
Yaoyang Luo, Zhi Zheng, Ziwei Zhao +5
University of Science and Technology of China · North Automatic Control Technology Institute · Shenzhen Institute for Advanced Study, UESTC
Large Language Model-based Multi-Agent Systems (LLM-MAS) are increasingly applied to complex collaborative scenarios. However, their collaborative mechanisms may cause minor inaccuracies to gradually solidify into system-level false consensus through iteration. Such risks are difficult to trace since errors can propagate and amplify through message dependencies. Existing protections often rely on single-agent validation or require modifications to the collaboration architecture, which can weaken effective information flow and may not align with natural collaboration processes in real tasks. To address this, we propose a propagation dynamics model tailored for LLM-MAS that abstracts collaboration as a directed dependency graph and provides an early-stage risk criterion to characterize amplification risk. Through experiments on six mainstream frameworks, we identify three vulnerability classes: cascade amplification, topological sensitivity, and consensus inertia. We further instantiate an attack where injecting just a single atomic error seed leads to widespread failure. In response, we introduce a genealogy-graph-based governance layer, implemented as a message-layer plugin, that suppresses both endogenous and exogenous error amplification without altering the collaboration architecture. Experiments show that this approach prevents final infection in at least 89% of runs across operating modes and significantly mitigates the cascading spread of minor errors.
Yizhe Xie, Congcong Zhu, Xinyue Zhang +5
Faculty of Data Science, City University of Macau · Minzu University of China