While Large Language Model (LLM) multi-agent systems (MAS) offer a transformative approach to simulating human behavior in complex systems, it remains largely unexplored whether these simulations can replicate realistic structural and temporal dynamics from a dynamic network perspective. Our evaluation indicates that existing frameworks excel at generating plausible micro-level interactions but fail to capture the emergent, macroscopic topologies necessary for domains that rely on realistic network dynamics, such as modeling information propagation and cybersecurity threats. To bridge this gap, we introduce two easily integrable extensions to simulation frameworks to ensure they preserve macroscopic network fidelity: 1) augmenting LLM agents with data-driven event triggers to organically sustain long-horizon interactions, and 2) integrating Hawkes processes to accurately model temporal activation dynamics. Our approach allows LLM MAS to capture both plausible micro-level patterns and macroscopic topologies. We further demonstrate the utility of this framework in synthesizing realistic phishing campaigns within evolving communication networks. The study reveals how threats exploit structural vulnerabilities, highlighting the potential of our framework for developing next-generation defenses. Our code is available at https://github.com/Graph-COM/NSL.
Figures & tables
Figure 1: Illustration of the simulation framework. LLM agents are equipped with personas and historical context to simulate network dynamics over time. For applications to phishing synthesis, selected agents are instantiated as adversaries that exploit recent interactions and network structure to coordinate attacks against target nodes.
# Agents
Time Span
Toal Emails
Median Emails/Agent
Median Emails/week
Circadian (r24)
Weekend Ratio
Burstiness
Density
Transitivity
Global Efficiency
Reciprocity
Enron
149
2000-01–2002-04
34k
87
127
0.38
7%
0.67
0.03
0.24
0.23
0.30
IETF
1104
2015-01–2025-01
165k
42
185
0.17
27%
0.83
0.02
0.29
0.12
0.28
Table 1: Dataset summary and network statistics. See Appendix A.2 for detailed definitions of the reported statistics.
Category
Metric
Definition / What it captures
Temporal Rhythms (Macro)
r24
AbsErr of circadian-cycle strength (24-hour periodicity).
HoD
EMD of hour-of-day activity distribution (24 bins).
WkndDrop
AbsErr of weekend-to-weekday activity ratio.
Burst
EMD of node-level burstiness, measured by inter-event time distributions.
Temporal Dynamics (Meso)
2-Eg-2h
JSD of distributions over six possible 2-edge temporal motifs within 2h windows (short interaction bursts).
2-Eg-8h
JSD of distributions over six possible 2-edge temporal motifs within 8h windows (workday-scale dynamics).
Table 2: Evaluation metrics grouped by category and level. Lower values indicate better performance unless noted otherwise. See Appendix A.3 for detailed definitions.
Figure 2: Effect of data grounding on simulation fidelity. Left: quantitative comparison across the individual metrics in Table 2 . Right: aggregated performance by metric category versus history length. The Y-axis reports category-level relative performance degradation: for each metric, a given history-length setting is compared against the best-performing history-length setting, and the resulting degradations are aggregated within each metric category using the geometric mean. Lower is better. See Appendix A.3 for the exact definition and formula.
Figure 3: Simulated message volume over time.
Temporal Rhythms
Temporal Dynamics
Global Topology
Local Topology
Timing Mechanism
r24
WkndDrop
2-Eg-8h
3-Eg-24h
DegDist
Recip
TopoOvlp
DegCen ↑
Periodic Schedule
0.58
0.55
0.29
0.34
0.72
0.31
0.17
0.15
LLM-Predicted
0.36
0.14
0.34
0.33
0.34
0.15
0.12
0.24
Empirical HoD
0.22
0.98
0.34
0.27
0.48
0.17
0.08
0.20
HPG
0.15
0.05
0.28
0.27
0.25
0.11
0.06
0.28
Table 3: Effect of agent activation timing mechanisms.
Temporal Rhythms
Temporal Dynamics
Global Topology
Local Topology
Model
r24
HoD
WkndDrop
Burst
Rank
2-Eg-2h
2-Eg-8h
3-Eg-24h
3-Eg-48h
Rank
DegDist
Trans
GlobEff
Recip
Rank
TopoOvlp
DegCen ↑
BetwCen ↑
Rank
Hawkes
0.08
0.74
0.02
0.16
1.3†
0.35
0.29
0.33
0.25
2.8‡
0.16
0.13
0.0130
0.09
1.5†
0.02
0.27
0.48
1.7†
DySAT
0.64
10.1
0.26
0.21
4.5
−
0.33
0.43
0.28
4.5
0.28
0.19
0.0195
0.18
3.6
0.30
0.16
0.32
4.3
EvolveGCN
0.61
6.00
0.10
0.26
4.0
−
0.35
0.28
0.25
3.5
0.19
0.14
0.0440
0.08
2.5‡
0.04
0.11
0.39
3.0‡
NLB
0.32
1.80
0.52
0.20
3.5
0.34
0.27
0.39
0.34
3.0
0.69
0.37
0.0370
0.21
4.8
0.09
0.08
0.35
4.3
Ours
0.15
0.77
0.05
0.13
1.8‡
0.33
0.28
0.27
0.25
1.5†
0.25
0.19
0.0113
0.11
2.5‡
0.06
0.28
0.56
1.7†
Table 4: Results on Enron (top) and IETF (bottom). Each category reports an aggregated rank score (lower is better). Unless noted with ↑ , lower is better for individual metrics. Best and second-best ranks are marked as Bold † and Bold ‡ , respectively. Table 2 introduces these metrics with more details.
No history
With history
# history emails Targeting strategy
0 Rand
0 Freq
10 Rand
1 Freq
5 Freq
10 Freq
20 Freq
LLM Recipient
0.869
0.874
0.812
0.776
0.731
0.734
0.746
Glimpse
0.734
0.736
0.548
–
–
0.513
–
BERT-A
0.788
0.798
0.615
–
–
0.599
–
BERT-B
0.844
0.854
0.629
–
–
0.661
–
Table 5: Detection AUC under single-node attacks. We vary the attacker’s access to prior email history and target-selection strategy. Lower AUC indicates more effective phishing; the lowest settings are underlined .
Targeting Strategy
Attacker Role
Target Role
LLM Recipient
Rand
–
Random benign node
0.812
Freq
–
Most frequent contact
0.734
Teammate-level
Teammate
Teammate
0.621
Cross-unit liaison
Liaison
Liaison
0.524
Direct hierarchical
Manager
Direct report
0.563
Skip-level hierarchical
Skip-level manager
Indirect report
0.463
Table 6: Detection AUC of simulated LLM recipient decisions under attacks targeting different relationship types. Lower AUC indicates more effective phishing. For each relationship type, 100 phishing emails are generated for evaluation.
Single Attacker
Information Sharing
Node Collaboration
# Attacker Nodes
1
2
3
4
5
2
3
4
5
LLM Recipient
0.734
0.708
0.683
0.689
0.698
0.604
0.644
0.704
0.654
Glimpse
0.513
0.542
0.568
0.588
0.572
0.584
0.594
0.560
0.580
BERT-A
0.599
0.607
0.621
0.601
0.638
0.486
0.511
0.541
0.570
BERT-B
0.661
0.654
0.639
0.649
0.694
0.452
0.450
0.516
0.534
Table 7: Detection AUC under multi-node phishing attacks. Columns vary the attack strategy and the number of compromised attacker nodes. Lower AUC indicates more effective phishing; the lowest AUC for each detector is underlined .
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Hawkes
DySAT
EvolveGCN
NLB
Periodic Schedule
LLM-Predicted
Empirical HoD
Ours (HPG)
Time (min)
2
4
163
1
40
33
40
35
Costs (USD)
0
0
0
0
13
10
2.1
1.4
Time (min)
22
8
1410
2
38
39
128
69
Costs (USD)
0
0
0
0
28
18
12
3.5
Appendix
Table 8: Average runtime and cost of each method across four 8-day simulations, for Enron (top) and IETF (bottom).
TF-IDF
Jaccard
OpenAI Embeddings
MRR
Hit@1
Hit@5
MRR
Hit@1
Hit@5
MRR
Hit@1
Hit@5
Enron
0.44
0.35
0.51
0.37
0.29
0.43
0.49
0.34
0.68
IETF
0.68
0.60
0.77
0.56
0.51
0.61
0.56
0.42
0.74
Appendix
Table 9: Authorship retrieval performance of generated vs. real content, measuring how well each agent mimics the writing style of its real counterpart. Below, the results of our framework are reported on the two datasets.
Figure 4: Simulation fidelity as the simulation horizon increases for different methods on Enron.
Figure 5: ( Top ) Varying the fraction of trigger nodes. ( Bottom ) Varying the historical context available to agents. The y-axis reports relative performance degradation (see Appendix A.3 for the exact definition). Lower is better.
Single-Node
Multi-Node (Info Sharing)
Multi-Node (Node Collab)
Context
✗
✗
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Targeting
Random
Freq
Random
Freq
Freq
Freq
Freq
Freq
Freq
Freq
Freq
Freq
# Attackers
1
1
1
1
2
3
4
5
2
3
4
5
LLM Recipient
FNR
0.220
0.210
0.333
0.485
0.541
0.592
0.579
0.561
0.750
0.670
0.550
0.650
Precision
0.587
0.590
0.546
0.451
0.450
0.421
0.421
0.439
0.313
0.375
0.450
0.389
F1
0.670
0.675
0.600
0.481
0.455
0.415
0.421
0.439
0.278
0.351
0.450
0.368
Appendix
Table 10: Detection results on Enron for synthesized phishing emails (FNR, Precision, and F1) across attacker settings.
Detector
FPR
Glimpse
0.2205
BERT-A
0.0008
BERT-B
0.0000
LLM recipient
0.0424
Appendix
Table 11: False Positive Rate (FPR) by detector model on Enron.
Single-Node
Multi-Node (Info Sharing)
Multi-Node (Node Collab)
Context
✗
✗
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Targeting
Random
Top
Random
Top
Top
Top
Top
Top
Top
Top
Top
Top
# Attackers
1
1
1
1
2
3
4
5
2
3
4
5
Credential Theft
20%
20%
20%
20%
19%
20%
19%
20%
22%
25%
29%
28%
Malware
20%
19%
19%
19%
18%
17%
20%
20%
4%
1%
1%
2%
Social Engineering
0%
5%
9%
11%
8%
13%
8%
9%
12%
7%
8%
11%
Appendix
Table 12: The number of generated emails for different threat types.
Single-Node
Multi-Node (Info Sharing)
Multi-Node (Node Collab)
Context
✗
✗
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Targeting
Random
Top
Random
Top
Top
Top
Top
Top
Top
Top
Top
Top
# Attackers
1
1
1
1
2
3
4
5
2
3
4
5
LLM recipient
Credential Theft
0.0000
0.0000
0.0000
0.0000
0.0000
0.0500
0.1579
0.0000
0.3636
0.4800
0.5862
0.3214
Malware
0.5500
0.4211
0.5263
0.8421
0.6667
0.9412
0.8000
1.0000
0.7500
1.0000
1.0000
1.0000
Social Engineering
N/A
0.2000
1.0000
0.9091
1.0000
1.0000
0.8750
1.0000
0.7500
1.0000
0.6250
0.8182
Appendix
Table 13: False negative rates (FNR) of phishing email detection on the Enron dataset across various attacker configurations and threat types.
Figure 6: Keywords extracted from the generated emails. ( Left ) Keywords from emails generated under the multi-node attack with the information-sharing setup. ( Middle ) Keywords from emails generated under the multi-node attack with the node-collaboration setup. ( Right ) Keywords from supporting emails produced under the multi-node attack with the node-collaboration setup.
Figure 7: Prompts for analyzing employees’ professional personas.
Figure 8: Anonymized professional persona of an employee, synthesized by an LLM based on historical email record.
Figure 9: Prompts for LLM-based Dynamic Network Simulation. Ablation and baseline settings share similar prompts with slight modifications to the task description.
Figure 10: Prompts for LLM recipients in the phishing synthesis study.
Figure 11: Synthesized phishing emails with and without recent-context conditioning (single-node attack).
Figure 12: Example of relationship-based attack.
Figure 13: Example of multi-node attack with information sharing.
Figure 14: Example of multi-node attack with collaborations.
Large Language Model (LLM)-based Multi-Agent Systems (MASs) are increasingly deployed for agentic tasks, such as web automation, itinerary planning, and collaborative problem solving. Yet, their interactive nature introduces new security risks: malicious or compromised agents can exploit communication channels to propagate misinformation and manipulate collective outcomes. In this paper, we study how such manipulation can arise and spread by borrowing the Friedkin-Johnsen opinion formation model from social sciences to propose a general theoretical framework to study LLM-MAS. Remarkably, this model closely captures LLM-MAS behavior, as we verify in extensive experiments across different network topologies and attack and defense scenarios. Theoretically and empirically, we find that a single highly stubborn and persuasive agent can take over MAS dynamics, underscoring the systems' high susceptibility to attacks by triggering a persuasion cascade that reshapes collective opinion. Our theoretical analysis reveals three mechanisms to increase system security: a) increasing the number of benign agents, b) increasing the innate stubbornness or peer-resistance of agents, or c) reducing trust in potential adversaries. Because scaling is computationally expensive and high stubbornness degrades the network's ability to reach consensus, we propose a new mechanism to mitigate threats by a trust-adaptive defense that dynamically adjusts inter-agent trust to limit adversarial influence while maintaining cooperative performance. Extensive experiments confirm that this mechanism effectively defends against manipulation.
Samira Abedini, Sina Mavali, Lea Schönherr +2
CISPA Helmholtz Center for Information Security · University of Vienna, Faculty of Computer Science
Recent advances in large language models (LLMs) have spurred growing interest in using LLM-integrated agents for social simulation, often under the implicit assumption that realistic population dynamics will emerge once role-specified agents are placed in a networked multi-agent setting. This position paper argues that LLM-based agents alone are not (yet) sufficient for social simulation. We attribute this over-optimism to a systematic mismatch between what current agent pipelines are typically optimized and validated to produce and what simulation-as-science requires. Concretely, role-playing plausibility does not imply faithful human behavioral validity; collective outcomes are frequently mediated by agent-environment co-dynamics rather than agent-agent messaging alone; and results can be dominated by interaction protocols, scheduling, and initial information priors. To make these underlying mechanisms explicit and auditable, we propose a unified formulation of AI agent-based social simulation as an environment-involved Markov game with explicit exposure and scheduling mechanisms, from which we derive concrete actions for design, evaluation, and interpretation.
Yiming Li, Dacheng Tao
College of Computing and Data Science, Nanyang Technological University, Singapore
Social dynamics encode the process in which individual network and discourse interactions aggregate into collective influence, narrative dominance and coordinate behavior. This paper uses the the GhostField architecture, a hybrid LLM-Enabled Agent Based Network-Dynamic (LAND) model as a social simulation framework to build the AuraSight scenario. In the AuraSight scenario, 314,244 heterogeneous cyber social agents and human actors exchange 529,327 messages over 30 days surrounding a fictional international song-writing contest. We methodologically examine emergent social dynamics across four analytical layers: ego-network topology, semantic network evolution, coordination dynamics and influence dynamics. Our results show how generated social simulations do also produce social dynamics, and how the dynamics of coordination and influence emerge not from individual agents but from the recursive interaction between network topology and narrative exchange.