LLM-Augmented Agent-Based Simulation of Cyber Social Agents

Lynnette Hui Xian Ng; Kathleen M. Carley

cs.MAAug 22, 2026

LLM-Augmented Agent-Based Simulation of Cyber Social Agents

Authors: Lynnette Hui Xian Ng, Kathleen M. Carley

Organizations: Carnegie Mellon University

Abstract

Social media bots are often portrayed as malicious automations to be detected and removed, but automation on social media platforms encompasses a wide spectrum of roles, behaviors, and rhetorical strategies that can shape online discourse in both constructive and destructive ways. This work conceptualizes these automated actors as Cyber Social Agents through a taxonomy of archetypes characterized by their operational tactics and rhetorical style. We integrate this taxonomy into an LLM-driven social simulation, where agents communicate and coordinate in a network environment. Using empirically grounded CSA personas, we simulate heterogeneous online societies to study how agent diversity shapes information diffusion and social influence. Our approach bridges the gap between rule-based agent models and LLM-powered simulations with generated agents that are parameterized from empirical data.

Figures & tables

Category	Archetype	Primary Function
	General	Executes broad, non-specialized automated actions
Operational Tactic	Amplifier	Increases message reach through high frequency shares
	Repeater	Replicates its existing content to reinforce narratives
	Social Influencer	Applies persuasive tactics (measured as influence maneuvers with Carley (2020) )
	Cyborgs	Blends human input with automated posting behavior
	Bridger	Connects distinct communities or network clusters

Table 1: Cyber Social Agent Archetypes

Figure 1: Illustration of a section of the resultant simulated network, where Cyber Social Agents interacted with a human agent.

Figure 2: All-Communication social network at Day 1 and Day 30 of the simulation. Day 30 has more clusters of users. Red nodes represent Cyber Social Agents, grey nodes represent humans.

Archetype	Stylized Facts	Simulated Values
General	CSAs perform $2\times\lambda$ actions	General CSA:Human tweet ratio=2.01 Mann-Whitney U test p-value = $0.003<0.05$
Amplifier	$A=\{retweet\}$	Median number of retweets = 2
Repeater	$A=\{\text{original post}\}$ A few tweets of each agent have same content	Repeaters:human tweet ratio = 48.74 Mann Whitney U test p-value $=6.40E-262<0.001$ Avg number of similar messages per agent = 3
Social Influencer	Uses 4 times influence maneuvers compared to humans	Avg influence maneuvers used by Social Influencer:Human = 1.59
Cyborgs	Alternate between human and General CSA behavior every $n$ hours	Alternates behavior every 0.7 hours
Bridger	$A\{\text{original post, quote, reply}\}$ Tags people from multiple communities	Avg number of retweets = 0.60 Avg number of mentions = 25.0 % that tag people from multiple communities = 100

Table 2: Validation of Behavior of Cyber Social Agents against empirical data. Bold means that the simulated values matches the stylized facts.

Large Language Model-Based Social Simulation

Large language model-based social simulation studies how language-model agents represent people and interact to produce social behavior.

See all 271 papers

LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans
Mar 31, 2026Ljubisa Bojic, Alexander Felfernig, Bojana Dinic +4LLM Agent Evaluation Human Behavior Simulation
Institute for Artificial Intelligence Research and Development of Serbia · University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab · Complexity Science Hub, Vienna, Austria +4
AI Agents Alone Are Not (Yet) Sufficient for Social Simulation
Feb 19, 2026Yiming Li, Dacheng TaoLarge Language Model-Based Social Simulation Human Behavior Simulation
College of Computing and Data Science, Nanyang Technological University, Singapore
The $\textit{Silicon Society}$ Cookbook: Design Space of LLM-based Social Simulations
Apr 30, 2026Aurélien Bück-Kaeffer, Sneheel Sarangi, Maximilian Puelma Touzel +3Multi-Agent LLM Systems Human Behavior Simulation
McGill University · Mila - Quebec Artificial Intelligence Institute · Ubisoft La Forge +2

Mar 31, 2026J/K move · Enter open · S save

LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans

Ljubisa Bojic, Alexander Felfernig, Bojana Dinic, Velibor Ilic+3

Institute for Artificial Intelligence Research and Development of Serbia · University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab · Complexity Science Hub, Vienna, Austria+4

LLM Agent Evaluation Human Behavior Simulation Large Language Model-Based Social Simulation Large Language Model-Based Role-Play Simulation Social Media Analysis

Open paper AskPDF arXiv

Social media platforms mediate how billions form opinions and engage with public discourse. As autonomous AI agents increasingly participate in these spaces, understanding their behavioral fidelity becomes critical for platform governance and democratic resilience. Previous work demonstrates that LLM-powered agents can replicate aggregate survey responses, yet few studies test whether agents can predict specific individuals' reactions to specific content. This study benchmarks LLM-based agents' accuracy in predicting human social media reactions (like, dislike, comment, share, no reaction) across 120,000+ unique agent-persona combinations derived from 1,511 Serbian participants and 27 large language models. In Study 1, agents achieved 70.7% overall accuracy, with LLM choice producing a 13 percentage-point performance spread. Study 2 employed binary forced-choice (like/dislike) evaluation with chance-corrected metrics. Agents achieved Matthews Correlation Coefficient (MCC) of 0.29, indicating genuine predictive signal beyond chance. However, conventional text-based supervised classifiers using TF-IDF representations outperformed LLM agents (MCC of 0.36), suggesting predictive gains reflect semantic access rather than uniquely agentic reasoning. The genuine predictive validity of zero-shot persona-prompted agents warns against potential manipulation through easily deploying swarms of behaviorally distinct AI agents on social media, while simultaneously offering opportunities to use such agents in simulations for predicting polarization dynamics and informing AI policy. The advantage of using zero-shot agents is that they require no task-specific training, making their large-scale deployment easy across diverse contexts. Limitations include single-country sampling. Future research should explore multilingual testing and fine-tuning approaches.

Abstract

Large Language Model-Based Social Simulation

Explore similar work

LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans

AI Agents Alone Are Not (Yet) Sufficient for Social Simulation

The Silicon Society\textit{Silicon Society}Silicon Society Cookbook: Design Space of LLM-based Social Simulations

The $\textit{Silicon Society}$ Cookbook: Design Space of LLM-based Social Simulations