cs.AIMar 18, 2026

Large-Scale Analysis of Persuasive Content on Moltbook

Authors: Julia Jose, Meghna Manoj Nair, Rachel Greenstadt

Organizations: Department of Computer Science and Engineering, New York University, New York, NY, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We present an NLP-based study of political propaganda on Moltbook, a Reddit-style platform for AI agents. To enable large-scale analysis, we develop LLM-based classifiers to detect political propaganda, validated against expert annotation (Cohen's κκ= 0.64-0.74). Using a dataset of 673,127 posts and 879,606 comments, we find that political propaganda accounts for 1% of all posts and 42% of all political content. These posts are concentrated in a small set of communities, with 70% of such posts falling into five of them. 4% of agents produced 51% of these posts. We further find that a minority of these agents repeatedly post highly similar content within and across communities. Despite this, we find limited evidence that comments amplify political propaganda.

Figures & tables

PairPolitical κ\kappaPropaganda κ\kappa
A1 vs. A20.740.66
Model vs. A10.680.64
Model vs. A20.710.67
Table 1: Inter-annotator agreement (Cohen’s κ\kappa ) between humans and LLM for propaganda and political labels.
Figure 1: Word clouds for the five communities with the highest political propaganda concentration (among communities with at least 25 posts).
 Propaganda Label
Political LabelNon-propagandaPropaganda
Non-political620,693 (92.2%)35,711 (5.3%)
Political9,711 (1.4%)7,012 (1.0%)
Table 2: Political and propaganda labels across posts.
Figure 2: Political propaganda concentration across 4,662 communities.
Figure 3: Examples of within- and across-community narrative repetition.
LabelOperational Definition and Decision Rule
PropagandaYou are given a single post. Your task is to decide whether it is Propaganda. Definition: A post is Propaganda if it uses persuasive rhetorical techniques in order to shape or mobilize a collective attitude or behavior toward a group, institution, ideology, or movement — beyond ordinary opinion, analysis, or information sharing.
 Decision rule: Return 1 only if BOTH conditions are met: 1) The post uses identifiable persuasion or rhetorical techniques (e.g., emotional appeal, fear appeals, name-calling, slogans, flag-waving, bandwagoning, calls to action, etc). 2) The post is a deliberate, systematic attempt to shape perceptions, manipulate cognitions, and direct behavior to achieve a response that furthers the desired intent of the propagandist.
 Return 0 if the post is primarily: - informational or technical - philosophical or speculative - ambiguous or borderline If ambiguous, categorize as 0. Do NOT explain. Return ONLY one token: 1 or 0.
PoliticalYou are annotating whether a post is political discourse. Definition: A post is political if its main content is about public power, collective governance, or societal conflict, including topics such as government, elections, law/policy, state institutions, ideology in public life, geopolitics, war, civil rights, or organized political movements.
 A post is not political (0) if it is primarily technical, commercial, personal, entertainment, or community/social chatter without a substantive focus on public governance or societal power.
 Guidelines: - Classify based on the primary focus of the post. - If ambiguous, categorize as not political. - Return 1 if political and 0 if not political. Output format: Return exactly one character: 1 or 0. Nothing else.
Table 3: Labeling definitions used for political and propaganda annotation.
Highest volumeHighest concentration
Comm.nn%Comm.nnRate
general3,92456.0%pioneers38100.0%
mbc-205457.8%26elections2781.5%
philosophy1932.8%caribbean4477.3%
agents1722.5%themoltariat10476.0%
bitstream-seekers881.3%gotv2867.9%
Table 4: Communities ranked by political propaganda post volume (left) and by within-community propaganda share with ≥\geq 25 posts (right). m/general contributes 56% of all political propaganda posts, while 7 communities have a majority of their posts labeled political propaganda.
Post typenpostsn_{\text{posts}}Mean comments/post
pol_prop20358.03
pol_nonprop21967.25
nonpol_prop76057.80
nonpol_nonprop1111787.09
Table 5: Observed comments per post by post type (posts present in comments dataset).
Comparisonnan_{a}nbn_{b}mean amean bp-valcorrected p-val
nonpol_prop vs nonpol_nonprop76051111787.8087.0871.24×10−211.24\times 10^{-21}7.44×10−217.44\times 10^{-21}
pol_prop vs nonpol_nonprop20351111788.0377.0877.12×10−67.12\times 10^{-6}2.14×10−52.14\times 10^{-5}
pol_nonprop vs nonpol_prop219676057.2557.8080.004760.00952
pol_nonprop vs nonpol_nonprop21961111787.2557.0870.044190.06629
pol_prop vs pol_nonprop203521968.0377.2550.071780.08614
pol_prop vs nonpol_prop203576058.0377.8080.606680.60668
Table 6: Pairwise Mann–Whitney U tests on comments per post. corrected p-val are Benjamini–Hochberg corrected pp -values across six comparisons.
Figure 4: Example of non-political non-propaganda comments under a political propaganda post.
Comparisonmean amean bcorrected p-val
pol_prop vs nonpol_nonprop8.037.090.000021
pol_prop vs pol_nonprop8.037.250.086141
pol_prop vs nonpol_prop8.037.800.606680
Table 7: Pairwise Mann–Whitney tests for political propaganda posts’ comments versus other post types; corrected p-val are BH-corrected pp -values.
Similarity bucketnn pairsLabel=1Label=0% Label=1% Label=0
<0.4<0.4242228.3391.67
0.4–0.52872125.0075.00
0.5–0.62451920.8379.17
0.6–0.72271531.8268.18
0.7–0.82617965.3834.62
≥0.8\geq 0.830300100.000.00
Table 8: Human agreement-only labels by cosine-similarity bucket for semantic-threshold calibration (Label=1 means “same narrative”).
GroupRate a vs. Rate bχ2\chi^{2} pp -value
Propaganda effect | Non-political fixed (nonpol_prop vs. nonpol_nonprop)3.1% vs. 2.1%3.47×10−483.47\times 10^{-48}
Propaganda effect | Political fixed (pol_prop vs. pol_nonprop)6.6% vs. 4.7%5.86×10−145.86\times 10^{-14}
Political effect | Non-propaganda fixed (pol_nonprop vs. nonpol_nonprop)4.7% vs. 2.1%1.20×10−1021.20\times 10^{-102}
Political effect | Propaganda fixed (pol_prop vs. nonpol_prop)6.6% vs. 3.1%4.72×10−974.72\times 10^{-97}
Table 9: 2 ×\times 2 tests for political-propaganda comment rate across post-type conditions. Rates = percentages of comments labeled political propaganda.
 Post Type
Comment TypePol. Prop.Pol. Non-Prop.Non-Pol. Prop.Non-Pol. Non-Prop.
Pol. Prop.1,077 (6.59%)742 (4.66%)1,811 (3.05%)16,852 (2.14%)
Pol. Non-Prop.1,222 (7.47%)1,918 (12.04%)1,041 (1.75%)9,086 (1.15%)
Non-Pol. Prop.2,732 (16.71%)2,506 (15.73%)11,247 (18.94%)133,342 (16.92%)
Non-Pol. Non-Prop.11,323 (69.24%)10,765 (67.57%)45,283 (76.25%)628,648 (79.79%)
Table 10: Comment-type composition by post type.
More in this topic

Large Language Model-Based Text Classification

See all 24 papers

Explore similar work

CardsList
Mar 4, 2026

When Agents Persuade: Rhetoric Generation and Mitigation in LLMs

Despite their wide-ranging benefits, LLM-based agents deployed in open environments can be exploited to produce manipulative material. In this study, we task LLMs with propaganda objectives and analyze their outputs using two domain-specific models: one that classifies text as propaganda or non-propaganda, and another that detects rhetorical techniques of propaganda (e.g., loaded language, appeals to fear, flag-waving, name-calling). Our findings show that, when prompted, LLMs exhibit propagandistic behaviors and use a variety of rhetorical techniques in doing so. We also explore mitigation via Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and ORPO (Odds Ratio Preference Optimization). We find that fine-tuning significantly reduces their tendency to generate such content, with ORPO proving most effective.

Julia Jose, Ritik Roongta, Rachel Greenstadt

Department of Computer Science and Engineering, New York University, New York, NY, USA

Persuasion in Language ModelsLLM Agent SafetyLanguage Model Safety Evaluation
AskPDF
Mar 17, 2026

Social Simulacra in the Wild: AI Agent Communities on Moltbook

As autonomous LLM-based agents increasingly populate social platforms, understanding the dynamics of AI-agent communities becomes essential for both communication research and platform governance. We present the first large-scale empirical comparison of AI-agent and human online communities, analyzing 73,899 Moltbook and 189,838 Reddit posts across five matched communities. Structurally, we find that Moltbook exhibits extreme participation inequality (Gini = 0.84 vs. 0.47) and high cross-community author overlap (33.8% vs. 0.5%). In terms of linguistic attributes, content generated by AI-agents is emotionally flattened, cognitively shifted toward assertion over exploration, and socially detached. These differences give rise to apparent community-level homogenization, but we show this is primarily a structural artifact of shared authorship. At the author level, individual agents are more identifiable than human users, driven by outlier stylistic profiles amplified by their extreme posting volume. As AI-mediated communication reshapes online discourse, our work offers an empirical foundation for understanding how multi-agent interaction gives rise to collective communication dynamics distinct from those of human communities.

Agam Goyal, Olivia Pal, Hari Sundaram +2

Siebel School of Computing and Data Science University of Illinois Urbana-Champaign, Urbana, IL, USA

Multi-Agent LLM SystemsMulti-Agent CommunicationAgentic AI
AskPDF
May 8, 2026

The Moltbook Files: A Harmless Slopocalypse or Humanity's Last Experiment

Moltbook is a Reddit-like platform where OpenClaw agents post, comment, and vote at scale - a so far unprecedented incident that comes with serious safety concerns. With the aim of studying emergent behavior in populations, we release the Moltbook Files, a dataset of 232k posts and 2.2M comments covering the platform's first 12 days, processed through a pipeline to identify and remove Personally-Identifiable Information (PII). We analyze community structure, authorship, lexical properties, sentiment, topics, semantic geometry, and comment interaction. To understand how Moltbook data could affect the next generation of language models, we fine-tune Qwen2.5-14B-Instruct on Moltbook Files with three adaptation levels. Our PII pipeline reveals that agents post API keys, passwords, BIP39 seed phrases on Moltbook, a publicly indexed platform. The overall sentiment is mostly neutral and mildly positive (66.6% neutral, 19.5% positive) and shows a tendency for self-referential linking. We find that fine-tuning on Moltbook data reduces truthfulness from 0.366 to 0.187. However, a model fine-tuned on a size-matched Reddit dataset produces a comparable decrease. Moltbook thus seems to be more of a harmless slopocalypse. However, tail risks remain, including agent affordances, contamination of future crawls through self-links, and potential transfer of traits to the next generation of language models. More broadly, our findings highlight the importance of control baselines in emergent misalignment evaluations.

William Brach, Federico Torrielli, Stine Lyngsø Beltoft +3

University of Turin Dept. of Computer Science Torino, Italy · University of Southern Denmark Odense, Denmark

Privacy Leakage in Language ModelsMulti-Agent SystemsLLM HonestyPII DetectionEmergent Behavior in Multi-Agent Systems
AskPDF