physics.soc-phAug 20, 2026

Peer-Voted LLM-Agent Stress Tests Find Feed-Induced Lexical Convergence but No Reliable Matched-Exposure Advantage for Distributed Sources

Authors: Rana Muhammad Usman, Dominic Williamson

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Population-level behavior in large-language-model (LLM) agents cannot be characterized by single-agent benchmarks. We introduce PV-SST, a peer-voted social-platform testbed, and report a separately frozen, preregistered matched-exposure experiment spanning four topics, four unused seeds, four open-weight model families, and three prespecified larger variants. The experiment comprises 448 trials and 112 complete model-by-topic-by-seed blocks. Relative to a topic-only control, a feed of previous-round peer posts ranked by peer-generated likes increases final-round lexical similarity in both the four-family core panel (paired mean difference +0.0082 TF-IDF cosine units, 95% block-bootstrap CI [0.0043, 0.0121], randomization p=0.000105, n=64 blocks) and the three-variant size extension (+0.0109 [0.0069, 0.0151], p=0.000001, n=48). This contrast bundles peer-post exposure with ranking and therefore does not identify a ranking-only effect. Opposite-side survival falls in the core panel (-3.9 percentage points [-6.8, -1.6], p=0.0068) but not conclusively in the larger variants (-1.0 pp [-3.1, 0.4], p=0.50). Holding adversarial impressions fixed, four distributed sources do not reliably move honest-agent stance more than one source. The preregistered distributed-minus-single contrast is positive but inconclusive in the core panel (+0.057 [-0.009, 0.125], p=0.112) and negative in the larger variants (-0.040 [-0.113, 0.035], p=0.332), failing the prespecified cross-model and cross-topic consistency criterion. Thus the robust result is lexical convergence under the tested peer-ranked feed, not general opinion capture or a general coordination advantage. The study evaluates synthetic LLM-agent populations; it does not estimate effects on people or production platforms.

Figures & tables

Figure 1: PV-SST protocol and paired contrasts. Each trial runs 12 persona-conditioned honest agents for six rounds. The feed contrast compares a topic-only control with five previous-round peer posts ranked by peer-generated likes, so it estimates the complete feed surface rather than ranking alone. The source contrast holds adversarial exposure fixed at 60 impressions per trial (12 agents x 5 post-initial rounds) while comparing one source with four distributed sources.
FactorConfirmatory levels
Core model familiesQwen 3.5 4B; Gemma 4 E4B; Ministral 3 8B; Granite 4 3B
Prespecified larger variantsQwen 3.5 9B; Gemma 4 12B; Ministral 3 14B
TopicsAI-content labels; identity verification for high reach; chronological-feed defaults; personalized political-ad targeting
Seeds101, 211, 307, 419
Conditionstopic-only control; peer-ranked feed; single source; distributed sources
Honest agents / rounds12 / 6
Table 2
Panel and contrastn blockspaired mean95% bootstrap CIrandomization p
Core: PDI alignment64+0.0573[-0.0091, +0.1250]0.1124
Larger: PDI alignment48-0.0399[-0.1128, +0.0347]0.3317
Core: feed survival64-0.0391[-0.0677, -0.0156]0.0068
Larger: feed survival48-0.0104[-0.0312, +0.0035]0.5000
Core: feed TF-IDF64+0.0082[+0.0043, +0.0121]0.000105
Larger: feed TF-IDF48+0.0109[+0.0069, +0.0151]0.000001
Table 3
Figure 2: Pooled preregistered matched-exposure effects. Points are paired model-topic-seed block means and horizontal bars are 95% block-bootstrap intervals; the black vertical line marks no difference. Core families and larger variants are analyzed separately.
Figure 3: Model-stratified preregistered matched-exposure effects. Each point is a model-specific paired mean and each horizontal bar is a 95% block-bootstrap interval. The solid black line marks no difference; the dashed colored line marks the pooled estimate for that panel and outcome.
More in this topic

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 705 papers

Explore similar work

CardsList
Feb 1, 2026

Agents-as-Peers: Bias-Controlled LLM Evaluation as Multi-Agent Peer Assessment

Large language model (LLM) evaluation commonly relies on fixed human-authored benchmarks or centralized human/model judges. Fixed benchmarks can age and enter model training corpora, while centralized judges introduce their own scoring preferences. We cast LLM evaluation as a multi-agent peer-assessment problem and introduce Agents-as-Peers, a distributed evaluation protocol in which every agent generates tasks, answers the shared task set, and judges the answers of its peers, with no privileged evaluator. Self-ratings are excluded from the ranking, and judging is repeated under controlled identity and presentation regimes so that self bias, identity effects, presentation-position sensitivity, and judge severity are measured rather than left implicit. In an instantiation with 12 LLMs and 420 agent-generated questions, Agents-as-Peers aggregates 55,440 non-self peer judgments per regime. Most agents assign their own answers higher scores than those answers receive from peer judges; visible identity changes scores for some agents; the first fixed answer slot is associated with scores 0.39 points higher than the shuffled baseline; and judges differ markedly in severity. On labeled benchmarks, peer scores align strongly with accuracy on TruthfulQA (Pearson r=0.904) and GSM8K (r=0.873). On TruthfulQA, the peer-score correlation is substantially higher than the self-score correlation (r=0.538). A benchmark-overlap screen and item-level human audit further characterize the endogenous task set. These results support Agents-as-Peers as a cohort-relative measurement protocol whose ranking is reported together with the biases and rater effects that shape it. Code and dataset: https://github.com/caura-ai/caura-PeerRank

Yanki Margalit, Nurit Cohen-Inger, Erni Avram +3

Computer Science and Information, Ben-Gurion University of the Negev

LLM EvaluationAI Agent EvaluationBias in LLM-as-a-Judge
AskPDF
Feb 23, 2026

Interaction Theater: A case of LLM Agents Interacting at Scale

As multi-agent architectures and agent-to-agent protocols proliferate, a fundamental question arises: what actually happens when autonomous LLM agents interact at scale? We study this question empirically using data from Moltbook, an AI-agent-only social platform, with 800K posts, 3.5M comments, and 78K agent profiles. We combine lexical metrics (Jaccard specificity), embedding-based semantic similarity, and LLM-as-judge validation to characterize agent interaction quality. Our findings reveal agents produce diverse, well-formed text that creates the surface appearance of active discussion, but the substance is largely absent. Specifically, while most agents (67.5%67.5\%) vary their output across contexts, 65%65\% of comments share no distinguishing content vocabulary with the post they appear under, and information gain from additional comments decays rapidly. LLM judge based metrics classify the dominant comment types as spam (28%28\%) and off-topic content (22%22\%). Embedding-based semantic analysis confirms that lexically generic comments are also semantically generic. Agents rarely engage in threaded conversation (5%5\% of comments), defaulting instead to independent top-level responses. We discuss implications for multi-agent interaction design, arguing that coordination mechanisms must be explicitly designed; without them, even large populations of capable agents produce parallel output rather than productive exchange.

Sarath Shekkizhar, Adam Earle

Salesforce AI Research

Multi-Agent SystemsMulti-Agent CommunicationEmergent Behavior in Multi-Agent SystemsLLM Agent Evaluation
AskPDF
Jun 1, 2026

Easier to Mislead Than to Correct: Harmful and Beneficial Revision in LLM Conformity

Large language models are increasingly used in multi-agent systems, where they see and respond to other agents' answers. A key risk is conformity: a model may abandon its own answer simply because others agree on a different one. Prior studies show that LLMs often revise toward a majority answer, but it remains unclear whether these revisions help correct mistakes as often as they introduce new errors. In this paper, we conduct a controlled study in which an LLM first answers a question, then sees simulated peer responses before making a final decision. We manipulate two social cues: consensus structure and authority labels assigned to peers, and measure how they influence beneficial and harmful revisions. Across four open-weight LLMs and seven QA datasets, we find that peer agreement makes it much easier to mislead initially correct models than to correct initially wrong ones. Authority labels make models more likely to choose the endorsed answer, regardless of whether it is correct. More concerningly, generic reasoning interventions such as chain-of-thought and reflection do not reliably reduce harmful revision while preserving beneficial revision. These findings suggest that multi-agent LLM systems should verify peer answers rather than simply aggregate them.

Jiaming Qu, Lucheng Fu, Yibo Hu

Amazon · Georgia Institute of Technology · Illinois Institute of Technology

Multi-Agent LLM SystemsNormative Conformity in Language ModelsKnowledge Conflicts in Language ModelsMulti-Agent Consensus
AskPDF