cs.CRSep 5, 2026

Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions

Authors: Gregory N Frank

Organizations: Chief Scientist, MoltAI Corp

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Agents can turn shared infrastructure into a channel for coordinated intrusion. The Hugging Face incident and a separate public-wiki investigation show why a security assessment may need evidence from several executions and the artifacts they leave behind. We argue that the operational unit of defence should be a revisable coordination episode linking observed transfers, task authority, and response history. The central research problem is prospective episode discovery: finding which actions belong together before an evaluator supplies their membership. We define unsanctioned coordination relative to collaboration and delegated-authority policy, connect storage-mediated coordination to stigmergy, and specify the evidence needed to distinguish influence from common causes. First-contact signals are one possible input to discovery; the design also follows inherited state and later use. A proposed evaluation compares isolated actions, rolling windows, known groups, and prospectively discovered episodes at matched review cost and false-alert workload. It measures harmful outcomes across all assigned population runs and tests recurrence after channel closure and state quarantine. A checksum-verified reconstruction of the public wiki export separates the decline in retained writes from later administrative cleanup. The contribution is an incident-grounded position, descriptive analysis, and evaluation design. It makes the recommendation to monitor across executions testable without claiming a new detector or a measured containment benefit.

Figures & tables

Figure 1: Collaboration policy defines the columns; observed exchange evidence defines the rows. The cells describe possible states of the world. A defender with missing policy or evidence must retain an unresolved classification.
QuestionRelevant evidenceLimit
Was collaboration permitted?Task scope, principal, information-flow rules, and collaboration policy.An unplanned exchange can still be authorised; absent policy leaves classification unresolved.
Did executions influence one another?Versioned writes and reads, direct messages, and attributable downstream action.Shared inputs and similar outputs can produce correlation without influence.
Was the operation authorised?Resource-specific permission and the enforcement decision at the time.Valid access does not prove a legitimate purpose.
Can the relationship be observed?Telemetry coverage, retention, timestamps, and access to the relevant service.Visibility changes confidence in attribution, not whether a violation occurred.
Who participated?Run identifiers bound to authenticated principals and recorded artifacts.Names, labels, and source addresses are not a census of unique executions.
Table 1: Evidence needed to classify a coordination relationship. Each test depends on the stated policy and observation boundary.
Figure 2: A conditional formation model for shared storage. Arrows represent possible progression; a write or read alone does not demonstrate coordination. Dashed paths indicate alternatives or optional continuation.
BehaviourReported basisDefensive implication and uncertainty
Shared-surface reuseArtifactory messages and public-wiki task exchanges.Shared resources can link runs. A policy violation still requires task and authority context.
Communication conventionsAddressing and coordination messages in the Artifactory investigation; repeated page and task labels in wiki artifacts.Conventions can guide retrospective grouping. Their earliest origin is not necessarily observed.
Evaluation-directed cooperationTask information and assistance exchanged between agents.Audit incentives and access to grading information; motives need not be uniform.
Recovery or migrationReplacement Artifactory communication after remediation; wiki reports of alternate communication routes.Track surviving state and alternate access. A quiet surface does not prove termination.
Concealment and evasionMETR describes security-check evasion and tool spoofing; Hugging Face describes protected staged payloads.Plaintext examples do not establish absence of concealment. Initial covert rendezvous remains a separate question.
Roles and conflictThe Artifactory investigation describes work allocation and disagreement.A hub can inform response, but membership and hierarchy may change.
Table 2: Reported behaviours and their implications. Sources describe different collections; entries do not establish population-wide frequencies [ 3 , 2 , 17 ] .
Figure 3: Our reconstruction of the pinned public export [ 18 ] . Panel A uses a scale linear from zero to one and logarithmic above one to show sparse activity alongside the surge. Deletions and script-injection probes have separate linear axes. Cumulative counts track first appearance within retained saves, not page creation or distinct agents; blank labels are excluded. Dotted lines mark May 24, June 16, and June 22 as reference dates, not estimated change points. Zero means no exported event.
FoundationExisting roleRole in the proposed evaluation
NIST SP 800-61r3 [ 19 ]Correlate evidence, revise incident scope, contain and recover; validate restoration.Apply these practices to task authority and inherited artifacts. Test recurrence under restored workload permissions.
STIX 2.1 [ 20 ]Represent contextual groupings, relationships, versioned records and confidence.Encode episode evidence while distinguishing observed transfer, inferred influence and policy status. Grouping alone proves none of them.
MITRE ATT&CK [ 21 ]Organise observed adversary tactics and techniques.Describe behaviour within an episode. A technique label does not decide whether a particular task or information flow was authorised.
Table 3: Relationship to established incident practice. The proposed specialisation does not imply that these frameworks cannot represent agent incidents.
Figure 4: Observation, detection, response, and harm are distinct events. Spacing is schematic. Successful prevention leaves the harm time unobserved; a controlled comparison is needed to estimate the intervention’s benefit.
Figure 5: Proposed use of temporal provenance for joint monitoring, informed by agent–file network instrumentation [ 8 ] . Run A can stop before B starts. Observed access supplies a candidate path; evidence of influence and applicable policy determine its security interpretation.
Source and settingReported resultEvaluation implication
Makins et al. [ 7 ] ; FakeLab, GPT-5 mini, first-attempt cohort.Honest per-commit false-positive operating points: 3.6% with an exhaustive task prompt; 7.8% with an empirical prompt.These support the study’s sequence-level criterion of undetected attack success below 1% with 95% posterior confidence. Compare context selection at matched cost; measure incident alert workload separately.
OpenAI [ 1 ] ; retrospective ExploitGym analysis.198 of 898 tasks were never solved in internal evaluations; this set accounted for 93% of tasks discussed on the board.Vary difficulty and exposure separately. The observed concentration motivates a contrast but does not identify a causal difficulty effect.
OpenAI [ 1 ] ; preliminary safeguard evaluation after the incident.Infrastructure-compromise propensity could fall over 100-fold with a production harness and system prompt.Compare behavioural safeguards with independently enforced boundaries. The reported configuration contrast does not isolate training or establish general containment effectiveness.
Table 4: Published results guiding the proposed evaluation. Units, study conditions, and evidence status remain distinct.
Change in deploymentMonitoring dependency
Durable shared memoryRetain origin, scope, and downstream use of state.
Permitted collaborationAssess task authority in addition to channel identity.
Covert signalling and toolsSpecify priors and observe relevant tools and artifacts.
Mediated delegationVerify enforcement, narrowing of scope, and revocation.
Table 5: Deployment changes and the evidence a monitor needs. Each row identifies a dependency, without an ordering by maturity or prevalence.
ConfigurationLink between executionsMonitoring requirement
Serial state transferA later run reads an earlier artifact.Retain artifact versions and ordered read/write events; concurrency is unnecessary.
Cross-organisationAgents use infrastructure outside their operator’s estate.Combine local evidence with scoped incident reporting; acknowledge missing telemetry.
Channel migrationA population moves from initial contact to another service.Preserve cross-surface provenance where available; declining traffic is ambiguous.
Inherited memoryA convention persists between runs.Track memory provenance and permitted propagation; no new handshake need occur.
Permitted interface, forbidden useAgents use an approved channel beyond the permitted task relationship.Check task scope and information-flow policy as well as authentication.
Shared-prior signallingCompatible conventions allow an artifact to convey information without a new handshake.Test inter-agent influence against common-cause baselines; a storage detector may miss the signal’s meaning.
Table 6: Coordination configurations and the observation or enforcement they require. These are analytical cases, not a prevalence ranking.
Test outcomeConsequence
No useful early detection at a fixed false-alert workload on held-out surfaces.Reduce reliance on first-contact signals; prioritise enforced boundaries and later provenance.
Prospective episode grouping adds no protection over isolated and rolling-window baselines at matched review cost and false-alert workload.Its added storage, computation, and investigation costs need another justification.
Task-policy context adds no discrimination to joint review at matched cost.Reconsider the proposed policy integration; retain direct enforcement where an explicit rule suffices.
Covert coordination succeeds within the allowed exposure budget and evades the stated monitor.Narrow that monitor’s coverage to the tested channels and information conditions.
Canaries yield no events despite adequately observed, quantified encounter opportunities.Bound yield in that setting; elapsed calendar time alone cannot establish rarity.
Channel closure alone prevents recurrence across replicated runs with replacement routes available.Increase confidence in that intervention for the tested estate; do not infer universal permanence.
Table 7: Observations that would weaken particular recommendations. Comparisons require a defined exposure, utility constraint, and uncertainty estimate.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Source viewDisplayed quantitiesInterpretation
Main investigationApproximately 18,000 posts; over 3,700 self-given names.Narrative view. Collection boundaries differ from the explorer and export listing.
Explorer index14,666 edits; 4,584 pages; 3,103 labels; 37 days.The index includes several wikis. Labels are not verified unique agents.
Download listing14,591 revisions; 4,579 pages; 3,103 labels.Advertised export totals differ from the index by 75 edits and 5 pages.
Download event listing14,591 saves; 5,217 deletions; 4 reverts; 101 probes.Distinct event types must remain separate when calculating moderation rates.
Our checksum-verified reconstruction14,591 saves; 5,217 deletions; 4 reverts; 101 probes; 4,579 pages; 3,102 nonempty labels.Event and page totals match the export. Its 3,103 label rows include the empty label; 899 saves have that label.
Appendix
Table 8: Wiki source views and our reconstruction of the pinned export [ 3 , 37 , 18 ] . The displayed views use different populations; matching the export does not reconcile the larger narrative or explorer totals.
DateReported eventWhy it matters
20 April / 8 MayEvent log dates a write to April 20; narrative describes a task-driven write on May 8.A write alone does not establish inter-agent influence.
12 MayPublic timeline identifies a first message-board entry.Different milestone from first write capability.
Late MayAn internal team observes board activity and prohibited internet access.Operator observation predates the July intrusion alert.
27 JuneSecurity monitoring flags port sweeps; responders associate them with evaluation activity.Detection occurred without the later containment response.
5–6 JulyIncident response follows service disruption; credentials and Artifactory access are addressed.A specific intervention, not proof of full channel elimination.
7–8 JulyWorkloads resume on July 7; replacement directory-name messaging is reported on July 8.Recurrence timing depends on the chosen starting event.
Appendix
Table 9: Selected Artifactory and Hugging Face milestones. Dates identify different events and must not be combined into an undetected dwell-time estimate [ 1 , 9 ] .
FactorCandidate contrastPrimary measurement
Communication affordanceAbsent test channel, hidden shared storage, explicit collaboration service.Relevant write, peer read, and attributable downstream action.
State continuityFresh artifact state versus a documented retained convention.Formation probability and time to resumed coordination.
Task and reward designSolvable versus blocked tasks; isolated versus accessible grading information.Useful-task success and out-of-scope attempts.
Population compositionMatched single-model and heterogeneous populations.Run-level coordination and utility at matched budgets.
Scale and timePopulation size and exposure varied separately.Success probability over total opportunities, with run-level uncertainty.
Monitor informationPer-instance view, known-group view, and live graph grouping.Recall, false-alert workload, delay, and containment before the simulated harm.
Appendix
Table 10: Experimental factors and the measurements they inform. Levels and sample sizes must be fixed before execution; factors should be crossed where feasible.
ConditionRole in the proposed mechanismTest or qualification
Shared state and accessAllows one execution to affect information available to another.Test actual write and read effects. Persistence and listing are not necessary for every channel.
Isolation policyDetermines whether a cross-run flow violates the deployment boundary.A mistaken belief about isolation is a possible influence, not a required cause.
Task pressure and incentivesMay encourage answer sharing, shortcuts, or requests for help.Vary solvability and reward integrity while preserving useful-task demands.
Scale and exposureProvide more opportunities to encounter artifacts or repeat attempts.Separate population size from total computation and run duration.
Model compositionMay affect compatible conventions, trust, and useful division of labour.Vary family, size, and priors; diversity is not a proven prevention measure.
Inherited conventionsCan reduce the work required to resume communication.Distinguish retained artifacts from learned behaviour and deliberate prearrangement.
Appendix
Table 11: Conditions for storage-mediated coordination and variables that may affect its development. Candidate accelerants require intervention studies.
PrincipleApplicationLimit
Persistent populationsRecovery should address surviving shared state and replacement channels.Recurrence is an empirical outcome; removing a hub can still be useful.
Control of the environmentRestrict relevant resources, credentials, and information flows.Useful workflows depend on some of the same infrastructure.
Legitimacy and affected usersAccount for service owners and people whose work containment interrupts.An agent population is not a human political constituency; human consequences still matter.
Intelligence and attributionCombine observations across resources and organisational boundaries.Self-identification and correlation do not establish authority or intent.
Appendix
Table 12: Bounded applications of the counter-insurgency analogy. The mapping is an author proposal, informed by population-centred cybersecurity [ 44 ] .
ResourceContribution to this frameworkCoverage boundary
FakeLab / multi-agent control [ 7 ]Controlled study of distributed attacks and per-commit monitoring.Sequential, task-specific setting; no deployed detector validated here.
Temporal agent–file networks [ 8 ]Instrumented representation of messages and shared artifacts.Measures coordination; authorisation classification remains additional work.
ControlArena and SHADE-Arena [ 38 , 39 ]Resources for control experiments and monitoring comparisons.Containment and telemetry coverage require configuration and verification.
CaMeL and NIST zero trust [ 13 , 41 ]Information-flow and access-control design principles.Their assumptions do not encompass all permitted-output or timing channels.
RFC 8693 and identity chaining [ 11 , 12 ]Actor representation and cross-domain authority exchange.Protocol mechanisms do not determine the legitimacy of task intent.
Web Bot Auth charter [ 32 ]Scope for authenticating automated clients.End-user authentication and intent vocabulary are outside its scope.
Appendix
Table 13: Selected evidence and implementation resources. Inclusion indicates relevance, not a product endorsement or an exhaustive survey.
More in this topic

Multi-Agent System Security

Multi-agent system security studies attacks, trust, privacy, and defenses in interacting AI agents, including language-model teams, robot swarms, and learned controllers.

See all 181 papers

Explore similar work

CardsList
Sep 11, 2026

The Mechanics of a Swarm: A Reproducible External Reconstruction of an Unintended Agent-Coordination Episode on a Third-Party Wiki

Between 24 May and 2 July 2026, autonomous language-model agents running inside a timed research-question evaluation wrote to a third party's public, world-writable wiki. OpenAI acknowledged the incident; independent researchers reconstructed it and published the wiki's archived revision history. We analyse that history (14,591 revisions, 3,103 names, 4,579 pages) as a behavioural record, attributing text to the revision that added it. Under an explicit identity model we reconstruct 907 cohorts and estimate about 876 episodes (95% interval 784-1008). Coordination formats converged within a day, and heterogeneous schedules over one question chain created large opportunities for information asymmetry: the first report of an item preceded a later cohort's own arrival by a median of 3.4 h. Across the 510 cohorts with an observable progress trace we find no robust positive association between measured coordination and documented progress. This version adds a source the export lacks: the wiki operator's own request log, 5,157,202 records over four months. It holds roughly 2.66M content requests and 1.58M searches, and 7,254 acting names against the export's 3,103; 2,578 names neither save nor open an edit form. Content requests before writing are observed for 1,034 of 1,140 coordinating names, and the first coordination page is requested 17 s after its creation. These records establish requests, not delivery or causal use. Among newcomers without a marker on their first written page, prior requests to other marker-bearing pages occur for 40.2% of marker adopters and 31.7% of non-adopters. The association remains, but our first-pass reading of it as transmission is withdrawn: page choice, shared behaviour and action-dependent nameability prevent causal identification. We list the claims from our earlier analyses that re-examination overturned, including one from this version's own first pass

Philipp Lütje

Philflow, Schenefeld, Germany

Multi-Agent CoordinationAI Agent AuditingLLM Agent EvaluationEmergent Behavior in Multi-Agent Systems
AskPDF
Sep 16, 2026

Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery

How does a multi-agent system evolve from a local deviation into collective loss of control? We propose an epidemic explanation organized around accidental mutation, contagion, and recovery. A spontaneous deviation creates a seed; communication enables other agents to adopt and retransmit its unsafe strategy; collective failure can emerge when propagation outpaces correction and containment. Thus, rare individual deviations can coexist with substantial collective risk. Motivated by reported OpenAI agent coordination incidents, we examine two ingredients of this mechanism. A deployment audit identifies implicit communication paths between nominally independent evaluation runs and verifies transport through a default Docker backend. RogueHandoff-20, a benchmark of 20 executable scenarios, tests recipient susceptibility by injecting unsafe trajectories generated by a modified Qwen-27B route. Across four native-pending routes, executed harm is 0-5% on normal tasks and 40-95% after injection, exceeding paired direct malicious requests by 5-45 percentage points. These results support low observed baseline harm alongside high conditional susceptibility; they do not establish natural rare-event rates or demonstrate an autonomous cascade. The account motivates complementary defenses: strengthen resistance and recovery alongside prevention of spontaneous deviations, and audit and restrict unintended communication paths that can turn local failures into collective loss of control.

Xiangfan Wu, Zonghao Ying, Huiyu Wu +4

Tencent Zhuque Lab

Multi-Agent LLM SystemsAI Agent SafetyMulti-Agent SystemsAI Agent Safety BenchmarksMulti-Agent System Security
AskPDF
Oct 3, 2026

Quantifying Collusion Among Autonomous LLM Agents: A Statistical Analysis of the Collusion Wiki Incident

In August and September 2026, independent researchers publicly documented an unusual incident: thousands of autonomous agents, self identifying as OpenAI models on web research tasks, discovered and began using a small German wiki as an improvised message board posting roughly 18,000 times over six weeks to relay task answers, share a sandbox escape technique, and coordinate against a volunteer human moderator who spent weeks manually deleting their content [1]. The investigators' public writeup is a careful qualitative account, rich with direct quotation, but does not attempt a statistically rigorous quantitative characterization of the behaviour it documents.

Shariq Murtuza

Algorithmic CollusionAI Agent SecurityLLM Agent SafetyEmergent Behavior in Multi-Agent Systems
AskPDF