Small-world Networks of Agents Brainstorm AI Risks to Support Ideation

Ke Zhou; Edyta Bogucka; Daniele Quercia

cs.HCSep 21, 2026

Small-world Networks of Agents Brainstorm AI Risks to Support Ideation

Authors: Ke Zhou, Edyta Bogucka, Daniele Quercia

Organizations: Nokia Bell Labs, UK · University of Nottingham, UK · Politecnico di Torino, Italy

Abstract

The ideation phase of participatory AI risk assessment often starts with a blank slate or a limited list of predefined risks, making it difficult to surface indirect or systemic harms. To address this limitation, we propose a three-stage ideation support tool. The tool complements participatory AI, rather than replacing it, and helps focus later engagement with affected communities. First, it dynamically discovers stakeholders depending on the given AI use and recursively expanding outward, allowing overlooked or indirect stakeholders to emerge. Second, it simulates these stakeholders with LLMs, connecting them into a network of a given topology, and having them ideate about risks. Third, it prioritizes risks using network centrality measures. In an initial evaluation, we found that betweenness centrality run through agents connected in a small-world network works best as it elevates risks raised by stakeholders who bridge disconnected groups, surfacing novel, systemic harms that traditional methods often miss. On an AI chatbot companion use case, this approach increased the novelty of the identified risks by approximately 1.1 points over single LLM brainstorming, and by 0.5 points over agentic LLM brainstorming, measured on a normalized five-point Likert scale, without reducing the plausibility or severity of the identified risks. To test whether our framework helps a human-led ideation session using the Futures Wheel approach, we divided 11 teams of non-western young chatbot users into two types: control (team) and treatment (team) in a participatory AI risk assessment. The control teams started from a list of risks generated by the 45 AI practitioners in the initial evaluation; the treatment teams started from a list generated by our framework. The treatment teams identified more risks overall, and more systemic, human-computer interaction, and environmental risks.

Figures & tables

Figure 1: Overview of the proposed framework and its two evaluations. (A) Our three-stage framework for generating risks. It involves identifying diverse stakeholders dynamically (Stage 1); building a stakeholder of in-silico agents network and generating risks by orchestrating the communication through a network with a given topology (Stage 2); and prioritizing risks using three centrality metrics: degree, betweenness, and closeness (Stage 3). (B) Evaluation of the framework’s risk generation quality by comparing system-generated risks with practitioners-generated risks from workshops. (C) Evaluation of the framework as a support tool for participatory AI risk assessment. Participants are assigned to either a control condition (report with practitioners-generated risks) or a treatment condition (report with network-generated risks), and use the provided reports as starting points for Futures Wheel ideation.

Dimension	Definition
Plausibility	Assesses whether the described risk could realistically arise in the given AI use context (1–5 Likert ( Diakopoulos and Johnson 2021 ) ).
Likelihood	Estimates the likelihood that a risk will occur (1–7 Likert, based on NIST AI risk assessment measures ( AI 2023 ) and foresight ( Fröhling et al. 2026 ) ).
Uniqueness	Evaluates whether the risk is specific to the particular AI use case, rather than broadly applicable across AI systems or domains (1–3 Likert ( Kieslich et al. 2026 ) ).
Novelty	Rates the degree of creativity and non-obviousness of the risk, ranging from mundane or checklist-like to imaginative and original (1–3 Likert ( Dean et al. 2006 ; Fröhling et al. 2026 ) , ).
Usability	Measures how clear, understandable, and practically engaging the risk (1–5 Likert ( Lewis and Sauro 2009 ; Brooke and others 1996 ) ).
Severity	Assesses the seriousness of potential harm and its downstream social, economic, or institutional consequences (1–5 Likert ( AI 2023 ; Fröhling et al. 2026 ) ).

Table 1: Evaluation rubric used for annotating AI risks. Each dimension corresponds to explicit annotation questions posed to evaluators, and is grounded in prior literature.

Figure 2: Our analysis utilizes two metrics: “Combined Score” aggregates Plausibility, Likelihood, Uniqueness, Usability and Severity while “Novelty” reflects risk diversity. All measures were normalized to the five-point Likert scale, and the top performing systems are bolded. A : Comparison of Network Topologies (using Snowballing Typical Roles and Degree Centrality): Small-World networks outperform both structured hierarchies and unstructured random networks. They significantly outperform the Baselines, demonstrating the value of structured interaction. B : Impact of Centrality Measures on Risk Quality (Small-World Network + Snowballing Typical Roles): Degree Centrality yields the highest quality (Combined Score) by capturing consensus; Betweenness Centrality is most effective at surfacing Novel risks by prioritizing risks from bridging stakeholders.

Figure 3: A: Risks and their novelty scores generated by the top 10 stakeholders ranked by Betweenness Centrality. B: The stakeholder network constructed through snowballing, within which in-silico LLM agents are connected and communicate to brainstorm risks. Many stakeholders who bridge diverse domains with top Betweenness Centrality such as Parent, Mental Health Researchers, Policy Makers and Clinical Pharmacist are the primary identifiers of highly novel risks (scores 4–5) such as psychological dependency and user safety during crisis.

Figure 4: Futures Wheel participatory ideation results. Teams seeded with our networked simulation generated risks (Treatment) identified more second- and third-order risks (A) than teams seeded with practitioner-generated risks (Control). They also surfaced a broader range of socio-technical harms, particularly human–computer interaction and socioeconomic and environmental risks (B).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Network Variant (Structure + Measure)	Roles (Stakeholder Strategy)	Combined Score	Novelty
Baselines
None	Single-turn LLM	3.29	2.32
Implicit DAG	Multi-turn Plurals	3.72	2.98
Hierarchical Network
Hierarchy	Fixed Typical Roles (5 roles)	3.15	2.21
Hierarchy	Fixed Comprehensive Roles	3.58	2.70

Appendix

Table 2: Performance of AI risk assessment system variants, organized by network variant (structure + measure). “Combined Score” aggregates Plausibility, Likelihood, Uniqueness, Engagement and Severity while “Novelty” solely reflects risk diversity. All measures were normalized to the five-point Likert scale, and the top-two performing systems are bolded. We observe that performance improves systematically with (i) small-world interaction structures, (ii) dynamic stakeholder expansion, and (iii) network-aware prioritization. Degree centrality yields the highest combined risk scores, whereas betweenness centrality surfaces the most novel and diverse risks.

Initial Seed	Expansion Strategy	Combined	Novelty
Typical (5 generic stakeholders)	Fixed (No expansion)	3.19	2.36
Nuanced (30 specific stakeholders)	Fixed (No expansion)	3.73	3.12
Typical (5 generic stakeholders)	Snowballing	3.99	3.48
Nuanced (30 specific stakeholders)	Snowballing	3.94	3.42

Appendix

Table 3: Effect of Seeding Strategy on Risk Prioritization (Small-World + Betweenness Centrality). Dynamic expansion (Snowballing) drastically improves performance over fixed stakeholders. Notably, snowballing from a small, generic set (Typical) slightly outperforms starting with a more specific set (Nuanced) on Combined Score (3.99 vs 3.94) and Novelty (3.48 vs 3.42), indicating that organic network growth effectively discovers critical bridging roles.

Characteristic	Number of Participants (%)
Gender
Men	15 (33.3%)
Women	30 (66.7%)
Age
18–24	8 (17.8%)
25–34	23 (51.1%)

Appendix

Table 4: Demographics of participants in the Chatbot Companion workshops for our first evaluation (blank slate human generated risks).

Figure 5: Risk impact assessment report provided to participant groups in the control condition of the second evaluation. The report contains risks generated by AI practitioners in blank-slate brainstorming workshops (Section 4.1 ), organized into technical, human interaction, and systemic risk categories.

Figure 6: Risk impact assessment report provided to participant groups in the treatment condition of the second evaluation. The report contains risks generated by our networked stakeholder simulation framework (Section 3 ), organized into technical, human interaction, and systemic risk categories.

Figure 7: Correlation matrix of various rubric dimensions demonstrates that these dimensions capture distinct evaluative signals rather than redundant measures. Novelty is marginally negatively correlated with Likelihood and Uniqueness, while weakly positively correlated with all other measures.

AI Risk Management

AI risk management studies how to identify, assess, reduce, and monitor harms from AI across development, deployment, and wider social systems.

See all 108 papers

An Agentic Multi-Agent Architecture for Cybersecurity Risk Management
Mar 20, 2026Ravish Gupta, Saket Kumar, Shreeya Sharma +2Cybersecurity AI Risk Management
BigCommerce · Lead Software Engineer, BigCommerce; IEEE Senior Member · University at Buffalo, The State University of New York, Buffalo, NY, USA +4
The Virtual Roundtable: Multi-Agent Personas Simulating the Dynamics of Human Brainstorming
Apr 17, 2026Tim Dorn, Saara A. Khan, Julie MumfordMulti-Agent Collaboration Multi-Agent Systems
Reality Labs, Meta, Menlo Park, California 94025, USA
ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation
Nov 27, 2025Nicoleta Tantalaki, Sophia Vei, Athena VakaliAlgorithmic Bias Human-Centered AI
Aristotle University, Thessaloniki, Greece

Mar 20, 2026J/K move · Enter open · S save

An Agentic Multi-Agent Architecture for Cybersecurity Risk Management

Ravish Gupta, Saket Kumar, Shreeya Sharma, Maulik Dang+1

BigCommerce · Lead Software Engineer, BigCommerce; IEEE Senior Member · University at Buffalo, The State University of New York, Buffalo, NY, USA+4

Cybersecurity AI Risk Management Multi-Agent LLM Systems LLMs for Cybersecurity LLM Agent Evaluation

Open paper AskPDF arXiv

Getting a real cybersecurity risk assessment for a small organization is expensive -- a NIST CSF-aligned engagement runs $15,000 on the low end, takes weeks, and depends on practitioners who are genuinely scarce. Most small companies skip it entirely. We built a six-agent AI system where each agent handles one analytical stage: profiling the organization, mapping assets, analyzing threats, evaluating controls, scoring risks, and generating recommendations. Agents share a persistent context that grows as the assessment proceeds, so later agents build on what earlier ones concluded -- the mechanism that distinguishes this from standard sequential agent pipelines. We tested it on a 15-person HIPAA-covered healthcare company and compared outputs to independent assessments by three CISSP practitioners -- the system agreed with them 85% of the time on severity classifications, covered 92% of identified risks, and finished in under 15 minutes. We then ran 30 repeated single-agent assessments across five synthetic but sector-realistic organizational profiles in healthcare, fintech, manufacturing, retail, and SaaS, comparing a general-purpose Mistral-7B against a domain fine-tuned model. Both completed every run. The fine-tuned model flagged threats the baseline could not see at all: PHI exposure in healthcare, OT/IIoT vulnerabilities in manufacturing, platform-specific risks in retail. The full multi-agent pipeline, however, failed every one of 30 attempts on a Tesla T4 with its 4,096-token default context window -- context capacity, not model quality, turned out to be the binding constraint.

Abstract

AI Risk Management

Explore similar work

An Agentic Multi-Agent Architecture for Cybersecurity Risk Management

The Virtual Roundtable: Multi-Agent Personas Simulating the Dynamics of Human Brainstorming

ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation