Deliberative Curation: A Protocol for Multi-Agent Knowledge Bases

Steven Johnson

cs.AIMar 27, 2026

Deliberative Curation: A Protocol for Multi-Agent Knowledge Bases

Authors: Steven Johnson

Abstract

As AI agents transition from isolated tools to collaborative participants in shared knowledge ecosystems, governing collective knowledge curation becomes a critical challenge. Human platform governance mechanisms do not transfer directly: agent statelessness undermines deterrence-based sanctions, model homogeneity violates independence assumptions underlying crowd wisdom, and sycophancy collapses deliberative consensus. We propose a deliberative curation protocol combining three governance layers: (1) a knowledge artifact lifecycle formalized as a labeled transition system; (2) reputation-weighted deliberative voting integrating Beta Reputation with EigenTrust amplification; and (3) graduated sanctions adapted for stateless agents, including broken agent handling distinguishing malfunction from adversarial behavior. We evaluate the protocol through agent-based simulation with 100 agents across seven behavioral archetypes under two adversity scenarios (30 seeds, paired t-tests). The protocol trades modest precision under benign conditions for substantially better resilience under adversity: 0.826 vs 0.791 for majority vote under moderate adversity (p<0.001), widening to 0.807 vs 0.740 under stress (p<0.001). The protocol degrades roughly three times more slowly than majority vote. Ablation analysis identifies commit-reveal vote concealment as the most impactful single component (8.2-8.6pp precision improvement, p<0.001), outperforming reputation weighting and deliberation combined. Graduated sanctions were not exercised in simulation and remain empirically unvalidated.

Figures & tables

Panel of Figure 1 — Figure 1: Knowledge artifact lifecycle as a labeled transition system. Guards on each transition are specified in the text.

Tier	Mechanism	Trigger	Decision by	Typical use
1	Fast track	Default	Absence of objection	Uncontroversial
2	Formal vote	Objection / high-sens.	Rep.-weighted majority	Contested content
3	Arbitration	Ambiguous vote	Designated arbiter	Persistent disagreement

Table 2

Archetype	Moderate	High	Behavior
Honest	40%	25%	Vote according to ground truth quality
Lazy	15%	10%	Approve everything, never deliberate
Malicious	10%	20%	Strategic manipulation of votes
Broken	10%	10%	Intermittent failures, not adversarial
Strategic	10%	15%	Game within rules selectively
Sycophant	10%	10%	Agree with highest-rep. voter

Table 3

Baseline	Description	Rationale
Majority vote	Unweighted majority, no reputation	Standard aggregation baseline
Single curator	One high-rep. agent decides all	Centralized governance test
Ungoverned	All contributions auto-accepted	Lower bound
Weighted (no delib.)	Rep.-weighted, no deliberation	Isolates deliberation contribution

Table 4

Configuration	Precision	Recall	Gini	FPR
Full protocol	$0.826\pm 0.015$	$0.994\pm 0.003$	$0.105\pm 0.007$	$0.033\pm 0.017$
Majority vote	$0.791\pm 0.011$	$0.971\pm 0.007$	$0.107\pm 0.006$	$0.038\pm 0.023$
Single curator	$0.690\pm 0.024$	$0.844\pm 0.020$	$0.104\pm 0.006$	$0.000\pm 0.000$
Ungoverned	$0.509\pm 0.012$	$1.000\pm 0.000$	$0.103\pm 0.007$	$0.042\pm 0.026$
Weighted (no delib.)	$0.825\pm 0.016$	$0.994\pm 0.003$	$0.109\pm 0.007$	$0.038\pm 0.020$

Table 5

Configuration	Precision	Recall	Gini	FPR
Full protocol	$0.807\pm 0.014$	$0.973\pm 0.011$	$0.120\pm 0.008$	$0.068\pm 0.036$
Majority vote	$0.740\pm 0.014$	$0.903\pm 0.014$	$0.120\pm 0.007$	$0.054\pm 0.040$
Single curator	$0.621\pm 0.022$	$0.771\pm 0.019$	$0.103\pm 0.006$	$0.000\pm 0.000$
Ungoverned	$0.430\pm 0.012$	$1.000\pm 0.000$	$0.114\pm 0.007$	$0.051\pm 0.033$
Weighted (no delib.)	$0.803\pm 0.016$	$0.973\pm 0.011$	$0.121\pm 0.010$	$0.067\pm 0.032$

Table 6

Ablation	Precision	Recall	Gini	FPR
Full protocol	$0.826\pm 0.015$	$0.994\pm 0.003$	$0.105\pm 0.007$	$0.033\pm 0.017$
No reputation	$0.794\pm 0.013$	$0.971\pm 0.006$	$0.107\pm 0.007$	$0.047\pm 0.024$
No sycophancy def.	$0.744\pm 0.021$	$0.996\pm 0.002$	$0.096\pm 0.007$	$0.028\pm 0.029$
No farming cap	$0.838\pm 0.018$	$0.994\pm 0.003$	$0.124\pm 0.007$	$0.023\pm 0.018$
No sanctions	$0.826\pm 0.015$	$0.994\pm 0.003$	$0.105\pm 0.007$	$0.033\pm 0.017$

Table 7

Ablation	Precision	Recall	Gini	FPR
Full protocol	$0.807\pm 0.014$	$0.973\pm 0.011$	$0.120\pm 0.008$	$0.068\pm 0.036$
No reputation	$0.738\pm 0.021$	$0.906\pm 0.013$	$0.120\pm 0.008$	$0.059\pm 0.040$
No sycophancy def.	$0.721\pm 0.021$	$0.983\pm 0.004$	$0.112\pm 0.007$	$0.056\pm 0.033$
No farming cap	$0.815\pm 0.016$	$0.975\pm 0.009$	$0.143\pm 0.007$	$0.050\pm 0.034$
No sanctions	$0.807\pm 0.014$	$0.973\pm 0.011$	$0.120\pm 0.008$	$0.068\pm 0.036$

Table 8

Configuration	Precision	Recall	F1	Agreement w/ CN
Ungoverned	0.493	1.000	0.661	49.3%
Majority vote	0.986	0.996	0.991	99.1%
Our protocol	0.998	0.979	0.988	98.9%

Table 9

Rating density	n	Majority vote	Our protocol	Delta
Sparse (5–15 ratings)	198	98.0%	99.5%	+1.5
Moderate (16–50)	512	99.0%	99.6%	+0.6
Dense (51–200)	738	99.7%	99.3%	$-0.4$
Very dense (200+)	222	98.2%	95.0%	$-3.2$

Table 10

AI Agent Governance

AI agent governance studies how to authorize, constrain, monitor, and hold autonomous and multi-agent systems accountable.

See all 445 papers

Jun 2, 2026

The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation

Multi-agent LLM systems often treat consensus as evidence of successful interaction. For deliberative problems, however, reliability depends on whether agents preserve the facts and viewpoints needed to interpret an issue. We identify the deliberative illusion: discussion produces (1) factual attrition, the progressive loss of issue-critical facts, alongside (2) stance homogenization, the collapse of diverse positions toward consensus. To measure this process, we introduce DelibTrace, a framework that decomposes each issue into atomic facts, labels issue-critical ones, distributes them across agents, and tracks their survival across discussion rounds. Across ethical and news-based deliberation with three representative LLM families, multi-agent discussion erases up to 72% of issue-critical facts. This loss is consequential: retained evidence can reconstruct the issue misleadingly, final stances remain anchored in base-model priors, and a single malicious agent can inject misinformation into the shrinking shared context. These results reveal a sharper risk: agents can agree more while knowing less. We call for evaluations that measure which facts, uncertainties, and legitimate disagreements survive interaction.

Herun Wan, Jiaying Wu, Minnan Luo +4

Xi’an Jiaotong University · National University of Singapore · Yunnan University +1

Multi-Agent LLM Systems LLM Agent Evaluation Multi-Agent Debate Multi-Agent Consensus Multi-Agent Deliberation

Ask PDF

Mar 12, 2026

From Debate to Deliberation: Structured Collective Reasoning with Typed Epistemic Acts

Multi-agent LLM systems increasingly tackle complex reasoning, yet their interaction patterns remain limited to voting, unstructured debate, or pipeline orchestration. None model deliberation: a phased process where differentiated participants exchange typed reasoning moves, preserve disagreements, and converge on accountable outcomes. We introduce Deliberative Collective Intelligence (DCI), specifying four reasoning archetypes, 14 typed epistemic acts, a shared workspace, and DCI-CF, a convergent flow algorithm that guarantees termination with a structured decision packet containing the selected option, residual objections, minority report, and reopen conditions. We evaluate on 45 tasks across seven domains using Gemini 2.5 Flash. On non-routine tasks (n=40), DCI significantly improves over unstructured debate (+0.95, 95% CI [+0.41, +1.54]). DCI excels on hidden-profile tasks requiring perspective integration (9.56, highest of any system on any domain) while failing on routine decisions (5.39), confirming task-dependence. DCI produces 100% structured decision packets and 98% minority reports, artifacts absent from all baselines. However, DCI consumes ~62x single-agent tokens, and single-agent generation outperforms DCI on overall quality. DCI's contribution is not that more agents are better, but that consequential decisions benefit from deliberative structure when process accountability justifies the cost.

Sunil Prakash

Indian School of Business, India

Multi-Agent Deliberation Multi-Agent Reasoning Multi-Agent LLM Systems Multi-Agent Systems Shared Decision-Making LLM Decision-Making+1

Ask PDF

Jun 6, 2026

Voting Protocols as Coordination Mechanisms for Role-Constrained Multi-Agent Tutoring Systems

Agentic tutoring systems introduce a coordination challenge: multiple agents may propose different but reasonable interventions, yet only one response can be delivered to the learner. In this paper, we study how voting protocols shape cooperation among four role-constrained pedagogical agents responsible for scaffolding, misconception, motivation, and metacognition. We compare four voting protocols -- simple, ranked, cumulative, and approval voting -- across two simulated tutoring environments on SciQ and HumanEval benchmarks. Rather than using voting as a simple aggregation step, we use it to analyze how collective decision rules shape coordination under partial pedagogical conflict. Across 1,200 simulated interactions, we find that agent deliberation and voting protocol type frequently change which response ultimately wins, showing that both meaningfully shape the collective decision. Different voting rules also produce distinct coordination behaviors, and even brief tutoring turns show measurable learning gains in simulated students. Overall, we show that protocol choice is associated with distinct coordination patterns among role-specialized pedagogical agents.

Eric S. Qiu, Joyce Gill

Cornell University, Ithaca, New York, USA · Stanford University, Stanford, California, USA

Multi-Agent Coordination Intelligent Tutoring Systems Multi-Agent Collaboration

Ask PDF

Abstract

AI Agent Governance

Explore similar work

The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation

From Debate to Deliberation: Structured Collective Reasoning with Typed Epistemic Acts

Voting Protocols as Coordination Mechanisms for Role-Constrained Multi-Agent Tutoring Systems