Collective intelligence through aggregation

Franz Dietrich; Christian List

cs.AIOct 5, 2026

Collective intelligence through aggregation

Authors: Franz Dietrich, Christian List

Organizations: We thank two anonymous referees and an editor for very helpful comments.

Abstract

Suppose a committee, expert panel, or other group is making judgments on some issues, where these may be not just yes/no-questions, such as whether a defendant is guilty, but also variables with many possible values, such as macroeconomic or meteorological variables or travel directions. Furthermore, there may be interconnections between different issues, as in the case of economic or climate variables. How can the group arrive at "intelligent" collective judgments, based on the group members' individual judgments? We investigate three challenges raised by this judgment-aggregation problem. First, reasonable methods of aggregation (such as defining the collective judgment for each issue as the average or median judgment) can produce inconsistent collective judgments. Secondly, many methods of aggregation are manipulable by strategic voting. Finally, not all methods of aggregation are conducive to tracking the truth on the issues in question. We prove new impossibility or possibility theorems on all three challenges, identifying what it takes to produce collective judgments in a consistent, non-manipulable, and truth-tracking manner and thereby to achieve collective intelligence through aggregation. Overall, the median method, though imperfect, performs reasonably well. We also note the relevance of our analysis for non-human group decisions.

Figures & tables

	One issue/variable	Many issues/variables
Binary judgments	Jury decisions (“guilty”, “not guilty”)	Making yes/no judgments on logically related propositions
Non-binary judgments	Evaluating or estimating one issue/variable Choosing a grazing spot, nest site, or travel direction	Estimating interconnected economic or climate variables Assigning probabilities or likelihoods to several events

Table 1: Examples of general aggregation problems

	Workforce productivity ( $x$ ) (EUR)	Employment level ( $y$ ) (number of workers)	GDP ( $z$ ) (EUR)
Expert 1	116 thousand	25 million	2900 billion
Expert 2	87 thousand	39 million	3393 billion
Expert 3	106 thousand	26 million	2756 billion
Median	106 thousand	26 million	2900 billion
Average	103 thousand	30 million	$\approx$ 3016 billion

Table 2: Experts’ estimates

	$p$	$p\rightarrow q$	$q$
Expert 1	True	True	True
Expert 2	True	False	False
Expert 3	False	True	False
Majority	True	True	False

Table 3: Experts’ judgments

Figure 1: Competence as a random variable

Explore similar work

A latent dimension of Condorcet's jury theorem for multiple AI advisers
Sep 13, 2026Kazutoshi Sasahara, Aoi Naito, Ryo FujieDisagreement Advice
Institute of Science Tokyo, Tokyo, Japan · Carnegie Mellon University, Pittsburgh, PA, USA
An evolutionary origin of collective decision making in humans and machines
Oct 5, 2026Guocheng Wang, Qi Su, Joshua B. PlotkinCollective Behaviors Majority Voting
Department of Biology, University of Pennsylvania, Philadelphia, PA 19104, USA · School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China · Meiji Institute for Advanced Study of Mathematical Sciences, Meiji University, Nakano, Tokyo 164-8525, Japan
When Do Institutions Beat Intelligence?
Aug 11, 2026Zhengye HanInstitutions Collective Behaviors
New York University Department of Electrical and Computer Engineering Brooklyn, New York, United States

Sep 13, 2026, cs.CYJ/K move · Enter open · S save

A latent dimension of Condorcet's jury theorem for multiple AI advisers

Kazutoshi Sasahara, Aoi Naito, Ryo Fujie

Institute of Science Tokyo, Tokyo, Japan · Carnegie Mellon University, Pittsburgh, PA, USA

Disagreement Advice

Open paper AskPDF arXiv

When the same question is asked of multiple AI advisers, as in self-consistency and LLM-as-a-judge panels, Condorcet's jury theorem predicts that adding independent, competent advisers makes the majority more reliable. The theorem, however, has a latent dimension when viewed from the user's vantage: adding advisers also makes disagreement more visible. A binomial model reveals that this ``visible dissent'' becomes nearly inevitable as the number of advisers grows, and that reliability and disagreement approach certainty at rates that cross at an adviser accuracy of 4/5 (0.8); below it, visible dissent eventually becomes more likely than a correct majority. Even ideal panels of independent and competent advisers can be correct in aggregate but appear divided; such disagreement does not by itself indicate aggregation failure. The way advisers split also provides a common basis for predictive multiplicity, reconciliation load, and reliance miscalibration. These results separate aggregation from disclosure and turn the latter into testable questions about how disagreement should be presented and interpreted.

Abstract

Explore similar work

A latent dimension of Condorcet's jury theorem for multiple AI advisers

An evolutionary origin of collective decision making in humans and machines

When Do Institutions Beat Intelligence?