cs.CLNov 12, 2026

Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation

Authors: David Herrera-Poyatos, Cristina Zuheros, Carlos Peláez-González, Andrés Herrera-Poyatos, Rosana Montes, Francisco Herrera

Organizations: Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Spain · Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain · Department of Algebra, University of Granada, Spain · Department of Software Engineering, University of Granada, Spain · 

Abstract

Small open-weight language models can run crowd decision-making (CDM) on an institution's own infrastructure, which keeps the checkpoint, the inference settings and the opinions under its control. Their judgements vary across repeated queries; whether repeating and aggregating them, often through the 2-tuple linguistic representation, settles the decision rather than the labels is untested. Five open-weight models of 8-36B total parameters on one consumer GPU and a hosted reference classify 474 TripAdvisor reviews of four London restaurants ten times each; three open-weight models repeat the protocol on 600 reviews of twelve Madrid restaurants. Continuous evaluations preserve resolution: snapping each review leaves 5 distinct values against 21-65, and snapping at every level at most two collective values. Sensitivity to the participants persists once run-to-run variability is reduced: in London, redrawing the reviewers keeps the winner on 47-81% of replicas against 86-100% for redrawing the runs. With simultaneous intervals over every pair of alternatives under crowd resampling, no system on either corpus names a single winner: in London every system keeps all four restaurants as candidates, and in Madrid the candidate sets hold five to eight of the twelve. Extra runs and output-level fusion stabilize the winner but do not remove that sensitivity, and the fusion does not beat its best member. The three larger open-weight models name the hosted reference's London winner and at least match its per-review correlation with the reviewers' stars, so small open-weight models can support local CDM, reported as candidate sets with margins rather than forced rankings.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Releasing the Inner Jev: A Simple Recipe for Open Decision Models

    Oct 12, 2026Jonathan Geuter, Andrew Cheng, David Alvarez-MelisLLM Decision-MakingLLM Fine-Tuning

  2. Judged in Company: In-Context Batching Makes LLM Verdicts Depend on Other Items

    Oct 29, 2026Peng Xie, Amr AlanwarLLM-as-a-JudgeLLM Evaluation

  3. The Wisdom of Artificial Deliberative Crowds

    Sep 18, 2026Federico Barrera-Lemarchand, Mariano Sigman, Joaquin NavajasMulti-Agent DeliberationMulti-Agent Consensus