Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation
Organizations: Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada, Spain · Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain · Department of Algebra, University of Granada, Spain · Department of Software Engineering, University of Granada, Spain ·
Abstract
Small open-weight language models can run crowd decision-making (CDM) on an institution's own infrastructure, which keeps the checkpoint, the inference settings and the opinions under its control. Their judgements vary across repeated queries; whether repeating and aggregating them, often through the 2-tuple linguistic representation, settles the decision rather than the labels is untested. Five open-weight models of 8-36B total parameters on one consumer GPU and a hosted reference classify 474 TripAdvisor reviews of four London restaurants ten times each; three open-weight models repeat the protocol on 600 reviews of twelve Madrid restaurants. Continuous evaluations preserve resolution: snapping each review leaves 5 distinct values against 21-65, and snapping at every level at most two collective values. Sensitivity to the participants persists once run-to-run variability is reduced: in London, redrawing the reviewers keeps the winner on 47-81% of replicas against 86-100% for redrawing the runs. With simultaneous intervals over every pair of alternatives under crowd resampling, no system on either corpus names a single winner: in London every system keeps all four restaurants as candidates, and in Madrid the candidate sets hold five to eight of the twelve. Extra runs and output-level fusion stabilize the winner but do not remove that sensitivity, and the fusion does not beat its best member. The three larger open-weight models name the hosted reference's London winner and at least match its per-review correlation with the reviewers' stars, so small open-weight models can support local CDM, reported as candidate sets with margins rather than forced rankings.
Figures & tables
| Property | London | Madrid |
| Corpus | TripR-2020Large | TripAdvisor Restaurant Reviews, version 2.0.0 |
| Alternatives / reviews / participants | 4 / 474 / 132 | 12 / 600 / 599 |
| Review coverage | 95–148 reviews per alternative | 50 reviews per alternative |
| Local models | Gemma 4 E4B (8.0B); Ministral 3 8B (8.9B); Gemma 4 31B (31.3B); Ornith 35B (34.7B); Qwen 3.6 35B (36.0B) | Gemma 4 E4B (7.5B); Ministral 3 8B (8.9B); Qwen 3.6 35B (36.0B) |
| Hosted reference / fixed fusion | GPT-5.2 / all five local models | None / all three local models |
| Serving runtime / collection | Ollama 0.31.1 (July collection) / April and July 2026 | Ollama 0.40.4 / 14 October 2026 |
| Corpus | Model | Lead | Run Top-1 | Crowd Top-1 | Candidates | |
| London | Gemma 4 E4B | 0.0171 | 85.8% | 47.4% | 4/4 | 0.764 |
| Ministral 3 8B | 0.0236 | 90.3% | 53.8% | 4/4 | 0.752 | |
| Gemma 4 31B | 0.0773 | 100.0% | 71.1% | 4/4 | 0.798 | |
| Ornith 35B | 0.0649 | 100.0% | 70.7% | 4/4 | 0.816 | |
| Qwen 3.6 35B | 0.0355 | 99.9% | 61.3% | 4/4 | 0.814 | |
| GPT-5.2 (hosted reference) | 0.1336 | 100.0% | 81.2% | 4/4 | 0.782 |
| Corpus | Configuration | Calls | Crowd Top-1 | 5th–95th | Same winner | Cost (s) | |
| London | Gemma 4 31B | 5 | 5 | 71.1% | 69.7–72.5 | 100% | 1.58 |
| Fixed fusion | 1 | 5 | 60.5% | 57.2–65.6 | 100% | 1.10 | |
| Gemma 4 31B | 10 | 10 | 71.1% | – | 100% | 3.17 | |
| Fixed fusion | 2 | 10 | 60.5% | 57.4–65.1 | 100% | 2.20 | |
| Fixed fusion | 10 | 50 | 60.8% | – | 100% | 10.98 | |
| Madrid | Qwen 3.6 35B | 1 | 1 | 87.3% | 68.7–94.6 | 100% | 0.07 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
LLM Decision-Making
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
See all 275 papers