Can Network Models of Science Do Without Pluralist Agents?
Authors: Jonas Hallgren, David Hyland, Mahault Albarracin
Organizations: Equilibria Network, Stockholm, Sweden · Uppsala University, Uppsala, Sweden · University of Oxford, Oxford, UK · Laboratoire d’Analyse Cognitive de l’Information, Université du Québec à Montréal, Québec, Canada · Centre of Excellence for AI and Robotics, Sheffield Hallam University, Sheffield, UK
Science is a process of useful models being proven wrong time and again, but despite its many flaws, it somehow works. Why is this? Paul Feyerabend argued that it works because people dare to hold and defend incompatible rival theories against the accepted one, since a theory's failures often lead to its abandonment in the presence of a more accurate or simpler alternative. He called this view theoretical pluralism. We introduce a theoretical and experimental model in which this can be examined. It compares communities of pluralist agents, which can hold rival theories side by side and maintain beliefs within and between those theories, with communities of monist agents, which can only incorporate what they hear or observe into a single model of their own. In a multi-agent variational inference setting, our experimental demonstrations illustrate how communities with rival theories can go through periods of uncertainty followed by a shift in the theories they hold true, how the information that is communicated between communities affects beliefs when those communities make different observations about the world, and how sharing support for theories and sharing beliefs within them contribute differently to collective belief dynamics.
Figures & tables
Figure 1: (a) Private observations generated from a common world state whose likelihoods are evaluated under different candidate theories; (b) Agent i retains a conditional belief distribution ρik over each candidate’s quantities and one model probability qik for that candidate; (c) Edges on a communication graph represent the exchange of these beliefs.
Figure 2: A demonstration of theory revision under full observation of observable quantities (extent of combustion, net gas uptake, and solid mass change). Mean oxygen probability (green) and mean individual candidate entropy (blue) after the mass signal changes. The dotted line marks the first changed observation at round 80, and the bands represent pointwise 95 percentile bootstrap confidence intervals for means across 50 simulation runs, using 2,000 resamples of whole runs.
Figure 3: Theory choice and numerical prediction under selective observation. All conditions use the same 50 observation streams, β=0.05 , and ω=0.3 . (a) Distance between the groups’ observable predictive means, where theory predictions are averaged using their current probabilities. (b) Oxygen support in the gas observer group (solid) and mass observer group (dotted), for only the models that maintain beliefs over both theories.
Figure 4: Communication channels after changed evidence on a complete graph. The four conditions begin from the same fully observed, fully communicating state after 80 updates and use the same subsequent observation streams. (a,b) Support for the oxygen theory among four fully observing and twenty restricted agents. The dotted line marks the common state before the first changed observation. (c) Full-signal MSE (i.e., the average squared error in an agent’s predictions across all three observable quantities) is averaged over completed states 451–600, by observation group. Restricted weights are (1,0.01,0.01) , ω=0.9 , and σ=1 .
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Reference dependencies for the release and oxygen candidates. Dashed nodes are candidate-specific latent quantities; C , G , and M are shared observables. Arrows give the signs of coefficients of magnitude 0.8 .
Condition
Wx
αm
Neither
I24
0
Within-theory only
W
0
Model support only
I24
0.02
Both
W
0.02
Appendix
Table 1: Communication settings from observation 80 in Fig. 4 . Local observation updates remain enabled in every condition. Here Wij=1/24 .
Figure 6: Belief revision of the restricted agents over candidate theories with both channels enabled across four networks. Black outlines around nodes in the topology depictions mark the fully observing agents. Bold curves average the restricted-group mean oxygen support across 50 runs. Bands represent pointwise 95% percentile-bootstrap intervals from 2,000 whole-run resamples. Thin curves show individual restricted agents in run 0.
Success-driven social learning, in which individuals preferentially adopt the ideas and methods that appear most successful, is a foundational principle of collective behavior across systems ranging from ant colonies to scientific communities. But science is a particular kind of collective search -- one in which the quality of an explanation is itself difficult to assess. Is success bias adaptive in this setting? In agent-based simulations of collective theory building, we find that it is not. Scientists in our model systematically overestimate the quality of their own theories, creating an illusion of understanding: a persistent gap between perceived and actual performance. Success bias amplifies this illusion; communities that favor apparently successful theories explore a narrower range of possibilities, efficiently filtering out poor explanations but failing to discover better ones. This effect intensifies with problem complexity, as scientists in more complex environments become increasingly unable to assess how well their theories actually perform. Most strikingly, when agents optimize their social behavior to maximize the perceived success of their theories, they paradoxically undermine their actual performance, and produce levels of inequality that mirror those found in real scientific communities.
Avery W. Louis, Marina Dubova
Systems, Stanford University, Stanford, CA 94305, USA · Santa Fe Institute, Santa Fe, NM 87501, USA
Scientific knowledge gains new uses when ideas, methods, and results move between research contexts. Understanding this process requires a model of both specialized knowledge production and the intermediary activities that connect it. We introduce Scientific Liquidity Agents (SLA), an agent-based model of knowledge arbitrage in scientific communities. SLA represents domain specialists and epistemic arbitrageurs within a shared environment of research problems, researcher interactions, and scientific artifacts. Individual decisions connect to collective state through a belief-desire-intention architecture, a prerequisite-based problem graph, a dynamic interaction network, and a persistent publication registry. The framework makes researcher roles and knowledge-transfer frictions explicit, providing a process-oriented account of scientific production and exchange. Two exploratory simulations with 10 and 18 agents produce 68 and 274 registered papers. In the smaller run, arbitrageurs take part in 16 of the 26 contact pairs that join this community into one connected component, while topical similarity alone splits it into seven. The same run yields three similarity edges, 26 contact pairs, and two coauthorship pairs, so measured community structure depends on what counts as a link. Paper production is abundant, but no recorded replication succeeds, which places the bottleneck in verification rather than production. Solved-problem sets correspond to the prerequisite reachability of the disciplines represented in each community. SLA makes knowledge liquidity, the movement and reuse of artifacts across domains, a quantity that agentic science can simulate and measure.
Koutian Wu, You Liu, Yue Liu +2
The University of Texas at Austin · TaciteAI · Independent Researcher +2
We argue that the AI community is now ready to move beyond benchmarking and consolidate scattered efforts in model analysis into a systematic discipline, a direction we term Model Science. Complex AI models now serve billions of users, yet our understanding of how they work lags far behind our ability to deploy them. Decades of benchmark-driven research have delivered remarkable progress: extensive leaderboards, a wide range of performance metrics, tracking capability gains across diverse tasks; yet this success has also revealed the limits of benchmarks as they tell us whether models perform but not why they succeed or fail, they miss critical failure modes, such as hallucinations or shortcuts. Precedents from established sciences point the way forward: cognitive science shows that understanding complex systems requires complementary levels of analysis; neuroscience demonstrates that deep study of single cases reveals what population studies miss; medicine teaches that specialised training must develop alongside research practice; and agriculture models how shared infrastructure and principles enable cumulative progress. These lessons inform three foundations for Model Science. First, we propose to consolidate research around four functional perspectives: Verify, Explore, Steer, and Refine that address complementary questions about model behaviour. Second, we discuss the required infrastructure for cumulative knowledge: catalogues of datasets, models and findings. Third, we highlight the need for deep analysis of individual model instances, not just model families, because single cases can reveal what population studies miss.
Przemyslaw Biecek, Luca Longo, Jianlong Zhou +3
Center for Credible AI · University of Warsaw · Warsaw University of Technology +7