The Subjectivity of Monoculture

Nathanael Jo; Nikhil Garg; Manish Raghavan

cs.CYFeb 27, 2026

The Subjectivity of Monoculture

Authors: Nathanael Jo, Nikhil Garg, Manish Raghavan

Organizations: MIT, Cambridge, USA · Cornell Tech, New York, USA

Abstract

Machine learning models -- including large language models (LLMs) -- are often said to exhibit monoculture, where outputs agree strikingly often. But what does it actually mean for models to agree too much? We argue that this question is inherently subjective, relying on two key decisions. First, the analyst must specify a baseline null model for what "independence" should look like. This choice is inherently subjective, and as we show, different null models result in dramatically different inferences about excess agreement. Second, we show that inferences depend on the population of models and items under consideration. Models that seem highly correlated in one context may appear independent when evaluated on a different set of questions, or against a different set of peers. Experiments on two large-scale benchmarks validate our theoretical findings. For example, we find drastically different inferences when using a null model with item difficulty compared to previous works that do not. Together, our results reframe monoculture evaluation not as an absolute property of model behavior, but as a context-dependent inference problem.

Figures & tables

Figure 1: Mean square error (MSE) and absolute mean of the pairwise residual correlations, as a function of $K$ : dimensions in the multidimensional IRT model. Left (right) shows results on the HELM ( HF ) dataset. As $K$ increases, residual correlations that are unexplained by the null model tend toward zero, meaning that increasingly expressive null models can arbitrarily absorb model correlations.

Figure 2: Residual correlation matrices for models in HELM (a) and HF (b), using different nulls: from left to right, a baseline from Kim et al. (2025) , from Goel et al. (2025) , from a 1D IRT with no item difficulty, and from a 1D IRT with item difficulties. Each item is a question-answer choice pair. The first three null models do not include item heterogeneity. As such, the corresponding excess correlation for the full IRT model is attenuated compared to the others because item difficulties absorb much of the apparent positive correlation.

Panel of Figure 3 — Figure 3: (a) Scatter plot of inferred model ability $\Theta$ in a two-dimensional IRT model, colored by accuracy over entire benchmark dataset ( HELM ). (b; top middle) Same as (a), but for HF data. (b; top right) Percentage of variance explained by principle component, when applying PCA on model accuracy stratified by question type. (b; bottom) Scatter plot of inferred ability $\Theta$ , disaggregated by contributor.

Figure 4: [Top] Inferred excess correlation matrix $\hat{\Sigma}$ from the two-stage procedure, on HELM (a) and ACSIncome (b) data. For HELM , the correlation matrix is shown for OpenAI models only. From left to right, more models are injected into the population for inference, starting from only OpenAI models to all models. For ACSIncome , the correlation matrix is shown for random forest models (RF) only, following a similar pattern as HELM . [Bottom] Histogram of inferred question difficulties $d$ for all questions in the dataset.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Mean square error (MSE) and summary statistics of the distribution of pairwise residual correlations, as a function of $K$ : the dimensions in the multidimensional IRT model. Top (bottom) row shows results on the HELM ( HF ) dataset. As $K$ increases, the residual correlations that are unexplained by the independent null model tend toward zero, meaning that a sufficiently expressive null model can arbitrarily absorb model correlations.

Figure 6: Residual correlation matrices for models in HELM (a) and HF (b), using different nulls: from left to right, a baseline from Kim et al. (2025) , from Goel et al. (2025) , from a 1D IRT with no item difficulty, and from a 1D IRT with item difficulties. Each item is a question. The first three null models do not include item heterogeneity. As such, the corresponding excess correlation for the full IRT model is attenuated compared to the others because item difficulties absorb much of the apparent positive correlation.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Linguistic Monoculture in LLM-Assisted Language Use
Jul 29, 2026Suhas Thejaswi, Juhi Kulshreshta, Lutz OettershagenPersonalized Language Models
Aalto University, Finland · University of Liverpool, United Kingdom
Narrowing the Horizon: Quantifying Topic Saliency Shifts in Generative Monoculture
Sep 28, 2026Oriane Peter, Elena Simperl, Kate DevlinLLM Auditing
King’s College London London, United Kingdom
Strategic Algorithmic Monoculture: Experimental Evidence from Coordination Games
Apr 10, 2026Gonzalo Ballestero, Hadi Hosseini, Samarth Khanna +1Multi-Agent Coordination Multi-Agent LLMs
Department of Economics, Pennsylvania State University · Department of Informatics and Intelligent Systems, Pennsylvania State University

Jul 29, 2026J/K move · Enter open · S save

Linguistic Monoculture in LLM-Assisted Language Use

Suhas Thejaswi, Juhi Kulshreshta, Lutz Oettershagen

Aalto University, Finland · University of Liverpool, United Kingdom

Personalized Language Models

Open paper AskPDF arXiv

Writing and communication are increasingly mediated by large language models (LLMs) that are being used to draft, revise and polish text. Although such assistance can improve clarity and help authors meet institutional expectations, widespread reliance on shared models may reduce population-level variation in linguistic form, a phenomenon we refer to as linguistic monoculture. We develop a mathematical framework in which authors and LLMs are represented as distributions over linguistic features and coevolve through repeated interaction. We analyze three interaction mechanisms: a shared model with a fixed linguistic distribution, a shared model recursively updated from author outputs, and personalized models updated through author-specific and population-level feedback. We characterize the resulting equilibria and convergence rates, showing that, shared models can drive authors toward a common norm, recursive feedback relocates the shared norm without altering pairwise spread under common conformity, and personalization can preserve a family of distinct author-model equilibria with nonzero linguistic diversity. We then endogenize conformity as a strategic choice trading off private benefits from clarity, legibility, and perceived fluency against distinctive style. Within this utility model, individually rational authors may conform more than is socially optimal because they do not internalize the value their distinctiveness provides to others, creating a negative externality and a price of monoculture that is finite for each fixed instance but can grow without bound when distinctiveness dominates authenticity. Synthetic simulations illustrate how fixed shared assistance, recursive feedback, and personalization produce different long-run diversity outcomes.

Abstract

LLM Evaluation

Explore similar work

Linguistic Monoculture in LLM-Assisted Language Use

Narrowing the Horizon: Quantifying Topic Saliency Shifts in Generative Monoculture

Strategic Algorithmic Monoculture: Experimental Evidence from Coordination Games