cs.LGNov 4, 2026

Who Pays for the Knowledge Models Copy? Model Collapse as Lossy Transmission and Rogers' Paradox

Authors: Nam H. Le

Organizations: Department of Computer Science and Vermont Complex Systems Institute, University of Vermont, Burlington, VT, USA

Abstract

Generative models trained on the output of earlier models lose knowledge across generations, rare knowledge first. The remedies that work all bring in information from outside the loop, such as fresh human data, verifiers, or curators, and the theory of model collapse treats that supply as given. We argue that cultural evolution has a theory of the supply, and that it distinguishes two ways copying loses knowledge. In a fixed world, copies are lossy, as in Henrich's account of skill loss in small populations. In a changing world, even perfect copies go stale, which is Rogers' paradox. We write both as one chain with separately measurable parameters and test it on lineages of language models (word trigrams and a character transformer on two books, and OPT-125m on wikitext-2, on arXiv abstracts written after its pretraining, and on a world of 2,000 fictional facts that change at a set rate). In a fixed world, neutral drift on word frequencies predicts, with no free parameter, the rare-word coverage of every trigram lineage to within 0.03. In the fact world, the model's accuracy falls and its outdated answers rise with the rate of change; the chain calibrated on the static world predicts both for mid- and low-frequency facts to within 0.03 on average, with the rate of change set, not fitted. Models that inherit their weights keep more in a static world and carry more outdated facts in a changing one, as Rogers' theory says of individual memory. Finally we close the loop with the people who write the data. When they can ask the model instead of finding out, the model ends up exactly as reliable as finding out is costly, and in both regimes someone must keep paying to learn; we give the share in closed form, extend it to lossy copying, heterogeneous costs, aggregating models, and users who check, and show when checking is a stable way out.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Sep 10, 2026

Break Step: Recursive Training Resonates with Replayed Sampling Noise

How fast does a language model degrade when trained on its own outputs? Theory traces it to gradually accumulating errors, while experiments report repeated phrases within ten generations. Under a fixed sampling seed in vLLM, the fast loss of lexical diversity comes from the sampler. When vLLM serves a batch from one seeded sampling configuration, every request receives the same random draws, and a fixed seed replays them every generation. Fine-tuning raises the tokens that won, and the replayed draws let them win by more. Sharing across requests and replay across generations matter only together. Remove either one, by changing the shared seed every generation or by giving each request its own seed that repeats every generation, and the unique-4-gram fraction of two StableLM checkpoints stays near its starting value of about 0.98 through generation 3. Keep both, and the replayed shared seed takes seven checkpoints from five families to between 0.045 and 0.38 by then. Three generations of replay write the favoured phrases into the weights: decoded with one seed per request, the generation-3 weights of the replayed StableLM-2-1.6B chain recover most of their diversity, yet the phrase that filled every sample under the shared seed still opens 46% of them. Without replay, five checkpoints drift slowly, consistent with the gradual accumulation that theory describes, and three turn incoherent though their diversity scores stay high. One peer-reviewed model-collapse pipeline that fine-tunes Gemma-2-27B samples identical prompts under one seeded configuration, and three quarters of the rows it released for one iteration repeat nearly as often as one such batch copies them. A seed per request restores the fresh sample that stability analyses assume.
May 21, 2026

Model Collapse as Cultural Evolution

Model collapse, the progressive degradation of LLMs trained on their own outputs, has been characterized statistically but lacks a linguistic explanation for which structures degrade, in what order, and why. We show that iterated learning theory from cultural evolution fills this gap. We derive five falsifiable predictions, distinguish those uniquely discriminative for the theory from confirmatory ones, and test them by self-training LLaMA-2-7B and Mistral-7B over 10 generations in English, German, and Turkish. The critical discriminative finding: compositionality follows a non-monotonic trajectory (initially rising, then falling) under unfiltered self-training. This signature persists with maximally regular seed data (ruling out noise removal) and is sustained only by task-grounded filtering, not random filtering, providing the first LLM-scale evidence for the compression-communication tradeoff. All predictions are confirmed with large effect sizes (Hedges' g>1.6g > 1.6; BF10>100\mathrm{BF}_{10} > 100), and LLM regularization gradients closely match human behavioral data (R2=0.94R^2 = 0.94). These results reframe model collapse as a cultural transmission phenomenon and yield concrete principles for self-training pipeline design.
Mar 12, 2026

Language Generation with Replay: A Learning-Theoretic View of Model Collapse

As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed. At the same time, widespread LLM usage increases the volume of machine-generated content on the web; together, these trends raise the likelihood of generated text re-entering future training corpora, increasing the associated risk of performance degradation often called model collapse. In practice, model developers address this concern through data cleaning, watermarking, synthetic-data policies, or, in some cases, blissful ignorance. However, the problem of model collapse in generative models has not been examined from a learning-theoretic perspective: we study it through the theoretical lens of the language generation in the limit framework, introducing a replay adversary that augments the example stream with the generator's own past outputs. Our main contribution is a fine-grained learning-theoretic characterization of when replay fundamentally limits generation: while replay is benign for the strongest notion of uniform generation, it provably creates separations for the weaker notions of non-uniform generation and generation in the limit. Interestingly, our positive results mirror heuristics widely used in practice, such as data cleaning, watermarking, and output filtering, while our separations show when these ideas can fail.