Model Collapse

Ask about this topic

Overview

Model collapse occurs when repeated training on generated or filtered data, or feedback among models, shifts predictions away from the original data and can erase rare patterns or modes. Researchers analyze these dynamics with statistical learning theory, dynamical systems, and empirical tests in language models, diffusion and image generators, and structured tasks such as regression and forecasting. Mitigations include retaining fresh real data, weighting synthetic examples, filtering for information or uncertainty, and using external verification or diverse human preferences. A central open question is when collapse is inevitable: outcomes depend on data quality, selection and training protocol, feedback structure, and whether evaluation detects losses that aggregate accuracy can hide.

Latest papers

CardsList

41 papers6 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

  1. Mitigating Model Collapse through Dreaming Learning

    Oct 21, 2026Matteo Benati, Alessandro Londei, Denise Lanzieri +2Model CollapseSelf-Training for Language Models

    DIAG, Sapienza University of Rome, Italy · Sony CSL Rome, CREF, Rome, Italy · Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark +2

  2. ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization

    Oct 21, 2026Yun Li, Ehsan Javanmardi, Simon Thompson +7Direct Preference OptimizationFine-Tuning Language Models

    Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan · TIER IV, Inc., Tokyo, Japan · Huawei

  3. Screening Collapse under Recursive Synthetic Data

    Oct 11, 2026Jia Zhang, Xin ChenFeature SelectionStatistical Learning Theory

    Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics, Chengdu 611130, China · Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen 518055, China

  4. No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse

    Oct 1, 2026Lewis MitchellLLM Fine-TuningModel Collapse

    Adelaide Data Science Centre School of Mathematical Sciences Adelaide University Adelaide SA 5005, Australia

  5. Feature Selective Model Collapse in Diffusion Models: Total Replacement versus Fixed-Budget Training

    Oct 1, 2026Hanna Malet, Gabriel TuriniciModel CollapseDiffusion Models

    CEREMADE CNRS Université Paris Dauphine - PSL Paris 75116 FRANCE · Cornell University Ithaca, NY, USA

  6. Source Identification Is Not Fitness Testing: Measuring the Limits of Synthetic-Data Attribution

    Sep 30, 2026Joss ArmstrongModel CollapseTraining Data Attribution

    Ericsson, Athlone, Ireland

  7. Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data

    Sep 16, 2026Matteo Marchi, João Pedro Silvestre, Bahman Gharesifard +1Synthetic Data PretrainingModel Collapse

    Electrical and Computer Engineering Department, University of California at Los Angeles, Los Angeles, CA 90095 USA · Department of Mathematics and Statistics at Queens University, Kingston, ON, Canada

  8. Break Step: Recursive Training Resonates with Replayed Sampling Noise

    Sep 10, 2026Yangze Liu, Zhongyi HanSelf-Training for Language ModelsModel Collapse

    Shandong University

  9. Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

    Sep 9, 2026Jichu li, Difan ZouSynthetic Data PretrainingHigh-Dimensional Linear Regression

    Renmin University of China · The University of Hong Kong

  10. Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization

    Aug 21, 2026Md Rezwanul Islam, Wael MohammedTime Series ForecastingModel Collapse

    Field Nation LLC, Minneapolis, MN, USA

  11. Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning

    Jul 19, 2026Xiaonan Luo, Yue Huang, Kehan Guo +4Self-Training for Language ModelsModel Collapse

    University of Notre Dame · Vanderbilt University · University of Pennsylvania

  12. World-Model Collapse as a Phase Transition

    Jun 30, 2026Xinyuan Song, Zekun CaiPhase TransitionsModel Collapse

    Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

  13. When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

    Jun 26, 2026Xinyuan Song, Zekun Cai, Liang ZhaoSelf-TrainingModel Collapse

    Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

  14. Model Collapse: On Recursion, Noise, and Uncharted Machine Visions

    Jun 22, 2026Violaine Boutet de MonvelModel CollapseGenerative Modeling

    LIRA, IRCAV · Bibliotheca Hertziana – Max Planck Institute for Art History · MPRG Impett – Machine Visual Culture: Artificial Intelligence and the History of Seeing

  15. Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization

    Jun 11, 2026Naïl B. Khelifa, Richard E. Turner, Ramji VenkataramananDiffusion SamplingModel Collapse

    University of Cambridge

  16. When Sample Selection Bias Precipitates Model Collapse

    Jun 11, 2026Xinbao Qiao, Xianglong Du, Wei Liu +4Synthetic Data PretrainingModel Collapse

    National University of Singapore (Chongqing) Research Institute · Chongqing Key Laboratory of Trusted Perception and Interaction Technology for Intelligent and Connected Vehicles, Chongqing, China · The Chinese University of Hong Kong +4

  17. Entropy Minimization without Model Collapse: Mitigating Prediction Bias in Medical Imaging

    Jun 1, 2026Tim Nielen, Sameer Ambekar, Johannes Kiechle +2Distribution Shift RobustnessTest-Time Adaptation

    School of Computation, Information and Technology, Technical University of Munich, Germany · Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany · Munich Center for Machine Learning (MCML) +3

  18. Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control

    May 29, 2026David MullettBenchmark DesignModel Collapse

    Independent Researcher

  19. Anti Mode-Collapse in Mean-Field Transformer via Auxiliary Variables

    May 28, 2026Masaaki Imaizumi, Masanori Koyama, Noboru Isobe +1TransformerSelf-Attention

    The University of Tokyo, Tokyo, Japan · RIKEN Advanced Intelligene Project, Tokyo, Japan · Kyoto University, Kyoto, Japan

  20. When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop

    May 28, 2026Yang Zhang, Xiukun Wei, Xueru ZhangLLM AlignmentSelf-Training

    Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio.

  21. Model Collapse as Cultural Evolution

    May 21, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuCultural EvolutionSelf-Training for Language Models

    The University of Hong Kong · Hong Kong, China · Stellaris AI Limited

  22. Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies

    May 20, 2026Ming LiuSelf-Training for Language ModelsModel Collapse

    Amazon

  23. When Does Model Collapse Occur in Structured Interactive Learning?

    May 19, 2026Yuchen Wu, Kangjie Zhou, Weijie SuModel CollapseGenerative Modeling

    School of Operations Research and Information Engineering, Cornell University · Department of Statistics, Columbia University · Department of Statistics and Data Science, University of Pennsylvania

  24. The Economics of Model Collapse: Equilibrium, Welfare, and Optimal Provenance Subsidies in Synthetic Data Markets

    May 19, 2026Gustav Olaf Yunus Laitinen-Fredriksson Lundström-ImanovData ProvenanceDataset Watermarking

    Department of Economics Stockholm University Stockholm, Sweden

  25. Silent Collapse in Recursive Learning Systems

    May 14, 2026Zhipeng ZhangModel CollapseAdaptive Learning

    China Mobile Research Institute, Beijing 100053, China · China Mobile GBA (Greater Bay Area) Innovation Institute, Guangzhou 510656, China

  26. Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences

    May 8, 2026Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson +1Synthetic Data PretrainingModel Collapse

    University of Waterloo · 2Rensselaer Polytechnic Institute.

  27. Human-AI Co-Evolution and Epistemic Collapse: A Dynamical Systems Perspective

    May 7, 2026Xuening Wu, Yanlan Kang, Qianya Xu +5Model CollapseHuman-AI Interaction

    Fudan University, Shanghai, China · Pfizer China, Shanghai, China · University of California San Diego, CA, USA +4

  28. On Semantic Loss Fine-Tuning Approach for Preventing Model Collapse in Causal Reasoning

    May 6, 2026Pratik Deshmukh, Atirek GuptaLLM Fine-TuningModel Collapse

    Technical University of Vienna, Austria · HCLTech, Noida, India

  29. Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities

    May 5, 2026Devon Jarvis, Richard Klein, Benjamin Rosman +2Model CollapseResponsible AI

    School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa · Machine Intelligence and Neural Discovery Institute, University of the Witwatersrand, Johannesburg, South Africa · Data Science and AI, Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg

  30. Epidemiology of Model Collapse: Modeling Synthetic Data Contamination via Bilayer SIR Dynamics

    Apr 14, 2026Xiangyu WangModel CollapseData Contamination in Language Models