cs.CLAug 22, 2026
SaveRAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authored
Organizations: Graphite Growth
Abstract
LLM responses are based on the internet (via training or RAG), and AI is now used to generate a significant amount of content online (Paredes et al., 2026), creating the potential for a self-reinforcing feedback loop. Prior work has shown that when LLMs are recursively trained on their own output, they experience model collapse (Shumailov et al., 2024): responses become less diverse, and eventually no longer resemble the original training data. In this paper, we show that a similar collapse occurs if LLM-based AI systems retrieve references they authored using a search tool. We call this RAG collapse. We conduct extensive experiments with three types of simulations of AI systems retrieving references they generated, using three model families, and 1,019 information-seeking prompts, totaling 1,528 simulations and over one million LLM API calls, and find that 79.6% (1,216/1,528) of simulations end in collapse. Surprisingly, even a single self-authored reference can trigger collapse because the LLM disproportionately cites its own content. This self-bias persists even after controlling for reference quality.
Figures & tables
Figure 1: Two responses to the prompt “Who are the best Twitch streamers currently?” from GPT-5.2 that differ in structure and streamers mentioned due to random sampling.
Figure 2: Entity visibility for “Who are the best Twitch streamers currently?” at round 1. “Kai Cenat” appears in all ten, while “Squeezie” appears in only two of ten.
Figure 3: Entity visibility for “Who are the best Twitch streamers currently?” at round 5. Every streamer now has either 0% or 100% visibility. The distribution has collapsed.
Figure 4: Overview of Replace All. In each round, every reference is replaced by a self-authored reference generated from a response in the previous round.
Figure 5: Overview of Replace One. Original references are replaced more slowly than in Replace All, although the design still assumes that self-authored references are retrieved.
Figure 6: Overview of the Search simulation. Self-authored references are added to a reference pool and retrieved using search, competing with the original references for retrieval.
Figure 7: Final-round entity visibility across nine independent runs for “Who are the best Twitch streamers currently?” Different entities survive in each run, illustrating run-to-run variance.
| Name | Questions |
|---|---|
| Entity ChatGPT | 843 |
| Editorial ChatGPT | 159 |
| Entity AI Overview | 60 |
| Editorial AI Overview | 45 |
Table 1: Dataset statistics.
| Overall | Entity | Editorial | |
|---|---|---|---|
| 1/26 ChatGPT | 38.9% [38.0, 39.9] | 41.1% [40.1, 42.1] | 20.6% [18.3, 23.0] |
| 6/26 ChatGPT | 42.7% [41.7, 43.7] | 45.8% [44.7, 46.8] | 27.5% [25.4, 29.7] |
Table 2: Estimated percentage of ChatGPT references classified as AI-generated, overall and by prompt type. Brackets show 95% Wilson confidence intervals over unique reference URLs.
| Simulation | Model | Dataset | Questions | Collapsed at Start | Collapsed at End | Rounds Collapsed |
|---|---|---|---|---|---|---|
| Replace All | GPT-5.2 Chat | Entity ChatGPT | 101 | 2.97% | 88.12% | 68.51% |
| Replace All | GPT-5.2 Chat | Editorial ChatGPT | 57 | 29.82% | 91.23% | 79.65% |
| Replace One | GPT-5.2 Chat | Entity ChatGPT | 101 | 2.97% | 88.12% | 67.18% |
| Replace One | GPT-5.2 Chat | Editorial ChatGPT | 57 | 21.05% | 94.74% | 81.67% |
| Search | GPT-5.2 Chat | Entity ChatGPT | 101 | 1.98% | 77.23% | 62.31% |
| Search | GPT-5.2 Chat | Editorial ChatGPT | 57 | 31.58% | 75.44% | 73.63% |
Table 3: Summary of experiments and collapse rates. The distinction between GPT-5.2 and GPT-5.2 Chat is explained in Section 5 .
Figure 8: Collapse rates across simulation designs using GPT-5.2 Chat.
Figure 9: Core metrics for entity questions across simulation designs. The numbers of unique words and entities decrease over rounds, indicating declining response diversity, while semantic and entity-ranking similarities increase.
Figure 10: Same-answer rate and self-authored citations for entity questions. The same-answer rate rises from less than 30% in round 1 to over 90%, while self-authored citations increase and generally exceed the rate expected from their share of the reference pool.
Figure 11: Observed and expected self-authored retrievals by round for entity questions in Search using GPT-5.2 Chat. As with citations, more self-authored references are retrieved than expected by chance. Replace All and Replace One do not perform retrieval.
Figure 12: Metrics for editorial questions across simulation designs. Unique words decrease while semantic similarity and the same-answer rate increase. Self-authored references are again cited disproportionately.
Figure 13: Entity visibility across selected rounds for “What are the best video games of all time today?” Initially, visibility is broadly distributed; after the first-round answers become references, the distribution shifts, and by round 5 every entity either always or never appears.
Figure 14: Round 1 responses: the first few games are the same, but the answers then diverge.
Figure 15: Round 10 responses: despite slight formatting differences, the games and their order are identical, and the text is nearly word-for-word the same.
Figure 16: Collapse by model for the Replace One simulation. All three models exhibit collapse, showing that the effect is not specific to OpenAI models. Gemini 3 Pro has lower collapse rates, though the confidence intervals overlap in most cases.
Figure 17: Core entity-question metrics by model for Replace One. Claude and Gemini initially include more entities than GPT-5.2 Chat, but unique entities and words decrease for every model. Semantic similarity increases more slowly for Gemini 3 Pro, while entity-ranking similarity increases at a similar rate across models.
Figure 18: Same-answer rate and self-authored citations by model for entity questions. Same-answer rates increase for every model, though more slowly for Gemini 3 Pro and Claude Sonnet 4.5, and every model disproportionately cites self-authored references.
Figure 19: Editorial-question metrics by model for Replace One. Unique words decrease for every model, while semantic similarity and same-answer rates increase, more slowly for Gemini 3 Pro. Every model disproportionately cites self-authored references.
Figure 20: Large-scale entity-question results for Search using GPT-5.2.
Figure 21: Retrieval and same-answer metrics for the large-scale entity-question Search experiment.
Figure 22: Large-scale editorial-question results for Search using GPT-5.2.
Figure 23: Retrieval and same-answer metrics for the large-scale editorial-question Search experiment.
Figure 24: Citation rate by reference type. AI-generated originals are cited only slightly more often than human-written originals (9.4% vs. 7.4%), whereas self-authored references are cited at 38.9%.
Figure 25: Mean reference-quality score by dimension. Self-authored references score higher on most dimensions, while the two reference types are tied in accuracy.
Figure 26: Citation rates in the quality-controlled subset. Self-authored references retain a 2.9 citation-rate advantage.
| Name | Pearson Correlation | |
|---|---|---|
| unique entities | 0.342 | |
| length of responses | 0.245 | |
| unique words | 0.223 | |
| entity ranking similarity | 0.164 | |
| same-answer % | 0.151 |
Table 4: Correlations between initial-response statistics and eventual collapse in entity-question Search simulations.
Figure 27: Entity dropout rates by initial visibility and position. Dropout decreases as initial visibility increases and as position improves.
Figure 28: Mean entity mentions in responses versus references, showing that response frequencies do not simply mirror reference frequencies.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 29: Complete Replace One results for GPT-5.2 Chat.
Figure 30: Complete Replace One results for Gemini 3 Pro.
Figure 31: Complete Replace One results for Claude Sonnet 4.5.
More in this topic
RAG Robustness
Retrieval-augmented generation robustness studies how to keep systems reliable when retrieved evidence is noisy, misleading, incomplete, or malicious.
See all 119 papers