The public text record -- the material from which both people and AI systems now learn -- is increasingly shaped by its own outputs. Generated text enters the public record, later agents learn from it, and the cycle repeats. Here we develop an exactly solvable mathematical framework for this recursive process, based on variable-order n-gram agents, and separate two forces acting on the public corpus. The first is drift: unfiltered reuse progressively removes rare forms, and in the infinite-corpus limit we characterise the stable distributions exactly. The second is selection: publication, ranking and verification filter what enters the record, and the outcome depends on what is selected. When publication merely reflects the statistical status quo, the corpus converges to a shallow state in which further lookahead brings no benefit. When publication is normative -- rewarding quality, correctness or novelty -- deeper structure persists, and we establish an optimal upper bound on the resulting divergence from shallow equilibria. The framework therefore identifies when recursive publication compresses public text and when selective filtering sustains richer structure, with implications for the design of AI training corpora.
Figures & tables
Figure 1: Recursive text as drift plus selection. A finite sample of the environment is used to fit a short-context generator. The generator produces synthetic text. What re-enters the environment can be unfiltered (drift) or filtered by a success criterion (selection).
Figure 2: Descriptive versus normative publication. Each panel shows both recursions over 40 generations with matched initial conditions. In the descriptive case (blue), the Kullback–Leibler divergence (the excess of cross-entropy over entropy) between the corpus r -gram distribution and the rollout implied by its induced order- n continuation law collapses to zero, so the public environment becomes n -shallow. In the normative case (orange), the recursion converges to a stable public environment, but the KL divergence stabilises at 2.57 bits—well within the extremal bound Llog2s=2log25≈4.64 bits—so the corpus does not become n -shallow.
Figure 3: Recursive resampling concentrates support; filtering redirects it in the Conan Doyle corpus. All panels are computed from the recursive loop run on a trigram model fitted to the public-domain Arthur Conan Doyle fiction corpus used for the main-text illustration. Under neutral recursion, finite resampling erodes weakly supported higher-order structure and pushes the generator towards more generic continuations. Under lookahead-filtered recursion, probability mass is redirected towards continuations that preserve short-horizon viability under full-order generation. The resulting gain in support comes at a cost in diversity, and the value of that trade-off depends on what rare contexts or traces are removed and on whether later learners need finished artefacts or the search process itself. Parameters: n=3 (trigram), lookahead horizon L=5 , generations T=10 , total synthetic budget M=120,000 tokens per generation, five independent replicates, and 1,000 fixed-seed evaluation rollouts of length 100 per generation. Curves show mean ± s.e.m. across replicates.
Figure 4: Vocabulary retention declines under recursive resampling. Across Doyle, Austen, and Darwin, stronger replacement fractions produce stronger contraction. Full replacement ( α=1 ) yields the steepest decline by generation 12.
Figure 5: High-order support contracts faster than vocabulary. Trigram-type retention falls more sharply than vocabulary retention, showing the fragility of higher-order continuation structure.
Figure 6: Active-vocabulary retention above the baseline threshold 1/M0 . Even thresholded active vocabulary shows the same monotone concentration pattern as α increases.
Figure 7: The positive side: orthographic standardisation in an Austen pilot (0.5% injection rate). Active variant forms decline after generation 1; canonical share rises; but rare clean-vocabulary retention also falls.
tokens s
order n
∣C∣=sn−1
dimFn
# extreme points
# Hamiltonian cycles
2
2
2
2
3
1
2
3
4
4
6
1
2
4
8
8
19
2
2
5
16
16
179
16
2
6
32
32
30,176
2,048
3
2
3
6
8
2
Table 2: Number of extreme points of the fixed-point polytope Fn for various numbers of tokens s and model orders n . Each extreme point corresponds to a simple directed cycle in the de Bruijn graph B(n−1,s) . The rightmost column gives the number of Hamiltonian cycles (de Bruijn sequences), computed from the closed-form formula (s!)sn−2/sn−1 28 ; these form a subset of all simple cycles and so provide a lower bound on the total count. For reference, modern large language models typically use token vocabularies of s≈32,000 – 128,000 and context windows exceeding 105 tokens; at such scales the number of extreme points is beyond any conceivable enumeration. † Lower bound from the Hamiltonian cycles alone.
Figure 8: Matched exact theorem-2 diagnostics. The descriptive recursion drives both the KL divergence and the L1 gap essentially to zero, while the normative recursion settles to a stable nonzero plateau. The entropy panels show that the corpus distribution and induced trigram continuation law can both stabilise even when the KL divergence remains positive.
Figure 9: The stable normative gap has visible structure. Left: the largest final 5 -gram probability mismatches between the corpus distribution and the rollout from its induced trigram continuation law. Right: the prefixes contributing most to the total L1 gap. The right panel visualises the prefix component only; in this genuinely 3 -deep target, the suffix-conditional component is also nonzero.
Theorem
Sign
Corpus/task
What is recursively removed or filtered
Main observable
Section
Theorem 1
Negative
Doyle, Austen, Darwin
Rare lexical tail and high-order support under fixed-size recursion
Vocabulary and trigram retention
Section 1
Theorem 1
Positive
Austen orthographic pilot
Low-support orthographic variants under fixed-size recursion
Active variants, canonical share
Section 1
Theorem 2
Fixed point
Synthetic exact runs
Soft vs. hard publication diagnostics; descriptive vs. normative KL split
KL divergence, ℓ1 convergence, entropy
Sections 4 – 5
Theorem 3
Both
Synthetic + Doyle
Environment reshaped by drift and acceptance filtering
Target conditional recovery by cross-entropy learners
Sections 4
Table 3: Matrix overview for appendix experiments and protocols.
Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing, where each interaction treats the model as stateless, generating responses independently without memory. Yet human writing is inherently longitudinal: authors' styles and cognitive states evolve across months and years. This raises a central question: can LLMs reproduce such temporal structure across extended time periods? We construct and publicly release a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012--2024 across three domains (academic abstracts, blogs, news) and compare them to trajectories generated by three representative LLMs under standard and history-conditioned generation settings. Using drift and variance-based metrics over semantic, lexical, and cognitive-emotional representations, we find temporal flattening in LLM-generated text. LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive-emotional drift relative to humans. These differences are highly predictive: temporal variability patterns alone achieve 94% accuracy and 98% ROC-AUC in distinguishing human from LLM trajectories. Our results demonstrate that temporal flattening persists regardless of whether LLMs generate independently or with access to incremental history, revealing a fundamental property of current deployment paradigms. This gap has direct implications for applications requiring authentic temporal structure, such as synthetic training data and longitudinal text modeling.
LLM responses are based on the internet (via training or RAG), and AI is now used to generate a significant amount of content online (Paredes et al., 2026), creating the potential for a self-reinforcing feedback loop. Prior work has shown that when LLMs are recursively trained on their own output, they experience model collapse (Shumailov et al., 2024): responses become less diverse, and eventually no longer resemble the original training data. In this paper, we show that a similar collapse occurs if LLM-based AI systems retrieve references they authored using a search tool. We call this RAG collapse. We conduct extensive experiments with three types of simulations of AI systems retrieving references they generated, using three model families, and 1,019 information-seeking prompts, totaling 1,528 simulations and over one million LLM API calls, and find that 79.6% (1,216/1,528) of simulations end in collapse. Surprisingly, even a single self-authored reference can trigger collapse because the LLM disproportionately cites its own content. This self-bias persists even after controlling for reference quality.
This study contributes to a growing line of research in comparing LLM-generated texts with human-authored text, in this case, English news text. We focus in particular on the evaluation of syntactic properties through formal grammar frameworks. Our analysis compares two generations of LLMs in the context of two human-authored English news datasets from two different years. Employing the Head-Driven Phrase Structure Grammar (HPSG) formalism, we investigate the distributions of syntactic structures and lexical types of AI-generated texts and contrast them with the corresponding distributions in the human-authored New York Times (NYT) articles. We use diversity metrics from ecology and information theory to quantify variation in grammatical constructions and lexical types. We show that English news text has changed little in the given time frame, while newer LLMs display reduced syntactic and, especially, lexical diversity compared to older, non-instruction-tuned models. These findings point to future work in studying effects of instruction tuning, which, while enhancing coherence and adherence to prompts, may narrow the expressive range of model output.
Adrián Gude, Roi Santos-Ríos, Francis Bond +3
Universidade da Coruña, CITIC · Palacký University, Olomouc · Independent Researcher