Drift and selection in LLM text ecosystems

Søren Riis

cs.CLMar 15, 2026

Drift and selection in LLM text ecosystems

Authors: Søren Riis

Organizations: Queen Mary University of London, London, United Kingdom

Abstract

The public text record -- the material from which both people and AI systems now learn -- is increasingly shaped by its own outputs. Generated text enters the public record, later agents learn from it, and the cycle repeats. Here we develop an exactly solvable mathematical framework for this recursive process, based on variable-order $n$ -gram agents, and separate two forces acting on the public corpus. The first is drift: unfiltered reuse progressively removes rare forms, and in the infinite-corpus limit we characterise the stable distributions exactly. The second is selection: publication, ranking and verification filter what enters the record, and the outcome depends on what is selected. When publication merely reflects the statistical status quo, the corpus converges to a shallow state in which further lookahead brings no benefit. When publication is normative -- rewarding quality, correctness or novelty -- deeper structure persists, and we establish an optimal upper bound on the resulting divergence from shallow equilibria. The framework therefore identifies when recursive publication compresses public text and when selective filtering sustains richer structure, with implications for the design of AI training corpora.

Figures & tables

Panel of Figure 1 — Figure 1: Recursive text as drift plus selection. A finite sample of the environment is used to fit a short-context generator. The generator produces synthetic text. What re-enters the environment can be unfiltered (drift) or filtered by a success criterion (selection).

Figure 2: Descriptive versus normative publication. Each panel shows both recursions over

40

generations with matched initial conditions. In the descriptive case (blue), the Kullback–Leibler divergence (the excess of cross-entropy over entropy) between the corpus

r

-gram distribution and the rollout implied by its induced order-

n

continuation law collapses to zero, so the public environment becomes

n

-shallow. In the normative case (orange), the recursion converges to a stable public environment, but the KL divergence stabilises at

2.57

bits—well within the extremal bound

L\log_{2}s=2\log_{2}5\approx 4.64

bits—so the corpus does not become

n

-shallow.

Figure 3: Recursive resampling concentrates support; filtering redirects it in the Conan Doyle corpus. All panels are computed from the recursive loop run on a trigram model fitted to the public-domain Arthur Conan Doyle fiction corpus used for the main-text illustration. Under neutral recursion, finite resampling erodes weakly supported higher-order structure and pushes the generator towards more generic continuations. Under lookahead-filtered recursion, probability mass is redirected towards continuations that preserve short-horizon viability under full-order generation. The resulting gain in support comes at a cost in diversity, and the value of that trade-off depends on what rare contexts or traces are removed and on whether later learners need finished artefacts or the search process itself. Parameters: $n=3$ (trigram), lookahead horizon $L=5$ , generations $T=10$ , total synthetic budget $M=120{,}000$ tokens per generation, five independent replicates, and $1{,}000$ fixed-seed evaluation rollouts of length $100$ per generation. Curves show mean $\pm$ s.e.m. across replicates.

Figure 4: Vocabulary retention declines under recursive resampling. Across Doyle, Austen, and Darwin, stronger replacement fractions produce stronger contraction. Full replacement (

\alpha=1

) yields the steepest decline by generation 12.

Figure 5: High-order support contracts faster than vocabulary. Trigram-type retention falls more sharply than vocabulary retention, showing the fragility of higher-order continuation structure.

Figure 6: Active-vocabulary retention above the baseline threshold

1/M_{0}

. Even thresholded active vocabulary shows the same monotone concentration pattern as

\alpha

increases.

Corpus	$\alpha$	Generation	Vocabulary retention	Trigram-type retention
Charles Darwin	0.25	12	0.825	0.656
Charles Darwin	0.50	12	0.808	0.621
Charles Darwin	1.00	12	0.408	0.165
Conan Doyle	0.25	12	0.860	0.717
Conan Doyle	0.50	12	0.836	0.683
Conan Doyle	1.00	12	0.473	0.211

Table 1: Theorem 1 negative endpoints under fixed-size recursive trigram resampling (generation 12).

Figure 7: The positive side: orthographic standardisation in an Austen pilot (0.5% injection rate). Active variant forms decline after generation 1; canonical share rises; but rare clean-vocabulary retention also falls.

tokens $s$	order $n$	$\|\mathcal{C}\|=s^{n-1}$	$\dim\mathcal{F}_{n}$	# extreme points	# Hamiltonian cycles
$2$	$2$	$2$	$2$	$3$	$1$
$2$	$3$	$4$	$4$	$6$	$1$
$2$	$4$	$8$	$8$	$19$	$2$
$2$	$5$	$16$	$16$	$179$	$16$
$2$	$6$	$32$	$32$	$30{,}176$	$2{,}048$
$3$	$2$	$3$	$6$	$8$	$2$

Table 2: Number of extreme points of the fixed-point polytope

\mathcal{F}_{n}

for various numbers of tokens

s

and model orders

n

. Each extreme point corresponds to a simple directed cycle in the de Bruijn graph

B(n{-}1,s)

. The rightmost column gives the number of Hamiltonian cycles (de Bruijn sequences), computed from the closed-form formula

(s!)^{s^{n-2}}/s^{n-1}

28 ; these form a subset of all simple cycles and so provide a lower bound on the total count. For reference, modern large language models typically use token vocabularies of

s\approx 32{,}000

–

128{,}000

and context windows exceeding

10^{5}

tokens; at such scales the number of extreme points is beyond any conceivable enumeration. † Lower bound from the Hamiltonian cycles alone.

Figure 8: Matched exact theorem-2 diagnostics. The descriptive recursion drives both the KL divergence and the

L^{1}

gap essentially to zero, while the normative recursion settles to a stable nonzero plateau. The entropy panels show that the corpus distribution and induced trigram continuation law can both stabilise even when the KL divergence remains positive.

Figure 9: The stable normative gap has visible structure. Left: the largest final

5

-gram probability mismatches between the corpus distribution and the rollout from its induced trigram continuation law. Right: the prefixes contributing most to the total

L^{1}

gap. The right panel visualises the prefix component only; in this genuinely

3

-deep target, the suffix-conditional component is also nonzero.

Theorem	Sign	Corpus/task	What is recursively removed or filtered	Main observable	Section
Theorem 1	Negative	Doyle, Austen, Darwin	Rare lexical tail and high-order support under fixed-size recursion	Vocabulary and trigram retention	Section 1
Theorem 1	Positive	Austen orthographic pilot	Low-support orthographic variants under fixed-size recursion	Active variants, canonical share	Section 1
Theorem 2	Fixed point	Synthetic exact runs	Soft vs. hard publication diagnostics; descriptive vs. normative KL split	KL divergence, $\ell_{1}$ convergence, entropy	Sections 4 – 5
Theorem 3	Both	Synthetic + Doyle	Environment reshaped by drift and acceptance filtering	Target conditional recovery by cross-entropy learners	Sections 4

Table 3: Matrix overview for appendix experiments and protocols.

Model Collapse

Model collapse studies how recursive training or feedback can erode a model’s diversity, accuracy, or fidelity to the target distribution.

See all 41 papers

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories
Apr 13, 2026Zhanwei Cao, YeoJin Go, Yifan Hu +1Language Model Generation Evaluation AI-Generated Text Detection
Northeastern University, Seattle, WA, USA
RAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authored
Aug 22, 2026Gregory Druck, Ethan SmithRAG Robustness LLM Reliability
Graphite Growth
More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMs
May 7, 2026Adrián Gude, Roi Santos-Ríos, Francis Bond +3LLM Evaluation
Universidade da Coruña, CITIC · Palacký University, Olomouc · Independent Researcher

Apr 13, 2026J/K move · Enter open · S save

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories

Zhanwei Cao, YeoJin Go, Yifan Hu, Shanu Sushmita

Northeastern University, Seattle, WA, USA

Language Model Generation Evaluation AI-Generated Text Detection

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing, where each interaction treats the model as stateless, generating responses independently without memory. Yet human writing is inherently longitudinal: authors' styles and cognitive states evolve across months and years. This raises a central question: can LLMs reproduce such temporal structure across extended time periods? We construct and publicly release a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012--2024 across three domains (academic abstracts, blogs, news) and compare them to trajectories generated by three representative LLMs under standard and history-conditioned generation settings. Using drift and variance-based metrics over semantic, lexical, and cognitive-emotional representations, we find temporal flattening in LLM-generated text. LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive-emotional drift relative to humans. These differences are highly predictive: temporal variability patterns alone achieve 94% accuracy and 98% ROC-AUC in distinguishing human from LLM trajectories. Our results demonstrate that temporal flattening persists regardless of whether LLMs generate independently or with access to incremental history, revealing a fundamental property of current deployment paradigms. This gap has direct implications for applications requiring authentic temporal structure, such as synthetic training data and longitudinal text modeling.

Abstract

Model Collapse

Explore similar work

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories

RAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authored

More Aligned, Less Diverse? Analyzing the Grammar and Lexicon of Two Generations of LLMs