Stability and Diversity of Networked Self-Consuming Generative Ecosystems

Xiukun Wei; Yang Zhang; Xueru Zhang

cs.LGOct 7, 2026

Stability and Diversity of Networked Self-Consuming Generative Ecosystems

Authors: Xiukun Wei, Yang Zhang, Xueru Zhang

Organizations: The Ohio State University

Abstract

The widespread deployment of generative AI has made it increasingly difficult to distinguish synthetic content from real data. Consequently, synthetic data is inevitably incorporated into the training pipelines of future model generations, forming a self-consuming training loop. Prior work has studied the effects of such recursive self-consuming training, but analyses have largely been limited to isolated models, where a model consumes only its own synthetic data, or to simplified interactions between two models. This paper takes a first step toward understanding networked self-consuming generative models, in which multiple models consume synthetic data generated by one another through complex interaction pathways. We introduce a theoretical framework representing models as nodes in a directed, weighted graph, with edge weights governing the flow of synthetic data among models. Using this framework, we analyze the long-term behavior of networked models under retraining dynamics, establishing conditions for convergence and characterizing the resulting fixed points. We further investigate how the system's long-term stability and diversity are shaped by each model's access to real data, cross-model data consumption, and the structure of the interaction graph.

Figures & tables

Figure 1 : From networked self-consuming models to graph abstraction. (1) Left: A generative ecosystem in which each model is trained on its own real-data distribution and a weighted mixture of synthetic data produced by other models. (2) Right: Abstraction of the system as a directed, weighted interaction graph that captures effective synthetic-data propagation among models. (3) Top right: An example local retraining rule, where Model $3$ is trained at iteration $t$ on $p^{\mathrm{data}}_{3}+\lambda_{3}\sum_{j=1}^{K}w_{3j}p_{\boldsymbol{\theta}_{j}^{t-1}}$ .

Panel of Figure 2 — Figure 2 : Stability experiments when all models trained on full CIFAR-10. Each row corresponds to a different interaction graph. For each row, the left figure illustrates the direction and strength of synthetic data flow, while the right figures report the FID between generated samples and real data for each model. Results are shown under different synthetic mixing strengths, with $\alpha_{i}=\alpha$ shared across models.

Model	Model Type	Stability	Diversity
A	VLB Diffusion	$\{0,\cdots,9\}$	$\{0,\cdots,9\}$
B	OT-CFM		$\{3,4\}$
C	iCFM		$\{5,6\}$
D	Hybrid Diffusion		$\{7,8,9\}$

Table 1 : Models and real-data assignments. The same data for all models is used for stability; the heterogeneous assignment is used for diversity experiments.

Figure 3 : Diversity across models under heterogeneous data with different interaction graphs. Systems are defined in Fig. 2 . Each figure reports the evolution of diversity

\mathcal{D}

across retraining rounds under different synthetic mixing ratios

\alpha_{i}=\alpha

shared across models.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

System	$\\|W\\|_{2}$	$\rho(J)$	$r_{\mathrm{emp}}$	$\rho(D)$	$\max_{C}\gamma(C)$
Sys. 1 (isolated)	1.00	0.900	0.900	0.900	0.900
Sys. 2 (weak ring)	1.00	0.606	0.606	0.700	0.450
Sys. 3 (strong ring)	1.00	0.582	0.582	0.674	0.604
Sys. 4 (dense)	1.00	0.600	0.600	0.900	0.225

Appendix

Table 2: Closed-form spectral validation at

\lambda=1

(

s=0.5

). The measured asymptotic rate

r_{\mathrm{emp}}

matches the exact Jacobian spectral radius

\rho(J)

, while

\rho(D)

upper-bounds

\rho(J)

. The last column reports the largest cycle gain and verifies the lower bound in Prop. 3.10 .

System	$s_{c}^{D}$	$s_{c}^{J}$
Sys. 1 (isolated)	0.556	0.556
Sys. 2 (weak ring)	0.714	0.825
Sys. 3 (strong ring)	0.742	0.860
Sys. 4 (dense)	0.556	0.833

Appendix

Table 3: Critical synthetic-data ratio

s=\lambda/(1+\lambda)

. The comparison-matrix threshold

s_{c}^{D}

is a sufficient stability certificate and is therefore no larger than the exact Jacobian threshold

s_{c}^{J}

in all tested systems.

$\#$	Model	Architecture	Stability	Diveristy	Retrain Steps
A	DDPM	MLP (hidden dimensions 128)	$t\in\{0,\cdots,7\}$	$t\in\{0,4\}$	$100$
B	CFM	MLP	$t\in\{0,\cdots,7\}$	$t\in\{2,6\}$	$50$
C	GAN	MLP	$t\in\{0,\cdots,7\}$	$t\in\{3,5\}$	$30$
D	DDPM	MLP (hidden dimensions 256)	$t\in\{0,\cdots,7\}$	$t\in\{1,7\}$	$100$

Appendix

Table 4: Model architectures and data access.

Panel of Figure 4 — Figure 4: Stability experiments when all models trained on full dataset. Each row corresponds to a different interaction graph. For each row, the left figure illustrates the direction and strength of synthetic data flow, while the right figures report the Wasserstein distance ( $\mathcal{W}\mathcal{D}$ ) between generated samples and real data for each model. Results are shown under different synthetic mixing strengths, with $\alpha_{i}=\alpha$ shared across models.

Panel of Figure 5 — Figure 5: Stability experiments when all models trained on full dataset (continued).

Panel of Figure 6 — Figure 6: Generated samples when all models trained on full dataset with different interaction graphs. All models use $\alpha_{i}=\alpha=1.0$ .

Panel of Figure 7 — Figure 7: Generated samples when all models trained on full dataset. All models use $\alpha_{i}=\alpha=0.2$ .

Panel of Figure 8 — Figure 8: Diversity across models under heterogeneous data with different interaction graphs. Each figure reports the system-level diversity metric $\mathcal{D}$ across retraining rounds under different synthetic mixing strengths $\alpha$ .

Panel of Figure 9 — Figure 9: Generated samples under the heterogeneous data with different interaction graphs. All models use $\alpha_{i}=\alpha=1.0$ .

Panel of Figure 10 — Figure 10: Generated samples under the heterogeneous data with different interaction graphs. All models use $\alpha_{i}=\alpha=0.2$ .

$\#$	Model	Variants	Stability	Diversity	Retrain Steps
A	DDPM	VLB	class $\{0,\cdots,9\}$	class $\{0,\cdots,9\}$	$100$
B	CFM	OT-CFM	class $\{0,\cdots,9\}$	class $\{3,4\}$	$600$
C	CFM	iCFM	class $\{0,\cdots,9\}$	class $\{5,6\}$	$600$
D	DDPM	hybrid	class $\{0,\cdots,9\}$	class $\{7,8,9\}$	$100$

Appendix

Table 5: Model architectures and data access.

Panel of Figure 11 — Figure 11: Stability experiments when all models trained on full dataset. Each row corresponds to a different interaction graph. For each row, the left figure illustrates the direction and strength of synthetic data flow, while the right figures report the recall between generated samples and real data for each model. Results are shown under different synthetic mixing strengths, with $\alpha_{i}=\alpha$ shared across models.

Panel of Figure 12 — Figure 12: Stability experiments when all models trained on full dataset. Each row corresponds to a different interaction graph. For each row, the left figure illustrates the direction and strength of synthetic data flow, while the right figures report the precision between generated samples and real data for each model. Results are shown under different synthetic mixing strengths, with $\alpha_{i}=\alpha$ shared across models.

Panel of Figure 13 — Figure 13: Generated samples under the heterogeneous real data setting with different interaction graphs. All models use $\alpha_{i}=\alpha=1.0$ .

Panel of Figure 14 — Figure 14: Generated samples under the heterogeneous real data setting with different interaction graphs. All models use $\alpha_{i}=\alpha=0.2$ .

Explore similar work

Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences
May 8, 2026Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson +1Synthetic Data Generative Models
University of Waterloo · 2Rensselaer Polytechnic Institute.
Self-Consuming Generative Models with Co-Evolving Human Preferences
Oct 7, 2026Xiukun Wei, Tian Xie, Ding Zhu +1Preference Datasets Generative Models
The Ohio State University
Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training
Feb 17, 2026Kevin Wang, Hongqian Niu, Didong LiMulti-Stage Training
Department of Biostatistics, University of North Carolina at Chapel Hill

May 8, 2026, cs.LGJ/K move · Enter open · S save

Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences

Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson, Lukasz Golab

University of Waterloo · 2Rensselaer Polytechnic Institute.

Synthetic Data Generative Models Retraining Collapse Theory

Open paper AskPDF arXiv

Recursive retraining of generative models poses a critical representation challenge: when synthetic outputs are curated based on a fixed reward signal, the model tends to collapse onto a narrow set of outputs that over-optimize that objective. Prior work suggests that such collapse is unavoidable without adding real data into the mix. We revisit this conclusion from an alignment perspective and show that collapse can be mitigated through curation based on multiple reward functions. We formalize the dynamics of recursive training under heterogeneous preferences and prove that, under certain conditions, the model converges to a stable distribution that allocates probability mass across competing high-reward regions. The limiting distribution preserves diversity and provably satisfies a weighted Nash bargaining solution, offering a formal interpretation of value aggregation in synthetic retraining loops.

Abstract

Explore similar work

Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences

Self-Consuming Generative Models with Co-Evolving Human Preferences

Can Generative Artificial Intelligence Survive Data Contamination? Theoretical Guarantees under Contaminated Recursive Training