The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior memory. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences over a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce sentence diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.
Figures & tables
Figure 1: Illustration of iterative LLM reprocessing ( Markovian generation chains ). Each node denotes the output after iteration i . Under greedy decoding , chains typically enter fixed points or short cycles, limiting sentence-level diversity. Under sampling-based decoding , stochasticity may yield longer transients and more distinct outputs
Round
Qwen2.5-7B-Instruct
Llama-3.1-8B-Instruct
0
We begin with a prologue.
We begin with a prologue.
1
We start with a prologue.
The story commences with a prologue.
2
We begin with a prologue.
The narrative begins with a prologue.
3
We start with a prologue.
The story commences with a prologue.
4
We begin with a prologue.
The narrative begins with a prologue.
5
We start with a prologue.
The story commences with a prologue.
Table 1: Illustrative greedy-decoding trajectories under iterative rephrasing. Qwen2.5-7B enters a 2-cycle after one step, while Llama-3.1-8B alternates between two near-paraphrases with different lexical realizations.
Figure 2: Average number of unique paraphrases generated over 50 iterative rephrasings across three datasets, comparing four instruction-tuned LLMs: GPT-4o-mini, Llama-3.1-8B, Mistral-7B, and Qwen-2.5-7B. Results are shown for greedy decoding (orange) and sampling-based decoding (purple). Error bars represent one standard deviation.
Figure 3: Evolution of text similarity metrics across 50 rephrasing iterations for the BookSum dataset using greedy decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Model (Decoding)
BookSum
ScriptBase
News2024
Llama (greedy)
0.17
-0.17
0.19
Llama (sampling)
0.17
0.03
0.11
Mistral (greedy)
0.32
0.07
0.11
Mistral (sampling)
0.44
-0.02
0.20
Qwen (greedy)
0.35
0.09
0.20
Qwen (sampling)
0.49
0.16
0.37
Figure 4: Pearson correlation r between seed length (words) and the number of distinct outputs over T=50 iterations.
Figure 5: Number of distinct sentences produced over 50 iterative rephrasings with GPT-4o-mini under different settings. P1 and P2 correspond to Listings 1 and 2 in the Appendix. Boxes denote the interquartile range, and the center line indicates the median.
Figure 6: Prompt heterogeneity across iterations ( GPT-4o-mini , sampling-based decoding). We compare a single fixed prompt against alternating prompts, and a mixed variant constructed from prompt halves.
Sentence
Frequency
Hooray for best friends!
50
I don’t think so.
50
You’re the ones being banished!"
50
Sue now has two children and is expecting a third.
46
Suddenly, the ghost appears.
41
Jo continues to write short romantic stories for the newspaper while secretly working on a novel.
34
Table 2: Highest recurrence frequency observed sentences within the 50-iteration paragraph-level runs ( GPT-4o-mini , sampling-based decoding, prompt P1) on 150 BookSum paragraphs.
Figure 7: Distinct-sentence counts under iterated round-trip translation for GPT-4o-mini (sampling-based decoding) and Google Translate (v3) .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Round
Llama-3.1-8B-Instruct
Qwen2.5-7B-Instruct
0
We begin with a prologue.
We begin with a prologue.
1
The narrative commences with a prologue.
We start with a prologue.
2
The story begins with a prologue that sets the scene.
We begin with a prologue.
3
The narrative commences with a prologue that establishes the backdrop for the story.
We start with a prologue.
5
The story starts with a prologue, which establishes the setting and lays the groundwork for the events that unfold in the narrative.
We start with a prologue.
10
The narrative begins with a prologue, which sets the scene and lays the groundwork for the events that unfold in the story that follows.
We begin with a prologue.
Appendix
Table 3: Examples of repeated rephrasing by different LLMs (temperature=0.7, top-p=0.9).
Round
French
English
0
We begin with a prologue.
1
Nous commençons par un prologue.
We begin with a prologue.
2
Nous commençons par un prologue.
We begin with a prologue.
3
Nous commençons par un prologue.
We begin with a prologue.
4
Nous commençons par un prologue.
We begin with a prologue.
Appendix
Table 4: Examples of iterative translation via GPT-4o-mini (temperature=0.7, top-p=0.9).
Round
French
English
0
Elizabeth reads through Darcy’s letter with a mixture of emotions.
1
Elizabeth lit la lettre de Darcy avec un mélange d’émotions.
Elizabeth read Darcy’s letter with a mix of emotions.
2
Elizabeth a lu la lettre de Darcy avec un mélange d’émotions.
Elizabeth read Darcy’s letter with a mix of emotions.
⋮
⋮
⋮
11
Elizabeth lut la lettre de Darcy avec un mélange d’émotions.
Elizabeth read Darcy’s letter with a mixture of emotions.
12
Elizabeth a lu la lettre de Darcy avec un mélange d’émotions.
Elizabeth read Darcy’s letter with a mixture of emotions.
Appendix
Table 5: Examples of iterative translation via GPT-4o-mini (temperature=0.7, top-p=0.9). The omitted lines are all identical to the content in the second round (the italicized sentences). The similar scenario also occurs after the 19th round.
Figure 8: Evolution of text similarity metrics across 50 rephrasing iterations for the News2024 dataset using greedy decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Figure 9: Evolution of text similarity metrics across 50 rephrasing iterations for the ScriptBase dataset using greedy decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Figure 10: Evolution of text similarity metrics across 50 rephrasing iterations for the BookSum dataset using sampling-based decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Figure 11: Evolution of text similarity metrics across 50 rephrasing iterations for the News2024 dataset using sampling-based decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Figure 12: Evolution of text similarity metrics across 50 rephrasing iterations for the ScriptBase dataset using sampling-based decoding. Each iteration compares the current rephrased text against the previous iteration’s text as reference.
Dataset
Model
r
p
R²
Slope
booksum
Llama greedy
0.171
3.611e-02
0.029
0.107
Llama sample
0.166
4.272e-02
0.027
0.191
Mistral greedy
0.324
5.135e-05
0.105
0.350
Mistral sample
0.438
2.046e-08
0.192
0.594
Qwen greedy
0.353
9.427e-06
0.125
0.125
Qwen sample
0.494
1.330e-10
0.244
0.616
Appendix
Table 6: Results of the Correlation Analysis. r represents the Pearson correlation coefficient, and R2 represents the coefficient of determination.
This paper presents a probabilistic perspective on large language models (LLMs), developed with the aim of bringing together, in a single self-contained account, tools that are usually treated separately across the literature. LLMs are described through probability measures on the set of sequences of tokens, specified via their autoregressive conditional distributions. Training is formulated as a maximum-likelihood estimation problem, addressed by stochastic gradient methods, while text generation is viewed as the sequential simulation of the resulting stochastic process. The role of the asymmetry of the Kullback--Leibler divergence in text generation is examined in relation with characteristic phenomena such as hallucination and the distinction between statistical plausibility and truth. As a complementary illustration of the same viewpoint, we also discuss diffusion models, built around the score function, which cast generation not as sequential token prediction but as the simulation of a reverse-time stochastic process transforming noise into data both in discrete and continuous time.
Large language models (LLMs) are increasingly used as interactive agents and simulators, yet it remains unclear whether they can recover latent sequential structure beyond surface action frequencies. This distinction is critical for behavioral simulation, where actions are often shaped by prior context rather than marginal frequencies alone. We study this question using controlled two-player Rock--Paper--Scissors interactions and a one-player stochastic n-gram continuation task. Across these experiments, we test whether LLMs can identify latent strategies, follow simple Markov rules, and sustain higher-order conditional dependencies. Our framework separates distribution matching from conditional rule following. Results show that longer context does not improve identification, correct recognition does not ensure faithful simulation, and higher-order dependencies substantially degrade rule recovery. Apparent behavioral fidelity can therefore mask incorrect generative mechanisms.
Jerry Wang, Zhengxiang Wang, Ting Yu Liu +3
University of Illinois Urbana-Champaign · Stony Brook University · National Chengchi University
Looped language models (LMs) reuse one block of layers several times per token generation, promising more latent computation at no additional storage cost. Despite their rising popularity, the analysis of looped LMs is still in its infancy: little is known about whether the iterative transformations actually improve their capabilities, or whether these models remain stable under prompt perturbation. We construct an inspection protocol rooted in Mechanistic Interpretability comprising five diverse lenses. We apply it to examine eight looped LMs from four families. Models differ in many aspects but the key one is whether the prompt is transmitted only within the looped state (carry loops: Ouro, LoopUS) or re-supplied at every iteration (re-read loops: Parcae, and Rec-OLMo & Huginn from the Raven family). The results of our study provide valuable insights concerning safety and efficient use of looped language models.
Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk +2
Samsung AI Center, Warsaw, Poland · IFTR, Polish Academy of Sciences, Warsaw, Poland · University of Warsaw, Poland