Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Philipp Emanuel Weidmann; Allen Roush; Judah Goldfeder; Sanjay Basu; Ravid Shwartz-Ziv

cs.CLAug 24, 2026

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Authors: Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

Organizations: Thoughtworks · New York University

Abstract

Large Language Models generate text autoregressively, but open-ended generation is prone to verbatim looping, in which models repeat spans already present in context. Standard defenses such as repetition, presence, and frequency penalties and n-gram blocking act on token recurrence rather than the sequential structure of a loop, and often suppress looping only at strengths that also degrade formatting or fluency. We propose Don't Repeat Yourself (DRY), a sampling-time logit adjustment that penalizes a candidate token only when generating it would extend the current suffix into an exact continuation of a span seen earlier in the context. Sequence breakers protect chat templates and formatting tokens. Across models from 1.5B to 120B parameters, nine prompt families, and a 600-pair human study, DRY reduces suffix-extension rate by 47% while improving lexical diversity. An intervention-matched placebo produces no comparable reduction, identifying suffix matching as the operative mechanism. On AWQ-quantized 70B and 120B models, DRY reduces loop rate by roughly half while preserving MT-Bench, MMLU, and GSM8K performance, whereas standard alternatives lose measurable ground. DRY has been adopted by popular open-source LLM inference frameworks including llama.cpp, ExLlamaV2, and text-generation-webui, highlighting its practical impact on text generation.

Figures & tables

Panel of Figure 1 — Figure 1: DRY mechanism. The current suffix matches an earlier span of length 3. Candidate token $v$ , which followed the earlier span, receives an exponential penalty that grows with match length. Sequence breakers (e.g., newline) halt matching at structural boundaries, preventing penalties on formatting tokens.

Method	SER@4 $\downarrow$ (95% CI)	SER@8 $\downarrow$	D-4 $\uparrow$	LFL $\uparrow$
No intervention	0.124 [.087, .160]	0.069	0.958	100.1
Repetition penalty	0.083 [.061, .106]	0.047	0.969	137.1
Presence penalty	0.112 [.077, .144]	0.064	0.959	98.8
Frequency penalty	0.097 [.063, .126]	0.056	0.967	134.5
No-repeat n-gram ∗	0.043 [.029, .056]	0.000	0.986	101.6
Contrastive decoding	0.145 [.137, .153]	0.079	0.957	58.7

Table 1: DRY achieves a 47% relative SER@4 reduction over no intervention while improving distinct-4, outperforming all soft baselines. Main results across 16,566 regime-A generations (macro-averaged over three models and three seeds). SER@4, SER@8: suffix-extension rates. D-4: distinct-4. LFL: loop-free length (tokens). 95% CIs (cluster bootstrap across model

\times

seed cells, 5000 resamples) shown for SER@4. Bold marks best among soft methods.

Figure 2: DRY reduces SER@4 relative to every penalty baseline on every model; no-repeat n-gram achieves lower raw SER@4 only by hard-blocking all n-gram reuse, including structurally necessary repetition. SER@4 per method and model (regime A, three seeds). Lower is better. DRY shown in dark blue, no-repeat n-gram in purple.

Figure 3: DRY sits on the Pareto frontier of loop suppression vs. output diversity alongside no-repeat n-gram, but with softer, graduated control rather than hard blocking. Loop suppression (SER@4,

x

-axis, lower is better) vs. output diversity (distinct-4,

y

-axis, higher is better). Each large marker is a method averaged across three models; small markers show per-model values. The ideal corner is upper-left.

Figure 4: DRY suppresses loops aggressively on loop-stress families while applying graduated, moderate suppression on legitimate-repetition families, preserving correct reuse where token-level penalties and hard blocking cannot. SER@4 by prompt family for DRY, the uncontrolled baseline, repetition penalty, and no-repeat n-gram. Families are ordered by baseline SER@4 (left = highest baseline loop rate).

Figure 5: DRY roughly halves SER@4 on both Llama-3-70B-Instruct and GPT-OSS-120B while improving distinct-4 and MAUVE; no-repeat n-gram edges DRY on raw SER@4 only through hard blocking, at a cost on capability scores (Figure 6 ). Frontier-scale evaluation. Top row: Llama-3-70B-Instruct. Bottom row: GPT-OSS-120B. DRY shown in dark blue, no-repeat n-gram in purple.

Figure 6: DRY is the only repetition control that preserves capability scores on Llama-3-70B-Instruct, tracking the uncontrolled baseline on MT-Bench, MMLU, and GSM8k while repetition penalty and no-repeat n-gram both incur measurable losses. Capability preservation on Llama-3-70B-Instruct. The dotted line marks the uncontrolled baseline; DRY shown in blue.

Figure 7: Human annotators prefer DRY over the uncontrolled baseline on loop avoidance (and tie on fluency and formatting), and prefer DRY over repetition penalty 1.15 across all three axes, with the largest margin on formatting preservation. Human A/B preference on MTurk (

N\,=\,600

pairs

\times

3 annotators). Left: DRY versus uncontrolled baseline. Right: DRY versus repetition penalty 1.15.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

	Qwen 2.5-1.5B		Llama 3.2-3B		Qwen 2.5-7B
Method	SER@4	D-4	SER@4	D-4	SER@4	D-4
No intervention	0.057	0.974	0.167	0.947	0.147	0.953
Repetition pen.	0.045	0.977	0.093	0.970	0.112	0.961
Presence pen.	0.053	0.976	0.147	0.947	0.136	0.956
Frequency pen.	0.039	0.984	0.127	0.960	0.124	0.956
No-repeat n-gram	0.018	0.995	0.058	0.984	0.052	0.979

Appendix

Table 2: Per-model results (regime A, three seeds). SER@4 and distinct-4 (D-4) for each method and model.

	Regime A		Regime B
Method	SER@4	D-4	SER@4	D-4
No intervention	0.124	0.958	0.121	0.970
DRY	0.065	0.975	0.067	0.980
Relative reduction	47.4%		44.6%

Appendix

Table 3: DRY and baseline SER@4 across two decoding regimes (macro-averaged over three models).

	Seed 7	Seed 42	Seed 123	Mean $\pm$ SD
DRY SER@4	0.067	0.065	0.065	0.066 $\pm$ 0.001
Baseline SER@4	0.121	0.126	0.121	0.123 $\pm$ 0.003

Appendix

Table 4: DRY SER@4 per random seed (regime A).

Method	Hard/soft	Structure-aware	Typical failure mode
Repetition pen.	Soft	No	Suppresses common or structural tokens
Presence pen.	Soft	No	Discourages any reuse, even expected
Frequency pen.	Soft	No	Flattens high-frequency structural tokens
No-repeat n-gram	Hard	No	Blocks legitimate fixed phrases and templates
DRY	Soft	Yes (breakers)	Depends on breaker choice, exact loops only

Appendix

Table 5: Detailed method taxonomy. The central distinction is whether a method acts on repeated tokens or on repeated continuations of the current suffix .

Method	Target	Type	Key parameter(s)
No intervention	–	Reference	–
Repetition penalty	Token recurrence	Soft, multiplicative	penalty $\in[1.02,1.30]$
Presence penalty	Token occurrence	Soft, additive	penalty $\in[0.10,1.50]$
Frequency penalty	Token count	Soft, additive	penalty $\in[0.10,1.50]$
No-repeat n-gram	N-gram recurrence	Hard blocking	$n\in\{4,5,6,8\}$
Contrastive decoding	Amateur–expert gap	Soft, model-based	$\alpha,\beta$ , amateur model

Appendix

Table 6: Compared methods. Primary baselines are the three token-level penalty types. The placebo tests mechanism specificity by applying matched-strength perturbation without suffix awareness.

Family	Count	Primary test
Stress dialogue	40	Multi-turn loop suppression
Long-context chat	40	Late-turn looping in long histories
Creative continuation	40	Loop suppression vs. style preservation
Structured formatting	20	Format token preservation
Necessary repetition	20	False-positive control (labels, refrains)
Exact copy / quotation	20	Copy fidelity under repetition control

Appendix

Table 7: Prompt families. The benchmark balances loop stress, quality preservation, false-positive control, and negative controls across nine families totaling 240 prompts (168 test, 72 dev).

Method	Settings
DRY	$\lambda{=}0.8$ , $\beta{=}1.75$ , $L{=}2$ , default breakers, range = 1024
Contrastive decoding	$\alpha{=}0.1$ , $\beta_{\text{cd}}{=}0.5$ , amateur = same family 1B
Repetition penalty	penalty $\in\{1.10,1.20\}$ (mean reported), range = 1024
Presence penalty	penalty = 0.50, range = 1024
Frequency penalty	penalty = 0.50, range = 1024
No-repeat n-gram	$n{=}4$

Appendix

Table 8: Tuned hyperparameter settings used in the main evaluation.

Configuration	SER@4 $\downarrow$	False Positive Penalty Rate $\downarrow$	Structural MAUVE $\uparrow$
No intervention	0.120	–	1.00 ∗
DRY (no breakers)	0.041	28.4%	0.61
DRY (standard breakers)	0.053	1.2%	0.94

Appendix

Table 9: Sequence-breaker ablation on Qwen 2.5-7B over structured-formatting prompts. Without breakers, DRY achieves marginally lower SER@4 but exhibits a 23

\times

higher false-positive rate on structural tokens, devastating distributional fidelity. ∗ Reference distribution for structural MAUVE.

Figure 8: Sequence-breaker ablation. Without breakers DRY achieves marginally lower SER@4 (left), but its False Positive Penalty Rate on structural tokens spikes from 1.2% to 28.4% (center), and structural MAUVE collapses from 0.94 to 0.61 (right). Breakers are the operative mechanism that lets DRY decouple verbatim repetition from legitimate structural reuse.

Figure 9: Loop-free generation length distributions per method and model (regime A). Violin plots show the full distribution. DRY shifts the distribution toward longer loop-free spans on all three models relative to the uncontrolled baseline.

Figure 10: SER@4, distinct-4, and loop-free length across model scales (1.5B, 3B, 7B). DRY (thick blue line) consistently outperforms penalty baselines across all three scales.

Method	SER@4 $\downarrow$ (95% CI)	SER@8 $\downarrow$	D-4 $\uparrow$	LFL $\uparrow$
No intervention	0.130 [.123, .143]	0.075	0.948	82.7
Repetition penalty ‡	0.123 [.122, .124]	0.064	0.963	109.6
Presence penalty †	0.125 [.106, .144]	0.079	0.974	95.0
Frequency penalty †	0.106 [.098, .113]	0.064	0.974	106.1
No-repeat n-gram ∗	0.042 [.040, .043]	0.000	0.981	83.0
DRY (ours)	0.079 [.069, .093]	0.027	0.962	84.2

Appendix

Table 10: Qwen 2.5-14B results (regime A). DRY, no-intervention, and no-repeat n-gram have three seeds. DRY’s 39% reduction is consistent with smaller-scale results. 95% CIs in brackets (cluster bootstrap over seeds).

Figure 11: DRY latency profile at extreme contexts. Left: stacked Time Per Output Token decomposition at 4K, 32K, 64K, and 128K tokens. Right: relative overhead as a fraction of the forward pass. DRY stays under a 3% budget across the full range and undercuts the multiplicative repetition penalty at every context length.

Method	Context = 4K	Context = 32K	Context = 64K	Context = 128K
Forward pass (no intervention)	10.45 ms/tok	11.12 ms/tok	11.95 ms/tok	12.30 ms/tok
Repetition penalty ( $O(V)$ logit pass)	+0.15 ms/tok	+0.22 ms/tok	+0.28 ms/tok	+0.41 ms/tok
DRY (reverse suffix match)	+0.04 ms/tok	+0.09 ms/tok	+0.18 ms/tok	+0.32 ms/tok
DRY relative overhead	0.4%	0.8%	1.5%	2.6%

Appendix

Table 11: Per-token latency benchmark (Llama 3.2-3B, RTX 4090, batch size 1). DRY’s reverse suffix match adds less than 3% overhead even at a 128K context.

	Llama-3-70B-Instruct			GPT-OSS-120B
Method	SER@4 $\downarrow$	D-4 $\uparrow$	MAUVE $\uparrow$	SER@4 $\downarrow$	D-4 $\uparrow$	MAUVE $\uparrow$
No intervention	0.088 [.082, .095]	0.942	1.00 ∗	0.076 [.071, .082]	0.951	1.00 ∗
Repetition penalty	0.065 [.059, .070]	0.948	0.79	0.058 [.051, .064]	0.954	0.81
Presence penalty	0.079 [.071, .085]	0.944	0.52	0.069 [.065, .073]	0.952	0.58
No-repeat n-gram	0.035 [.033, .038]	0.971	0.93	0.031 [.029, .033]	0.975	0.94
DRY (ours)	0.042 [.038, .046]	0.959	0.96	0.039 [.035, .042]	0.966	0.97

Appendix

Table 12: Frontier-scale evaluation (Llama-3-70B-Instruct and GPT-OSS-120B, AWQ 4-bit, 3 seeds, 4,500 generations per model). DRY halves the loop rate while maintaining or improving distinct-4 (D-4) and MAUVE. 95% CIs from cluster bootstrap over seeds. ∗ The MAUVE column on this table uses the uncontrolled baseline as the reference distribution and therefore measures the distribution shift a method induces relative to the uncontrolled model, complementing the human-reference MAUVE reported in Section 5.2 and Appendix P . The two reference choices answer different questions and need not coincide in ranking.

Method	MT-Bench $\uparrow$	MMLU (gen, 5-shot) $\uparrow$	GSM8k (0-shot CoT) $\uparrow$
No intervention	8.95 $\pm$ 0.04	79.2%	92.1%
Repetition penalty	8.61 $\pm$ 0.12	76.8%	88.4%
No-repeat n-gram	8.12 $\pm$ 0.15	71.4%	81.2%
DRY (ours)	8.97 $\pm$ 0.05	79.1%	92.0%

Appendix

Table 13: Downstream capability evaluation on Llama-3-70B-Instruct. DRY is the only repetition control that preserves core reasoning and instruction-following scores across all three suites.

Method	Qwen 2.5-1.5B	Llama 3.2-3B	Qwen 2.5-7B	Macro-avg
No intervention	0.074	0.092	0.063	0.077
Repetition penalty	0.049	0.071	0.147	0.089
Presence penalty	0.052	0.107	0.098	0.086
Frequency penalty	0.020	0.113	0.053	0.062
No-repeat n-gram	0.053	0.094	0.057	0.068
Placebo control	0.086	0.070	0.099	0.085

Appendix

Table 14: MAUVE versus the WikiText-103 human reference (higher is closer to human text). Per-model and macro-averaged across the three primary models. DRY achieves the highest macro-average MAUVE, exceeds the uncontrolled baseline on all three models, and exceeds every penalty baseline in the macro average. Absolute values are low because of the genre mismatch between the prompt suite (creative continuation, dialogue, structured formatting) and the encyclopedic reference. The table should therefore be read as a ranking on a fixed reference distribution, not as an absolute human-likeness score.

Figure 12: Composability: SER@4 for non-DRY configurations (gray) versus DRY stacking configurations (blue). DRY improves every configuration it is added to. Macro-averaged over Qwen 2.5-1.5B and Llama 3.2-3B.

Configuration	SER@4 $\downarrow$	D-4 $\uparrow$	LFL $\uparrow$
No intervention (regime A)	0.106	0.959	100.2
No interv. + temp 0.7 + top- $p$ 0.9	0.136	0.949	67.5
DRY alone	0.059	0.975	97.1
DRY + temp 0.7	0.074	0.969	78.3
DRY + top- $p$ 0.9	0.073	0.969	85.1
DRY + temp 0.7 + top- $p$ 0.9	0.079	0.964	73.5

Appendix

Table 15: Composability: DRY stacked with common decoding configurations (macro-averaged over Qwen 2.5-1.5B and Llama 3.2-3B, regime A). DRY improves every configuration it is added to without degrading diversity.

Repetition in Language Generation

See all 23 papers

When Do Repetition Loops Lock? Truncation Sampling, Bounded Penalties, and a Blind Spot in DRY
Oct 17, 2026Yichuan Chen, Xin WangRepetition in Language Generation
School of Art and Design, Zhejiang Sci-Tech University, Hangzhou, China
Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generation
Aug 20, 2026Eric Bigelow, Amir Zur, Satchel Grant +7LLM Uncertainty Estimation Uncertainty Quantification
Dynamic Context Evolution for Scalable Synthetic Data Generation
Apr 8, 2026Ryan Lingo, Rajeev ChhajerRepetition in Language Generation Output Diversity in Language Models
Honda Research Institute, USA, Inc.

Oct 17, 2026J/K move · Enter open · S save

When Do Repetition Loops Lock? Truncation Sampling, Bounded Penalties, and a Blind Spot in DRY

Yichuan Chen, Xin Wang

School of Art and Design, Zhejiang Sci-Tech University, Hangzhou, China

Repetition in Language Generation

Open paper AskPDF arXiv

Language models sometimes repeat verbatim until the generation limit. We study the exit rather than the entrance: given that a span has already been repeated

n

times, when is escape still possible? Along the loop the context is deterministic, so loop survival under any sampler can be computed exactly from one forward pass. Pure sampling leaves a positive escape probability at every step; top-

p

with threshold

π>1/2

locks a position exactly when the continuation probability reaches

π

. A bounded penalty can leave a position locked; a penalty that outgrows the model's preference for the loop over unseen tokens guarantees escape under top-

p

. The same analysis predicts a blind spot in the widely deployed DRY sampler: its sequence breakers cap the penalty at

λb^{g-1-a}

for loops whose longest breaker-free run has

g

tokens. On eleven open models, 66% of locked units with

g=6

stay locked under DRY and 1% with

g=8

. In free generation with llama.cpp, all 109 loops that survive DRY have

g\le 7

. We propose DRY-C, which adds a term that grows exponentially with the number of verbatim recurrences, up to its look-back window. With it, none of 720 structured-output generations looped to the length limit, and task completion stayed close to DRY's.

Abstract

Repetition in Language Generation

Explore similar work

When Do Repetition Loops Lock? Truncation Sampling, Bounded Penalties, and a Blind Spot in DRY

Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generation

Dynamic Context Evolution for Scalable Synthetic Data Generation