LoopGuard: Breaking Self-Reinforcing Attention Loops via Dynamic KV Cache Intervention

Dongjie Xu; Hao Wu; Weijie Shi; Yue Cui; Yuanjun Liu; Jiawei Li; Haolun Ma; An Liu

cs.AIApr 11, 2026

LoopGuard: Breaking Self-Reinforcing Attention Loops via Dynamic KV Cache Intervention

Authors: Dongjie Xu, Hao Wu, Weijie Shi, Yue Cui, Yuanjun Liu, Jiawei Li, Haolun Ma, An Liu,

Organizations: Soochow University · Tongji University · The Hong Kong University of Science and Technology · Alibaba Group ·

Abstract

Through systematic experiments on long-context generation, we observe a damaging failure mode in which decoding can collapse into persistent repetition loops. We find that this degeneration is driven by collapsed attention patterns, where a subset of heads locks onto a narrow suffix of the history, and is further stabilized by inference-time KV cache reuse. Crucially, since many existing KV cache policies rely on attention-based importance, this collapse can produce spuriously high scores for repetitive tokens, causing cache management to inadvertently amplify repetition. To study this phenomenon in a controlled and reproducible manner, we introduce LoopBench, a benchmark with explicit loop-inducing conditions and loop-oriented metrics that quantify repetition severity and generation instability beyond downstream task scores. Building on these insights, we propose LoopGuard, a lightweight, plug-in KV cache guard that detects loop onset online and disrupts the feedback cycle by pruning repetitive tail spans under a fixed cache budget. Experiments on LoopBench show that LoopGuard reduces loop incidence by over 90 percentage points, while restoring output diversity and reducing token waste.

Figures & tables

Figure 1: Illustration of self-reinforcing repetition loops and our mitigation strategy. (a) Attention heads lock onto narrow repetitive spans, creating a positive feedback loop. (b) Existing attention-based eviction policies (e.g., H2O) inadvertently strengthen this loop by preserving high-score repetitive tokens while discarding diverse context. (c) LoopGuard detects loop onset and prunes the repetitive tail to break the feedback cycle and restore diversity.

Figure 2: Diversity Collapse and Scale-dependent Survival under Repetition Loops. (a) Diversity metric over decoding steps with a clear collapse point. (b) Survival rate (non-collapsed runs) across model scales under loop-inducing prompts.

Figure 3: Overview of LoopGuard. (a) Repetition loops driven by attention locking on a narrow tail and stabilized by KV-cache feedback. (b) Online detection via debounced statistical and tail-repetition signals. (c) Adaptive KV cache pruning to break the loop and restore diversity.

Dataset	Model	Method	CR $\uparrow$	TTR $\uparrow$	Avg Len $\downarrow$	Loop Rate $\downarrow$
LoopBench-DC	Qwen3-1.7B	Full Cache	0.064	0.086	2495	100%
		StreamingLLM	0.085	0.118	2496	100%
		H 2 O	0.052	0.079	2495	100%
		LoopGuard (Ours)	0.219	0.528	1326	1.3%
	Llama3.2-1B	Full Cache	0.044	0.065	2498	100%
		StreamingLLM	0.058	0.087	2498	100%

Table 1: Main results on LoopBench under a fixed KV budget of 1024 tokens and a shared maximum generation budget of

T_{\max}=2500

. We report loop rate and loop-oriented metrics (CR/TTR) together with average generation length (Avg Len) as an auxiliary token-cost measure. Best results are in bold .

Method	F1 $\uparrow$
Full Cache	50.49
StreamingLLM	21.05
H 2 O	33.73
LoopGuard (Ours)	53.61

Table 2: Performance on 2WikiMultiHopQA (LongBench) measured by F1 under the same KV budget setting as LoopBench.

Figure 4: LoopGuard intervention behavior under loop-inducing prompts. (a) Distribution of the number of interventions per sequence. (b) Distribution of intervention trigger positions.

Variant	Loop Rate (%) $\downarrow$	Avg. #Int. $\downarrow$	Early (%) $\downarrow$
No Guard	100.0	0.00	–
Always-on Pruning	79.8	12.5	57.6
Single-signal ( $p_{1}$ )	35.4	7.6	21.2
Full LoopGuard	1.3	4.8	1.7

Table 3: Ablation study of LoopGuard variants on LoopBench. We report loop rate, average number of interventions per sequence (Avg. #Int.), and the fraction of early triggers. Lower is better for all metrics.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Barcode-like attention patterns under repetition collapse. Visualization of attention maps from selected heads across multiple layers (L16, L20–L22) during persistent repetition loops. Each panel exhibits narrow, vertically aligned high-attention stripes, indicating head-level locking onto a small repetitive suffix of the history.

Repetition in Language Generation

See all 23 papers

ART: Attention Run-time Termination for Efficient Large Language Model Decoding
Apr 15, 2026Chen Qiu, Guozhong Li, Cristian McGee +2Long-Context Language Model Inference LLM Inference Acceleration
King Abdullah University of Science and Technology · University of Central Florida
Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time
Aug 24, 2026Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder +2Repetition in Language Generation
Thoughtworks · New York University
Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models
Jul 17, 2026Wenjie Fan, Bin Ma, Dong LiLanguage Model Decoding Autoregressive Language Modeling
Yotta Labs

Apr 15, 2026J/K move · Enter open · S save

ART: Attention Run-time Termination for Efficient Large Language Model Decoding

Chen Qiu, Guozhong Li, Cristian McGee, Aritra Dutta+1

King Abdullah University of Science and Technology · University of Central Florida

Long-Context Language Model Inference LLM Inference Acceleration KV-Cache Management Efficient Attention

Open paper AskPDF arXiv

Long-context decoding in Large Language Models (LLMs) is constrained by the cost of accessing and processing the Key-Value (KV) cache. Despite evidence that attention outputs depend jointly on keys and values, most existing KV management methods rely on key-only pruning, since incorporating values incurs prohibitive overhead. In this paper, we propose Attention Run-time Termination (ART), a lightweight run-time mechanism that tracks accumulated attention outputs during kernel execution and terminates subsequent KV block accesses once further contributions become negligible. Rather than replacing KV selection, ART dynamically terminates redundant KV traversal on top of existing dense or sparse attention policies. We introduce a stability-based criterion that monitors both magnitude and directional changes of intermediate attention outputs and provideds a theoretical characterization of the resulting truncation error. Experiments on the LongBench and RULER Needle-in-a-Haystack tasks show that ART increases the generation throughput of existing KV-cache methods by up to 20%, without compromising the result quality.

Abstract

Repetition in Language Generation

Explore similar work

ART: Attention Run-time Termination for Efficient Large Language Model Decoding

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Bayesian Repetition Penalty: A Principled Adjacent-Conditional Framework for Reversing Attention Collapse in Autoregressive Language Models