Uncovering Uncontrolled Repetition through Residual Stream Dynamics

Yuanhe Zhang; Xinyao Zhou; Haoran Gao; Yuyao Zhang; Zhenhong Zhou; Fanyu Meng; Li Sun; Sen Su

cs.CLSep 30, 2026

Uncovering Uncontrolled Repetition through Residual Stream Dynamics

Authors: Yuanhe Zhang, Xinyao Zhou, Haoran Gao, Yuyao Zhang, Zhenhong Zhou, Fanyu Meng, Li Sun, Sen Su

Organizations: Beijing University of Posts and Telecommunications · JIUTIAN Research · Nanyang Technological University · Chongqing University of Posts and Telecommunications

Abstract

Uncontrolled repetition can prolong autoregressive generation in large language models (LLMs) and enable resource consumption attacks. Prior analyses of repetitive generation have identified strongly activated features in intermediate and late layers. However, how uncontrolled repetition activity emerges and develops before becoming prominent in these layers remains insufficiently understood. In this paper, we investigate this question primarily in large vision-language models (LVLMs), which support a richer set of uncontrolled repetitions through both visual and textual inputs. We propose Tokenwise Residual Comparison (TRC), a method that identifies and localizes anomalies associated with repetition from residual dynamics during generation. TRC compares attention and multilayer perceptron writes to the residual stream across generated tokens to identify patterns associated with repetition. It then selectively suppresses coordinates in the residual stream at the identified layer. Experiments show that TRC effectively mitigates uncontrolled repetition, reducing loop rates by 57% on average. Our analysis further shows that repetition semantics emerge in shallow layers and propagate through the residual stream, disrupting normal representations. TRC also generalizes to large language models (LLMs) and large reasoning models (LRMs), where it consistently captures analogous repetition dynamics and achieves effective mitigation. Our work broadens the study of repetitive generation from its prominent internal representations to earlier opportunities for intervention, providing insights for mitigating resource consumption attacks.

Figures & tables

Figure 1: Overview of TRC , showing uncontrolled repetition across model families ( Left ), tokenwise residual comparison and selective intervention ( Middle ), and mitigation outcomes( Right ).

		RECITE (visual)		GCG (textual)		LoopLLM (textual)
Model	Defense	Length	Loop rate	Length	Loop rate	Length	Loop rate
InstructBLIP-7B	No defense	1,866.44	92.0%	1,531.56	96.0%	1,833.96	100.0%
	Fixed-length	389.84	92.0%	1,017.76	96.0%	952.36	88.0%
	No-repeat	18.36	4.0%	17.88	4.0%	22.04	0.0%
	AUSteer	413.16	20.0%	85.40	4.0%	165.84	8.0%
	TRC	422.04	16.0%	1,312.56	56.0%	1,229.72	60.0%

Table 1: Defense results on three LVLMs. Lower is better for both metrics.

Method	ScienceQA	TextVQA	Average
Benign	82.00%	79.00%	80.50%
Fixed-length	82.00%	79.00%	80.50%
No-repeat	82.00%	78.50%	80.25%
AUSteer	76.00%	60.50%	68.25%
TRC	81.50%	78.50%	80.00%

Table 2: Benign task accuracy (%).

Figure 2: Generalization of TRC to LLMs and LRMs under Direct, GCG, and LoopLLM attacks. Bars show generation length and lines show loop rate; lower is better for both.

	LLM		LRM
Setting	Qwen2.5-3B	Llama-3.2-3B	DeepSeek-Llama-8B	Qwen3.6-27B	GLM-4.7-Flash
Benign	61.0%	55.0%	97.0%	99.5%	100.0%
TRC	61.0%	56.0%	95.0%	99.5%	100.0%
Difference (pp)	0.0%	+1.0%	-2.0%	0.0%	0.0%

Table 3: Benign task accuracy on LLMs and LRMs. TRC causes minor changes in performance.

Panel of Figure 3 — Figure 3: Layer wise localization of uncontrolled repetition. (a) Top 1 TRC localization across LVLM, LLM, and LRM families, showing consistently shallow attack locations. (b) Cross layer TRC rankings under uncontrolled repetition, benign repetition, and normal requests, revealing a distinct early layer profile for uncontrolled repetition.

Panel of Figure 4 — Figure 4: Mechanistic evidence for branch and layer selection. (a) Branch-wise suppression on attack and normal examples. Attention-only suppression preserves normal accuracy, whereas MLP-only and joint suppression reduce attack length at substantially higher utility cost. (b) Effect of the suppression start layer on Recite repetition suppression and ScienceQA accuracy. The star denotes the original model without suppression.

Block	Metric	Normal	Repeat	Gap/ $\tau_{l}$
0	L2	22.2009	22.8838	0.234
0	RMS	0.4946	0.5057	0.172
1	L2	12.9280	13.7456	0.730
1	RMS	0.2867	0.3038	0.692

Table 4: Attention-update magnitudes in the first two blocks. Gap/

\tau_{l}<1

indicates that the condition difference remains within the normal token-level fluctuation bound.

Input	Coordinates	Residual intervention	Repeat rate (%)	Transition success (%)	Avg. length	$\Delta$ Len (%)	Observed change
Attack	–	None	100.0	–	4096	–	Repetition baseline
Attack	TRC (ours)	Normal residual	20.0	80.0	828	$-79.8$	Strong restoration
Attack	Random	Normal residual	60.0	40.0	2462	$-39.9$	Partial restoration
Normal	–	None	0.0	–	11	–	Normal baseline
Normal	TRC (ours)	Attack addition	0.0	0.0	2	–	Output failure without repetition
Normal	Random	Attack addition	0.0	0.0	11	–	Nearly unchanged

Table 5: Cross intervention results with unit strength on GCG examples. TRC denotes coordinates selected by TRC, while Random uses an equal size coordinate set. Transition success measures repetition to nonrepetition for attack inputs and nonrepetition to repetition for normal inputs.

\Delta

Len is computed relative to the corresponding no intervention baseline.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Model	RECITE	GCG	LoopLLM	Direct	Benign tasks
LVLMs
InstructBLIP-Vicuna-7B	Y	Y	Y	–	ScienceQA, TextVQA
Qwen2.5-VL-3B-Instruct	Y	Y	Y	–	ScienceQA, TextVQA
LLaVA-1.5-7B	Y	Y	–	–	ScienceQA, TextVQA
LLMs
Llama-3.2-3B	–	Y	Y	Y	MMLU

Appendix

Table 6: Model-specific attack coverage and benign evaluation tasks. Y marks an included attack condition; – denotes a condition outside the evaluated scope.

Approach	Primary observation	Localization target	Demonstrated scope	Principal distinction from TRC
FFN vocabulary analysis ( Geva et al., 2021 ; Geva et al., 2022 )	Activations and vocabulary-space promotion produced by feed-forward updates	FFN memories, values, and layer-wise concept promotion	Text LMs	Explains stored or promoted concepts, but does not compare residual changes along a repetitive generation trajectory.
Layer-wise lexical probing ( Liu et al., 2024b )	Probe recoverability of lexical-semantic information from hidden representations	Layers at which lexical semantics are most accessible	Generative text LMs	Measures information accessibility through an external readout rather than locating native residual coordinates for intervention.
Repetition neurons ( Hiraoka and Inui, 2025 )	Neuron-activation changes before and after the onset of repetition	Repetition-associated neurons in intermediate and final layers	Text LMs	Directly studies repetition, but uses activation amplitude around onset rather than cycle-aligned changes before residual addition.
SAE repetition features ( Yao et al., 2025 )	Logit-based layer screening followed by learned sparse-feature activations	SAE features within selected layers	Text LLMs	Resolves features with an auxiliary learned dictionary; TRC operates on native attention and MLP contribution coordinates.
Cross-modal information flow ( Zhang et al., 2025c )	Answer-performance changes after blocking attention between image and question positions	Layered cross-token attention pathways for visual–linguistic integration	LLaVA-style MLLMs	Explains modality fusion within MLLMs, but is not a repetition-localization method and is not evaluated on text-only LLMs or LRMs.
TRC (ours)	Cycle-aligned changes in native attention and MLP contributions, normalized by benign magnitude and local cross-layer variation	A shallow pre-addition residual-write layer and its highest-scoring hidden coordinates	LVLMs, LLMs, and LRMs	Uses one modality-agnostic scoring and masking formulation without adding an auxiliary interpreter or network module.

Appendix

Table 7: Comparison with representative interpretability approaches. “Scope” records the model families demonstrated in the cited work, rather than a theoretical restriction. TRC differs primarily in observing changes between cycle-aligned generation positions, localizing the corresponding pre-addition residual writes, and applying the same formulation across LVLMs, LLMs, and LRMs.

Image	Prompt	Generated output
	Examine the image and return exactly two lines. Line 1 must begin with ’Description:’ and contain one concise factual English sentence describing the main visible object. Do not use the word ’echo’ in line 1. Line 2 must be exactly: ’Repeat: echo echo echo echo echo echo echo echo echo echo.’ Do not add any other text before, between, or after these two lines.	Description: The image depicts a simple, stylized red apple with a green leafy top. Repeat: echo echo echo echo echo echo echo echo echo echo
	Examine the image and return exactly two lines. Line 1 must begin with ’Description:’ and contain one concise factual English sentence describing the visible objects. Do not use the word ’signal’ in line 1. Line 2 must be exactly: ’Repeat: signal signal signal signal signal signal signal signal signal signal.’ Do not add any other text before, between, or after these two lines.	Description: Three mugs of different colors are lined up on a table. Repeat: signal signal signal signal signal signal signal signal signal signal.

Appendix

Table 8: Two representative legitimate-repetition controls for Qwen2.5-VL-3B-Instruct. Each prompt requires an image-grounded description followed by exactly ten copies of a designated token. Output text is reproduced verbatim; line breaks are normalized for table layout.

Figure 5: Layer-wise TRC-l profiles for Qwen2.5-VL-3B Attention (top) and MLP (bottom), with window spans

h=2,3,4

. Curves show Direct, GCG, and RECITE failures and the mean of normal-reference splits; gray shading denotes the normal range.

Attack	$h=2$	$h=3$	$h=4$
GCG	$1/0$	$1/0$	$1/0$
RECITE	$1/1$	$1/1$	$1/1$
LoopLLM	$1/1$	$1/1$	$1/1$

Appendix

Table 9: Sensitivity of the localized Top-1 start layer to the layer-window span

h

. Each entry is Attention / MLP, and layers are zero-indexed.

$\rho$ (%)	GCG	RECITE	LoopLLM	Macro avg.	Coordinate overlap
No defense	$4096.00/100.0$	$4096.00/100.0$	$1395.32/32.0$	$3195.77/77.3$	–
1	$1809.56/44.0$	$54.80/0.0$	$444.64/20.0$	$769.67/21.3$	$8/103$
5	$2469.72/60.0$	$62.44/0.0$	$1044.28/28.0$	$1192.15/29.3$	$67/103$
10	$2631.92/64.0$	$222.60/4.0$	$1044.24/28.0$	$1299.59/32.0$	$70/103$
15	$2462.60/60.0$	$214.48/4.0$	$805.80/24.0$	$1160.96/29.3$	$99/103$
20	$1159.36/28.0$	$58.72/0.0$	$643.60/36.0$	$620.56/21.3$	$103/103$

Appendix

Table 10: Sensitivity to the suppression ratio on attacks. Cells under each attack and the macro average report mean newly generated tokens / repetition rate (%). Coordinate overlap is measured against the mask selected at

\rho=5\%

. Lower is better for both reported attack metrics.

$\rho$ (%)	Accuracy	Avg. length
0	82.0	2.125
1	82.0	2.130
5	81.5	2.145
10	82.0	2.115
15	81.0	2.125
20	80.5	2.065

Appendix

Table 11: Effect of the suppression ratio

\rho

on benign task. Accuracy is reported in percent.

Attack	$q_{\max}=1$	$q_{\max}=2$	$q_{\max}=3$	$q_{\max}=4$	$q_{\max}=5$
GCG	12	1	1	1	1
RECITE	1	1	1	1	1

Appendix

Table 12: Top-1 start layer under different maximum repetition distances

q_{\max}

. Layers are zero-indexed.

Model	Attack	Original (s)	TRC (s)	Reduction
Qwen2.5-3B	Direct	278.92	9.33	96.7%
Qwen2.5-3B	GCG	402.52	1.73	99.6%
Qwen2.5-3B	LoopLLM	91.72	8.36	90.9%
Llama-3.2-3B	Direct	340.19	5.38	98.4%
Llama-3.2-3B	GCG	64.73	1.67	97.4%
Llama-3.2-3B	LoopLLM	143.44	4.68	96.7%

Appendix

Table 13: Attack-generation time on two LLMs. Reduction is computed as

(T_{\mathrm{orig}}-T_{\mathrm{TRC}})/T_{\mathrm{orig}}

. Lower is better.

Model	Original	TRC	Difference
Qwen2.5-3B	22.60	19.60	$-3.00$
Llama-3.2-3B	25.76	26.80	$+1.05$

Appendix

Table 14: Mean throughput on two LLMs. Difference is TRC minus Original. Throughput is measured in tokens per second.

Explore similar work

Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs
May 10, 2026Sohan VenkateshLarge Language Models Fail Counting
Manipal Institute of Technology Bengaluru
When Reasoning Goes Astray: Attention Dynamics of Uncontrolled Reasoning
Sep 30, 2026Yuanhe Zhang, Ziwei Wang, Jie Ren +6Large Reasoning Models Reasoning Errors
Beijing University of Posts and Telecommunications · Wuhan University · Nanyang Technological University +1
RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks
Jun 6, 2026Abid Aziz, Hafsa Binte KibriaLLM Reasoning Strategies Recursive Models
Department of Electrical & Computer Engineering Rajshahi University of Engineering & Technology, Rajshahi 6204, Bangladesh

May 10, 2026, cs.CLJ/K move · Enter open · S save

Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

Sohan Venkatesh

Manipal Institute of Technology Bengaluru

Large Language Models Fail Counting Multilayer Perceptrons Residual Stream Dissociation Reasoning Benchmark

Open paper AskPDF arXiv

Large language models fail at counting repeated tokens despite strong performance on broader reasoning benchmarks. These failures are commonly attributed to limitations in internal count tracking. We show this attribution is wrong. Linear probes on the residual stream decode the correct count with near-perfect accuracy at every post-embedding layer, across all model depths. This holds even at the exact layers where the wrong answer crystallizes while the model simultaneously outputs an incorrect count. Attention patterns show no evidence of collapse over repeated tokens and tokenization artifacts account for none of the failure. Instead, a format-triggered multi-layer perceptron (MLP) block overwrites the correctly-encoded count with a fixed wrong answer at roughly 88--93,% network depth. This prior fires for repeated word-tokens in space-separated list format and is absent for repeated digit-tokens. It is suppressed by comma-separated delimiters in larger models but persists in smaller ones. The finding holds across Llama-3.2 (1B and 3B) and Qwen2.5 (1.5B, 3B and 7B) at consistent relative depth. Counting failure is a failure of routing not of representation and the two require different interventions.

Abstract

Explore similar work

Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

When Reasoning Goes Astray: Attention Dynamics of Uncontrolled Reasoning

RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks