UpLoop: Upcycling Large Language Models with Gated Loop

Tieyuan Chen; Huabin Liu; Yingtao Zhang; Shijie Li; Tengyu Pan; Weiyao Lin; Jianguo Li

cs.LGOct 18, 2026

UpLoop: Upcycling Large Language Models with Gated Loop

Authors: Tieyuan Chen, Huabin Liu, Yingtao Zhang, Shijie Li, Tengyu Pan, Weiyao Lin, Jianguo Li

Organizations: Inclusion AI, Ant Group · Shanghai Jiao Tong University · Tsinghua University

Abstract

Looped large language models trade parameter storage for recurrent computation, but most are trained from scratch. In practice, pretraining with looped depth is expensive, making post-training upcycling of a pretrained non-looped LLM an attractive alternative. We introduce UpLoop, a post-training recipe based on layer-wise recurrence. Instead of looping a whole model or a middle block, UpLoop loops each selected layer locally, preserving the pretrained depth hierarchy and avoiding feature-space mismatch. To make loop execution useful, we add hidden-wise gates on the attention and MLP residual branches, so later loops act as progressive refinements rather than redundant rewrites. Empirically, UpLoop breaks the commonly observed saturation at two loops ( $R=2$ ) in prior looped models. Starting from Qwen3-4B Base and applying SFT, UpLoop improves the average accuracy across 20 benchmarks by 6.17% at $R=3$ . We use $R=3$ as the main setting for the best accuracy--cost trade-off, while both native and extrapolated $R=4$ yield further gains.

Figures & tables

		Comparison under $R=3$				Loop count under UpLoop
Category	Dataset	Non-looped	Loop w/o gate	UpLoop	$\Delta$ (UpLoop - Non-loop)	$R=2$	$R=4$
Math	Math500	81.40	82.20	85.60	+4.20	84.00	84.00
	GSM-Plus	81.28	83.61	84.97	+3.69	83.63	85.03
	OlympiadBench	53.19	56.46	60.81	+7.62	58.22	59.00
	AIME 2025	26.67	31.72	36.61	+9.94	33.18	39.53
	AIME 2026	26.67	26.98	36.09	+9.42	32.45	39.27

Table 1: Main results (%). The left group compares different strategies under the main

R=3

setting, while the right group compares different loop counts for UpLoop. Layer-wise loop without gates serves as a reasonable baseline for post-training loop design, but it does not yield a clear and consistent gain. This motivates gated UpLoop, which delivers markedly stronger and consistent improvements. Although performance continues to improve at

R=4

, the gain beyond

R=3

is relatively small, making

R=3

the best accuracy–cost trade-off.

Table 2: Evidence for progressive refinement across loops. Left: later loops become more confident, with lower entropy and higher top-1 probability. Right: later loops still revise a non-trivial fraction of decisions.

Metric	With gate	Without gate
Hidden $\Delta$ -RMS $r=1\rightarrow r=2$	0.47	1.19
Hidden $\Delta$ -RMS $r=2\rightarrow r=3$	0.70	0.92
$r=2\rightarrow r=3$ update cosine	0.43	0.34
$r=2\rightarrow r=3$ attention KL	0.005	0.001
$r=2\rightarrow r=3$ output KL	0.31	0.23

Table 3: Round-wise update statistics with and without gates. Following LoopCoder-v2 ( Yang et al., 2026 ) , hidden

\Delta

-RMS measures the magnitude of the recurrent hidden-state update, update cosine measures directional agreement of adjacent recurrent updates, attention KL measures the change in attention distributions, and output KL measures the change in token-level predictive distributions.

Figure 1: Late-layer zoom (layers 24–36) comparing the non-looped baseline, gated UpLoop, and ungated UpLoop (loop=3). Left: last-token hidden update RMS. Middle: cosine similarity between adjacent hidden states. Right: cosine similarity to the final decoder output. Gating suppresses late-layer write amplification, preserves better hidden-state continuity, and still allows gradual progress toward the final representation.

Figure 2: Gate-modulated residual write magnitude across looped layers. $r=1$ is the base write, while $r=2$ and $r=3$ are gated refinement writes. Later-round writes remain more modest than the base pass, showing that recurrent refinement operates through controlled residual corrections.

Figure 3: Learned static gate values for later loops in the $R=3$ model. The $r=2$ attention gates are often larger, while the $r=3$ MLP gates are often more prominent, suggesting that the second loop contributes relatively more to contextual re-access and the third loop relatively more to feature consolidation.

Figure 4: Hidden-dimension clusters of later-round gate profiles. Each hidden dimension is represented by its four-value profile

[\mathrm{r=2\ Attn},\mathrm{r=2\ MLP},\mathrm{r=3\ Attn},\mathrm{r=3\ MLP}]

, standardized, and clustered with K-means (

k=6

). Different clusters exhibit distinct round- and branch-dependent patterns, indicating that recurrent refinement is distributed non-uniformly across hidden dimensions.

Category	Dataset	loop ( $R=3$ )	KV-share $(3\leftarrow 2)$	KV-share $(2,3\leftarrow 1)$	$\Delta$ share $(3\leftarrow 2)$
Math	Math500	85.60	84.40	83.00	-1.20
	GSM-Plus	84.97	84.74	83.72	-0.23
	OlympiadBench	60.81	60.41	58.76	-0.40
	AIME 2025	36.61	36.82	33.23	+0.21
	AIME 2026	36.09	34.84	32.86	-1.25
Code	HumanEval+	68.29	67.07	65.24	-1.22

Table 4: KV-reuse study on the benchmark suite. We compare the standard

R=3

UpLoop model against two approximations: reusing the

r=2

KV cache at

r=3

, and reusing the

r=1

KV cache at both

r=2

and

r=3

. The last column reports the difference between standard

R=3

and the

r=3\leftarrow r=2

KV-reuse variant.

Loop pair	Q	K	V	PV	Attn out	Top-1	Top-8	Mass	$X$
$1\rightarrow 2$	0.9566	0.9656	0.8799	0.8507	0.7637	0.8597	0.8126	0.8404	0.9373
$2\rightarrow 3$	0.9882	0.9907	0.9758	0.9616	0.8948	0.9465	0.9059	0.9308	0.9853

Table 5: Similarity and overlap statistics between adjacent loops. “PV” denotes projected value similarity, “Mass” denotes attention-mass overlap, and

X

denotes hidden-state cosine similarity.

Category	Dataset	( $R=3$ )	Extrapolated ( $R=4$ )	Native ( $R=4$ )	$\Delta$ (Extrapolated - Native)
Math	Math500	85.60	84.60	84.00	+0.60
	GSM-Plus	84.97	84.85	85.03	-0.18
	OlympiadBench	60.81	59.52	59.00	+0.52
	AIME 2025	36.61	34.66	39.53	-4.87
	AIME 2026	36.09	33.99	39.27	-5.28
Code	HumanEval+	68.29	71.95	73.78	-1.83

Table 6: Inference-time loop extrapolation from training depth

R=3

to test-time depth

R=4

. The

r=4

is not trained; it reuses the round-3 gates for both attention and MLP residual writes. We additionally compare this extrapolated model with a natively trained

R=4

model.

Dataset	Non-looped	UpLoop ( $R=3$ )	$\Delta$
Math500	74.80	78.40	+3.60
GSM8K-Plus	70.77	76.57	+5.80
OlympiadBench	49.63	53.78	+4.15
HumanEval+	77.59	80.49	+2.90
MBPP+	66.68	68.25	+1.57
LiveCodeBench v6	23.51	25.66	+2.15

Table 7: Generalization of UpLoop to an MoE backbone on 17 benchmarks. We evaluate Ling-3.0 Tiny, a hybrid dense–MoE model. UpLoop consistently improves over the non-looped baseline, showing that the proposed recurrent-depth upcycling recipe also generalizes to routed MoE architectures.

Recurrent Language Models

See all 36 papers

Closing the Loop: Practical Training Recipes for Looped Language Models
Sep 30, 2026Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin +5Efficient Language Model Training LLM Training
RND NLP, DAIMLD, Russian Federation
LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models
May 10, 2026Taekhyun Park, Yongjae Lee, Dohee Kim +1Recurrent Transformers LLM Post-Training
Department of Data Science Pusan National University Busan, Republic of Korea · Department of Industrial Engineering Pusan National University Busan, Republic of Korea · Department of Artificial Intelligence Engineering Changwon National University Changwon, Republic of Korea
Allocating Recurrent Compute in Looped Language Models
Aug 18, 2026Ruhai Lin, Yiyang Guo, Rui-Jie Zhu +2Recurrent Language Models Recurrent Transformers
University of California, Santa Cruz

Sep 30, 2026J/K move · Enter open · S save

Closing the Loop: Practical Training Recipes for Looped Language Models

Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova+4

RND NLP, DAIMLD, Russian Federation

Efficient Language Model Training LLM Training Recursive Language Models Recurrent Language Models

Open paper AskPDF arXiv

Looped language models increase effective depth by repeatedly applying a shared block of layers, but existing large-scale recipes require multi-stage training over trillions of tokens, while the benefits of recurrence remain difficult to separate from differences in data and training. In this work, we establish practical training recipes for looped language models, with three main results. (1) We develop a compute-efficient from-scratch pipeline that reduces the training budget from 7.7T tokens in Ouro to 310B tokens while retaining strong reasoning performance. Pretraining followed by high-quality mid-training, together with learning-rate warmup and stronger exit-gate regularization, enables stable recurrent training without prior multi-stage schedules. (2) Under controlled comparisons, our 1.4B LoopLM outperforms a parameter-matched dense model trained on the same data and token budget on all 12 evaluated benchmarks, including +14 points on GSM8K, +10 on MATH, and +22 on DROP. At matched inference compute, it approaches a 3.9B dense model on mathematical reasoning and reading comprehension while using only 36% as many parameters. (3) We introduce a minimal recipe for converting pretrained dense models into looped ones: a single learned input-mixing scalar and a smoothed exit loss, with no step-specific parameters. Applied to Qwen3-1.7B-Base, Looped Qwen improves over an identically continued dense baseline on every evaluated benchmark across two data regimes, with statistically clear gains on GSM8K, MATH, and MMLU-Pro on the curated mixture. Together, these results make looped language models substantially cheaper to train from scratch and practical to introduce into existing pretrained checkpoints, while isolating the gains due to recurrence itself.

Abstract

Recurrent Language Models

Explore similar work

Closing the Loop: Practical Training Recipes for Looped Language Models

LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models

Allocating Recurrent Compute in Looped Language Models