Agreement rather than normalisation keeps depth reusable in looped language models

Ivan Viakhirev; Kirill Borodin; Maxim Abramov; Grach Mkrtchian

cs.LGOct 22, 2026

Agreement rather than normalisation keeps depth reusable in looped language models

Authors: Ivan Viakhirev, Kirill Borodin, Maxim Abramov, Grach Mkrtchian

Organizations: lab260, Yerevan, Armenia · BitmanagerAI, Dubai, UAE · St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

Abstract

Looped language models apply one block of layers repeatedly, and the design pays off only if depth is reusable: loop steps beyond the trained budget must remain usable at inference. We ask whether that rests on normalising the state between loop steps, as the released Ouro models do, or on agreement between the loop a model is trained with and the loop it is run with. On released weights this loop-boundary normalisation can be removed without adding parameters and with the readout path bit-identical, and the operation is an identity at one loop step, which serves as an implementation check. Removing it takes Ouro-1.4B and Ouro-2.6B to zero accuracy by the trained budget on a balanced binary task (chance 0.500). State-norm growth, the failure that prior accounts predict, is not the cause: the norm stays within a factor of 2.021 while the head commits to a single wrong token. Yet in 19M-parameter models trained from scratch on two synthetic tasks, 16 seeds per configuration, training without the normalisation is never worse than training with it and extrapolates better: the two are closest at the trained depth and separate on both sides of it (Cliff's delta up to 0.867 beyond the budget). Both results follow if agreement, and not the normalisation itself, is what keeps depth reusable. The damage sits in the readout together with the last layer of the block: in the models trained from scratch, refitting the head over the frozen modified loop recovers a third of the lost accuracy and plateaus, while refitting the last block layer with it recovers all of it. Finally, comparing looped with stacked depth at an equal inference loop count is confounded by unequal overshoot of the trained budget; at matched overshoot the advantage of looping vanishes on one task and shrinks on the other.

Figures & tables

Figure 1: Rearranging the loop-boundary normalisation on released weights, four in-context examples in the prompt; blue as released, red with the boundary normalisation removed, line style marks the model. Left and middle: accuracy against the inference-time loop count

r

; the shaded band lies beyond the trained budget

r_{0}=4

and the dotted line is chance inside the answer alphabet (on hops the majority floor, 0.072, is higher; the

y

ranges differ). Right: the same states of the arm with the normalisation removed, at the trained budget on circuit , read by the model’s own head and by a linear probe fitted to them, with the floor from a probe fitted to shuffled labels. The probe sits at its floor while the output sits at zero, below anything an answer within the alphabet would give (Sec. 4.2 ).

Figure 2: Mechanism on circuit ; colours and line styles as in Fig. 1 , shaded band beyond the trained budget. Left: readout entropy in bits, dotted line at the one bit of a uniform guess over two answers. Middle: state norm at

r

over state norm at

r=1

. Right: Ouro-2.6B with the boundary normalisation applied every

k

-th loop step and always on the last, at three depths; the response is a threshold. The arms with the normalisation removed lose the readout while their state scale stays within a factor of 2.021.

Figure 3: Models trained from scratch with the loop topology fixed before training, on circuit (left) and one cell of hops (right); means over 16 seeds, bands one standard deviation. Both configurations share

r_{0}=4

, so the comparison at equal

K

is free of the overshoot confound of Sec. 4.5 . The curve with a boundary normalisation peaks at its trained depth and falls away on both sides; the curve without one stays above 0.948 from

K=2

K=16

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

role	model	parameters	loop topology
released	Huginn-3.5B	3.5B	no boundary norm measured
released	Ouro-1.4B	1.4B	boundary norm, shared with readout
released	Ouro-2.6B	2.6B	boundary norm, shared with readout
from scratch	LoopedLM, $d{=}512$	19–76M	either, dedicated, set before training

Appendix

Table 1: Models. The released checkpoints carry the scale and the models trained from scratch carry the causal claims; see Sec. 3.1 for why the two halves answer different questions.

setting	value
width $d$ , heads, MLP multiplier	512, 8, 4
layers $\times$ trained loop steps	$6\times 4$ (normalisation axis and loop arm); $12\times 2$ ; $24\times 1$
boundary normalisation	RMSNorm between loop steps, skipped on the last; on or off
readout normalisation	RMSNorm before the unembedding, always on
context, tokens per batch, training tokens	256, 65 536, $3\times 10^{8}$
optimiser, peak learning rate, precision	AdamW, $3\times 10^{-4}$ , bfloat16

Appendix

Table 2: Training hyperparameters of the models trained from scratch. All configurations share data, tokeniser, schedule and seeds; they differ only in the row that names them.

Figure 4: Top-1 minus top-2 logit against the number of loop steps for both task families. This quantity is noisy across

r

and is interpretable only together with the modal share; Sec. 4.2 cites it only at the trained budget and at

r=32

Figure 5: Three placements of a matched depth of 24 layer applications per token, trained from scratch on circuit . Lines are means over 6 seeds, bands one standard deviation. Left: accuracy against the absolute loop count

K

, which is how such comparisons are usually reported, with each configuration shown from its own trained budget upwards. Right: the same runs against the budget overshoot

K/r_{0}

, restricted to the overshoots all three configurations reach. The ordering on the left is absent on the right; that difference is the point of Sec. 4.5 .

configuration	$K/r_{0}{=}1$	$K/r_{0}{=}2$	$K/r_{0}{=}4$	$K/r_{0}{=}8$
$4\times 6$ (loop)	0.999	0.994	0.971	0.907
$2\times 12$	0.998	0.985	0.958	0.918
$1\times 24$ (stack)	0.995	0.992	0.959	0.828
$\delta$ , loop vs $2\times 12$	+0.22	+0.39	+0.11	-0.11
$p$ , loop vs $2\times 12$	0.589	0.310	0.818	0.818
$\delta$ , loop vs $1\times 24$ (stack)	+0.50	+0.25	+0.25	+0.33

Appendix

Table 6: Answer-token accuracy at matched budget overshoot

K/r_{0}

for three placements of 24 layer applications per token, averaged over 6 training seeds on circuit . Each column compares the three configurations after leaving their trained budgets by the same factor.

configuration	$K/r_{0}{=}1$	$K/r_{0}{=}2$	$K/r_{0}{=}4$	$K/r_{0}{=}8$
$4\times 6$ (loop)	1.000	0.998	0.998	0.957
$2\times 12$	1.000	1.000	0.948	0.733
$1\times 24$ (stack)	1.000	0.997	0.925	0.737
$\delta$ , loop vs $2\times 12$	+0.00	-0.17	+0.58	+0.53
$p$ , loop vs $2\times 12$	1.000	0.699	0.132	0.180
$\delta$ , loop vs $1\times 24$ (stack)	+0.00	+0.03	+0.56	+0.78

Appendix

Table 7: Looped against stacked depth on one cell of hops : answer-token accuracy at matched budget overshoot

K/r_{0}

for the three placements of 24 layer applications per token, averaged over 6 seeds, with Cliff’s

\delta

and the exact Mann–Whitney

p

against the four-step loop. All three are trained without boundary normalisation; the four-step loop is the no-normalisation architecture of Table 5 on a separate set of six seeds, and its higher value at eight times the budget lies within the seed spread of Fig. 3 . Unlike on circuit (Table 6 ), the loop retains an advantage over the stack at eight times the budget (

p=0.026

over 6 seeds, uncorrected), and none over

2\times 12

(

p=0.180

arm	task	$r{=}1$	$r{=}2$	$r{=}3$	$r{=}4$	$r{=}6$	$r{=}8$	$r{=}12$	$r{=}16$	$r{=}24$	$r{=}32$
O26-base	circuit	0.530	0.584	0.627	0.642	0.620	0.598	0.588	0.590	0.591	0.583
O26-base	hops	0.074	0.107	0.164	0.166	0.154	0.142	0.147	0.138	0.122	0.106
O26-fbraw	circuit	0.530	0.500	0.058	0.000	0.000	0.000	0.000	0.000	0.000	0.000
O26-fbraw	hops	0.074	0.071	0.049	0.082	0.038	0.000	0.000	0.000	0.000	0.000
O14-base	circuit	0.471	0.591	0.600	0.619	0.561	0.550	0.537	0.530	0.527	0.524
O14-base	hops	0.074	0.070	0.121	0.154	0.140	0.152	0.149	0.149	0.128	0.117

Appendix

Table 9: Accuracy against the number of loop steps for the released arms with four in-context examples, averaged over strata of

d^{\star}

and

w

with

n=100

instances per stratum. Arms named -fbraw have the boundary normalisation removed. The trained budget is

r_{0}=4

circuit	$K{=}1$	$K{=}2$	$K{=}4$	$K{=}8^{*}$	$K{=}16^{*}$	$K{=}32^{*}$
as trained	0.578	0.742	0.998	0.864	0.621	0.601
loop-boundary norm removed	0.578	0.626	0.639	0.701	0.708	0.710
+ readout refit	0.671	0.718	0.756	0.765	0.728	0.729
recovered	—	78%	33%	39%	—	—
$p$ vs removed	0.031	0.062	0.031	0.031	0.625	0.625
$p$ vs as trained	0.031	0.312	0.031	0.094	0.562	0.562

Appendix

Table 10: Three arms on the same loop weights of the models trained from scratch with boundary normalisation: as trained, after the normalisation is removed, and after refitting the readout against the modified loop. The refit trains only the final normalisation and the unembedding, untied from the input embedding, for 300 steps with the loop frozen. Entries are medians over the first 6 training seeds (11–66), a subset of the seeds of Table 4 , so the “as trained” row is a six-seed median and differs from the sixteen-seed means there. “Recovered” is the share of the accuracy lost to the operation that the refit returns, reported where the operation costs accuracy at all. Both

p

values are the exact signed-rank test over seeds, paired by seed: against the arm with the normalisation removed, and against the arm as trained. An asterisk marks depths beyond the trained budget

r_{0}=4

circuit, refit steps	0	30	100	300	1000	3000
unembedding, accuracy	0.639	0.729	0.744	0.751	0.755	0.761
unembedding, recovered	0%	25%	29%	32%	33%	34%
+ final norm, accuracy	0.639	0.726	0.742	0.756	0.756	0.765
+ final norm, recovered	0%	25%	29%	33%	33%	35%
+ last loop block, accuracy	0.639	0.791	0.869	0.940	0.982	0.996
+ last loop block, recovered	0%	43%	65%	85%	96%	100%

Appendix

Table 12: Recovery of the accuracy lost to the operation, by number of refit steps and by what is unfrozen: the unembedding alone; plus the final normalisation; plus the last loop block, i.e. the last layer of the recurrent block, shared across loop steps (4.2M parameters). Loop weights below it stay frozen throughout. Medians over the first 6 seeds at the trained depth; “recovered” is the share of the loss returned, computed from unrounded medians. Step 0 is the untouched head and coincides with the arm after the operation by construction, checked per seed. The readout alone plateaus at a third on both tasks; with the last block the loss is recovered in full.

Recurrent Language Models

See all 36 papers

Closing the Loop: Practical Training Recipes for Looped Language Models
Sep 30, 2026Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin +5Efficient Language Model Training LLM Training
RND NLP, DAIMLD, Russian Federation
Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models
Oct 10, 2026Seyedarmin Azizi, Erfan Baghaei Potraghloo, Arya Fayyazi +2Recurrent Language Models Refusal Behavior in Language Models
University of Southern California (USC), USA
Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching
Aug 10, 2026Kristian Schwethelm, Daniel Rueckert, Georgios KaissisLLM Inference LLM Inference Scheduling
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM), Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1

Sep 30, 2026J/K move · Enter open · S save

Closing the Loop: Practical Training Recipes for Looped Language Models

Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, Inessa Fedorova+4

RND NLP, DAIMLD, Russian Federation

Efficient Language Model Training LLM Training Recursive Language Models Recurrent Language Models

Open paper AskPDF arXiv

Looped language models increase effective depth by repeatedly applying a shared block of layers, but existing large-scale recipes require multi-stage training over trillions of tokens, while the benefits of recurrence remain difficult to separate from differences in data and training. In this work, we establish practical training recipes for looped language models, with three main results. (1) We develop a compute-efficient from-scratch pipeline that reduces the training budget from 7.7T tokens in Ouro to 310B tokens while retaining strong reasoning performance. Pretraining followed by high-quality mid-training, together with learning-rate warmup and stronger exit-gate regularization, enables stable recurrent training without prior multi-stage schedules. (2) Under controlled comparisons, our 1.4B LoopLM outperforms a parameter-matched dense model trained on the same data and token budget on all 12 evaluated benchmarks, including +14 points on GSM8K, +10 on MATH, and +22 on DROP. At matched inference compute, it approaches a 3.9B dense model on mathematical reasoning and reading comprehension while using only 36% as many parameters. (3) We introduce a minimal recipe for converting pretrained dense models into looped ones: a single learned input-mixing scalar and a smoothed exit loss, with no step-specific parameters. Applied to Qwen3-1.7B-Base, Looped Qwen improves over an identically continued dense baseline on every evaluated benchmark across two data regimes, with statistically clear gains on GSM8K, MATH, and MMLU-Pro on the curated mixture. Together, these results make looped language models substantially cheaper to train from scratch and practical to introduce into existing pretrained checkpoints, while isolating the gains due to recurrence itself.

Abstract

Recurrent Language Models

Explore similar work

Closing the Loop: Practical Training Recipes for Looped Language Models

Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models

Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching