How Much Gradient Does Truncated Backprop Lose in Looped Transformers?

Aur Shalev Merin

cs.LGOct 20, 2026

How Much Gradient Does Truncated Backprop Lose in Looped Transformers?

Authors: Aur Shalev Merin

Organizations: Independent researcher

Abstract

Looped transformers reuse one block of layers many times and are often trained by backpropagating through only the last k loops. We measure how much of the weight gradient this throws away, exactly and for every k from one backward pass, on nine looped language models, and report three findings. First, on a trained model this lost gradient is set near the output: once the loop count is well above k, it barely changes with the loop count (within about 0.03 from 32 to 64 loops on the three models we tested, as a pre-registered test predicted for Parcae). Where the backward signal shrinks, $ρ^k$ , with $ρ$ the signal's fitted shrink per loop, approximates it at moderate loop counts but overstates it at long ones, because the fitted $ρ$ drifts up as the fit takes in more loops. On Ouro the signal grows, and keeping two of four loops still loses 81% of the final output's gradient. Second, the lost gradient falls during fine-tuning, by 0.07-0.22 within 300 steps on three of four models, on held-out problems too. The fall is not just the read-out's share of the gradient; training only the layers outside the loop reproduced up to half of it, and it also falls under truncated training. So a trained model's value overstates what truncation removes later. Third, we could not show that it predicts the training cost of truncation better than rules that need no measurement: on two new models it tied Parcae's cost at the same k, in a test too small to separate them.

Figures & tables

Figure 1: Size of the backward signal at each loop, counting back from the output and relative to the last loop (median over sequences; log scale). The legend gives the median

R^{2}

of a straight-line fit per sequence. Ouro’s signal grows.

model	trained with backprop through	$R$	$\rho$	lost gradient at $k=1/2/4$	rule
Huginn-3.5B ∗	last 8 loops	32	0.88	0.90 / 0.80 / 0.62	0.88 / 0.77 / 0.59
Parcae-1.3B ∗	last half of the loops	16	0.84	0.80 / 0.65 / 0.47	0.84 / 0.71 / 0.51
Parcae-370m	last half of the loops	16	0.81	0.82 / 0.67 / 0.45	0.81 / 0.66 / 0.43
Parcae-140m	last half of the loops	16	0.80	0.81 / 0.66 / 0.42	0.80 / 0.64 / 0.41
SmolLM2, converted (A)	last 4 loops	8	0.35	0.30 / 0.11 / 0.015	0.35 / 0.12 / 0.015
SmolLM2, converted (B)	last 4 loops	8	0.38	0.32 / 0.12 / 0.021	0.38 / 0.15 / 0.022

Table 1: Lost gradient vs the rule

\rho^{k}

(median over held-out sequences of a single sequence’s lost gradient. The rule column is the median of each sequence’s

\rho_{i}^{k}

, so it can differ slightly from the tabulated

\rho

raised to the power

k

model	$R$	$\rho$	lost gradient at $k=1/2/4/8$	$\rho^{k}$ at $k=1/2/4/8$
Parcae-140m	8	0.77	0.81 / 0.65 / 0.39	0.77 / 0.60 / 0.36
Parcae-140m	16	0.80	0.81 / 0.66 / 0.42 / 0.18	0.80 / 0.64 / 0.41 / 0.17
Parcae-140m	32	0.83	0.82 / 0.68 / 0.44 / 0.20	0.83 / 0.68 / 0.46 / 0.21
Parcae-140m	48	0.85	0.82 / 0.68 / 0.43 / 0.21	0.85 / 0.73 / 0.53 / 0.28
Parcae-140m	64	0.87	0.82 / 0.68 / 0.43 / 0.21	0.87 / 0.76 / 0.58 / 0.33
Parcae-370m	16	0.81	0.82 / 0.67 / 0.45 / 0.21	0.81 / 0.66 / 0.43 / 0.19

Table 2: The lost gradient and

\rho

across loop counts (median over held-out sequences; all parameters for Parcae, part of the loop for Huginn).

Figure 2: Measured lost gradient vs

\rho^{k}

at every measured

k

. (a) The loop counts of Table 1 , for the six models whose backward signal shrinks (the SmolLM2 conversions are ours); open markers: gradient measured over part of the loop. Crosses: Parcae-140m and SmolLM2 (A) with the dial of Section 2 (

a<1

), measured with the same protocol on GSM8K validation text (Parcae-140m at

R

32 with

a=0.5

and 0.7,

R

48 with

a=0.4

R

64 with

a=0.3

; SmolLM2 (A) at

R=8

a=0.5/0.3/0.2

). (b) Parcae-140m, Parcae-370m and Huginn at their other measured loop counts, marker shape by loop count; at 48 and 64 loops (S10) the rule overstates the loss on Parcae. Ouro-1.4B is not shown (

\rho=1.88

model	statistic	trained	full backprop	last 2 loops	no loop credit
Parcae-140m	all	0.80 / 0.41	0.61 / 0.22	0.62 / 0.21	0.71 / 0.34
	loop only	0.82 / 0.42	0.69 / 0.25	0.69 / 0.24	0.74 / 0.35
Part II, sampled loop count	all	0.76 / 0.29	0.63 / 0.22	0.65 / 0.25	0.67 / 0.26
	loop only	0.81 / 0.31	0.72 / 0.25	0.73 / 0.28	0.76 / 0.29
Part II, fixed 5 loops	all	0.83 / 0.39	0.72 / 0.30	0.74 / 0.37	0.79 / 0.38
	loop only	0.87 / 0.40	0.79 / 0.33	0.81 / 0.40	0.85 / 0.40

Table 3: The lost gradient during fine-tuning, on held-out problems (medians over the first 64 GSM8K test problems at

k=1/4

, over all parameters and with the loop-only statistic of “Where it falls”; after 300 of 600 steps of each fine-tune; S15, S16. Table adds

k=2

Figure 3: Training cost against lost gradient for every setting we trained on Parcae-140m (29) and SmolLM2 (A) (6), lost gradient measured as in Table (a batch average, not the per-sequence measure the S5 and S6 curve uses). Filled markers:

k=1

. No cutoff is implied. The two models of Appendix , whose lost gradient was measured per sequence, are in Table .

Figure 4: Observed vs predicted training cost, mean

\pm

1 standard error over 5 seeds. Left: Parcae-140m, predictions locked before training; “

k

alone” is the average cost at the same

k

in earlier runs. Right: SmolLM2 converted (A), blended settings only (note the smaller y-axis); “

k

alone” is this model’s own cost without blending.

Truncated Backpropagation Through Time

See all 8 papers

Oct 19, 2026

Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation

Looped transformers reuse a shared block of layers to increase computational depth without increasing the number of trainable parameters. They are trained with backpropagation through time (BPTT). To save training compute, BPTT is often truncated to the last few loops, but truncation yields biased gradients that may weaken the benefit of looping. In this work, we study the value of truncation in looped transformer training in a systematic way: we jointly vary model size, training tokens, and forward and backward loop counts across nearly 600 training runs and fit a unified scaling law that captures the effects of looping and truncation through effective parameters. Despite the training-compute savings of truncation, our fitted law favors full BPTT when forward and backward loop counts are jointly optimized. In matched-compute runs, full BPTT through fewer loops consistently outperforms truncated settings of the same compute. The law further shows that at fixed model size, the compute-optimal loop count and training-token budget both grow with compute. When model size is also free, the law identifies two fully trained loops as the best looped setting. Experimentally, this setting achieves lower validation loss than the compute-optimal non-looped model with about 27% fewer non-embedding parameters and only 8% more training compute.

Xuyuan Xiong, Siyu Chen, Zihan Shao +5

MMLab, The University of Hong Kong · Yale University

Truncated Backpropagation Through Time Efficient Language Model Training Recurrent Transformers Language Model Scaling Laws Recurrent Neural Networks

Ask PDF

Apr 22, 2026

How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models

We measure how much one recurrence is worth to a looped (depth-recurrent) transformer, in equivalent unique parameters. From an iso-depth pretraining sweep across recurrence counts

r \in \{1, 2, 4, 8\}

spanning

{\sim}50\times

in training compute, we fit a joint scaling law

L = E + A\,(N_\text{once} + r^{\varphi} N_\text{rec})^{-α} + B\,D^{-β}

and measure a recurrence-equivalence exponent

\varphi = 0.46

. Intuitively,

\varphi

tells us whether looping a block

r

times is equivalent in validation loss to

r

unique blocks of a non-looped model (full equivalence,

\varphi{=}1

) or to a single block run repeatedly with no capacity gain (

\varphi{=}0

). Our

\varphi = 0.46

sits in between, so replacing unique blocks with shared recurrences increases validation loss at matched training compute. For example, at

r{=}4

a 410M looped model performs on par with a 580M non-looped model, but incurs the training cost of a 1B non-looped one. We demonstrate the utility of

\varphi

as a diagnostic tool on two case studies: commonly used truncated backpropagation lowers

\varphi

0.38

, indicating that the loop mechanism is poorly trained under truncation, even though validation loss decreases. Conversely, hyperconnections raise

\varphi

0.65

, a genuine capacity gain. Our method separates true loop improvements from training-side gains, a distinction raw validation loss cannot make.

Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis

Chair for AI in Healthcare and Medicine, Technical University of Munich, Germany · Department of Computing, Imperial College London, UK · Munich Center for Machine Learning (MCML), Germany +1

Truncated Backpropagation Through Time Language Model Scaling Laws Recurrent Neural Networks Recurrent Transformers

Ask PDF

Sep 15, 2026

Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers

Looped Transformers offer a parameter-efficient route to test-time scaling by reusing shared layers for iterative latent reasoning. However, additional iterations can reduce support for a reference answer, leaving unclear whether an update's direction is locally unhelpful or its full displacement moves too far. We study this distinction by analysing reference utility, which measures this support, along the model's own update direction, varying the fraction of the proposed displacement supplied to the readout. This reveals finite-step failures in which a locally improving direction produces a harmful full update. A pathwise curvature decomposition characterises how initial progress is lost, while a local quadratic model predicts full-step gains and useful step scales. Bounds based on accumulated curvature variation characterise the approximation error of these predictions. Experiments across two model families reveal this separation on mathematical and commonsense tasks. A fixed quarter step produces positive gains in reference utility for 72.2--83.2% of selected failures across four settings. These findings identify a mismatch between update direction and step scale as a mechanism of lost progress, explaining how some harmful updates retain useful computation.

Zhihao Guo, Zonghan Wu, Haizhou Du +4

University of Technology Sydney, Australia · East China Normal University, China · Shanghai University of Electric Power, China +2

Inference-Time Scaling Recurrent Transformers

Ask PDF

Abstract

Truncated Backpropagation Through Time

Explore similar work

Truncate or Not? A Scaling Study of Looped Transformers with Truncated Backpropagation

How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models

Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers