Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data

Matteo Fischetti

cs.LGSep 12, 2026

Minibatch persistency, eight years later: what batch reuse costs in steps and joules, and what it saves in data

Authors: Matteo Fischetti

Organizations: Department of Information Engineering University of Padova, Italy

Abstract

Minibatch persistency reuses data instead of reading it: rather than drawing a fresh minibatch at every optimizer step, it takes K consecutive steps on the same one. Absorbed into data echoing in 2019, it has carried one objection -- that reuse merely imitates a larger learning rate -- and no baseline tuned as carefully as the method itself. This paper runs the missing test. A pre-registered study trains a 49M-parameter Transformer on FineWeb-Edu at minibatch size B in {32, 128, 512}, 8 seeds per cell, tuning the learning rate separately for every batch size and every arm, against a reuse-free control that changes the sampling and nothing else. Each headline claim is a cost to reach a fixed loss, read on four axes: optimizer steps, fresh tokens, seconds, and joules at the socket. We then replicate on new seeds and a newer GPU generation, and put the three arms on one schedule in steps. The outcome of our study is that what minibatch reuse buys is neither speed nor energy but data, and only at large minibatch size: at B = 32 it reads more fresh tokens than the baseline, not fewer. On steps, seconds and joules it is at best free; and at B = 512, where it looks best, a registered control cannot separate the effect of reuse from the position on the learning-rate schedule at n = 8 seeds. The technique is therefore worth using where fresh data rather than compute is the binding cost: a corpus that runs out, a pipeline that pays per sample, a stream that cannot be rewound. Where the data can simply be read again, spaced epochs do as well or better.

Figures & tables

card	what ran on it: stage, runs, card-hours	driver	torch (CUDA)	precision	micro-batch	energy method
Kaggle T4	coarse sweep, 31 runs, 14.3 h; confirmatory stage, 138 runs, 234.7 h; learning-rate-matched arm, 9 runs, 30.4 h	not recorded	2.10.0+cu128 (12.8)	fp16 autocast, dynamic loss scaling	8	NVML total-energy counter
Modal L4	coarse sweep, 5 runs, 1.0 h	not recorded	not recorded	bf16	8	NVML total-energy counter
RTX PRO 6000 Blackwell (university HPC)	confirmatory group $(B=512,\text{seed }7)$ , 6 runs, 1.0 h; bridge, 24 runs, 2.0 h; E1 coarse pass, 24 runs, 0.5 h; E1, 96 runs, 8.0 h; E2, 144 runs, 25.0 h; E3, 48 runs, 16.8 h; E3-lr, 12 runs, 2.1 h	610.57.04	2.13.0+cu130 (13.0)	bf16	32	NVML total-energy counter

Table 1: Platforms and software. Card-hours are the wall time of training plus evaluation, summed over the runs of one card within a stage and never across cards; the per-card totals are in Section 4.1 . The CUDA version is the one of the torch build; the driver of the free-tier cards is not recorded in the run headers. Micro-batch is the number of sequences per forward/backward pass, gradient accumulation supplying the rest of

B

instrument	where	scope	what we found
NVML total-energy counter	Kaggle T4 (the confirmatory study); university HPC (RTX PRO 6000 Blackwell, server edition: one confirmatory group, the bridge, the extensions)	one card, integrated in hardware	present; the source of every joule in Section 6 ; no sampling, no coverage clause
NVML power sampling	departmental cluster (A40, RTX 3090)	one card, sampled	the only per-device instrument on these cards; used once, on the exclusive-node run of the last row, and no run of the confirmatory study or of an extension contributed a joule from it
IPMI, shared node	university HPC	whole node	overstates the job by about $11\times$ ; the differential reading does not recover it while other jobs hold the remaining cards; exclusive allocation not grantable
IPMI, exclusive node	departmental cluster	whole node, attributable	$210.9$ kJ at the node against $54.5$ kJ sampled on the card over one run ( $3.9\times$ ); differential $+349$ W against $+270$ W, i.e. about $80$ W of host-side power NVML cannot see; about sixteen hours of queueing per data point

Table 2: The energy instruments available to this study and what each measures. The per-device counter is the only energy axis of the results; the exclusive-node reading is one run on other hardware, used to size what the counter cannot see; the shared-node reading is not an instrument for this question at all.

$B$	$\rho(B)$	grid argmin	reading (thresholds fixed before the data)
$32$	$1.21$	$2.99$	$\geq 0.8$ : not explained by a larger learning rate
$128$	$1.72$	$1.00$	$\geq 0.8$ : not explained by a larger learning rate
$512$	$0.68$	$1.00$	between $0.5$ and $0.8$ : indeterminate
$32$ / $128$ / $512$ , E2	$0.33$ / $1.00$ / $1.00$	—	two-point grid at $\tau^{\prime}$ , majority of 8 seeds (Section 6.5 )
$512$ , E3-lr	$1.00$	—	extension card, $\tau^{\prime}$ , one seed, argmins at $\eta_{1}$ (Section 6.6 )

Table 3: The tuned learning rate under reuse, relative to the tuned baseline—the higher the further from a mere re-parametrization. Coarse stage: one seed, a quarter of the budget, four rates on a geometric grid of spacing 2.99, read by best validation loss;

\rho

is the ratio of the interpolated optima

\eta^{\star}

, the grid column the ratio of the raw argmins, a sensitivity. Descriptive, single seed, no interval. Were persistency nothing but a larger step size,

\rho

would sit near

1/K=0.25

. The E2 row is

\rho^{\prime}(B)

on the replication’s inherited two-point grid: the majority best rate of the persistency arm over that of the baseline, resolved to a factor 2.99, a different estimator on a different card and depth (Section 6.5 ). The last row is the reading of Extension E3 (Section 6.6 ) on the replication’s card at its deeper target.

$B$	$\tau(B)$	adjusted $\tau(B)$	floor-check adjustments
$32$	$5.3400$	$5.3400$	none
$128$	$6.0772$	$5.7372$	$17$
$512$	$7.0681$	$6.6681$	$20$

Table 4: Targets, in nats of validation loss, frozen before the confirmatory stage; the adjusted column is the mechanical re-adjustment of Section 4.4 , computed once on the closed confirmatory stage onto a new file, the last column counting the adjustment steps it took. At

B=128

and

B=512

the adjustment is forced by the persistency arm alone, whose first-evaluation minima are 5.7606 and 6.6956 nats against 6.6932 and 7.6964 for the baseline and 6.7631 and 7.7232 for the A/A arm; on the baseline and A/A arms alone no adjustment would fire. The frozen

\tau(B)

is exogenous to every treatment arm and carries the registered verdict; the adjusted target is not, and is read beside it.

Figure 1: The registered quantity: the penalty $\hat{g}(B)$ against batch size, per seed, with the geometric mean and its 95% interval, the halving threshold of the decision rule, and the min-max band of the A/A arm’s per-seed penalties. Drawn once, on the closed seed set, as the plan required. At $B=128$ and $B=512$ every seed’s persistency cost is that of the arm’s first evaluation, so the penalties drawn there are ceilings (Section 6.2 ); the replication of Section 6.5 removes the ceiling, and Figure 7 follows the ratio to deeper targets.

$f$	$\tau^{\prime}(512)$ (nats)	registered $R(512)$	replication $R(512)$	95% interval	$n$	joules
0.25	6.909	1.19	1.018	$[0.962,\,1.078]$	8	1.023
0.375	6.498	1.01	0.952	$[0.879,\,1.031]$	8	0.953
0.5	6.266	0.93	0.923	$[0.854,\,0.997]$	8	0.926
0.625	6.087	0.88	0.878	$[0.805,\,0.957]$	8	0.879
0.75	5.960	0.82	0.812	$[0.720,\,0.915]$	8	0.812

Table 5: The registered map from the target fraction

f

to the depth

\tau^{\prime}(512)

and to the mean paired step ratio

R(512)

that the 106 primary runs then on disk predicted at each depth (Extension E2 of the registration), beside the replication’s own reading at the same five depths: geometric mean over its seeds of the persistency-to-baseline steps to target, the 95%

t

-interval, the pairs entering, and the joule ratio. The registered fraction is

f=0.625

(bold), the depth at which every arm reaches the target in every observed seed; the other four rows are descriptive.

Figure 2: The replication’s four cost axes: cost to reach the deeper target $\tau^{\prime}(B)$ , relative to the tuned baseline, against batch size—below one means reuse is cheaper. Each panel is one cost axis (optimizer steps, fresh tokens, training seconds, GPU joules). Each point is the estimator of Table 6 : the paired geometric mean over seeds of the per-seed ratio, with its 95% $t$ -interval on the logarithm, $n=8$ pairs in every cell. Seconds and joules are all on the Blackwell, so no pair is left out on any axis. No cell is a ceiling: no seed crosses at its first evaluation (Table 12 ), so every marker is filled. The A/A calibration arm (blue, dashed) sits on the line of no effect on every axis, fresh tokens included, since the evaluation grid of this study removes the offset of Section 6.8 .

arm	$B$	fresh tokens	steps (= compute)	seconds	joules
persistency	$32$	$1.410^{\,\blacktriangle}$	$5.638^{\,\blacktriangle}$	$5.589^{\,\blacktriangle}$	$5.647^{\,\blacktriangle}$
		$[1.396,\,1.424]$	$[5.582,\,5.693]$	$[5.454,\,5.727]$	$[5.509,\,5.788]$
persistency	$128$	$0.357^{\,\blacktriangledown}$	$1.425^{\,\blacktriangle}$	$1.417^{\,\blacktriangle}$	$1.423^{\,\blacktriangle}$
		$[0.347,\,0.367]$	$[1.386,\,1.464]$	$[1.383,\,1.452]$	$[1.383,\,1.465]$
persistency	$512$	$0.221^{\,\blacktriangledown}$	$0.878^{\,\blacktriangledown}$	$0.875^{\,\blacktriangledown}$	$0.879^{\,\blacktriangledown}$
		$[0.204,\,0.241]$	$[0.805,\,0.957]$	$[0.803,\,0.955]$	$[0.799,\,0.967]$

Table 6: The replication’s twin of Table 8 : cost-to-target ratios of the persistency arm and of the A/A calibration arm to the tuned baseline at

\tau^{\prime}(B)

, on the Blackwell, seeds 8–15: geometric mean over 8 pairs with the 95%

t

-interval beneath. Below one favors reuse;

\blacktriangledown

marks an outright win (interval entirely below one),

\blacktriangle

an outright loss (interval entirely above one), no mark an interval covering one. No cell is a ceiling: no seed crosses at its first evaluation (Table 12 ). Descriptive; the replication’s one confirmatory test is on the step axis.

$B$	arm	seconds	kJ	mean W
$32$	baseline	174	72.0	413
$32$	persistency	974	406.9	418
$32$	A/A	175	72.5	415
$128$	baseline	177	74.0	417
$128$	persistency	251	105.4	419
$128$	A/A	178	74.4	419

Table 7: Absolute costs to

\tau^{\prime}(B)

in the replication, on the RTX PRO 6000 (power limit 600 W, NVML total-energy counter): mean over 8 seeds per cell of training seconds and GPU kilojoules at the arm’s cost-minimizing rate, and mean power as kilojoules over seconds. Never pooled with the T4 absolutes of Section 6.8 .

Figure 3: The tuning evidence, on the coarse grid where the learning rate is actually chosen: best validation loss against peak learning rate, one panel per batch size. One seed per point, at the coarse stage’s budget, best validation loss over the run. Thin curves: the registered estimator of Amendment 2, the parabola in $\log\eta$ through the three grid points around the argmin, its vertex marked as the tuned rate. The largest gap between the A/A arm and the baseline over the grid, 0.08, 0.20 and 0.05 nats at $B=32$ , $128$ and $512$ , is the noise scale of a single point. Every cell has its optimum in the interior of the swept range—the sweep is wide enough to have found it, in all nine cells. Without this figure the 2018 criticism repeats itself, which is why it is a main-text figure and not an appendix one.

arm	$B$	fresh tokens	steps (= compute)	seconds	joules
persistency	$32$	$0.9732$	$3.886^{\,\blacktriangle}$	$3.852^{\,\blacktriangle}$	$3.866^{\,\blacktriangle}$
		$[0.9449,\,1.002]$	$[3.773,\,4.002]$	$[3.739,\,3.968]$	$[3.76,\,3.975]$
persistency	$128$	$0.3948^{\,\blacktriangledown}$	$1.554^{\,\triangle}$	$1.501^{\,\triangle}$	$1.515^{\,\triangle}$
		$[0.3766,\,0.4139]$	$[1.483,\,1.63]$	$[1.433,\,1.572]$	$[1.448,\,1.584]$
persistency	$512$	$0.3784^{\,\blacktriangledown}$	$1.419^{\,\triangle}$	$1.402^{\,\triangle}$	$1.442^{\,\triangle}$
		$[0.3644,\,0.393]$	$[1.367,\,1.474]$	$[1.296,\,1.516]$	$[1.408,\,1.476]$

Table 8: Cost-to-target ratios of the persistency arm and of the A/A calibration arm to the tuned baseline, on every cost axis, at the frozen targets: geometric mean over seeds, with the 95%

t

-interval beneath. Below one favors reuse.

\blacktriangledown

marks an outright win (interval entirely below one) and

\blacktriangle

an outright loss (interval entirely above one); no mark means the interval covers one.

\triangle

marks a cell whose interval lies above one but whose persistency cost is the cost of the arm’s first evaluation, a ceiling (Section 6.2 ): no outright loss is assigned there. The fresh-token cells at

B=128

and

B=512

rest on the same ceiling, and their

\blacktriangledown

is read a fortiori: a smaller cost only widens the win. Compute is a deterministic transform of steps at fixed

B

and shares its column. The table is descriptive in the sense of the registered plan: the primary study runs one confirmatory test, on the step axis, and no

\alpha

is spent here. Every cell has 8 pairs, on the closed stage; at

B=512

one of them is on the Blackwell (Section 6.4 ), and the seven-pair reading is in the text.

arm	$B$	fresh tokens	steps (= compute)	seconds	joules
persistency	$32$	$0.973$	$3.886^{\,\blacktriangle}$	$3.852^{\,\blacktriangle}$	$3.866^{\,\blacktriangle}$
		$[0.945,\,1.002]$	$[3.773,\,4.002]$	$[3.739,\,3.968]$	$[3.760,\,3.975]$
persistency	$128$	$0.293^{\,\blacktriangledown}$	$1.157^{\,\blacktriangle}$	$1.138^{\,\blacktriangle}$	$1.146^{\,\blacktriangle}$
		$[0.281,\,0.307]$	$[1.107,\,1.209]$	$[1.070,\,1.211]$	$[1.084,\,1.212]$
persistency	$512$	$0.263^{\,\blacktriangledown}$	$0.995$	$0.993$	$1.014$
		$[0.247,\,0.280]$	$[0.934,\,1.059]$	$[0.900,\,1.096]$	$[0.960,\,1.071]$

Table 9: Table 8 at the re-adjusted targets of Table 4 , the targets the registered floor check prescribes once the confirmatory stage is closed: the same estimator, pairs and marks on the same 144 runs. No cell is a ceiling at these targets. The re-adjusted targets at

B=128

and

B=512

are set by the persistency arm’s own first evaluations and are not exogenous to the treatment (Section 6.2 ); the frozen targets carry the registered verdict.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Learning curves at $B=32$ : validation loss against fresh tokens (left) and against GPU joules (right), one run per arm. The dashed line is $\tau(32)$ . The persistency arm (red) starts below the baseline (black) on fresh tokens and is overtaken shortly after both have passed the target; on joules it is displaced to the right by a factor that grows over the run. The A/A arm (blue, dashed) tracks the baseline on both axes. One run per arm, unpaired: the run with the lowest final validation loss in its cell over learning rates and seeds, among the runs on the card that carries the cell.

Figure 5: Learning curves at $B=128$ , same layout as Figure 4 . The persistency arm is below the baseline on fresh tokens at every evaluation, and below $\tau(128)$ already at its first one; on joules it is displaced to the right of the baseline for the whole run. One run per arm, unpaired, chosen by the lowest final validation loss over learning rates and seeds among the runs on the card that carries the cell.

Figure 6: Learning curves at $B=512$ , same layout as Figure 4 . On fresh tokens the persistency arm is below the baseline at every evaluation and below $\tau(512)$ at its first one; on joules the three curves lie on top of one another over the range where they overlap, the persistency arm continuing further because the same fresh-token budget lasts 4 times as many steps. One run per arm, unpaired, chosen by the lowest final validation loss over learning rates and seeds among the runs on the card that carries the cell; the $(512,7)$ group of Section 6.4 , which ran on the Blackwell, is excluded.

Figure 7: Cost-to-target ratio of the persistency arm (red) and of the A/A arm (blue, dashed) to the tuned baseline, as a function of the target validation loss, deeper targets to the right; one column per batch size, one row per cost axis; bands are 95% $t$ -intervals over seeds on the logarithm. Shaded: the grid-quantized stretch, where every arm crosses at its first evaluation and the ratio is fixed by the evaluation grid. Solid vertical line: the frozen $\tau(B)$ ; dash-dotted: the re-adjusted $\tau(B)$ of Table 4 ; dotted: the replication’s $\tau^{\prime}(B)$ . Below one favors reuse. Drawn on the 144 runs of the confirmatory stage.

$B$	arm	steps per run	evals per run	first eval. (step)	steps to $\tau(B)$	seeds reached	at first eval.
$32$	baseline	3052	16	191	408–435	8/8	0/8
$32$	persistency	12205	16	761	1570–1689	8/8	0/8
$32$	A/A	3049	16	189	414–433	8/8	0/8
$128$	baseline	763	16	48	112–129	8/8	0/8
$128$	persistency	3049	16	189	189	8/8	8/8
$128$	A/A	761	16	45	112–127	8/8	0/8

Appendix

Table 10: Steps, evaluations and crossings per cell at the frozen targets, on the 144 runs of the confirmatory stage. “Steps per run” and “evals per run” are ranges over the runs of the cell (both learning rates, all seeds); “first eval.” is the optimizer step of the first evaluation; “steps to

\tau(B)

” is the range over seeds of the cost at the cost-minimizing learning rate, the quantity every ratio in the paper is built from; “seeds reached” counts the seeds that reached the target; “at first eval.” counts, among those, the seeds whose crossing is the first evaluation, where the cost is a ceiling read at the resolution of one evaluation interval. Generated by mbp.plots --crossing-table ; the file records its provenance.

arm	$B=32$	$B=128$	$B=512$
baseline $K=1$	8.96e-04, 2.68e-03	3.00e-04, 8.96e-04	8.96e-04, 2.68e-03
persistency $K=4$	8.96e-04, 2.68e-03	8.96e-04, 2.68e-03	3.00e-04, 8.96e-04
A/A	8.96e-04, 2.68e-03	8.96e-04, 2.68e-03	8.96e-04, 2.68e-03

Appendix

Table 11: The two learning rates per cell carried from the coarse stage (Section 4.2 ) into the final stage, from the run archive. The A/A arm’s pair coincides with the persistency arm’s at

B=32

and

128

and with the baseline’s at

B=512

; which of the two wins is chosen per (cell, seed), as the text states where the winner matters.

$B$	arm	steps per run	evals per run	first eval. (step)	steps to $\tau^{\prime}(B)$ horizon rule	steps to $\tau^{\prime}(B)$ 2-eval. rule	seeds reached	at first eval.
$32$	baseline	3052	64	47	1722–1782	1722–1782	8/8	0/8
$32$	persistency	12205	259	47	9600–10113	9600–10113	8/8	0/8
$32$	A/A	3049	64	47	1703–1825	1703–1825	8/8	0/8
$128$	baseline	763	69	11	430–475	430–475	8/8	0/8
$128$	persistency	3049	277	11	608–673	608–673	8/8	0/8
$128$	A/A	761	69	11	431–480	431–480	8/8	0/8

Appendix

Table 12: Steps, evaluations and crossings per cell in the replication of Section 6.5 , at its targets

\tau^{\prime}(B)

, on the replication’s own 144 runs (seeds 8–15). Columns as in Table 10 , with the crossing read twice: over the registered horizon of 191 / 48 / 12 steps (“horizon rule”) and under the unmodified two-evaluation rule of Section 3 (“2-eval. rule”). Note the last column, zero throughout, against every seed at

B=128

and

B=512

in Table 10 . Generated by mbp.plots --crossing-table --confirm-horizon ; the file records its provenance.

Data Repetition in Language Model Training

Data repetition in language model training studies how reusing examples affects learning, efficiency, memorization, and generalization.

See all 34 papers

Oct 11, 2026

Scaling Repeated-Data Pretraining: Batch Size, Regularization, and GPU-Time Tradeoffs

We study pretraining when compute is abundant but unique training data are finite, requiring repeated training on the same data. We first revisit infinite-compute pretraining (Kim et al. 2026) and show that its frontier depends critically on choices previously fixed or studied only in isolation. In particular, optimizing batch size together with epoch counts reveals a U-shaped relationship between validation loss and batch size, favoring smaller batches than typical pretraining, likely because small-batch gradient noise provides implicit regularization. We further find that training recipes, previously optimized under a single-pass assumption, can be tuned, e.g., repacking documents into new sequences every epoch, weight tying, and positive embedding weight decay. We then relax the infinite-compute assumption. In particular, smaller-than-typical batch sizes can require prohibitively long sequential training, motivating to treat GPUs and time as separate resources. Tighter deadlines favor larger batches, and latency-oriented parallelism can trade additional GPUs for shorter training. Nevertheless, even under a fixed time budget, a smaller batch with fewer GPUs can achieve lower validation loss than a larger batch with more GPUs, challenging the common practice of increasing data parallelism to maximize GPU utilization under a deadline. Together, our results identify underexplored factors in repeated-data training and provide concrete guidance on how training recipes and resource allocation should change from single-pass pretraining.

Sewon Min

UC Berkeley & Allen Institute for AI

LLM Training Data Repetition in Language Model Training Efficient Language Model Training Distributed Training

Ask PDF

Oct 4, 2026

What Is a Repeated Token Worth? The Scaling Geometry of Multi-Epoch Pretraining

As pretraining increasingly repeats data, every run faces three questions: how many epochs to take, how that number should change with model size, and whether anything besides the epoch count matters. We answer them by pricing a repeated token against two references: one epoch on the same data, which gives its value, and fresh data at equal compute, which gives its cost. Against fresh data, the cost of repetition follows a single variable, the number of extra epochs divided by the unique tokens per parameter. Against the same data, a second epoch is worth nearly as much as a fresh one, and repeated tokens fall to half the value of fresh ones after a critical epoch count that grows with the training budget per parameter but hardly with model size. With unique data fixed, the predicted compute-optimal run grows model size and epochs together until loss stops improving, near the critical epoch count. The same variable accounts for the direction of size trends that appear to conflict: larger models tolerate fewer epochs when the corpus is fixed, from about 15 at 127M to 4 at 2B parameters, but not when unique data grow with the model. Counts alone do not determine loss: at identical counts, replaying shards consecutively raises loss by up to 0.46~bits per byte, concentrating repeats on fewer samples also raises it, lower-entropy sources degrade faster with repetition, and re-tokenizing repeats helps only under heavy repetition. These results offer an empirical guide to pretraining when unique data, rather than compute, are the binding constraint.

Yekun Chai, Haoyi Xiong

FloatAI · ETH Zurich · Independent Researcher

Language Model Pretraining Language Model Scaling Laws Data Repetition in Language Model Training

Ask PDF

Oct 23, 2026

LSTM-Style Looped Transformers with Shared KV Caches

Recurrent transformers reuse weights to perform deep computation with fewer parameters. We study an LSTM-style transformer that also shares one key--value cache across its recurrent core. At width 1024, it uses 40.4% fewer parameters than an untied control at nearly matched forward FLOPs, and reduces persistent cache sets from fifteen to four. Parallel training supplies cache entries from a standard-attention pass, whereas continuation reads entries produced recursively using shared attention. We address this mismatch by aligning the two training passes' cache entries with a consistency loss during the final 10% of a fixed training-token budget. Across three model widths on FineWeb, this removes 81--89% of the continuation-loss penalty relative to full-cache evaluation. The improvement replicates on disjoint documents for two training seeds. Across four widths and 1.05--4.19B training tokens, the loss cost of weight tying is smallest at lower token budgets, suggesting an opportunity to use recurrent computation when data and model storage are limited.

Andreas Kirsch

Recurrent Transformers KV-Cache Management

Ask PDF

Abstract

Data Repetition in Language Model Training

Explore similar work

Scaling Repeated-Data Pretraining: Batch Size, Regularization, and GPU-Time Tradeoffs

What Is a Repeated Token Worth? The Scaling Geometry of Multi-Epoch Pretraining

LSTM-Style Looped Transformers with Shared KV Caches