Looped language models such as Ouro and Raven apply one weight-shared block several times per token, so autoregressive decoding (AR) runs every loop of every token in sequence. Pipelined self-speculative decoders overlap these loops by drafting tokens from shallow readouts and verifying them at full depth. Yet every loop of a looped LM ends in a readout of the next token, and draft tokens pass through every depth anyway, so each depth can check them without an extra pass. We introduce LoopSkim, in which the depth of the pipeline is the verifier. Draft tokens copied from the context, as in prompt lookup decoding, enter after the first loops, any later readout that contradicts one replaces it at that depth, and only full-depth readouts commit tokens. LoopSkim fixes the random draws of every position in advance, so each check compares two tokens and decoding stays lossless in exact arithmetic under sampling for any number of checks and draft chains. Without training, its speedup over AR averages 1.73x that of LoopSpec across greedy decoding and sampling, and its checks between the first loops and full depth speed it up on every model. An optional draft head adds a second chain and raises this average to 1.99x.
Figures & tables
Figure 1: (a) A greedy decode of Ouro-1.4B ( R=4 ) with both draft chains: 8 tokens take 7 one-loop steps, against 8×4=32 loops for AR. (b) In step 4, the n-gram chain has more admitted tokens than the head’s and is kept. (c) At T>0 , all readouts at position i add the same Gumbel noise g , so each check compares two tokens.
Figure 2: Geometric mean over a model’s tasks, whiskers: range over three repeats. (a, b) Ratio of speedups over LoopSpec on (a) the main tasks and (b) four held-out datasets, two on Raven-TinyLlama ( Section C.1 ). (c) Throughput over LoopSpec’s schedule run in our engine ( Section C.3 ). Numbers in (a–c): LoopSkim. (d) Loops per output token relative to AR ( D ). LoopAhead: as reported in its Figure 3a ( Section C.5 ).
Model
Method
GSM8K
MATH500
BBH
HumanEval+
MBPP+
Geo. mean
Greedy decoding
LoopSpec
2.88 ± 0.07
3.13 ± 0.06
2.78 ± 0.06
3.19 ± 0.15
2.87 ± 0.02
2.97 ± 0.03
CopyLoop
3.39 ± 0.20 (1.18)
3.31 ± 0.11 (1.06)
3.23 ± 0.12 (1.16)
5.49 ± 0.18 (1.72)
2.74 ± 0.10 (0.95)
3.53 ± 0.08 (1.19)
LoopSkim
3.81 ± 0.02 (1.32)
3.79 ± 0.14 (1.21)
5.44 ± 0.17 (1.96)
6.90 ± 0.22 (2.16)
3.18 ± 0.02 (1.11)
4.44 ± 0.06 (1.49)
Ouro- 1.4B
+ head
5.01 ± 0.02 (1.74)
5.24 ± 0.20 (1.67)
5.20 ± 0.13 (1.87)
8.57 ± 0.16 (2.69)
4.38 ± 0.12 (1.53)
5.52 ± 0.08 (1.86)
LoopSpec
3.27 ± 0.04
3.39 ± 0.04
2.94 ± 0.11
3.23 ± 0.04
3.17 ± 0.14
3.20 ± 0.04
Table 1: Speedup ( × ) over AR in each method’s engine, mean ± standard deviation of three repeats ( Section A.4 ). Parentheses: ratio of speedups over LoopSpec. Bold: fastest per model and column. –: not run, as in LoopSpec. LoopSkim: no training. BBH: heads trained without BBH questions.
Figure 3: (a) Throughput without one component relative to with it, geometric mean over the five main tasks. Intermediate checks: greedy, five models ( Table 7 ). Others: Ouro-1.4B and Raven-Llama. Circles: Ouro. Triangles: Raven. (b) Speedup of LoopSkim + head over our AR, geometric mean over GSM8K, MATH500 and MBPP+ ( Section C.9 ). (c) Ratio of speedups over LoopSpec ( Section C.11 ). Numbers: LoopSkim. Whiskers: range over three repeats.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Data
Prompts
Output limit
Subset
Prompt format
GSM8K ( Cobbe et al., 2021 )
1,319
512
test set
LoopSpec’s
MATH500 ( Hendrycks et al., 2021 ; Lightman et al., 2024 )
500
2,048
all
LoopSpec’s
BBH ( Suzgun et al., 2023 )
540
1,024
first 20 per subtask
LoopSpec’s
HE+ ( Liu et al., 2023 ; Chen et al., 2021 )
164
1,024
all
LoopSpec’s
MBPP+ ( Liu et al., 2023 ; Austin et al., 2021 )
378
1,024
all
LoopSpec’s
SVAMP ( Patel et al., 2021 )
1,000
512
all, train and test
as GSM8K
Appendix
Table 2: Main tasks (top), held-out datasets and other data (middle, Sections C.1 and C.9 ) and the thinking tasks (bottom, Section C.2 ). Prompts: count after filtering. Output limit: maximum number of new tokens. Held-out samples are drawn with seed 24, and every prompt stops at the stop strings of its task.
Setting
Ouro
Raven
Draft tokens per chain G
8
8
Stride d
1
greedy: 2, T>0 : 4
Check depths Λ
1, 2, 3, 4
greedy: 2, 4, 8, 16, 32 T>0 : 4, 8, 16, 32
Tokens per pass
36 (45)
greedy: 144 (153) T>0 : 72 (81)
Head draws, T>0
at 0.7×T
at T
Pruning margin, T>0
3 nats
–
Appendix
Table 3: Default settings. Tokens per pass: (C+R/d−1)(1+G) , with the draft head in parentheses. Pruning margin: readouts below full depth drop tokens more than 3 nats below their top token.
Checks below R
Method
Draft source
Draft chains
Draft tokens / step
Ouro
Raven
On contra- diction
Sampling
LoopSpec
readout
1
1
–
–
–
✓
WaveFront
readout
1
1
–
–
–
✗
DAS-Wave
readout
1
≤ 24
–
–
–
✗
LoopAhead
drafter ∗
tree
≤ 3
2
4, 8 †
paused
✓
CopyLoop
context
1
K
1
4
replaced
✗
Appendix
Table 4: Self-speculative decoders for looped LMs. Checks below R : depths, in loops, at which a check can stop a draft token ( R=4 on Ouro, 32 on Raven). Sampling: evaluated with lossless sampling (✗: not evaluated). –: none. ∗ Trained. † 2, 8 on Raven-TinyLlama. ‡ Plus an optional draft head. § 4, 8, 16 under sampling.
Over LoopSpec
Over CopyLoop
Held-out datasets
Held-out datasets
Model
Method
SVAMP
Olymp.
MMLU-Pro
BCB
Mean
Spec-Bench
SVAMP
Olymp.
MMLU-Pro
BCB
Mean
Spec-Bench
Greedy
Ouro-1.4B
LoopSkim
1.36
1.26
1.39
2.27
1.52
1.46
1.10
1.15
1.42
1.35
1.25
1.18
LoopSkim + head
1.71
1.68
1.44
2.41
1.78
1.49
1.38
1.53
1.47
1.43
1.45
1.20
Ouro-2.6B
LoopSkim
1.36
1.84
1.26
2.13
1.61
1.46
0.99
1.23
1.12
1.21
1.13
1.22
Appendix
Table 5: Ratio of speedups, mean of three repeats. Mean: geometric mean over the held-out datasets. Spec-Bench: geometric mean over its six categories. Olymp.: OlympiadBench. BCB: BigCodeBench.
Model
Method
GSM8K
MATH500
BBH
HumanEval+
MBPP+
Geo. mean
Greedy decoding
LoopSpec
2.88 ± 0.07
3.12 ± 0.06
2.78 ± 0.06
3.19 ± 0.15
2.86 ± 0.01
2.96 ± 0.03
CopyLoop
3.38 ± 0.20 (1.17)
3.22 ± 0.11 (1.03)
3.20 ± 0.12 (1.15)
5.49 ± 0.18 (1.72)
2.68 ± 0.10 (0.94)
3.48 ± 0.08 (1.18)
LoopSkim
3.80 ± 0.02 (1.32)
3.73 ± 0.14 (1.20)
5.43 ± 0.17 (1.95)
6.90 ± 0.22 (2.16)
3.10 ± 0.02 (1.08)
4.40 ± 0.06 (1.49)
Ouro- 1.4B
+ head
5.01 ± 0.02 (1.74)
5.16 ± 0.20 (1.65)
5.19 ± 0.13 (1.87)
8.57 ± 0.16 (2.69)
4.31 ± 0.12 (1.51)
5.48 ± 0.08 (1.85)
LoopSpec
3.27 ± 0.04
3.27 ± 0.05
2.94 ± 0.11
3.23 ± 0.04
3.17 ± 0.14
3.17 ± 0.04
Appendix
Table 6: As Table 1 , without the prompts on which the greedy output of any of its decoders falls into a repetition loop in any repeat. The same prompts are left out at T=1 .
Greedy
T=1 , top-p 0.9
Ouro- 1.4B
Ouro- 2.6B
Raven- Llama
Raven- OLMo
Raven- TinyLlama
Ouro- 1.4B
Raven- Llama
Raw decode throughput (tok/s) in Table 1
SGLang’s AR
141.9
70.8
68.5
42.3
62.8
144.1
68.8
Our AR
164.2
83.3
67.1
60.3
58.9
162.0
66.1
LoopSpec
420.5
226.5
380.3
222.4
373.5
345.1
268.6
LoopSkim
728.8
433.6
692.6
700.0
648.2
608.5
416.5
Appendix
Table 7: Raw throughput in Table 1 (top) and the controls of Section C.3 . Geometric mean over tasks of the per-cell means over three repeats. Ratios in our engine: per repeat, the geometric mean over tasks of the per-cell throughput ratio, then the mean over repeats. ∗ Raven, greedy: stride 4 without the checks, 2 with them.
Admis- sion check: rejected (%)
Inter- mediate depths: replaced
Full depth: replaced
Below full depth (%)
Full depth: rate (%)
Mean depth
Greedy
Ouro-1.4B
33.6
9,493
1,247
88.4
1.10
2.5
Ouro-2.6B
25.3
5,759
719
88.9
0.69
2.5
Raven-Llama
19.9
6,897
207
97.1
0.16
6.7
Raven-OLMo
18.8
13,582
387
97.2
0.18
6.4
Raven-TinyLlama
22.4
2,733
72
97.4
0.14
6.6
Appendix
Table 8: Replacements per check depth of LoopSkim on the prompts of Section C.3 , pooled over each model’s tasks. Rejected: share of the draft tokens checked. Below full depth: share of the later replacements made at an intermediate depth. Rate: full-depth replacements per decoded token. Mean depth: mean check depth, in loops, of the later replacements ( R=4 on Ouro, 32 on Raven). All: pooled over models. All + head: the same for LoopSkim + head. T=1 : seed 0.
Greedy
T=1 , top-p 0.9
Ouro- 1.4B
Ouro- 2.6B
Raven- Llama
Raven- OLMo
Raven- TinyLlama
Ouro- 1.4B
Raven- Llama
Loops per output token, D
LoopSpec
0.2853
0.2725
0.0773
0.0954
0.0840
0.3032
0.1291
CopyLoop
0.1984
0.1475
0.0633
0.0619
0.0815
–
–
LoopSkim
0.1833
0.1498
0.0395
0.0380
0.0511
0.2151
0.0961
LoopSkim + head
0.1299
0.1078
0.0321
0.0310
0.0399
0.1628
0.0769
Appendix
Table 9: The factors of Equation 1 on the prompts of Table 1 , pooled over tasks, and the loops over all token positions ( Section C.5 ). Last block: LoopAhead’s Figures 3a and 3b (greedy). LoopSpec’s time per loop: its official code over SGLang’s AR. Bold: lowest per column in each block but τ .
Ouro-1.4B
Raven-Llama
Removed component
Greedy
T=1
Greedy
T=1
Multi-token admission ( G=0 )
0.521 (0.512–0.526)
0.624 (0.618–0.633)
0.585 (0.582–0.589)
0.681 (0.667–0.693)
Admission check ∗
0.606 (0.603–0.608)
0.629 (0.621–0.638)
0.582 (0.577–0.586)
0.559 (0.543–0.583)
N-gram chain
0.731 (0.726–0.741)
0.793 (0.787–0.803)
0.681 (0.675–0.692)
0.735 (0.715–0.757)
Redrafting after a replacement
0.984 (0.972–0.994)
0.973 (0.968–0.981)
0.982 (0.980–0.986)
0.981 (0.975–0.991)
LoopSkim
Appendix
Table 10: Throughput with one component removed, relative to with it, for LoopSkim + head (top) and LoopSkim (bottom). Geometric mean over tasks, mean of three repeats (range). T=1 : seed 0. ∗ Head chain alone, relative to it with the admission check. † Raven, greedy: stride 4 without the checks, 2 with them. ‡ Stride 2 in both arms. § Noise below full depth drawn independently of full depth’s.
Mode
R
LoopSkim + head / our AR
LoopSpec / our AR
LoopSkim + head / LoopSpec
Greedy
8
4.52
2.36
1.92
16
6.97
3.67
1.90
32
10.91
5.88
1.86
T=1
8
2.52
1.59
1.58
16
4.06
2.43
1.67
32
6.97
3.75
1.86
Appendix
Table 11: Raven-Llama run with R loops. Speedup over our AR of LoopSkim + head and of LoopSpec’s schedule in our engine, and their ratio. Geometric mean over tasks, mean of three repeats.
Our AR
LoopSkim + head
Model
GSM8K
MATH500
BBH
HE+
MBPP+
GSM8K
MATH500
BBH
HE+
MBPP+
Ouro-1.4B
78.5
72.6
69.6
65.9
64.8
79.1
73.2
70.7
66.5
64.8
Ouro-2.6B
86.0
57.4
77.8
73.2
67.7
86.2
56.0
77.4
72.6
69.0
Raven-Llama
51.9
35.4
32.8
11.0
23.8
52.0
35.8
31.5
9.8
24.6
Raven-OLMo
46.1
28.0
28.0
10.4
12.2
45.3
27.6
28.7
11.0
11.6
Raven-TinyLlama
43.7
–
–
14.0
23.0
43.1
–
–
14.0
23.8
Appendix
Table 12: Accuracy (%), each decoder run to its end token or the output limit. T=1 : mean over three seeds.
Greedy
T=1 , top-p 0.9
Ouro- 1.4B
Ouro- 2.6B
Raven- Llama
Raven- OLMo
Raven- TinyLlama
Ouro- 1.4B
Raven- Llama
LoopSpec
2.59
2.75
5.21
5.03
6.07
2.26
4.05
CopyLoop
4.00 (1.54)
4.81 (1.75)
8.28 (1.59)
8.75 (1.74)
8.76 (1.44)
–
–
LoopSkim
4.78 (1.85)
5.64 (2.05)
7.96 (1.53)
10.69 (2.13)
9.35 (1.54)
3.94 (1.74)
5.49 (1.36)
+ head
6.39 (2.47)
7.65 (2.78)
9.17 (1.76)
13.17 (2.62)
11.17 (1.84)
4.91 (2.17)
6.34 (1.57)
LoopSpec in our engine
2.77
2.91
5.22
5.47
6.19
2.35
3.84
Appendix
Table 13: Speedup ( × ) over AR in each method’s engine on an RTX 5090, 100 prompts per cell, mean of three repeats. Parentheses: ratio of speedups over LoopSpec. Bold: fastest per column.
Budgets of 4,096–16,384 tokens
Budget of 32,768 tokens
GSM8K
MATH500
AIME24
AIME25
AIME24
AIME25
MATH500
T=1 , top-p 0.7
LoopSpec / SGLang AR
2.25
1.95
1.60
1.53
1.58
1.53
1.80
LoopSkim/ our AR
3.04
3.20
2.99
2.96
3.05
3.15
3.46
LoopSkim + head/ our AR
3.40
3.63
3.15
3.10
3.31
3.41
4.01
LoopSkim/ LoopSpec
1.35
1.64
1.87
1.93
1.93
2.06
1.92
Appendix
Table 14: Ouro-1.4B-Thinking on LoopSpec’s thinking tasks. Speedup over own AR, mean of three repeats, and ratios of speedups, each the quotient of the two speedups it names. LoopSkim + head: the Ouro-1.4B draft head. Bold: largest speedup and largest ratio over LoopSpec per column and block. D : loops per output token relative to AR.
GSM8K
MATH- 500
BBH
Human- Eval+
MBPP+
Geo. mean
All prompts
LoopSkim
4.40
10.55
6.16
9.98
7.90
7.42
LoopSkim + head
5.56
11.55
6.57
10.24
8.80
8.24
CopyLoop
4.14
6.46
4.83
7.61
6.34
5.74
LoopSpec
3.28
3.74
3.28
4.39
4.14
3.74
Without repetition loops
Appendix
Table 15: Huginn-0125 under greedy decoding. Speedup ( × ) over AR in each method’s engine, mean of three repeats. Geometric mean over tasks per repeat, then the mean. Lower block: without the prompts on which any method’s output falls into a repetition loop. Bold: fastest per column and block.
1K
2K
4K
8K
16K
32K
Geo. mean
Over LoopSpec
Ouro-1.4B
1.44
1.60
1.62
1.65
1.51
1.30
1.51
Ouro-2.6B
1.33
1.39
1.38
1.44
1.54
1.33
1.40
Raven-Llama
1.66
1.85
1.91
1.87
2.08
1.77
1.85
Over CopyLoop
Ouro-1.4B
1.38
1.67
1.79
1.91
1.75
1.50
1.66
Appendix
Table 16: Ratio of speedups of LoopSkim + head over each baseline on PG-19 contexts, greedy decoding with the end token ignored, mean of three repeats. Each speedup is over AR in the method’s own engine, and each baseline runs its fastest configuration per length. Geo. mean: over lengths per repeat, then the mean.