cs.LGNov 15, 2026

LoopSkim: Lossless Multi-Depth Speculative Decoding for Looped Language Models

Authors: Muhan Xiao, Shijian Li

Organizations: College of Artificial Intelligence Zhejiang University · College of Computer Science and Technology Zhejiang University

Abstract

Looped language models such as Ouro and Raven apply one weight-shared block several times per token, so autoregressive decoding (AR) runs every loop of every token in sequence. Pipelined self-speculative decoders overlap these loops by drafting tokens from shallow readouts and verifying them at full depth. Yet every loop of a looped LM ends in a readout of the next token, and draft tokens pass through every depth anyway, so each depth can check them without an extra pass. We introduce LoopSkim, in which the depth of the pipeline is the verifier. Draft tokens copied from the context, as in prompt lookup decoding, enter after the first loops, any later readout that contradicts one replaces it at that depth, and only full-depth readouts commit tokens. LoopSkim fixes the random draws of every position in advance, so each check compares two tokens and decoding stays lossless in exact arithmetic under sampling for any number of checks and draft chains. Without training, its speedup over AR averages 1.73x that of LoopSpec across greedy decoding and sampling, and its checks between the first loops and full depth speed it up on every model. An optional draft head adds a second chain and raises this average to 1.99x.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DLoop: Looped Speculative Decoding

    Oct 6, 2026Geonmo Gu, Byeongho Heo, HeeJae Jun +4Speculative DecodingLLM Inference Acceleration

  2. LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

    Nov 14, 2026Siqi Zhu, Suozhi Huang, Yi Zhao +1Speculative DecodingLLM Inference Acceleration