A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives

Prateek Kumar Sikdar

cs.CLAug 28, 2026

A rigor-matched audit of periodic-step layer skipping for efficient llm inference: conflayers versus swift, with a supplemental analysis of trained routing alternatives

Authors: Prateek Kumar Sikdar

Organizations: Accenture

Abstract

Layer-skipping methods for efficient LLM inference decide, at some granularity, which transformer layers to execute for a given input. We present a rigor-matched, three-seed audit of two periodic-step, search-based methods that make this decision online at inference time and re-evaluate it every few generation steps: a confidence-gated early-exit baseline (ConfLayers) and genuine self-speculative decoding (SWIFT, Xia et al. 2024), together with vanilla autoregressive decoding, across Qwen2.5-0.5B and Qwen2.5-1.5B (Yang et al. 2024) on GSM8K (Cobbe et al. 2021) and CNN/DailyMail (Nallapati et al. 2016; See et al. 2017). SWIFT is strongest on accuracy in three of four cells, while ConfLayers is dominated everywhere, with particularly large deficits on GSM8K at 1.5B. Separating online-search overhead from pure inference cost, we find that SWIFT's true inference speed is faster than ConfLayers's in all four cells (5-21%), reversing the naive wall-clock ranking in three; ConfLayers's search overhead is small and stable (1-2% of cost), whereas SWIFT's is larger and more variable across seeds (up to 28.7%). We additionally examine two trained routing methods, LayerRoute (Sikdar, 2026), a per-sequence, input-conditioned hard gate, and LayerDrop (Fan et al. 2020), a fixed, input-independent pruning pattern, as a supplemental analysis rather than a head-to-head comparison because both operate at a coarser decision granularity. Under a verified protocol with genuine per-input gating, a genuine full-model baseline, and genuine inference-time compute skipping, both show modest real speedups (1.08-1.33x) but accuracy well below the periodic-step methods, including near-total LayerRoute collapse on GSM8K at 1.5B (0.003 mean exact-match across three seeds). We release the full audit protocol as a template for rigor-matched efficiency comparisons.

Figures & tables

Method	Training	Decision unit	Re-evaluation cadence
ConfLayers	None (online search)	Per token	Every 30 steps
SWIFT	None (online search)	Per token	Every 25 steps
LayerRoute	Trained (LoRA + gate)	Per sequence	Once, input-conditioned
LayerDrop	Trained (structured dropout)	N/A	Never (fixed, input-independent)

Table 1: Layer-skipping methods by decision granularity. ConfLayers and SWIFT share the same regime (periodic-step, search-based) and form our main comparison; LayerRoute and LayerDrop occupy two different, coarser regimes and are reported as a supplemental analysis rather than head-to-head peers.

Task	Scale	Vanilla	ConfLayers	SWIFT
GSM8K	0.5B	0.180 [0.14–0.26]	0.147 [0.13–0.18]	0.303 [0.27–0.36]
GSM8K	1.5B	0.413 [0.38–0.46]	0.077 [0.05–0.10]	0.307 [0.29–0.33]
CNN/DM	0.5B	0.169 [0.167–0.174]	0.178 [0.175–0.184]	0.190 [0.185–0.196]
CNN/DM	1.5B	0.204 [0.193–0.213]	0.215 [0.204–0.222]	0.219 [0.205–0.230]

Table 2: Accuracy, mean over 3 seeds (range in brackets). GSM8K is exact-match accuracy; CNN/DailyMail is ROUGE-L. Bold marks the best method per cell.

Figure 1: Accuracy across the full 2 (task)

\times

2 (scale)

\times

3 (method) matrix. Error bars show the observed 3-seed range. ConfLayers is the consistent worst performer in every cell.

Task/Scale	Method	Pure inference (ms)	Search overhead	Wall-clock total (ms)
GSM8K/0.5B	ConfLayers	9194	2.2%	10338.5
GSM8K/0.5B	SWIFT	8771	15.1%	11564.5
GSM8K/1.5B	ConfLayers	12366	1.3%	12485.4
GSM8K/1.5B	SWIFT	9913	12.4%	11628.8
CNN-DM/0.5B	ConfLayers	10889	1.2%	10231.4
CNN-DM/0.5B	SWIFT	8610	8.7%	10598.2

Table 3: Per-query wall-clock latency (ms), mean over 3 seeds. Both methods shown decomposed into pure inference and online-search overhead (Section 6.2.1 ).

Figure 2: Per-query wall-clock latency across the full matrix.

Figure 3: Search-overhead decomposition. Once online-search cost is separated from pure model-forward cost, SWIFT is the faster inference engine in every cell, despite appearing slower on unsplit wall-clock time in three of four cells.

Task	Scale	LayerRoute acc.	LayerRoute speedup	LayerDrop acc.	LayerDrop speedup
GSM8K	0.5B	0.137 [0.11–0.16]	1.08 $\times$	0.010 [0.00–0.02]	1.30 $\times$
GSM8K	1.5B	0.003 [0.00–0.01]	1.32 $\times$	0.060 [0.05–0.07]	1.25 $\times$
CNN/DM	0.5B	0.102 [0.099–0.107]	1.33 $\times$	0.091 [0.085–0.097]	1.29 $\times$
CNN/DM	1.5B	0.150 [0.139–0.165]	1.09 $\times$	0.134 [0.131–0.136]	1.25 $\times$

Table 4: Supplemental analysis: accuracy and speedup, mean over 3 seeds (range in brackets), LayerRoute and LayerDrop versus a genuine full-model baseline. GSM8K is exact-match; CNN/DailyMail is ROUGE-L. Speedup

>1

indicates the routed/pruned pass is faster than the full-model baseline.

Figure 4: Supplemental results. Left: accuracy under genuine gating and a genuine full-model baseline. Right: speedup versus that same genuine baseline, including genuine inference-time compute skipping (Section 7.2 ).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: ConfLayers: top, the full layer stack; bottom, detail of the per-layer confidence check. A per-layer confidence estimate $c_{l}$ is thresholded against $\tau_{l}$ , discovered online via Bayesian optimization; pure inference and search overhead are logged and reported separately (Section 6.2.1 ).

Panel of Figure 6 — Figure 6: SWIFT: the target model self-drafts by skipping a searched subset $S$ of its own layers, then the full model verifies all draft positions in a single batched pass; the longest accepted prefix is kept and the rest resampled.

Panel of Figure 7 — Figure 7: LayerRoute: top, the full 24-layer stack; bottom, detail of a single gated layer $l$ . The router computes a scalar score from the mean-pooled input hidden state $\bar{h}_{l-1}$ , thresholds it into a hard gate $z_{l}$ via a straight-through estimator, and the layer output is either the LoRA-adapted block’s output ( $z_{l}{=}1$ ) or an unchanged residual pass-through ( $z_{l}{=}0$ ), matching Equations 7 – 8 . Discussed in the supplemental analysis, Section 7 .

Panel of Figure 8 — Figure 8: LayerDrop: layers are stochastically dropped during training to make the backbone and LoRA adapters robust to any layer’s removal; at inference, one fixed set of layers, chosen once from training statistics, is pruned identically for every input — no router, no per-input adaptivity. Discussed in the supplemental analysis, Section 7 .

Scale	Seed	Vanilla	ConfLayers	SWIFT
0.5B	2024	0.26	0.13	0.36
	42	0.14	0.18	0.28
	123	0.14	0.13	0.27
1.5B	2024	0.40	0.05	0.29
	42	0.46	0.08	0.33
	123	0.38	0.10	0.30

Appendix

Table 5: GSM8K exact-match accuracy at each of the three evaluation seeds, main comparison.

Scale	Seed	Vanilla	ConfLayers	SWIFT
0.5B	2024	0.167	0.175	0.190
	42	0.174	0.184	0.196
	123	0.167	0.178	0.185
1.5B	2024	0.193	0.204	0.205
	42	0.206	0.218	0.223
	123	0.213	0.222	0.230

Appendix

Table 6: CNN/DailyMail ROUGE-L at each of the three evaluation seeds, main comparison.

Scale	Seed	LayerRoute	LayerDrop
0.5B	2024	0.14	0.00
	42	0.16	0.02
	123	0.11	0.01
1.5B	2024	0.00	0.07
	42	0.01	0.05
	123	0.00	0.06

Appendix

Table 7: GSM8K exact-match accuracy at each of the three evaluation seeds, supplemental analysis.

Scale	Seed	LayerRoute	LayerDrop
0.5B	2024	0.100	0.085
	42	0.099	0.092
	123	0.107	0.097
1.5B	2024	0.146	0.136
	42	0.139	0.131
	123	0.165	0.134

Appendix

Table 8: CNN/DailyMail ROUGE-L at each of the three evaluation seeds, supplemental analysis.

Layer Skipping in Language Model Decoding

See all 11 papers

LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference
Sep 12, 2026Prateek Kumar SikdarEfficient Transformer Inference LLM Inference Acceleration
Accenture
vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains
Sep 29, 2026Wei Da, Yavuz Ferhatosmanoglu, Evangelia KalyvianakiLLM Inference Acceleration LLM Serving
The University of Cambridge, United Kingdom
ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding
Apr 16, 2026Walaa Amer, Uday das, Fadi KurdahiSpeculative Decoding LLM Inference Acceleration
Department of Electrical Engineering and Computer Sciences, University of California Irvine, Irvine, USA · Advanced Micro Devices, San Jose, USA

Sep 12, 2026J/K move · Enter open · S save

LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference

Prateek Kumar Sikdar

Accenture

Efficient Transformer Inference LLM Inference Acceleration Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

We introduce LayerRoute, a parameter-efficient method for adaptive transformer layer-skipping that combines per-layer hard-gated routing (trained via a straight-through estimator) with joint LoRA fine-tuning. LayerRoute augments each of the 24 transformer blocks in Qwen2.5-0.5B-Instruct with a lightweight per-layer router (~21.5K parameters) and LoRA adapters (rank 8, ~1.08M parameters), training both jointly under a gate-regularized language-modeling objective. Across 10 independently-seeded training runs, LayerRoute converges to an identical skip-pattern structure in every run - a consistent set of 9 middle layers (8-16) becomes skip-eligible in all 10 seeds - and delivers genuine, verified wallclock speedup in every run (1.02x-1.06x, mean 1.04x). Quality is preserved or improved in every configuration tested: joint LoRA adaptation yields a perplexity improvement over the unmodified backbone in all 10 seeds (mean delta = -1.16 and -1.11 across the two evaluation splits used). We further verify the router performs genuine, non-trivial per-input computation: gate decisions in skip-eligible layers change the actual skip/run outcome for 87-100% of held-out samples, confirming real input-dependent routing rather than a fixed pruning pattern. LayerRoute trains in under 7 minutes on a single A100 and adds negligible overhead beyond the routing decision itself. We report our full reproducibility methodology, including a systematic diagnostic investigation into what determines the router's per-input decisions, as part of this work.

Abstract

Layer Skipping in Language Model Decoding

Explore similar work

LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference

vSkipper: Translating Dynamic Layer Skipping into LLM Serving Gains

ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding