Programs-of-Layers in LLMs through the Lens of Cortical Areas

Justus Westerhoff; Stephan Olbrich; Hatem Oraby; Matthew Evan Larkum; Felix Alexander Gers

cs.AISep 25, 2026

Programs-of-Layers in LLMs through the Lens of Cortical Areas

Authors: Justus Westerhoff, Stephan Olbrich, Hatem Oraby, Matthew Evan Larkum, Felix Alexander Gers

Organizations: Berliner Hochschule für Technik (BHT), Berlin, Germany · Humboldt-Universität zu Berlin, Berlin, Germany

Abstract

Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human brain does not work this way: using the thalamus as a central hub, it routes information flexibly to all regions of the cortex according to demand. Li et al. (2026) recently showed, with a system they call program-of-layers (PoLar), that transformers can be given an analogous flexibility if their layers are treated as a library of functions rather than a fixed sequence. Performance improves over the standard forward pass when each input is dynamically routed through an adaptive sequence of skipped or repeated contiguous layer blocks. We reconstructed PoLar's diagnostic MCTS in more detail than the original paper and applied it across 5 models. We reproduced several of PoLar's findings: skipping outperformed the standard pass, repeating outperformed skipping, and combining both outperformed either alone. Shorter programs sufficed for easier questions, while harder questions required more layer repeats. However, we failed to replicate the main claim regarding their learned router for single-shot inference: its top-ranked prediction consistently collapsed back to the standard pass, even though its top-k predicted programs, taken together, did show a real accuracy gain. Beyond reproduction, we find that a small number of generic programs are enough to solve most of the questions. We also provide a much deeper analysis of these programs' structure and robustness: for example, we found that programs that correct errors are highly brittle: undoing even a single edit inside a program typically breaks the correction. Connecting this to the brain's routing mechanisms, PoLar mirrors principles of thalamo-cortical coordination between cortical-area-like transformer layers. We publicly release the code at https://datexis.github.io/RE-PoLar/

Figures & tables

Figure 1: A PoLar , predicted by a lightweight router, executes a schedule of skip , keep , and repeat operations over frozen, pretrained transformer layers (left). We read this coordinating role as functionally analogous to the thalamus routing computation across cortical areas (right). A transformer’s layer order is fixed only by training, not by necessity: a model trained to run its layers in order can nonetheless be usefully executed out of that order at inference time, a flexibility the brain already has by default.

Finding	Description	Status
\ftag F0	Reproducing Li et al. ’s baseline numbers	↯
\ftag F1	Skip & Repeat combined beats either alone	✓
\ftag F2	Occam’s razor: most samples have shorter valid programs than a standard forward pass	✓
\ftag F3	Harder inputs need larger programs (test-time scaling via recurrence)	✓
\ftag F4	Valid programs concentrate on short, mostly single-recurrence segments	↯
\ftag F5	A small menu of programs generalizes across many questions	$\bigstar$

Table 1: Every finding in section 4 , checked against Li et al. (2026) ’s own study. ✓ = reproduced, ↯ = not reproduced (we tested it and got a contradicting result),

\bigstar

= a new finding of ours with no equivalent in the original paper to check against. \ftag F1– \ftag F4 are Li et al. ’s own diagnostic Findings 1–4.

Panel of Figure 2 — Figure 2: Simply repeating a well-chosen middle block (layers 18–22), no retraining, no other intervention, already improves Qwen3-8B’s accuracy on a 6-domain subset of MMLU-Pro ( Wang et al., 2024 ) . The representational pattern in (a) and the behavioral pattern in (b) agree on two boundaries: layer 6, where mixing layers from before and after collapses accuracy under both repeat and skip, and layer 25, past which layers grow mutually redundant, so skipping there turns from harmful to roughly neutral.

Figure 3: One simulation of algorithm 1 , traced end to end on a toy tree (

D{=}7

). (a)

N_{0}

is already widening-capped, so selection descends by

\arg\max

UCB into child

B

rather than expanding

N_{0}

further. (b)

B

is still under its own cap (

1

2

children allowed at

v{=}1

), so SelectExpand expands it directly, popping a random untried edit to create

G

. (c)

x

is scored under

G

’s edit set (

\{\textsf{skip}[5,6),\,\textsf{repeat}[2,4)\!\times\!2\}

), producing reward

r

. (d)

r

backpropagates along the traversed chain

N_{0}\!\to\!B\!\to\!G

, and the global counter

V

increments once.

Model			DM-1		DM-2		DM-3		DM-4		DM-5
Qwen1.5-MoE-A2.7B	Base	$\tau{=}0$	22.6	(-15.4)	24.2	(-0.4)	21.2	(+0.6)	18.0	(+2.8)	12.2	(-2.0)
		p@1	21.6	(-13.8)	24.2	(+2.2)	21.2	(+6.6)	18.2	(+5.6)	11.4	(+4.8)
		p@5	34.4	(-5.6)	36.2	(+10.6)	30.8	(+12.2)	28.6	(+13.6)	21.2	(+9.4)
	MCTS		73.0	(-0.2)	71.0	(+11.3)	67.8	(+17.4)	63.2	(+25.0)	43.9	(+11.5)
	Router	p@1	22.6	(-21.2)	24.2	(-2.0)	21.2	(-0.8)	18.0	(+0.8)	12.2	(-3.2)
	Router	p@5	35.6	(-26.4)	37.4	(-6.6)	34.2	(+1.2)	30.6	(+5.2)	19.0	(-4.2)

Table 2: \ftag F0 \ftag F8 The reproduction gap against Li et al. (2026) ’s own numbers is model-dependent (colored

\Delta=

ours

-

their matching tables), and Router p@1 equals Base (

\tau{=}0

) exactly on every model and difficulty: the router always predicts identity. Accuracy (%) by DART-Math difficulty: Base

\tau{=}0

/p@1/p@5 and Router p@1/p@5 are pass@

k

on the

500

-question held-out test split per difficulty; MCTS is the share of samples that have at least one valid program, computed over the full

2{,}000

-question pool.

Panel of Figure 4 — Figure 4: \ftag F2 Occam’s razor on Qwen3-8B, by DART-Math difficulty: (a) the share of all questions admitting a program shorter than the standard forward pass; (b) that shortest program’s actual executed length, bar = mean, black dot = median, dotted line marks the identity length $D$ . Same measurement as figs. 13 and 14 , all 5 models, in the appendix.

Panel of Figure 5 — Figure 5: \ftag F3 Of every question that is solvable at all, the share that needed recurrence (solid) or needed skipping (dashed) to be solved. We are using the same per-model panel as Li et al. (2026) for our reproduction.

Panel of Figure 6 — Figure 6: \ftag F3 Qwen3-32B: share of shortest-valid programs that skip (a) or repeat (b) each layer, by layer index and DART-Math difficulty. Easier difficulties concentrate more on skipped layers; harder difficulties spread repeats across more of the stack. Same measurement as figs. 17 and 18 , all 5 models, in the appendix.

Panel of Figure 7 — Figure 7: \ftag F4 Program-editing structure by segment length, grouped by model.

Panel of Figure 8 — Figure 8: \ftag F5 Coverage and visualization of the menu behavior.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Variant	Grader	Accuracy	\errtag unboxed	\errtag prompt template echo
Qwen1.5-MoE-A2.7B	polar_default	PoLar	16.3%	0.3%	2.70%
	drllm (faithful)	mathruler	2.0% ∗∗∗	89.2%	0.00%
	drllm_chat_prefill	mathruler	15.6%	0.0%	0.00%
	polar_oneshot	PoLar	19.3% ∗∗∗	2.0%	0.00%
	drllm_chat_oneshot	mathruler	13.2% ∗∗∗	18.3%	0.00%
	drllm_chat_oneshot_prefill	mathruler	15.3% ∗	0.0%	0.00%

Appendix

Table 4: Accuracy, \errtag unboxed rate, and \errtag prompt template echo rate, all

6\times 6=36

model/prompt-variant combinations, on our

10{,}000

-query DART-Math split ( section A.1 ). Bold = best accuracy for that model. Grader : polar_default / polar_oneshot use PoLar , Li et al. (2026) ’s own last- \boxed{} -match DART-Math extraction (byte-identical to their vendored copy); the four drllm variants use mathruler , Heakl et al. ’s own grading library ( Heakl et al., 2025 ) – each graded the way its own source protocol would grade it. polar_oneshot repeats the instruction-and-answer block twice inside the 50-token budget, risking a hallucinated third, incomplete \boxed{...} span that a naive last-match extraction would grab instead of the real answer; we truncate before grading at the first complete span, so it is, in effect, graded on the first \boxed{...} span rather than the last ( polar_default ’s single, unrepeated block never needs this). Superscripts are McNemar’s paired test against that model’s own polar_default row: ∗∗∗

p<0.001

, ∗

p<0.05

, unmarked = not significant. Table 5 gives the exact wording of every variant swept here.

Variant	Template
polar_default	“ Solve the following math problem and output ONLY the final answer directly, formatted strictly as \boxed{ANSWER}. ### Problem Start <q> ### Problem End Answer: ”
drllm (faithful) †	User: “ Question: <q> The final answer MUST BE put in \boxed{} and no explanation. ”
drllm_chat_prefill †	User: “ Question: <q> The final answer MUST BE put in \boxed{} and no explanation. ” Assistant: “ \boxed{ ”
polar_oneshot	“ Solve the following math problem and output ONLY the final answer directly, formatted strictly as \boxed{ANSWER}. ### Problem Start What is 1+1? ### Problem End Answer: \boxed{2} ” “ Solve the following math problem and output ONLY the final answer directly, formatted strictly as \boxed{ANSWER}. ### Problem Start <q> ### Problem End Answer: ”
drllm_chat_oneshot †	User: “ Question: What is 1+1? The final answer MUST BE put in \boxed{} and no explanation. ” Assistant: “ \boxed{2} ” User: “ Question: <q> The final answer MUST BE put in \boxed{} and no explanation. ”
drllm_chat_oneshot_prefill †	User: “ Question: What is 1+1? The final answer MUST BE put in \boxed{} and no explanation. ” Assistant: “ \boxed{2} ” User: “ Question: <q> The final answer MUST BE put in \boxed{} and no explanation. ” Assistant: “ \boxed{ ”

Appendix

Table 5: The 6 prompt variants swept in table 4 (question text abbreviated as <q> ), every message spelled out in full. Rows marked

\dagger

are chat-templated ( section A.3 ); the literal chat-template markup itself (special tokens etc.) is tokenizer-specific and omitted, but every turn’s exact text content is shown.

Model	no chat template	chat template	$\Delta$ (pp)
Qwen1.5-MoE-A2.7B	$22.5\%$	$7.0\%$	$-15.5$
Qwen2.5-3B	$22.0\%$	$20.0\%$	$-2.0$
Qwen2.5-7B	$44.0\%$	$36.5\%$	$-7.5$
LLaMA-3.2-3B	$20.5\%$	$2.5\%$	$-18.0$
Qwen3-8B	$34.0\%$	$33.5\%$	$-0.5$
Qwen3-32B	$40.0\%$	$1.0\%$	$-39.0$

Appendix

Table 6: Chat template ablation: Base (

\tau{=}0

) accuracy with vs. without the model’s native chat template, prompt content held fixed at polar_oneshot ,

200

-question pooled sample (seed

42

) per model. “chat template” adds the model family’s own system message where Li et al. (2026) ’s code applies one (Qwen2.5-family, Qwen1.5-MoE-A2.7B); Qwen3-8B’s chat setting has no system message, matching Li et al. ’s own polar/eval.py found at ( https://github.com/tianyi-lab/PoLar/blob/main/polar/eval.py ).

Model	Metric	DM-1		DM-2		DM-3		DM-4		DM-5
LLaMA-3.2-3B	$\tau{=}0$	18.8	(-23.6)	14.8	(-13.8)	17.2	(-10.0)	20.0	(-7.6)	24.4	(-4.2)
	p@1	19.2	(-21.4)	14.8	(-13.8)	16.2	(-11.2)	19.0	(-9.0)	24.4	(-4.8)
	p@2	22.6	(-21.4)	18.0	(-17.0)	20.2	(-9.4)	21.2	(-9.2)	28.2	(-4.6)
	p@3	24.6	(-21.6)	21.2	(-17.6)	21.8	(-9.4)	25.0	(-6.4)	32.0	(-2.4)
	p@4	25.2	(-21.8)	23.4	(-18.0)	23.4	(-8.8)	26.4	(-5.8)	34.6	(-0.2)
	p@5	27.2	(-20.4)	25.0	(-18.2)	25.2	(-7.6)	28.0	(-4.8)	35.8	(+0.2)

Appendix

Table 7: Full Base (

\tau{=}0

) and Base (sampling) p@1–p@5 breakdown (%) on our reconstructed DART-Math split ( section A.1 ),

500

questions/difficulty, all 6 models, polar_oneshot , no chat template ( section A.3 ). Colored

\Delta=

ours

-

Li et al. (2026) ’s matching table, where one exists; Qwen2.5-7B and Qwen3-32B are new here (see \ftag F0 for the reproduction-gap pattern).

Model	MMLU-Pro (overall)	ASDiv		MAWPS
LLaMA-3.2-3B	31.4	64.4	(-14.0)	70.6	(-0.9)
Qwen1.5-MoE-A2.7B	24.4	52.7	(-6.4)	62.3	(+20.6)
Qwen2.5-3B	35.2	68.1	(+18.6)	62.9	(+26.7)
Qwen2.5-7B	42.3	75.3		67.9
Qwen3-8B	42.1	78.7	(+11.6)	71.5	(+19.4)
Qwen3-32B	48.7	77.0		77.7

Appendix

Table 8: Base (

\tau{=}0

) accuracy (%): MMLU-Pro (overall, all 14 categories, forced-choice logit scoring) and OOD transfer targets ASDiv (

n{=}2{,}305

) and MAWPS (

n{=}520

, generation-based, same prompt/mechanism as table 7 ), all 6 models. Colored

\Delta=

ours

-

Li et al. (2026) ’s matching table, where one exists.

Model	pass@ $1$	pass@ $2$	pass@ $3$	pass@ $4$	pass@ $5$
Qwen1.5-MoE-A2.7B	$19.6\%$	$21.4\%$	$23.6\%$	$27.9\%$	$31.4\%$
Qwen2.5-3B	$22.2\%$	$25.2\%$	$29.0\%$	$32.1\%$	$34.6\%$
Qwen2.5-7B	$37.8\%$	$40.6\%$	$43.3\%$	$47.4\%$	$50.2\%$
Qwen3-8B	$33.2\%$	$36.3\%$	$38.6\%$	$41.2\%$	$43.6\%$
Qwen3-32B	$35.0\%$	$40.8\%$	$45.2\%$	$48.9\%$	$52.1\%$

Appendix

Table 9: Router ’s real online-execution pass@

1

–

5

, Drop-CE recipe, averaged across all

5

DART-Math difficulty splits, all

5

models in this study; the intermediate pass@

2

–

4

ranks here are not in table 2 , which reports only pass@

1

/pass@

5

per difficulty level rather than pooled across difficulty. Router ’s pass@

1

equals Base (

\tau{=}0

) exactly: it always predicts identity. The pass@

5

gap over pass@

1

is the real, non-random rescue signal documented in section 4.4 , not something Router ’s own top-

1

choice can access.

Model	DM	Solvable	No-id %	Examples	Id %	Keep %
Qwen1.5-MoE-A2.7B	1	$923$	$68.1\%$	$4{,}196$	$7.0\%$	$80.1\%$
	2	$883$	$68.2\%$	$4{,}037$	$6.9\%$	$79.9\%$
	3	$825$	$74.2\%$	$3{,}380$	$6.3\%$	$80.0\%$
	4	$801$	$67.8\%$	$3{,}592$	$7.2\%$	$80.2\%$
	5	$553$	$73.6\%$	$1{,}946$	$7.5\%$	$80.4\%$
Qwen2.5-3B	1	$1{,}032$	$60.8\%$	$8{,}538$	$4.5\%$	$84.0\%$

Appendix

Table 10: Router training-data composition, Drop treatment, full per-difficulty breakdown,

1{,}250

-question train slice (data composition depends only on the data treatment, not the loss function). Solvable = questions with

\geq\!1

MCTS -verified-correct program; no-id % = share of those with no valid identity program at all; examples = the resulting number of training program-examples; id % / Keep % as table 3 .

	Validation loss		Pass@ $5$
Model	Drop-CE	Crop-CE	Drop-CE	Crop-CE
Qwen1.5-MoE-A2.7B	$1.071$	$1.086$	$31.4\%$	$30.5\%$
Qwen2.5-3B	$1.024$	$1.034$	$34.6\%$	$32.7\%$
Qwen2.5-7B	$1.088$	$1.105$	$50.2\%$	$49.6\%$
Qwen3-8B	$1.008$	$1.032$	$43.6\%$	$44.9\%$
Qwen3-32B	$0.892$	$0.915$	$52.1\%$	$56.4\%$

Appendix

Table 11: Cross-model recipe comparison: validation loss (winning hyperparameter configuration) and real online-execution pass@

1

/pass@

5

, averaged across all

5

difficulty splits, for the four comparable variants (Drop/Crop

\times

CE/focal). Drop-CE is the recipe used throughout the rest of this paper. Drop-CE(pass@

5

) below is corrected to the real full-

500

-question test split (same backfill as tables 9 and 2 ); Crop-CE(pass@

5

) still uses the original

n{=}250

half, not yet backfilled, so that one column is not apples-to-apples with the rest of the table (verified against the repo’s own result files: no full-

500

Crop-CE backfill exists yet).

Model	Split	DM-1	DM-2	DM-3	DM-4	DM-5
Qwen1.5-MoE-A2.7B	train	$8$	$10$	$9$	$5$	$9$
	val	$8$	$9$	$8$	$5$	$9$
	test	$8$	$9$	$9$	$5$	$9$
Qwen2.5-3B	train	$12$	$13$	$11$	$13$	$11$
	val	$8$	$10$	$9$	$13$	$11$
	test	$10$	$11$	$10$	$13$	$10$

Appendix

Table 12: The \ftag F9 small-fixed-menu property holds on every split: the number of distinct programs appearing anywhere across all questions’ top-

5

decoded candidate sets, Drop-CE recipe, per model and DART-Math difficulty level, on training (

n{=}1{,}250

/difficulty), validation (

n{=}250

/difficulty), and test (

n{=}500

/difficulty) data.

Stage	GPU-hours	Architecture
DART-Math track
MCTS (5 difficulties $\times$ 5 models)	$\sim$ 800	B200
Router training & recipe selection ( section A.8 ), incl. the test-split backfill above	$\sim$ 865	A100
MMLU-Pro track
MCTS (14-domain, Qwen3-8B)	$\sim$ 40	A100
Router training & recipe selection	$\sim$ 150	A100

Appendix

Table 13: GPU-hours by what the compute went toward (rounded).

Model	\errtag wrong final answer	\errtag unboxed	\errtag prompt template echo	\errtag malformed boxed	Total
Qwen1.5-MoE-A2.7B	7,953 (98.7%)	87 (1.1%)	1 (0.0%)	16 (0.2%)	8,057
Qwen2.5-3B	7,668 (99.0%)	74 (1.0%)	0 (0.0%)	0 (0.0%)	7,742
Qwen2.5-7B	6,299 (100.0%)	0 (0.0%)	0 (0.0%)	3 (0.0%)	6,302
Qwen3-8B	6,599 (99.9%)	0 (0.0%)	2 (0.0%)	4 (0.1%)	6,605
Qwen3-32B	2,548 (39.1%)	3,857 (59.3%)	102 (1.6%)	2 (0.0%)	6,509
All models	31,067 (88.2%)	4,018 (11.4%)	105 (0.3%)	25 (0.1%)	35,215

Appendix

Table 14: Error-category breakdown for every identity-wrong record, all 5 DART-Math difficulties pooled, per model. Taxonomy and methodology above.

Model	Related $n$	Related rescued	Unrelated $n$	Unrelated rescued
Qwen1.5-MoE-A2.7B	6,653	57.9%	1,215	41.3%
Qwen2.5-3B	6,822	64.3%	784	36.9%
Qwen2.5-7B	5,661	66.3%	587	37.6%
Qwen3-8B	5,841	63.5%	700	43.6%
Qwen3-32B	2,355	83.1%	185	62.7%
All models	27,332	64.6%	3,471	41.3%

Appendix

Table 15: Rescue rate (share of \errtag wrong final answer records for which MCTS found at least one valid fixing program) by closeness label, all 5 difficulties pooled, per model. Closeness methodology above.

Panel of Figure 10 — Figure 10: (a) Accuracy gained by allowing skip and repeat together, over the standard forward pass alone, at each DART-Math difficulty level (1 = easiest, 5 = hardest); same Gain column as table 16 above. (b) Share of questions with at least one valid program, as more extra latent execution steps (recurrence) are allowed beyond the standard forward pass depth, pooled over all difficulties, the recurrence-budget half of \ftag F3 . One line per model in both panels.

Model	Difficulty	Base	Skip	Repeat	Skip&Repeat	Gain
Qwen1.5-MoE-A2.7B	DM-1	22.7%	34.8%	67.1%	73.0%	+50.2
	DM-2	22.8%	35.8%	64.8%	71.0%	+48.2
	DM-3	19.6%	33.5%	61.4%	67.8%	+48.3
	DM-4	20.0%	32.2%	58.4%	63.2%	+43.3
	DM-5	12.2%	19.4%	39.1%	43.9%	+31.7
Qwen2.5-3B	DM-1	31.6%	54.8%	76.5%	82.7%	+51.0

Appendix

Table 16: Combining skip and repeat always beats either alone ( \ftag F1 ): Skip&Repeat’s gain over Base ranges from

+31.7

+68.7

accuracy points across all 5 models and every DART-Math difficulty. Base = identity already correct. Skip = identity or a skip-only valid program exists. Repeat = identity or a repeat-only valid program exists. Skip&Repeat = identity or any valid program exists. Gain = Skip&Repeat

-

Base .

Model	$n$ segments	Single-layer	At-most-two-layers
Qwen1.5-MoE-A2.7B	$9{,}097$	$46.5\%$	$73.8\%$
Qwen2.5-3B	$12{,}575$	$32.6\%$	$58.4\%$
Qwen2.5-7B	$12{,}410$	$37.4\%$	$63.1\%$
Qwen3-8B	$13{,}503$	$32.7\%$	$58.5\%$
Qwen3-32B	$18{,}825$	$21.1\%$	$42.7\%$
PoLar (reference)	–	$54.5\%$	$>66.7\%$

Appendix

Table 17: Segment-length share of the shortest valid program per solved question, pooled over all 5 DART-Math difficulties, per model: the \ftag F4 comparison. PoLar reports

54.5\%

single-layer and

>66.7\%

(over two-thirds) at-most-two-layers.

Panel of Figure 12 — Figure 12: The per-model, per-difficulty view behind \ftag F5 : menu generalization degrades with difficulty on every model, coverage still climbing at $K{=}100$ on the hardest difficulty but nearly flat by $K{=}30$ on the easiest. Menu coverage of solved questions as the menu size $K$ grows, one line per DART-Math difficulty, one panel per model. $K$ on a log scale.

Panel of Figure 13 — Figure 13: The per-model, pooled-over-difficulty counterpart to \ftag F2 (Occam’s razor): share of all questions (not split by whether the standard forward pass already answered correctly) admitting a program shorter than it, by DART-Math difficulty. PoLar ’s F2 uses the C $\to$ C/W $\to$ C split of this measure instead, pooled over difficulty. The prose numbers for that split are in section 4.2 and section 4 . Qwen3-8B is omitted here since it already has its own dedicated figure in the main body ( fig. 4 ). One panel per remaining model, all four sharing the leftmost panel’s y-axis (0–100%, same scale throughout).

Panel of Figure 14 — Figure 14: The \ftag F2 split’s program-length counterpart. Same split as fig. 13 , but showing the shortest valid program’s actual executed length rather than just whether it beats identity. The dotted line marks the identity (standard forward pass) length $D$ for reference. Qwen3-8B again omitted ( fig. 4 in the main body). One panel per remaining model, by DART-Math difficulty, all four sharing the leftmost panel’s y-axis.

Panel of Figure 15 — Figure 15: The baseline/rescued/unsolved split behind \ftag F5 ’s menu coverage. Of every question: the share already answered correctly by the standard forward pass (baseline); the share the standard forward pass got wrong but for which the search found at least one valid program (rescued); the share for which nothing valid was found (unsolved). One panel per model, by DART-Math difficulty, all five sharing the leftmost panel’s y-axis.

Panel of Figure 16 — Figure 16: The \ftag F1 solvability split by operation class. Of every solved question, whether the standard forward pass already works (identity), a program using only skips works, a program using only repeats works, and a program using both works (shortest-valid-program lens). These four checks are independent, so a single question can count toward more than one bar and the four bars do not need to sum to 100%. One panel per model, by DART-Math difficulty, all five sharing the leftmost panel’s y-axis.

Panel of Figure 17 — Figure 17: The per-model layer-position evidence behind \ftag F3 : the earliest layers are almost never skipped in any model, consistent with the early load-bearing region we find independently in fig. 2 . This panel shows the share of shortest-valid programs that skip each layer, by layer index and DART-Math difficulty. Qwen3-32B is omitted here since it already has its own dedicated figure in the main body ( fig. 6 ).

Panel of Figure 18 — Figure 18: The repeat-side counterpart to fig. 17 , also part of the \ftag F3 layer-position evidence. Share of shortest-valid programs that repeat each layer, by layer index and DART-Math difficulty. Qwen3-32B again omitted ( fig. 6 in the main body).

	pass@ $1$	pass@ $2$	pass@ $3$	pass@ $4$	pass@ $5$
Identity	$43.9\%$	–	–	–	–
Random baseline	$17.0\%\pm 9.7\%$	$25.9\%\pm 8.8\%$	$36.1\%\pm 6.4\%$	$40.4\%\pm 7.8\%$	$45.3\%\pm 8.8\%$
Router (Drop-CE)	$43.9\%$	–	–	–	$62.6\%$ (identical, all 5)

Appendix

Table 18: The Drop-CE router beats a random-valid-program control at pass@

5

(

62.6\%

vs.

45.3\%\pm 8.8\%

), but identically across all 5 independently seeded retrainings, a structurally fixed margin, not per-question adaptation. Qwen3-8B,

2{,}800

-question held-out test split, same online-execution grading protocol both rows. “Random baseline” draws

5

shared valid programs from the router’s action grammar, independent of training, repeated over

5

seeds (mean

\pm

std). Drop-CE router retrains that recipe’s winning hyperparameter configuration

5

independent times (weight-init/data-order seed only); all

5

runs give the identical figure.

Transformer Inference

Transformer inference studies how trained Transformer models produce outputs efficiently, accurately, and securely across hardware and context lengths.

See all 57 papers

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
Jun 4, 2026Ziyue Li, Yang Li, Tianyi ZhouLLM Inference LLM Inference Acceleration
University of Maryland, College Park, MD, USA · 2MBZUAI, Abu Dhabi, UAE.
LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference
Sep 12, 2026Prateek Kumar SikdarEfficient Transformer Inference LLM Inference Acceleration
Accenture
MACRO: Markov Chain Routing of Transformer Layers
Aug 6, 2026Paweł Batorski, Abtin Pourhadi, Akylgali Aitaza +2Markov Models LLM Inference Acceleration
Heinrich Heine University Düsseldorf · Jagiellonian University · IDEAS Research Institute

Jun 4, 2026J/K move · Enter open · S save

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

Ziyue Li, Yang Li, Tianyi Zhou

University of Maryland, College Park, MD, USA · 2MBZUAI, Abu Dhabi, UAE.

LLM Inference LLM Inference Acceleration LLM Reasoning

Open paper AskPDF arXiv

Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic-depth methods, often while executing fewer layers, and that these gains persist under out-of-distribution evaluation. Our results suggest that fixed-depth execution captures only a narrow subset of an LLM's latent reasoning capacity.

Abstract

Transformer Inference

Explore similar work

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference

MACRO: Markov Chain Routing of Transformer Layers