Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation

Yibei Guo; Rui Liu

cs.LGOct 7, 2026

Input-Blind Controls Produce Substantial Oracle Headroom for Layer Programs in Multiple-Choice Evaluation

Authors: Yibei Guo, Rui Liu

Organizations: College of Aeronautics and Engineering, Kent State University

Abstract

Adaptive computation aims to improve language-model inference by tailoring execution to each input. For layer programs, oracle evaluations use known answers to estimate the potential gain from this flexibility, before a practical selector is available. However, a gain from selection does not by itself explain why the chosen programs help. This study examines this distinction using 32 layer-skipping and repetition programs on two models and 4,413 multiple-choice items. The analysis compares their gains over a fixed action selected without the evaluation prompt with those of input-blind perturbations at the same sites, re-evaluating selections on another prompt. With shared option order, the controls give 10.2-11.8 and 15.6-19.4 percentage points of headroom on Qwen3-4B-Base and Llama-3.1-8B, exceeding the real programs' 9.0 and 10.1 in all three random-direction draws per model. They match answer-change rate only, and the ordering depends on the menu: in post hoc comparisons, real programs lead on Llama's repeat-only menu in every draw. A smaller KL-calibrated comparison, including an input-dependent control, favours real programs in point estimate, with inconclusive corrected tests. Fixed letter offsets produce headroom of similar scale. Rotating options sharply reduces both families' headroom, while leaving positive real-minus-control differences of 1.4-2.3 and 3.7-4.5 points; their magnitudes and statistical support depend on further adjustments and the reference. A supplementary generated-answer test finds that search-selected programs keep a 26.0-point advantage over programs selected for other problems after rewording, without a placebo comparison. These results show that substantial headroom can persist across prompts with shared option order without establishing a benefit specific to the selected layer computation; neither ordering against these controls identifies that benefit.

Figures & tables

Figure 1: Cross-prompt oracle comparison (Qwen3-4B-Base; original RD draw). (A) Each family offers 32 edits and the unmodified pass

a_{0}

. The illustrated real action repeats layers 11–14; RD-fixed preserves the unmodified execution and adds an input-independent update at the segment exit. Its direction, norm profile and scale are shared across items and prompts; calibration matches answer-change rate only, on separate data. (B) For each item, all accuracy-maximizing actions on Prompt 1 are retained and their correctness averaged on Prompt 2. Only the few-shot examples change; the question and ordered options stay the same. Target answer labels are used for scoring and are withheld from the model. The full study uses 4,413 items per model and averages both selection directions. (C) All 40 qualifying items for one post-hoc selected action pair:

a_{0}

is wrong on both prompts and both edits are correct on Prompt 1. The counts describe this conditional subset, not the full oracle menu. Coordinates are correct-option log-probability margins against the strongest incorrect option, on common linear

-3

5

axes; positive values indicate correctness. Arrows connect unmodified and edited outputs, not hidden states. Detail 01 enlarges the printed question’s tile without adding an observation ( Appendix G ).

Figure 2: Different explanations, identical observed margins. A and B illustrate alternative decompositions of one layer program’s effect. The white circles track question

i

: replacing

(g_{i},h_{i})

(g_{i}+\delta_{i},h_{i}-\delta_{i})

, with

\delta_{i}>0

, preserves

g_{i}+h_{i}

. With all other terms fixed, both points project to the same observed margin pair

(U_{i}^{1},U_{i}^{2})

, shown in green. Height represents the computation-specific contribution

h

, defined relative to the chosen class of non-specific perturbations. The surfaces are schematic, not fitted functions or successive computations; A/B are not real/placebo conditions. The prompts change the few-shot examples while preserving the question and ordered options. Equality concerns margins, not full token distributions.

Property	RD-fixed	RD-item	SH	Rot
Study menu	32 programs	3 programs	3 programs	3 programs
Effect-size match	answer changes	KL	KL	KL
Direction shared by all items	yes	no	no	no
Same perturbation on shared text	yes	yes	yes	no
Model-generated update	no	no	other input’s	own, rotated
Reads current input	no	no	no	yes

Table 1: Placebo definitions.

Qwen3-4B-Base
	Same prompt $G_{\mathrm{raw}}$		G repro : tie average		$G_{\mathrm{repro}}$ : $a_{0}$ kept
Family	Fixed	Rotated	Fixed	Rotated	Fixed	Rotated
Real	$29.1\,{\color[rgb]{0.22,0.22,0.22}[28.0,30.3]}$	$\mathbf{29.3}\,{\color[rgb]{0.22,0.22,0.22}[28.3,30.3]}$	$9.0\,{\color[rgb]{0.22,0.22,0.22}[8.1,9.9]}$	$\mathbf{-3.3}\,{\color[rgb]{0.22,0.22,0.22}[-4.0,-2.7]}$	$11.4\,{\color[rgb]{0.22,0.22,0.22}[10.5,12.3]}$	$\mathbf{1.3}\,{\color[rgb]{0.22,0.22,0.22}[0.7,2.0]}$
RD-fixed	$\mathbf{29.2}\,{\color[rgb]{0.22,0.22,0.22}[28.1,30.4]}$	$29.0\,{\color[rgb]{0.22,0.22,0.22}[28.0,30.0]}$	$\mathbf{11.8}\,{\color[rgb]{0.22,0.22,0.22}[10.9,12.8]}$	$-5.7\,{\color[rgb]{0.22,0.22,0.22}[-6.3,-5.1]}$	$\mathbf{13.9}\,{\color[rgb]{0.22,0.22,0.22}[13.0,14.8]}$	$-0.7\,{\color[rgb]{0.22,0.22,0.22}[-1.3,-0.1]}$
$\Delta$	$-0.1\,{\color[rgb]{0.22,0.22,0.22}[-1.0,0.8]}$	$0.3\,{\color[rgb]{0.22,0.22,0.22}[-0.5,1.1]}$	$-2.8\,{\color[rgb]{0.22,0.22,0.22}[-3.8,-1.9]}$	$2.3\,{\color[rgb]{0.22,0.22,0.22}[1.6,3.0]}$	$-2.5\,{\color[rgb]{0.22,0.22,0.22}[-3.5,-1.6]}$	$2.0\,{\color[rgb]{0.22,0.22,0.22}[1.3,2.7]}$
Llama-3.1-8B

Table 2: Large-menu study: headroom and design sensitivity (original RD draw).

Figure 3: Cross-prompt headroom depends on the action menu. Left: counts of skip and repeat programs, each on a common 0–16 scale. Every menu also includes

a_{0}

. Low-churn membership uses the real programs’ answer-change rates of at most 30% on independent calibration items; the paired RD menu uses the same program IDs within each model. Right: headroom over each family/menu/task’s development-selected static action, with fixed option order, tie averaging and equal task weights. Points and 95% task-stratified paired item-bootstrap intervals use 4,413 held-out items per model and the original RD draw. Menus overlap; the models share the program-grid rule but use different physical layer segments. Calibration matches churn only: program-level median KL(RD)/median KL(Real) ranges from 0.19 to 4.84 for Qwen and from 0.22 to 7.71 for Llama, measured against the unmodified full-vocabulary output at the answer position.

Figure 4: Option-order changes affect cross-prompt headroom. (A) The lunar-eclipse example from Figure 1 illustrates a nonzero cyclic rotation on prompt 2: correct content moves from D to A. This is a schematic intervention, not a new measured outcome for that item. (B) Same-prompt and cross-prompt headroom over each family’s development-selected static action. Open circles show shared order; filled circles show rotated order, with horizontal segments spanning the measured differences on a common numerical scale. Results use the full 32-program menu plus

a_{0}

, tie averaging, 4,413 held-out items per model and equal task weights. Intervals are 95% task-stratified paired item-bootstrap intervals (original RD draw). Zero denotes equality with the static reference, not a validated null for computation-specific effects. Figure 8 gives the mixture and reference sensitivity analyses, including Llama’s item-decoupled mixture contrast whose interval contains zero.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$a_{0},F,K$	Unmodified pass, action family, and number of edited candidates (excluding $a_{0}$ ).
$U_{i}^{v}(a),\mathrm{Acc}_{i}^{v}(a)$	Correct-letter margin and 0/1 correctness for item $i$ , prompt $v$ and action $a$ .
$A_{i}^{s},a_{S}$	Best-accuracy tie set on selection prompt $s$ ; static action selected without evaluation outcomes.
$G_{\mathrm{raw}},G_{\mathrm{repro}}$	Headroom over the static action when selection and scoring use the same prompt, or different prompts.
$C$	Fixed-order half-headroom contrast: $G_{\mathrm{repro}}(\text{{RD-fixed}})-\tfrac{1}{2}G_{\mathrm{repro}}(\mathrm{Real})$ .
$\Delta,\rho$	Real minus control headroom; control divided by real headroom.

Appendix

Table 3: Notation used in the comparisons.

Action	Placebo	Mean difference (nats) placebo $-$ real; ref. 0	Noise variance ratio placebo / real; ref. 1
skip 16–19	RD-item	$+0.19\,{\color[rgb]{0.22,0.22,0.22}[+0.14,+0.24]}$	$0.80\,{\color[rgb]{0.22,0.22,0.22}[0.72,0.91]}$
	SH	$+0.16\,{\color[rgb]{0.22,0.22,0.22}[+0.11,+0.21]}$	$1.08\,{\color[rgb]{0.22,0.22,0.22}[0.91,1.26]}$
	Rot	$+0.11\,{\color[rgb]{0.22,0.22,0.22}[+0.06,+0.16]}$	$0.96\,{\color[rgb]{0.22,0.22,0.22}[0.85,1.08]}$
repeat 16–19	RD-item	$-0.11\,{\color[rgb]{0.22,0.22,0.22}[-0.16,-0.06]}$	$0.92\,{\color[rgb]{0.22,0.22,0.22}[0.83,1.02]}$
	SH	$-0.05\,{\color[rgb]{0.22,0.22,0.22}[-0.09,-0.004]}$	$0.71\,{\color[rgb]{0.22,0.22,0.22}[0.65,0.78]}$
	Rot	$-0.08\,{\color[rgb]{0.22,0.22,0.22}[-0.13,-0.04]}$	$0.85\,{\color[rgb]{0.22,0.22,0.22}[0.76,0.94]}$

Appendix

Table 4: Mean effects and noise variances.

Action	Placebo	Target KL	Scale $c_{k}$	KL ratio (placebo / real)
Action	Placebo	Target KL	Scale $c_{k}$	Calibration	Evaluation
skip 16–19	RD-item	$0.496$	$1.607$	$1.011$	$1.007$
	SH	$0.496$	$1.014$	$1.026$	$1.015$
	Rot	$0.496$	$1.551$	$1.005$	$0.981$
repeat 16–19	RD-item	$0.0753$	$1.137$	$0.997$	$1.096$
	SH	$0.0753$	$0.980$	$0.997$	$1.011$

Appendix

Table 5: Calibration and matching check.

Analysis	Role and specification relative to outcomes
Initial half-headroom and equivalence hypotheses	Specified before development outcomes; the original joint power requirement stopped evaluation. No equivalence claim is made.
Half-headroom confirmation	Confirmation specified after development estimates and before untouched evaluation; the threshold, menu and scoring rules were retained.
Original descriptive checks	Unmodified-pass reference, original letter-shift diagnostic, menu-size curves and cluster sensitivity were planned.
Rotation and other diagnostic extensions	Exploratory relative to the original study: rotated options, alternative ties, fitted letter offsets, decoupled and mixture references, covariance tests, and later paired decompositions.
Two fresh RD draws	Robustness contrasts specified after original findings and before the new draws; evaluation items were reused.
Three-draw means and expanded submenus	Post hoc summaries of stored outputs, with conditional pointwise intervals and no new confirmation rule.

Appendix

Table 6: Role and timing of the analyses.

Figure 5: Answer-letter probabilities for the lunar-eclipse example (Qwen3-4B-Base, ARC-Challenge; original RD draw). Options are A: Earth–Moon–Sun, B: Earth–Sun–Moon, C: Moon–Sun–Earth, D: Moon–Earth–Sun (correct). Filled and open bars use different few-shot examples with identical question/options. The unmodified model predicts A on both; repeating layers 11–14 and its paired RD-fixed control predict D on both. Probabilities normalize over the four letters on a shared 0–1 scale. This selected example illustrates output changes, not population frequencies.

Qwen3-4B-Base
		$G_{\mathrm{repro}}$ (pp): chosen static		$G_{\mathrm{repro}}$ (pp): static $a_{0}$
Task	$\mathrm{Acc}(a_{0})$ (%)	Real	RD-fixed	Real	RD-fixed
MMLU-Pro	$45.2$	$12.7$	$\mathbf{16.3}$	$12.5$	$\mathbf{16.3}$
BBH	$51.5$	$11.0$	$\mathbf{15.4}$	$10.7$	$\mathbf{15.4}$
ARC-Challenge	$89.9$	$3.3$	$\mathbf{3.7}$	$2.8$	$\mathbf{3.1}$
Llama-3.1-8B

Appendix

Table 7: Large-menu study: results by task.

Qwen3-4B-Base
Menu ( $K$ )	Draw	Real	RD-fixed	$\Delta$	$C$
Repeats (16)	1	7.3 [6.5, 8.2]	7.2 [6.4, 8.0]	0.1 [−0.7, 1.0]	3.5 [2.8, 4.3]
	2	7.3 [6.5, 8.2]	8.8 [7.9, 9.6]	−1.4 [−2.3, −0.5]	5.1 [4.3, 5.9]
	3	7.3 [6.5, 8.2]	9.9 [9.0, 10.8]	−2.6 [−3.5, −1.7]	6.3 [5.5, 7.0]
Skips (16)	1	5.6 [4.8, 6.5]	9.8 [8.8, 10.7]	−4.1 [−5.1, −3.2]	6.9 [6.1, 7.8]
	2	5.6 [4.8, 6.5]	7.5 [6.6, 8.3]	−1.8 [−2.7, −1.0]	4.6 [3.9, 5.4]

Appendix

Table 8: Submenu headroom across draws: shared option order.

Qwen3-4B-Base
Menu ( $K$ )	Draw	Real	RD-fixed	$\Delta$
Repeats (16)	1	−1.1 [−1.7, −0.5]	−2.5 [−3.0, −1.9]	1.4 [0.7, 2.1]
	2	−1.1 [−1.7, −0.5]	−2.4 [−2.9, −1.8]	1.3 [0.6, 2.0]
	3	−1.1 [−1.7, −0.5]	−2.2 [−2.8, −1.7]	1.1 [0.5, 1.8]
Skips (16)	1	−6.7 [−7.3, −6.0]	−8.4 [−9.0, −7.8]	1.7 [1.0, 2.4]
	2	−6.7 [−7.3, −6.0]	−7.6 [−8.2, −6.9]	0.9 [0.2, 1.6]

Appendix

Table 9: Submenu headroom across draws: rotated option order.

Model	Source family	Source headroom (pp)	Offset headroom (pp)	Offset / source (%)
Qwen3-4B-Base	Real	9.0 [-1pt][8.1, 9.9]	8.8 [-1pt][8.0, 9.6]	98.1 [-1pt][87.5, 109.6]
	RD-fixed	11.8 [-1pt][10.9, 12.8]	12.2 [-1pt][11.3, 13.1]	103.2 [-1pt][95.3, 112.0]
Llama-3.1-8B	Real	10.1 [-1pt][9.1, 11.0]	10.9 [-1pt][10.1, 11.8]	108.6 [-1pt][99.0, 119.4]
	RD-fixed	15.6 [-1pt][14.5, 16.6]	13.8 [-1pt][12.9, 14.8]	88.8 [-1pt][83.1, 94.8]

Appendix

Table 10: Letter-offset menus relative to their source families.

Figure 6: Cross-prompt headroom over development-selected static actions, with fixed option order (4,413 held-out items per model; original RD draw). (a) K counts edited actions, with a 0 included separately; the horizontal axis is logarithmic (base 2). For K < 32, points average 200 paired random submenus with static actions reselected within each submenu. Only the full-menu points have 95% item-bootstrap intervals; these do not measure variation across submenus. (b) Separate 32-action menus add each family’s development-fitted letter offsets to a 0 ’s scores, with their own development-selected static actions; bars are 95% item-bootstrap intervals. Both panels share the scale, weight tasks equally and stratify item resampling by task.

Figure 7: Fixed-order cross-prompt gains by the unmodified correct-letter margin averaged over prompts (original RD draw). Margin is the correct letter’s log probability minus the largest incorrect letter’s log probability. (a) Mean oracle-minus-static accuracy gain within each bin, with pointwise 95% within-bin item-bootstrap intervals (2,000 replicates). The shaded Llama left tail (†) has 18 items; its mean and interval are omitted because n < 30. (b) Each bin’s summed gain divided by all 4,413 items; bars sum to the family’s total headroom, including the left tail. The bottom row shows item counts on a shared scale; outer bins extend to infinity. Mean margin sign does not imply identical correctness on both prompts.

Qwen3-4B-Base
	Fixed: $G_{\mathrm{repro}}$ (pp)			Rotated: $G_{\mathrm{repro}}$ (pp)
Static	Real	RD-fixed	$\Delta$	Real	RD-fixed	$\Delta$
Dev.	$\mathbf{7.3}\,{\color[rgb]{0.22,0.22,0.22}[6.4,8.2]}$	$7.0\,{\color[rgb]{0.22,0.22,0.22}[6.2,7.8]}$	$0.3\,{\color[rgb]{0.22,0.22,0.22}[-0.6,1.2]}$	$\mathbf{-3.1}\,{\color[rgb]{0.22,0.22,0.22}[-3.7,-2.4]}$	$-4.0\,{\color[rgb]{0.22,0.22,0.22}[-4.6,-3.4]}$	$0.9\,{\color[rgb]{0.22,0.22,0.22}[0.2,1.6]}$
Family	$7.6\,{\color[rgb]{0.22,0.22,0.22}[6.7,8.4]}$	$\mathbf{7.9}\,{\color[rgb]{0.22,0.22,0.22}[7.0,8.7]}$	$-0.3\,{\color[rgb]{0.22,0.22,0.22}[-1.2,0.6]}$	$\mathbf{-3.0}\,{\color[rgb]{0.22,0.22,0.22}[-3.6,-2.3]}$	$-3.7\,{\color[rgb]{0.22,0.22,0.22}[-4.3,-3.1]}$	$0.7\,{\color[rgb]{0.22,0.22,0.22}[0.03,1.4]}$
$a_{0}$	$7.0\,{\color[rgb]{0.22,0.22,0.22}[6.2,7.8]}$	$7.0\,{\color[rgb]{0.22,0.22,0.22}[6.2,7.8]}$	$0.0\,{\color[rgb]{0.22,0.22,0.22}[-0.6,0.6]}$	$\mathbf{-3.5}\,{\color[rgb]{0.22,0.22,0.22}[-4.2,-2.9]}$	$-4.3\,{\color[rgb]{0.22,0.22,0.22}[-4.9,-3.7]}$	$0.8\,{\color[rgb]{0.22,0.22,0.22}[0.3,1.2]}$
Llama-3.1-8B

Appendix

Table 11: Large-menu study: letter-offset adjustment.

Qwen3-4B-Base
	Headroom $G_{\mathrm{repro}}$ (pp)		Drop (pp)	Offset-menu $G_{\mathrm{repro}}$ (pp)
Family	Fixed	Rotated	Fixed $-$ rotated	Fixed	Rotated
Real	$9.0\,{\color[rgb]{0.22,0.22,0.22}[8.1,9.9]}$	$\mathbf{-3.3}\,{\color[rgb]{0.22,0.22,0.22}[-4.0,-2.7]}$	$12.3\,{\color[rgb]{0.22,0.22,0.22}[11.5,13.2]}$	$8.8$	$-1.8$
RD-fixed	$\mathbf{11.8}\,{\color[rgb]{0.22,0.22,0.22}[10.9,12.8]}$	$-5.7\,{\color[rgb]{0.22,0.22,0.22}[-6.3,-5.1]}$	$17.5\,{\color[rgb]{0.22,0.22,0.22}[16.6,18.4]}$	$12.2$	$-2.8$
Llama-3.1-8B
Real	$10.1\,{\color[rgb]{0.22,0.22,0.22}[9.1,11.0]}$	$\mathbf{-2.5}\,{\color[rgb]{0.22,0.22,0.22}[-3.2,-1.8]}$	$12.5\,{\color[rgb]{0.22,0.22,0.22}[11.6,13.5]}$	$10.9$	$-3.0$

Appendix

Table 12: Large-menu study: option order and letter offsets.

Qwen3-4B-Base; shared order
Rule	Family	All correct	All wrong	Several	Unique	Total
Tie average	Real	0.0	0.1	6.6	2.4	9.0
	RD-fixed	0.0	0.1	8.6	3.2	11.8
	$\Delta$	0.0	0.0	−2.0	−0.8	−2.8
$a_{0}$ kept	Real	0.0	0.0	9.0	2.4	11.4
	RD-fixed	0.0	0.0	10.7	3.2	13.9

Appendix

Table 13: Contributions by the selection prompt's tie type.

Figure 8: Real-minus-RD cross-prompt headroom under two references (original RD draw). A: one static action selected on development data per family and task. B: other items’ leave-one-out selection weights within the same family and task, recomputed in bootstrap. These are different estimands. Filled circles use a nonzero cyclic option rotation on prompt 2. Open circles reweight stored outputs by

1/n

original order and

1-1/n

rotated order, where

n

is the option count. This post-hoc mixture adds no observations and is not a randomized or fully balanced design. Bars are 95% paired, task-stratified item-bootstrap intervals. Both models use the same scale, the full 32-program menu plus

a_{0}

, tie averaging, 4,413 items and equal task weights. Llama’s B-mixture interval includes zero. Zero denotes equal headrooms under the specified reference, not a validated null for computation-specific effects.

	Headroom (pp)		Placebo / Real		$\Delta$ (pp)
Family	$G_{\mathrm{raw}}$	$G_{\mathrm{repro}}$	$\rho_{\mathrm{same}}$	$\rho$	Real $-$ placebo
Real	$16.1$	$\mathbf{6.7}$	—	—	—
RD-item	$15.3$	$4.6$	$0.95\,{\color[rgb]{0.22,0.22,0.22}[0.87,1.04]}$	$0.69\,{\color[rgb]{0.22,0.22,0.22}[0.50,0.94]}$	$2.1\,{\color[rgb]{0.22,0.22,0.22}[0.4,3.7]}$
SH	$15.9$	$5.3$	$0.99\,{\color[rgb]{0.22,0.22,0.22}[0.91,1.08]}$	$0.79\,{\color[rgb]{0.22,0.22,0.22}[0.61,1.00]}$	$1.4\,{\color[rgb]{0.22,0.22,0.22}[0.01,2.9]}$
Rot	$\mathbf{16.7}$	$5.4$	$1.03\,{\color[rgb]{0.22,0.22,0.22}[0.95,1.13]}$	$0.81\,{\color[rgb]{0.22,0.22,0.22}[0.61,1.11]}$	$1.3\,{\color[rgb]{0.22,0.22,0.22}[-0.7,2.9]}$

Appendix

Table 14: Three-program study: headroom, ratios, and paired differences.

Figure 9: Cross-prompt headroom in the three-program study (Qwen3-4B-Base; 1,800 held-out items). Each column selects oracle actions and static references using its own metric: accuracy (pp) or correct-letter margin (nats). References maximize the pooled mean metric on the selection prompt and are reselected in each unstratified paired item-bootstrap replicate; intervals are 95%. Dashed lines mark Real point estimates. No paired accuracy contrast survives Holm correction. Rot rotates hidden updates, not answer options. Rescale multiplies a 0 ’s option log-probabilities by positive factors, preserving every answer ranking: accuracy headroom is zero, while a margin oracle can select among factors. These factors were fitted on evaluation items and held fixed in bootstrap; the skip program’s KL target was not reached. Rescale is a descriptive comparison, not a matched control.

		Accuracy (%)			$T_{\mathrm{rescue}}$ (pp)
Rewordings	$n$	Unmodified	Own program	Donors	Own $-$ donors
All kept rewordings	$200$	$19.0$	$\mathbf{41.5}$	$15.5$	$26.0\,{\color[rgb]{0.22,0.22,0.22}[20.9,31.4]}$
Same executed length	$136$	$22.1$	$\mathbf{43.4}$	$16.5$	$26.8\,{\color[rgb]{0.22,0.22,0.22}[21.3,32.1]}$
Sentence by sentence	$200$	$18.0$	$\mathbf{49.5}$	$15.8$	$33.7$
Restructured	$199$	$20.1$	$\mathbf{33.2}$	$14.9$	$18.3$

Appendix

Table 15: Transfer to reworded problems: confirmation and descriptive styles.

Draw	Calibration	Matching fixed	Matching rotated	$G_{\mathrm{repro}}(\text{{RD-fixed}})$	$C$	$C$ cluster lower bound
Qwen3-4B-Base
1	32/32	Pass (32/32)	Pass (32/32)	$11.8\,{\color[rgb]{0.22,0.22,0.22}[10.9,12.8]}$	$7.3\,{\color[rgb]{0.22,0.22,0.22}[6.5,8.2]}$	5.8
2	32/32	Pass (32/32)	Pass (32/32)	$10.2\,{\color[rgb]{0.22,0.22,0.22}[9.3,11.1]}$	$5.7\,{\color[rgb]{0.22,0.22,0.22}[4.9,6.5]}$	4.4
3	32/32	Pass (32/32)	Pass (32/32)	$11.4\,{\color[rgb]{0.22,0.22,0.22}[10.4,12.3]}$	$6.9\,{\color[rgb]{0.22,0.22,0.22}[6.1,7.7]}$	5.5
Llama-3.1-8B
1	32/32	Pass (32/32)	Pass (32/32)	$15.6\,{\color[rgb]{0.22,0.22,0.22}[14.5,16.6]}$	$10.5\,{\color[rgb]{0.22,0.22,0.22}[9.6,11.5]}$	8.7

Appendix

Table 16: Matching and fixed-order headroom across random-direction draws. Calibration: programs within churn tolerance; evaluation matching: the full churn rule and the within-tolerance count for fixed and rotated option order. Headroom and

C=G_{\mathrm{repro}}(\text{{RD-fixed}})-\tfrac{1}{2}G_{\mathrm{repro}}(\mathrm{Real})

are in percentage points; brackets are 95% paired item-bootstrap intervals. The last column gives the clustered 95% lower bound for

C

Draw	$\Delta$ fixed	$\Delta$ rotated	Rotated $\Delta$ cluster interval	Same-prompt $G_{\mathrm{raw}}(\text{{RD-fixed}})$
Qwen3-4B-Base
1	$-2.8\,{\color[rgb]{0.22,0.22,0.22}[-3.8,-1.9]}$	$2.3\,{\color[rgb]{0.22,0.22,0.22}[1.6,3.0]}$	[1.5, 3.1]	$29.2\,{\color[rgb]{0.22,0.22,0.22}[28.1,30.4]}$
2	$-1.2\,{\color[rgb]{0.22,0.22,0.22}[-2.1,-0.3]}$	$1.4\,{\color[rgb]{0.22,0.22,0.22}[0.7,2.2]}$	[0.4, 2.4]	$28.0\,{\color[rgb]{0.22,0.22,0.22}[26.9,29.2]}$
3	$-2.4\,{\color[rgb]{0.22,0.22,0.22}[-3.3,-1.5]}$	$1.4\,{\color[rgb]{0.22,0.22,0.22}[0.7,2.1]}$	[0.5, 2.2]	$28.9\,{\color[rgb]{0.22,0.22,0.22}[27.8,30.1]}$
Llama-3.1-8B
1	$-5.5\,{\color[rgb]{0.22,0.22,0.22}[-6.6,-4.4]}$	$3.7\,{\color[rgb]{0.22,0.22,0.22}[2.8,4.5]}$	[2.8, 4.5]	$35.5\,{\color[rgb]{0.22,0.22,0.22}[34.3,36.7]}$

Appendix

Table 17: Real-minus-placebo contrasts and same-prompt headroom.

\Delta=G_{\mathrm{repro}}(\mathrm{Real})-G_{\mathrm{repro}}(\text{{RD-fixed}})

; all entries are percentage points. Brackets are 95% paired item-bootstrap intervals; the separate cluster column gives the clustered interval for rotated

\Delta

. Same-prompt

G_{\mathrm{raw}}(\text{{RD-fixed}})

uses the fixed-order design.

Model	Contrast	Mean [item interval]	Draw SD	Draw range
Qwen3-4B-Base	$C$ , shared	6.6 [6.0, 7.2]	0.85	5.7 to 7.3
	$\Delta$ , shared	−2.1 [−2.9, −1.4]	0.85	−2.8 to −1.2
	$\Delta$ , rotated	1.7 [1.1, 2.3]	0.55	1.4 to 2.3
Llama-3.1-8B	$C$ , shared	13.0 [12.1, 13.8]	2.11	10.5 to 14.4
	$\Delta$ , shared	−7.9 [−8.9, −6.9]	2.11	−9.3 to −5.5
	$\Delta$ , rotated	4.0 [3.3, 4.8]	0.41	3.7 to 4.5

Appendix

Table 18: Three-draw means and the two sources of variation.

Explore similar work

Jun 4, 2026cs.LG

Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic-depth methods, often while executing fewer layers, and that these gains persist under out-of-distribution evaluation. Our results suggest that fixed-depth execution captures only a narrow subset of an LLM's latent reasoning capacity.

Ziyue Li, Yang Li, Tianyi Zhou

University of Maryland, College Park, MD, USA · 2MBZUAI, Abu Dhabi, UAE.

LLM Inference LLM Inference Acceleration LLM Reasoning

Ask PDF

Oct 3, 2024cs.CL

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer internally. We expose this latent knowledge via the Query--Key (QK) score, defined for an attention head as the inner product between the last-token query and the key at the end-of-line token following option

i

, evaluated before rotary positional embedding is applied. Its argmax identifies a universal class of select-and-copy heads in middle layers that perform option selection through semantic query--key alignment, mechanistically distinct from induction and copy-suppression heads (Olsson et al., 2022): they are invariant to label symbols, and solve a synthetic task with zero surface overlap---properties no positional-copy account explains and that critically require stripping RoPE. Across 24 models from 1.5B to 72B parameters (LLaMA-2/3/3.1/3.3, Qwen-2.5, Gemma, Phi-3.5, DeepSeek-R1-Distill), a single head's QK-score exceeds the model's own zero-shot accuracy by up to

+27.4

pp on HellaSwag and

+49.8

pp on HaluDialogue; causal zero-ablation collapses MCQA accuracy to near-random. To remove any dependence on labeled validation data, we introduce an unsupervised HeadScore that ranks heads from unlabeled inputs and recovers the supervised top-

k

heads on every tested model. Against four positional-debiasing baselines (e.g., PriDe, Wiegrefe, Wang), QK-score is complementary by construction: debiasing re-weights output logits, whereas QK-score reads the model's selection from a middle-layer head before decoding. We release a one-line drop-in HeadScore script and per-model head indices, making every result one-command reproducible across all 24 models and four benchmarks.

Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5

Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2

Self-Attention Multiple-Choice Question Answering LLM Interpretability Attention Mechanisms Multi-Head Attention Language Model Probing

Ask PDF

Sep 1, 2026cs.CL

StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions

Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented framings. We investigate whether these discrepancies arise from different internal representations induced by the two framings. We introduce a dual-framing protocol with minimally varied prompts that use either support- or elimination-oriented framing while keeping the evaluation target fixed. To probe the internal computation, we append an untrained special token, [STATE], and treat its residual-stream activation as an intervention interface. Across both models, the two framings induce separable [STATE] activations concentrated in intermediate layers. Swapping these activations between paired prompts systematically changes predictions and improves cross-framing agreement, providing intervention-based evidence that the activations are behaviorally relevant. Beyond instance-level substitution, mean-difference steering directions derived from the dual-framing contrast exhibit more bounded layer-wise responses than matched contrastive activation addition directions under the evaluated protocol.

Chao Gao, Haijiang Liu, Qiyuan Li +3

Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China · Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, China

Prompt Sensitivity Multiple-Choice Question Answering LLM Interpretability Framing Effects in Language Models Causal Interventions in Language Models Activation Patching

Ask PDF