When Every Choice Is Correct: A Causal Study of Numerical Preference in Language Models
Organizations: Department of Mathematics and Statistics, University of New Mexico · Independent researcher
Abstract
When a language model chooses a number between 1 and 30, every integer in the range is valid, yet the model can favor particular answers. We use numerical choice to study how preference is processed across layers and tokens. Perturbation probing and gradient attribution nominate neurons for activation interventions. In a selected Qwen3-4B case, a pre-answer edit changes 17 to 15 while both runs first emit 1. After the edit ends, selective cache and attention-output replacements trace part of its continuing influence on the second digit. Replacing one attention head's output or the other heads' outputs jointly produces similar immediate readout contributions; both interventions restore 17 but have different final log-odds effects on 7 versus 5. Holding later multilayer perceptron (MLP) outputs at their values from the edited run without either replacement reduces this effect gap from 0.119 to 0.0025 nat. Later computation therefore changes how similar local contributions affect the continuation. Retrospective estimates across eighteen contexts capture much of the probability movement but miss the selected answer switch. Prompt tests also separate influence from answer changes: one held-out prompt retains influence without switching, and only one of twelve fresh wordings qualifies for the reader/MLP test; there, joint MLP clamping again reduces the gap between the two reader effects while the answer remains 17. These results use numerical choice as a controlled case for studying persistent state and conditional downstream processing; they do not establish the origin or a complete circuit of natural numerical preference.
Figures & tables
| Intervention | Changed object and reference |
|---|---|
| Source dose | Add to the selected pre-down-projection scalar at the last pre-answer position; remove the hook before processing the first digit. |
| Cache replacement | Copy selected source-position keys or values from the stated donor cache into the recipient. Other entries remain unchanged. |
| Reader replacement | Copy selected L33 head-output slices before the attention output projection, while processing the common first digit. |
| Late-MLP clamp | Replace L34–35 MLP output vectors at that query with their source-edited, no-reader values. Attention and final normalization remain active. |
| Target | Cohort | Predicted | Unresolved | Mean (control) |
|---|---|---|---|---|
| L29:8172 | Historical | 24/24 | 0 | 0.636 (0.060) |
| L29:8172 | Extensions | 10/12 | 2 | 0.533 (0.065) |
| L34:2836 | Historical | 24/24 | 0 | 0.317 (0.007) |
| L34:2836 | Extensions | 12/12 | 0 | 0.357 (0.004) |
| Selected retained-prefix assay | Baseline | Positive L29 edit |
|---|---|---|
| 0.596142 | 0.492439 | |
| 0.403735 | 0.507433 | |
| (nat) | 15.028371 | 15.431160 |
| Candidate-event mode | 17 | 15 |
| Transfer into edited recipient | Head 26 | Complement |
|---|---|---|
| L33 attention-output change, norm | 1.739 | 1.733 |
| Immediate readout contribution (nat) | ||
| Final 7/5 margin movement (nat) |
| MLP output held | Head effect | Head | Complement effect | Complement |
|---|---|---|---|---|
| Neither | ||||
| L34 only | ||||
| L35 only | ||||
| L34 and L35 |
| Setting | Starting | Source change | Edited | Reply |
|---|---|---|---|---|
| Selected prompt | ||||
| Same prompt, +5% dose | ||||
| Held-out wording |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Target | Positive dose | Control | Positive control dose |
|---|---|---|---|
| L29:8172 | 1.81884765625 | L29:8369 | 1.897157100217 |
| L34:2836 | 1.81298828125 | L34:1261 | 2.305332032952 |
| Context | Exact recipient prompt |
|---|---|
| P1 | Give me a number between 1 and 30 |
| P2 | Give me a number between 1 and 30. |
| P3 | Give me a number from 1 to 30 |
| P4 | Give me a number from 1 to 30. |
| P5 | Pick a number between 1 and 30 |
| P6 | Pick a number between 1 and 30. |
| Test | Recorded boundary or control | Consequence for the claim |
|---|---|---|
| Original 4B validation | Protected 1–20 answer changes from 13 to 5 | No all-criteria validation success; keep substantial 1–40 effects separate. |
| Original 2507 validation | increases by 0.28, 0.03 and 0.00 at 1–35, 1–38 and 1–45 | Extension-interval reach criterion fails. |
| Original 8B validation | Some extension baselines already favor 17 | The assumed baseline condition does not apply there. |
| First-digit diagnostic | FP32 pooled share 0.698 versus a 0.70 criterion (bf16: 0.702) | Threshold fails in FP32; distinct from fitted-predictor accuracy. |
| Zeroing specificity | 100 controls matched on down-projection-column norm | Does not match activation-dependent removed writes. |
| Natural instruction swaps | 16 matched control neurons give comparably small effects | Weak transfer is not unique to the selected source. |
| Zeroing measurement | Qwen3-4B | Qwen3-8B |
|---|---|---|
| Mean absolute change in | 0.0074 | 0.0044 |
| Mean answer-distribution TV | 0.019 | 0.006 |
| Items with | 3.4% | 1.3% |
| Largest absolute change | 0.62 | 0.50 |
| Study | Measurements and records | Presentation |
|---|---|---|
| Preference and interface | Conditional baseline scores and fixed-question serialization comparison | Section 2; App. A.4 |
| Discovery and selection | PP/gradient scores, causal eligibility and corrected 4B ranks | Section 2 |
| Wording and interval tests | Historical cohorts, sampling settings and aggregate predictions | App. C.6 |
| Fixed-dose direction | Two targets, matched controls and frozen 17/23 predictions | Section 3.2 |
| Branch and continuation | Saved-event KL, retrospective predictions and paired event masses | Fig. 1 ; Apps. C.3 – C.4 |
| Computed-answer controls | Historical continuous scores and sensitive tails | App. A.5 |
| Context | Reader | Normal | L34 held | L35 held | Both held |
|---|---|---|---|---|---|
| Selected | Head 26 | ||||
| Selected | Complement | ||||
| Reused | Head 26 | ||||
| Reused | Complement |
| Context | Reader | L35 free | L35 held | Interaction |
|---|---|---|---|---|
| Selected | Head 26 | |||
| Selected | Complement | |||
| Reused | Head 26 | |||
| Reused | Complement |
| Context | Exact prompt | Choice | Included |
|---|---|---|---|
| F1 | Please name one integer from the inclusive range of 1 to 30. | 15 | No |
| F2 | Choose a single whole number in the closed interval from 1 to 30. | 15 | No |
| F3 | I need one integer: choose any value from 1 through 30, endpoints included. | 23 | No |
| F4 | Which integer would you choose from 1 to 30, with both endpoints allowed? | Capped | No |
| F5 | Provide one whole number that is at least 1 and at most 30. | 25 | No |
| F6 | Select an integer from the set of whole numbers starting at 1 and ending at 30. | Capped | No |
| Incremental 7/5 effect | Normal | Joint L34–35 clamp |
|---|---|---|
| Head 26 | ||
| Complement | ||
| Head minus complement |
| Program | Anchor | One new eligible wording |
|---|---|---|
| No edit | 17 | 17 |
| Terminal root edit, L35:787 | 24 | 23 |
| Earlier state edit, L28:5371 | 27 | 27 |
| Terminal suffix edit, L35:4145 | 14 | 14 |
| Earlier state plus terminal suffix edits | 24 | 24 |
| Fixed-dose arm (18 contexts each) | Eligible | |||
|---|---|---|---|---|
| Positive source | 0.0200922 | 0.0290863 | 59.1% | 18 |
| Negative source | 0.0070568 | 0.0286929 | 80.3% | 18 |
| Positive control | 0.0001596 | 0.0025334 | 94.1% | 15 |
| Negative control | 0.0001890 | 0.0024630 | 92.9% | 15 |
| Self replacement | — | 0 |
| Context | Baseline loss | Prediction loss | Reduction | TV error |
| P1 | 98.35% | |||
| P2 | 99.46% | |||
| P3 | 89.93% | |||
| P4 | 73.20% | |||
| P5 | 99.02% | |||
| P6 | 98.09% |
| Site | All-30 correlation | 10–19 correlation | 10–19 amplitude |
|---|---|---|---|
| L29:8172 | 0.9826 | 0.9705 | 0.20478 |
| L29:8369 (control) | 0.7794 | 0.9592 | 0.02781 |
| L34:2836 | 0.9994 | 0.9943 | 0.01483 |
| L34:1261 (control) | 0.8695 | 0.9652 | 0.00352 |
| Interval | Total KL | Branch KL | Continuation KL | Branch share |
|---|---|---|---|---|
| 1–30 | 0.025055 | 0.0000767 | 0.024979 | 0.306% |
| 1–40 | 40.395118 | 40.236408 | 0.158710 | 99.607% |
| 1–30 | 1–40 | |||||
|---|---|---|---|---|---|---|
| Template | Total KL | Branch share | Total KL | Branch share | ||
| P1 | 99.333% | 100.000% | ||||
| P2 | 99.190% | 100.000% | ||||
| P3 | 92.147% | 99.999% | ||||
| P4 | 0.066% | 81.475% | ||||
| P5 | 95.969% | 99.992% | ||||
Mechanistic Interpretability
Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.
See all 427 papers