When a language model chooses a number between 1 and 30, every integer in the range is valid, yet the model can favor particular answers. We use numerical choice to study how preference is processed across layers and tokens. Perturbation probing and gradient attribution nominate neurons for activation interventions. In a selected Qwen3-4B case, a pre-answer edit changes 17 to 15 while both runs first emit 1. After the edit ends, selective cache and attention-output replacements trace part of its continuing influence on the second digit. Replacing one attention head's output or the other heads' outputs jointly produces similar immediate readout contributions; both interventions restore 17 but have different final log-odds effects on 7 versus 5. Holding later multilayer perceptron (MLP) outputs at their values from the edited run without either replacement reduces this effect gap from 0.119 to 0.0025 nat. Later computation therefore changes how similar local contributions affect the continuation. Retrospective estimates across eighteen contexts capture much of the probability movement but miss the selected answer switch. Prompt tests also separate influence from answer changes: one held-out prompt retains influence without switching, and only one of twelve fresh wordings qualifies for the reader/MLP test; there, joint MLP clamping again reduces the gap between the two reader effects while the answer remains 17. These results use numerical choice as a controlled case for studying persistent state and conditional downstream processing; they do not establish the origin or a complete circuit of natural numerical preference.
Figures & tables
Figure 1
Intervention
Changed object and reference
Source dose
Add δ to the selected pre-down-projection scalar at the last pre-answer position; remove the hook before processing the first digit.
Cache replacement
Copy selected source-position keys or values from the stated donor cache into the recipient. Other entries remain unchanged.
Reader replacement
Copy selected L33 head-output slices before the attention output projection, while processing the common first digit.
Late-MLP clamp
Replace L34–35 MLP output vectors at that query with their source-edited, no-reader values. Attention and final normalization remain active.
Table 1: Operations in the selected continuation study. All model weights remain fixed. Cache and reader donors are specified with each comparison; replacement directions do not prescribe a particular sign or answer. Source zeroing in the earlier assays has its separately stated position scope.
Target
Cohort
Predicted
Unresolved
Mean ∣Δ∣ (control)
L29:8172
Historical
24/24
0
0.636 (0.060)
L29:8172
Extensions
10/12
2
0.533 (0.065)
L34:2836
Historical
24/24
0
0.317 (0.007)
L34:2836
Extensions
12/12
0
0.357 (0.004)
Table 2: Frozen directional predictions at two individually edited sites. Each prompt supplies positive and negative target doses. Parentheses give mean absolute effects for the site’s matched control; effects are in nats. Historical contexts informed selection; the six extensions are outside that discovery set. These dependent conditions do not estimate a population success rate.
Figure 1: A shared response captures probability movement while missing the selected switch. A: all eighteen source responses at the same dose, centered across the thirty candidate log probabilities; the black curve is their coordinate-wise median. E1 and E2, initially favoring 23 and 15, remain visible. B: the selected prompt’s baseline, leave-one-out estimate from the other seventeen responses, and observed source distribution. The estimate retains mode 17; the observed source favors 15. For the full thirty-candidate distribution in B, KL prediction error falls by 88.9%, although the estimated mode remains 17. E1 uniquely has no pre-answer text after the chat template. This retrospective comparison is distinct from the earlier fitted interval predictor.
Figure 2: The eighteen-condition dose scan. A: source and matched-control candidate-event margins. B: the source’s two leading event probabilities. Markers are measured factors; straight segments are visual guides. Actual source replies are 17 through factor 0.875 and 15 from factor 1 onward; control replies remain 17. This is one selected prompt.
Selected retained-prefix assay
Baseline
Positive L29 edit
q(17)
0.596142
0.492439
q(15)
0.403735
0.507433
q(23)
1.77260×10−7
9.78779×10−8
log[w(17)/w(23)] (nat)
15.028371
15.431160
Candidate-event mode
17
15
Table 3: The same dose improves one comparison while changing the winner against another competitor. Probabilities and log ratios are derived from the same full-precision event-weight vector for each condition, then rounded for display. Separately computed directional-panel margins differ by less than 5×10−5 nat here; the 0.403-nat conclusion is unchanged. Sampled reply frequencies require a separate generation assay. The dose scan independently checks generated answers within the same experimental programme.
Figure 3: Partial localization across an unchanged token. The pre-answer edit changes later cached state at position p . With the hook removed, both runs process 1 at p+1 and predict the next digit. The source contrast is 17 versus 15; reader replacements in the specified hybrid backgrounds can instead restore 17. Arrows summarize nested interventions and processing order, not exclusive edges or transition probabilities. Late-MLP response clamps change relative reader effects without changing their measured answers. Other cache windows and the complement remain influential.
Figure 4: A partial contribution and its boundaries. A: head-26 transfer relative to all L33 readers and to the full source effect, computed separately in each setting. These ratios are not probabilities and are not multiplied along Figure 3 . B: replacing the complement can cross the selected 7/5 boundary as well. Points record actual intervention margins; no prompt-population error bars are implied.
Transfer into edited recipient
Head 26
Complement
L33 attention-output change, ℓ2 norm
1.739
1.733
Immediate readout contribution (nat)
+0.02237
+0.02322
Final 7/5 margin movement (nat)
+0.15263
+0.03374
Table 4: Similar immediate reader contributions produce different final effects in the selected prompt. Saved-state accounting uses the projected attention-output change and an inferred recipient normalization for the immediate contribution. Final movements are relative to the edited no-reader recipient. Both interventions restore the completed answer 17. Figure 5 tests the role of subsequent MLP responses.
Figure 5: Later MLP responses change the relative influence of the two reader interventions. Points show head-26 and complement effects on the 7/5 margin relative to the matching no-reader recipient; lines connect interventions for comparison, not an ordered computational path. The output clamp holds L34–35 MLP vectors at recipient values. A: the normal head/complement gap nearly disappears. B: the ordering reverses in the reused prompt, although both effects remain positive. Every reader condition shown continues with 17. The experiment comprised twelve core conditions and eight reference/identity conditions across two reused contexts; the figure displays the head and complement effects on the same vertical scale.
MLP output held
Head effect
Head m
Complement effect
Complement m
Neither
+0.152637
+0.122656
+0.033738
+0.003757
L34 only
+0.088187
+0.058206
+0.062106
+0.032125
L35 only
+0.068869
+0.038888
+0.010002
−0.019979
L34 and L35
+0.033326
+0.003345
+0.030854
+0.000873
Table 5: Separate late-MLP readouts for both readers in the selected prompt, in nats. All entries use the same FP64 reconstruction and corresponding no-reader baseline. Normal and joint-clamp arms also have measured continuations, all 17; the single-clamp arms have only reconstructed readouts. Small differences from the measured FP32 margins elsewhere in the text reflect arithmetic. A negative m means 5 outranks 7, without establishing a full-vocabulary winner or completed answer.
Setting
Starting m
Source change
Edited m
Reply
Selected prompt
+0.389778
−0.419754
−0.029976
17→15
Same prompt, +5% dose
+0.389778
−0.441185
−0.051407
17→15
Held-out wording
+10.930855
−0.360317
+10.570538
17→17
Table 6: Starting competition and measured source movement after the common first digit 1. Margins are 7/5 next-token log odds in nats; completed continuations supply the reply column. The held-out wording uses the higher dose and recaptured donors. These comparisons do not isolate starting margin from the other context changes.
Figure 6: Scalar and residual replacements on the same ordinary/random pairs, using the same candidate-event readout. The broader residual replacement produces greater donorward progress. This is a matched comparison within the instruction assay; it does not identify a minimal mediator.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Target
Positive dose
Control
Positive control dose
L29:8172
1.81884765625
L29:8369
1.897157100217
L34:2836
1.81298828125
L34:1261
2.305332032952
Appendix
Table 7: Fixed target and residual-write-matched control doses. Each site was tested individually with both the displayed positive dose and its exact negative. Control magnitudes are rounded to twelve decimal places; the archive retains full precision.
Context
Exact recipient prompt
P1
Give me a number between 1 and 30
P2
Give me a number between 1 and 30.
P3
Give me a number from 1 to 30
P4
Give me a number from 1 to 30.
P5
Pick a number between 1 and 30
P6
Pick a number between 1 and 30.
Appendix
Table 8: The eighteen recipient prompts in the fixed-dose panel. P1–P12 retain the historical pre-answer prefixes; E1–E6 use newly generated prefixes. Range-35 donor prompts replace the upper endpoint 30 by 35. Punctuation is part of the prompt.
Test
Recorded boundary or control
Consequence for the claim
Original 4B validation
Protected 1–20 answer changes from 13 to 5
No all-criteria validation success; keep substantial 1–40 effects separate.
Original 2507 validation
q(17) increases by 0.28, 0.03 and 0.00 at 1–35, 1–38 and 1–45
Extension-interval reach criterion fails.
Original 8B validation
Some extension baselines already favor 17
The assumed baseline condition does not apply there.
First-digit diagnostic
FP32 pooled share 0.698 versus a 0.70 criterion (bf16: 0.702)
Threshold fails in FP32; distinct from fitted-predictor accuracy.
Zeroing specificity
100 controls matched on down-projection-column norm
Does not match activation-dependent removed writes.
Natural instruction swaps
16 matched control neurons give comparably small effects
Weak transfer is not unique to the selected source.
Appendix
Table 9: Qualifications belong to their own cohorts and endpoints. The table combines documented historical criteria with later frozen follow-up checks; it does not pool them into a success rate.
Zeroing measurement
Qwen3-4B
Qwen3-8B
Mean absolute change in P(correct)
0.0074
0.0044
Mean answer-distribution TV
0.019
0.006
Items with ∣ΔP(correct)∣>0.05
3.4%
1.3%
Largest absolute P(correct) change
0.62
0.50
Appendix
Table 10: Historical computed-answer controls. Small average movement coexists with a sensitive tail. The task, prefix and zeroing intervention differ from the selected additive-dose continuation experiment.
Study
Measurements and records
Presentation
Preference and interface
Conditional baseline scores and fixed-question serialization comparison
Section 2; App. A.4
Discovery and selection
PP/gradient scores, causal eligibility and corrected 4B ranks
Section 2
Wording and interval tests
Historical cohorts, sampling settings and aggregate predictions
App. C.6
Fixed-dose direction
Two targets, matched controls and frozen 17/23 predictions
Section 3.2
Branch and continuation
Saved-event KL, retrospective predictions and paired event masses
Fig. 1 ; Apps. C.3 – C.4
Computed-answer controls
Historical continuous scores and sensitive tails
App. A.5
Appendix
Table 11: Named study families connect the main results to their experimental records. Appendix A gives cohort and control boundaries; each study retains its own numerical profile and endpoint.
Context
Reader
Normal
L34 held
L35 held
Both held
Selected
Head 26
+0.152637
+0.088187
+0.068869
+0.033326
Selected
Complement
+0.033738
+0.062106
+0.010002
+0.030854
Reused
Head 26
+0.101275
+0.072788
+0.061985
+0.019258
Reused
Complement
+0.056680
+0.046034
+0.017404
+0.029239
Appendix
Table 12: All four reader/context comparisons under common FP64 reconstruction. Entries are incremental 7/5 effects in nats relative to the matching no-reader background. The two contexts were reused; separate-clamp columns have no generated replies. Opposing single-clamp changes relative to normal occur only for the selected complement.
Context
Reader
L35 free
L35 held
Interaction
Selected
Head 26
+0.064449
+0.035542
+0.028907
Selected
Complement
−0.028367
−0.020852
−0.007515
Reused
Head 26
+0.028487
+0.042727
−0.014240
Reused
Complement
+0.010646
−0.011835
+0.022482
Appendix
Table 13: Conditional L34 effects derived from the same reconstructed readouts as Table 12 , in nats. The reused complement changes sign with the L35 background. These are defined output-clamp contrasts, not additive natural-mediation fractions or generated replies.
Context
Exact prompt
Choice
Included
F1
Please name one integer from the inclusive range of 1 to 30.
15
No
F2
Choose a single whole number in the closed interval from 1 to 30.
15
No
F3
I need one integer: choose any value from 1 through 30, endpoints included.
23
No
F4
Which integer would you choose from 1 to 30, with both endpoints allowed?
Capped
No
F5
Provide one whole number that is at least 1 and at most 30.
25
No
F6
Select an integer from the set of whole numbers starting at 1 and ending at 30.
Capped
No
Appendix
Table 14: All twelve fresh screening outcomes. “Capped” means no completed numerical choice within 80 tokens. All nine completed choices passed the format and alignment checks; eight were excluded because they were not 17. F10 passed the subsequent common-digit and write-cap checks. No reader/MLP intervention was collected for the other eleven contexts.
Incremental 7/5 effect
Normal
Joint L34–35 clamp
Head 26
+0.168781
+0.029083
Complement
+0.021667
+0.018753
Head minus complement
+0.147114
+0.010330
Appendix
Table 15: Fresh-context effects in F10, in nats. Its source-off and source-on 7/5 margins are +5.852978 and +5.437187 . The frozen primary contrast subtracts the clamped head/complement gap from the normal gap, giving +0.136784 nat. The 93.0% reduction describes this gap, not a fraction of natural preference mediated by the MLPs. All ten measured suffixes were 17**. , completing the retained prefix I choose ** .
Program
Anchor
One new eligible wording
No edit
17
17
Terminal root edit, L35:787
24
23
Earlier state edit, L28:5371
27
27
Terminal suffix edit, L35:4145
14
14
Earlier state plus terminal suffix edits
24
24
Appendix
Table 16: A bounded illustration of composing interventions at different positions. Both root interventions select 2, but their continuations differ; the suffix intervention then changes the earlier edit’s 27 to 24. These are completed, reviewed choices rather than predictions from a fixed-prefix diagnostic.
Figure 7: Downstream-response measurements. A: attenuation of a direct response at 21–40 in two distinct configurations; the FP32 check uses a different bank, prefix, dose and operating point. B: contextual response difference removed by blocking ten nominated readouts or the broader later MLPs, across the three checkpoints. The groups differ in size and coverage; this is not a matched-specificity test. No per-prompt observations or uncertainty are reconstructed from aggregate summaries.
Figure 8: Two historical first-digit examples after zeroing the selected source at 1–40. A: signed numerator contributions divided by the total numerator change. “Remaining” includes the other recorded contributions. B: the six largest absolute unit contributions, summed with their signs and divided by the corresponding net layer contribution. The panels use different denominators; markers are offset horizontally for visibility. Both recorded replies change from 23 to 17; neither example is a new held-out replication. The accounting does not test serial causal links between units.
Fixed-dose arm (18 contexts each)
∑Ba
∑Ca
∑Ca/∑(Ba+Ca)
Eligible
Positive source
0.0200922
0.0290863
59.1%
18
Negative source
0.0070568
0.0286929
80.3%
18
Positive control
0.0001596
0.0025334
94.1%
15
Negative control
0.0001890
0.0024630
92.9%
15
Self replacement
1.68×10−12
6.95×10−11
—
0
Appendix
Table 17: Offline KL accounting in nats, with all eighteen contexts included in each sum. “Eligible” counts cells with total KL at least 10−8 nat, the prespecified numerical reporting threshold for per-cell fractions in this reanalysis. Absolute components are retained below it. The ratios describe these archived conditions rather than a population or source-specific property.
Figure 9: Absolute branch and continuation KL components for the eighteen positive-dose L29:8172 interventions and their L29:8369 matched controls, on a common logarithmic scale. P1–P12 are the earlier templates; E1–E6 are later extensions. E1 and E2 initially favor 23 and 15; the other candidate modes are 17. E4 is the subsequently selected switching prompt. Source and matched-write control use separate panels. Small components remain visible instead of being converted into large-looking percentages. No prompt-level uncertainty is estimated.
Context
Baseline loss Dj
Prediction loss Lj
Reduction
TV error
P1
2.68×10−7
4.43×10−9
98.35%
6.69×10−8
P2
3.19×10−7
1.71×10−9
99.46%
4.01×10−8
P3
2.17×10−8
2.19×10−9
89.93%
5.07×10−8
P4
5.04×10−6
1.35×10−6
73.20%
6.37×10−6
P5
2.04×10−6
2×10−8
99.02%
6.79×10−7
P6
2.84×10−7
5.43×10−9
98.09%
1.39×10−7
Appendix
Table 18: Retrospective leave-one-out prediction for each saved positive source edit. Losses are in nats; TV compares predicted with observed candidate-event distributions. P1–P12 are historical templates and E1–E6 extensions. These related contexts are not independent replications. The selected E4 is the only mode mismatch; an unchanged-baseline predictor also matches the other seventeen modes.
Site
All-30 correlation
10–19 correlation
10–19 amplitude
L29:8172
0.9826
0.9705
0.20478
L29:8369 (control)
0.7794
0.9592
0.02781
L34:2836
0.9994
0.9943
0.01483
L34:1261 (control)
0.8695
0.9652
0.00352
Appendix
Table 19: Positive-dose output responses at four separately edited sites. Correlation entries are medians over eighteen per-context Pearson correlations with the leave-one-out coordinate-wise median response. The 10–19 columns restrict each response to that integer range. Amplitude is the median root mean square of the response centered within 10–19, in nats. This amplitude statistic is distinct from PP ranking aggregation. Equal write sizes apply within each source/control pair, not across layers.
Interval
Total KL
Branch KL
Continuation KL
Branch share
1–30
0.025055
0.0000767
0.024979
0.306%
1–40
40.395118
40.236408
0.158710
99.607%
Appendix
Table 20: Archived zeroing comparisons, with twelve discovery templates per interval. Values are summed KL components in nats and their branch share; these are previously used contexts with context-dependent zeroing doses.
1–30
1–40
Template
Total KL
Branch share
Δ17/15
Total KL
Branch share
Δ17/15
P1
1.48×10−6
99.333%
−0.879
6.22
100.000%
+1.693
P2
1.72×10−6
99.190%
−0.692
6
100.000%
+1.479
P3
6.02×10−7
92.147%
−1.786
4.12
99.999%
+0.977
P4
0.000396
0.066%
−1.556
0.639
81.475%
+2.033
P5
1.17×10−5
95.969%
−1.170
1.67
99.992%
+1.048
Appendix
Table 21: Per-template zeroing results, rounded for display. Total KL is in nats; branch share is the branch component divided by that template’s total. Δ17/15 is the change in candidate log-odds of 17 against 15. P1–P12 match the historical template labels in Figure 9 . These are discovery contexts with context-dependent doses.
Figure 10: Validation and prediction for the selected coordinates. A: sampled fractions on eligible new wordings (16, 17 and 20 prompts); connecting segments pair the two conditions within each checkpoint. Bootstrap uncertainty on the increases is given in the text. B: the frozen predictor and a separate first-token diagnostic on 21 new intervals. Model order is fixed throughout. No point-level data or uncertainty absent from the retained historical summaries is reconstructed.
Figure 11: Post hoc diagnostics of instruction swaps. A: correlation and calibration slope of the scaled-zeroing projection approximation. B: mean absolute and signed projected transfer. Opposing effects reduce the signed average. The panels show aggregates for the specified projection readout. JS-based donorward progress and independent validation require different measurements; per-pair observations are not reconstructed here.
Figure 12: Mean KL to uniform in the separate instruction-bank analysis. The oracle changes branch weights while holding conditional distributions fixed. Lines connect model summaries for readability; they do not fit a scaling law. This sampling-law comparison is separate from the matched natural-transfer assay in Figure 6 .
Irrelevant numbers in a prompt can shift language model judgments, producing anchoring effects in numerical reasoning. We study where this anchor-sensitive signal is carried inside language models using a controlled multiple-choice setup with shared answer options. We define a logit-difference metric comparing the correct answer option with the answer option corresponding to the anchor, and validate that it tracks behavioral anchoring. Using attribution-based circuit localization on 7B--8B Qwen and Llama base and instruction-tuned models, we find that edge-level methods recover this signal more faithfully than node-level methods. Low- and high-anchor circuits transfer strongly within a model, suggesting shared pathway structure across anchor direction. However, sparse transfer across base and instruction-tuned variants is less reliable, indicating that post-training changes which pathways matter most. Overall, our results provide a mechanistic account of how anchoring-related decision signals are carried inside language models.
Hillary N. Owusu, Sarah Wiegreffe, Naomi H. Feldman
When a language model answers a factual question, where does its internal state go? We asked four open models (Llama-3.1-8B, Qwen2.5-7B, Gemma-2-9B and Llama-3.2-1B) 2,256 factual questions whose answers recur, such as the country a person is a citizen of. At every layer, we recorded the state from which the answer is produced. Questions that receive the same answer converge on a shared region of state space, which we summarise by its centre, the answer centroid. The answer becomes readable within a single layer. In every model, the share of questions whose state lies nearest its own answer's centroid jumps at one layer, between 47% and 71% of the way through the network. The centroids follow the answer the model gives rather than the question: the centroid of a country learned from questions about people's citizenship also identifies that country in questions about towns and films. The model also uses the centroids. After the jump, moving the state of a wrongly answered question to the right answer's centroid makes the model give the right answer in 83%-97% of cases late in the network, against 2%-4% for a random move of the same size. Before the jump, moves of the same size almost never work. Near misses, in which the model answers wrongly but picks the right option in multiple choice, can be seen from the onset layer, where the right answer's centroid is already among the nearest to the state. This signal is about as informative as the right answer's rank in the model's final output, and more informative than the model's confidence in its reply.
We measure, without sampling error, the probability a language model assigns to each integer from 1 to N when asked for a random number. For a fixed prompt the model defines one probability vector, which we compute by summing over each integer's scored spellings rather than estimate by sampling, removing the noise that limits earlier studies [1, 4]. We evaluate 18 open-weight model lines at five ranges up to N = 100, under 29 decoding configurations, on two inference stacks; 15 of them have matched base and instruction-tuned checkpoints that both give usable answers. Instruction tuning makes the distribution less uniform in all 15 pairs. It installs one heuristic, round-number avoidance, and sharpens another, endpoint avoidance. Base checkpoints do not avoid round numbers once the endpoints are removed from those sets; tuned checkpoints avoid them, multiples of 10 almost completely. Base checkpoints return the endpoints 1 and N at 0.30 to 0.60 of the interior rate; tuned checkpoints almost never return them. A preference for odd numbers is consistent across tuned checkpoints only at N = 100, where base checkpoints also show it weakly. Endpoint avoidance and the odd preference fit the account Kubovy and Psotka [6] proposed for human number choice; round-number avoidance is not part of that account, and the midpoint avoidance it predicts is essentially absent, with one slight exception at N = 100. Tuning does not install a new favourite integer shared across models: above N = 10 no modal integer is shared by more than 42 percent of tuned conditions, and P (17) rises in 10 of 15 pairs and falls in 5. The concentration is present in the model's own distribution: at temperature 1.0, nucleus truncation at top-p 0.95 adds a median of 0.035 to the total variation distance.