When writing code, the same computation can be expressed in different programming languages and through different implementation strategies. We show that the activations of LLMs capture this shared structure in reusable and composable abstract representations when they generate code. These representations emerge in the middle layers, are reusable across programming languages, and afford the model an impressive amount of generalization ability when generating code under various interventions. Using activation steering, we show that function, programming language, and implementation strategy can be independently manipulated. The independent interventions on language and function can be flexibly composed to control code generation, even in cases when the prompt contains no information about language and function. We additionally find that finetuning on examples of arbitrarily defined functions in one programming language transfers to other languages, consistent with the models reusing abstract features of code generation that are shared across different examples. Finally, to study how such abstract representations come to be, we implement a minimal model of shared, reusable, and composable structure in a synthetic setting, and show that transformers trained on such data learn representations that reuse these parts flexibly across contexts.
Figures & tables
Figure 1: Abstract representations enable control and generalization in programming. (a) : The factored world hypothesis motivates additive activation states; when multiple factored sources reuse the same mechanism, a reusable representation avoids redundancy. (b,c) : Testing composability and reusability of coding representations. In (b), we find a steering vector for a fixed function without ever asking the model to output Haskell code (separately, we also find a steering vector for Haskell, without ever asking the model to implement the target function.) In (c) we use both steers at once to implement the target function in Haskell.
Figure 2: Modern models compile coding specifications. (a) : The most capable coding models are more resistant to not being allowed to attend to the description after prefill. These models also mix local or linear attention with global, or are fully recurrent. (b) : This also largely correlates with the recency of the model.
Figure 3: Function, language, and strategy support causal reuse. For each condition, we report the fraction of evaluation cells in which the generated program both runs correctly and exhibits the injected factor. Function and language factors estimated outside the recipient context generally recover most of the effect of same-context-only interventions, and strategy factors likewise causally control recursive versus iterative realization. Function–language composition remains strong, while simultaneous composition of function, language, and strategy is substantially more difficult. The Pythia control shows no corresponding causal reuse. Interventions with s have 72 trials, all others have 36 (since Python is never a recipient).
Figure 4: A toy training dataset with a shared mechanism. (a) A function (here sum_mod_3 , one of three) generates a string of symbols (orange). A language turns that string into tokens: each symbol is usually written with a token shared by all languages (gray) and sometimes with a token of that language’s own (colored; language A writes 2 as c , language B as γ ). The function is a shared mechanism, analogous to how digit sum is shared by its Python and Rust implementations. The transformer is trained on the tokens alone. (b) The composition test of Section 3.2 : from a prompt that specifies neither factor, we mask the description and add, at the blank suffix, a function edit estimated only from other languages and a language edit estimated only from other functions. (c) With an eight-token attention window, the model passes transport, installation and composition in nearly every trial (compose: 0.99). A dense-attention model with the same loss does not (0.09): even its own prompt, once masked, leaves little at the suffix (0.08), so accurate prediction alone does not guarantee a reusable state. Leave-one-context-out edits, three seeds (marker: mean; bar: seed range); control is the stronger of the random-direction and wrong-value edits. Full results in Fig. 8 .
Figure 5: A single example provides a transferable representation. (a) : A state harvested from the suffix prompt with a single example in one language transplanted into an example-free prompt with a different language specified produces a correct program in the new language. (b) : Models trained on a single algorithm in Python learn that algorithm in several held-out languages.
Figure 6: Cross-language strategy transport can also impart language specific information. Prompting Gemma to write Java, and then steering it with an iterative strategy donor from Haskell induces Java-native syntax: a program using a reduction/stream implementation rather than reproducing Haskell syntax. However, the resulting Java implementation (center) is different from the program that comes from a prompt that directly asks for iteration in Java (right). The steered program is a “Haskel-like” approach to writing iteration in Java.
visible
masked
installed
Model
Identical
Correct
Identical
Correct
Identical
Correct
Gemma-4-31B
39/40
40/40
0/40
31/40
0/40
32/40
Gemma-4-26B-A4B
35/40
40/40
1/40
30/40
1/40
30/40
Qwen3.6-27B
40/40
40/40
0/40
18/40
0/40
18/40
RWKV-7 13B
35/40
40/40
—
—
35/40
40/40
Table 1: The form of the generated program differs when the model is asked to generate without being able to re-read the example and when it is patched.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
machine
state
QM
ηM(∅)
emits
conditions on
language λ
z
{C,F}
(1,0)
c∈{S,Lλ,□}
–
function f
g
Qf
ηf(∅)
k∈{0,1,2,R,□}
r
watcher W
m
{∅,0,01}
(1,0,0)
r∈{cont,R,□}
c,k
Appendix
Table 2: The three machines in a common format: state variable, state set, initial vector, the subtoken each emits, the subtokens of the other machines on which its operator is conditioned. Qf and ηf(∅) depend on the function and are given at the end of this section.
Figure 7: Three coupled machines generate the observed tokens. The function emits subtokens, the language selects shared or language-specific tokens and blank intervals, and the watcher triggers returns with a probability that rises after a pattern match. A return resets the function and watcher; blanks preserve the function and watcher states. The machines communicate through internal subtokens, combined by a deterministic map into the observed token stream. Function states in the diagram are schematic.
Figure 8: Function and language support causal reuse in the toy model’s suffix state. Three training seeds are pooled (marker: mean; bar: seed range). Each row shows the target prompt performance (dashed orange); The top line of each column shows target prompt with its description masked (gray diamond, averaged within the columns because the drop is about the same on every row). Filled blue: leave-one-context-out edit; open blue: same-context-only edit; gray: random-direction and wrong-value controls. Black tick: Bayesian reference with uniform priors over unknown factors and the remaining factors known. We compare with explicit prompting of the target task, the target task prompt under the same mask, and the unedited, masked recipient sequence. We also include a wrong-value control that adds an edit toward an incorrect factor value while retaining the original target test.
Figure 9: With sixteen blanks the windowed model shows the same behavior without direct description access. Same layout and conventions as Fig. 8 , including the Bayesian reference ticks. For the windowed model, masking the description is provably inert, so the diamond sits exactly on the dashed line.
Figure 10: Prompts, cache operations and readouts on the minimal process. A prompt is a beginning-of-sequence token, a 36 -token description containing ordinary symbols, returns and blanks, and a suffix of B shared blanks. After prefill the description’s cached keys and values are masked out of attention and a function displacement ΔF , a language displacement ΔL or their sum is added to the blank suffix’s keys and values at every layer; the input tokens and the beginning-of-sequence cache do not change. The model then produces next-token distributions from the edited cache, and every test is a test on those distributions; no completion is sampled. The language is scored from the distribution immediately after the last blank. Each function test appends its own probe to a separate copy of the edited cache and scores the distribution after the probe.
element
code model
minimal process
τθ
instructions name the computation and language
36 tokens: statistics indicate function; tokens identify language
? factors
omit the function or language instruction
use an ambiguous function sequence or only shared tokens, respectively
σ
shared formatting instruction, with an empty think block
a string of shared blank tokens that hold function dynamics fixed
Appendix
Table 3: Main differences between code prompts and the minimal process.
model
row
leave-one- context-out
same- context-only
unedited
random direction
wrong value
windowed
transport
0.94 (0.94–0.94)
0.94
0.01
0.01
0.01
installation
0.96 (0.95–0.96)
0.96
0.33
0.33
0.02
composition
0.95 (0.95–0.96)
0.96
0.33
0.34
–
dense (control)
transport
0.33 (0.33–0.34)
0.33
0.33
0.33
0.33
installation
0.33 (0.33–0.34)
0.33
0.33
0.33
0.33
composition
0.33 (0.33–0.34)
0.33
0.33
0.33
–
Appendix
Table 4: Estimated belief in the target function on the four-blank trials of Fig. 8 , for the leave-one-context-out edit (mean over three training seeds, seed range in parentheses), the same-context-only edit, the unedited prompt, and the random-direction and wrong-value controls. A model certain of the target scores 1 , an uninformed one 1/3 , one committed to the wrong function 0 . Prompted models score 0.99.
Figure 11: (a) : The different types of prompts and the included factors. (b) : How the prompt is organized.
Figure 12: We compare our breadth-based coding benchmark to published scores on LiveCodeBench v6.
tier
task
specification (as prompted: “Write a function in λ that …”)
easy
digit_sum
returns the sum of the digits of a non-negative integer
easy
count_divisors
returns the number of positive divisors of a positive integer
easy
int_sqrt
returns the floor of the square root of a non-negative integer, computed exactly (beware floating point at large inputs)
easy
largest_prime_factor
returns the largest prime factor of an integer greater than 1
easy
binary_gap
returns the length of the longest run of zeros strictly between two ones in the binary representation of a positive integer
easy
alt_digit_sum
returns the alternating sum of the digits of a positive integer, starting with a plus sign on the most significant digit
Appendix
Table 5: The 31 tasks of the coding-ability bench. Each task is requested in every one of the 10 languages (310 cells per model); a cell passes only if the generated function returns the exact reference output on all six battery inputs. verlix , dromber and trelk ( † ) are invented specifications, so the bank cannot be solved from memory alone.
model
function
kind
compiles
no example
mean over donors
single donor
Gemma-4-31B
thulep
regular
87/90
0/10
10/10
87/87
Gemma-4-31B
vantril
regular
81/90
0/10
10/10
80/81
Gemma-4-31B
kanex
nonce
50/90
0/10
0/10
42/50
Gemma-4-31B
faromin
nonce
64/90
0/10
1/10
54/64
Gemma-4-26B-A4B
thulep
regular
80/90
0/10
10/10
80/80
Gemma-4-26B-A4B
vantril
regular
77/90
0/10
8/10
72/77
Appendix
Table 6: In-context transplant per algorithm. compiles : donor–target pairs (of 90) whose suffix state still yields the function when attention to the example is masked (RWKV-7: unmasked pairs). no example : the recipient alone; mean over donors : the mean state of the admissible donors, one cell per target language; single donor : one donor’s state per (admissible donor, target) cell. A cell passes only if all six unit tests pass in the requested language. The random-edit control passes 0 cells.
Name
Definition for a non-negative integer n
vantril
Number of distinct decimal digits appearing in n .
thulep
Largest decimal digit of n minus the smallest decimal digit of n .
kanex
Sum of d+1 over all decimal digits d of n (the digit sum plus the number of digits).
faromin
Sum of d+1 over the distinct decimal digits d of n , each digit value counted once.
Appendix
Table 7: Digit-based properties of a non-negative integer n .
Hyperparameter
Value
Optimizer
AdamW
LR schedule
Cosine decay
Warmup
3%
Gradient clipping
1.0
Precision
bf16
Sharding
DeepSpeed ZeRO-3
Appendix
Table 8: Fine-tuning hyperparameters used across experiments.
Dose
Learning rate
Epochs
1×
5×10−7
36
4×
2×10−6
36
16×
8×10−6
36
Appendix
Table 9: Fine-tuning dosage sweep.
Figure 13: The full generalization results across all the invented algorithms.
Figure 14: (a) : Pretrained models are substantially worse than their instruct-tuned counterparts at generating correct code with a masked out description. (b) : The prompt we use for the base-models to trigger their completion.
Figure 15: On Mistral-Small-3.2, the same intervention site that worked well on the main models largely fails.
Figure 16: (a) : As more instructions are provided, a longer suffix length is needed for the model to store the program state. (b) : Not all instructions are equal. Instructions less relevant for computation are more often early re-read as opposed to being part of the computation state. With an unablated description, all instructions are honored.
Figure 17: For each model pair and intervention, when strategy steering succeeds from a single donor, how is the resulting program expressed? Blank squares imply strategy steering did not work.
Figure 18: For each model, concrete examples where the steered recipient expresses a program differently from when it is prompted for the same strategy.
Figure 19: In the middle layers of the tested models, the function being implemented accounts for a larger share of the variance than the language it is implemented in.
Figure 20: Function state has locally reusable but anisotropic causal geometry. (a) A square modifier estimated only from sum induces squared-product and squared-max at intermediate dose; a norm-matched random edit does not. (b) Moving along the displacement between two naturally elicited function states produces a sharp transition from source to target behavior, with validity falling near the transition and under over-extrapolation. (c) Valid computation is preserved much farther along the semantic direction than along its reversal or a matched random direction. (d) The same modifier across different bases is aligned, related modifiers on the same base are more strongly aligned, while modifier directions are approximately orthogonal to generic function differences. Fsum,Fprod,Fmax denote the corresponding measured Z states.
Figure 21: Computational modifiers generalize across held-out contexts. (a) Target-correct transfer across 160 cells per model. (b) Outcomes among unsuccessful pooled-donor interventions. Some failures still visibly express the modifier in its erroneous computation. Especially RWKV, which does so in 48 of its 49 misses.
Figure 22: The effectiveness of different steering strengths α for language and function transport and install.
Figure 23: The paraphrasing interventions largely still succeed, albeit at a lower rate than when the descriptions are kept fixed between donor and recipient.
Code LLMs often default to particular programming languages and libraries under neutral prompts. We investigate whether these preferences are encoded as approximately linear directions in activation space that can be manipulated at inference time. Using a difference-in-means method, we estimate layer-wise steering vectors for five language/library pairs and add them to model hidden states during generation. Across three open-weight code LLMs, these interventions substantially increase generation toward the target ecosystem under neutral prompts and often remain effective even when prompts explicitly request the opposite choice. Steering strength varies by model and target, with common ecosystems easier to induce than rarer alternatives, and overly strong interventions can reduce output quality. Overall, our results suggest that code-style preferences in LLMs are partly represented by compact, steerable structure in activation space.
Md Mahbubur Rahman, Arjun Guha, Harshitha Menon
Iowa State University · Northeastern University · Lawrence Livermore National Laboratory
LLMs are widely used for code generation and mathematical reasoning tasks where they are required to generate structured output. They either need to reason about code, generate code for a given specification, or reason using programs of thought. The typical approach to code generation is to prompt the model and generate samples until an appropriate program is obtained. Within this process, sampling n programs from the language model requires n GPU compute-intensive generations which becomes prohibitively expensive for larger values of n. In this work, we address this limitation by exposing the LLM's distribution within the generated programs themselves. We propose a novel test-time framework we dub probabilistic programs of thought to obtain more samples from the model with fewer LLM generations. Given a program generated by a model and the associated next-token probabilities, we build a probabilistic program that compactly represents exponentially many deterministic programs. Since performing probabilistic reasoning in this probabilistic program is much cheaper, our approach allows sampling new programs without any additional GPU compute and little CPU overhead. We instantiate our approach on benchmarks for code generation, code understanding and mathematical reasoning and report improvements in performance with fewer generations from the LLM.
Poorva Garg, Renato Lui Geh, Daniel Israel +3
University of California Los Angeles · Allen Institute for AI
Large language models (LLMs) achieve high pass rates on code generation benchmarks, yet whether they can transfer this ability to languages absent from pretraining remains poorly understood. We introduce PyLang, a minimal imperative language absent from all pretraining corpora, and evaluate frontier models zero-shot and fine-tuned Qwen3 (4B, 8B, 32B) on 352 problems. We find that fine-tuning quickly teaches syntax but fails to transfer semantic competence: Python outperforms PyLang by up to 19% across all configurations, and no intervention (multi-task learning, preference tuning, code infilling, or latent-space objectives) closes the gap. An LLM judge reveals that frontier models select an identical algorithm to Python 80% of the time, yet cannot translate it into a working PyLang implementation., and CKA analysis confirms that fine-tuned models converge to nearly identical internal representations across languages (CKA > 0.97) while diverging at the output stage. We term this the implementation fidelity gap: models possess language-agnostic algorithmic understanding but cannot express it in an unfamiliar language. Our findings highlight the need for training methods that decouple reasoning from language-specific realization.