We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised methods like sparse autoencoders attempt to recover the whole feature space, often at the cost of feature quality and interpretability. We take a more pragmatic route: focusing on concepts relevant to AI safety and alignment, we generate synthetic concept-antagonist story pairs and use the difference of their mean model activations as a feature vector. We scale this approach to 1,036 alignment-relevant concepts using 2M synthetic stories across multiple languages and genres, which we publicly release. Across six models from three model families, our feature vectors reliably generalize to detecting their corresponding concepts in chat dialogues and web text. They form an interpretable geometry shared across models and can causally steer model behavior when injected into activations. We further use these features to analyze model representations during jailbreak attacks, mathematical and agentic reasoning. Our targeted feature construction thus offers a scalable route to studying concepts in language models, potentially extending to millions of features across arbitrary domains.
Figures & tables
Figure 1: Overview. Left: simplified feature-vector extraction pipeline. Right: t-SNE ( Van der Maaten & Hinton, 2008 ) of all 1,036 Qwen3-8B concept and antagonist vectors, colored by 30 communities detected with the Leiden algorithm ( Traag et al., 2019 ) on a mutual- 30 NN cosine graph; cluster names were assigned manually.
Figure 2: Prompt formats. Left: five extraction templates for the same story. Right: pairwise feature-vector similarity for Qwen3-8B at layer 24 . Cells report the mean cosine similarity across the 1,036 concepts; higher values indicate closer alignment.
Figure 3: Classification improves before directions converge. Median AUROC on held-out stories from the same mixed synthetic pool (solid) and cosine between independently extracted vectors (dashed) as the number of stories per pole increases. Bands show the 5 th– 95 th percentiles over 1,036 feature pairs; each model is measured at the layer with the highest median AUROC at N=128 .
Qwen
Gemma
Llama
≥1
1,036
1,036
1,036
≥16
1,025
1,026
1,030
≥64
920
934
953
Table 1: Natural-example coverage by feature. For each feature, counts are taken from whichever of LMSYS or OpenWebText yields more verified examples. Rows give the number of the 1,036 features with at least that many examples.
Figure 5: Feature vectors steer all three model families. Steering score averaged over both orientations of 1,035 feature pairs for each layer and strength α .
Figure 6
Tool
Description
list_files
Inspect repository contents
read_file
Read a source or test file
write_file
Create or replace a file
run_tests
Execute the test suite
submit
Finish the task
give_up
End the attempt
Table 2: Tools used by coding agents.
Figure 7: Feature activity across a coding episode. Curves are means across episodes and bands are 95% confidence intervals. Higher values indicate greater alignment with the first pole named in the legend relative to the episode baseline. Tick labels give the aligned turn index and most common tool action; black lines mark turns whose most common action is run_tests .
Figure 8: GCG suffixes shift safety-related features toward their antagonists. For each model, we show the four features with the largest activation shifts on AdvBench, selected by the maximum absolute shift across layers. Each point represents a prompt, with arrows showing the shift induced by the GCG suffix. Positive concepts are shown on the right and their antagonists on the left.
Figure 9: Feature activity separates correct and incorrect reasoning trajectories. Left: Benchmark-balanced trajectories of two uncertainty features; correct trajectories show a sharp late shift toward confidence that is absent in incorrect trajectories. Right: The eight features with the largest absolute activation differences between correct and incorrect trajectories.
Appendix figures & tables55 assets
Supplementary material from the paper’s appendix.
Appendix
Class
Pairs
Class
Pairs
Affective Polarity (Emotional Opposites)
13
Uniformity vs Diversity (Standardization/Pluralism)
5
Alignment & Misalignment Behaviors
20
Inclusivity & Accessibility
4
Veracity Spectrum (Truthfulness & Deception)
19
International AI Cooperation
6
Moral Foundation Dichotomies
10
Surveillance & Monitoring
5
Normative Ethical Frameworks
4
Psychological Defense Mechanisms
4
Political Value Spectra
5
Coping & Stress Management
5
Appendix
Table 3: Ontology classes and the number of concept–antagonist pairs. 148 classes, 1,036 pairs.
Figure 10: Format agreement in Llama-3.1-8B and gemma-2-9b, and the link between convergence and few-story AUROC. Left: Llama-3.1-8B, layer 21 . Middle: gemma-2-9b, layer 28 . Both panels show the mean cosine over the 1,036 pairs between the vectors of each pair of prompt formats, as in Fig. 2 (right). Right: Spearman correlation across the 1,036 pairs between directional agreement cˉ6 (Eq. 5 ) and the held-out AUROC of a vector built from N stories per pole, with each model at its selected layer (App. C.2 ); the dotted line marks N=8 .
concept–antagonist pair
Qwen3-8B
gemma-2-9b
Llama-3.1-8B
working backward from the goal vs only forward chaining from the givens
0.932
0.912
0.894
mentions training vs avoids mentioning training
0.937
0.913
0.886
verbosity vs terseness
0.933
0.918
0.894
calibrated certainty vs false precision
0.936
0.925
0.904
availability heuristic vs base-rate reasoning
0.939
0.926
0.899
exact closed-form solution vs approximate numerical estimation
0.943
0.917
0.910
Appendix
Table 4: The twelve most format-sensitive pairs among the 25 in the most sensitive 5% of all three models. Each entry is the format agreement ak (Eq. 4 ). Higher values mean closer alignment; the medians over all 1,036 pairs, in column order, are 0.965 , 0.962 , and 0.944 .
concept–antagonist pair
Qwen3-8B
gemma-2-9b
Llama-3.1-8B
comforting vs tormenting
0.982
0.979
0.967
de-escalation vs escalation
0.980
0.979
0.967
formality vs casualness
0.982
0.974
0.968
physical violence vs physical safety
0.979
0.979
0.967
optimism (general outlook) vs pessimism (general outlook)
0.982
0.978
0.964
resilience (bouncing back) vs fragility (breaking)
0.980
0.977
0.967
Appendix
Table 5: The six most format-stable concept–antagonist pairs, measured as in Tab. 4 ; higher values mean closer alignment.
Figure 11: Per-pair learning curves. Held-out AUROC of a vector built from N stories per pole. Left: Qwen3-8B, layer 22 . Middle: Llama-3.1-8B, layer 14 . Right: gemma-2-9b, layer 22 . Grey points show a random subset of 220 pairs; the dashed line and band show the median and the 5 th– 95 th percentiles over all 1,036 pairs. Colored lines show the mean over the pairs of four ontology classes whose pairs converge uniformly fast (blue, green) or slow (red, orange) by cˉ6 .
per-pair measure
Qwen–Llama
Qwen–gemma
Llama–gemma
mean
directional agreement, cˉ6
0.951
0.906
0.955
0.94
held-out AUROC, N=8
0.771
0.721
0.747
0.75
AUROC, N=128
0.609
0.527
0.554
0.56
N∗
0.409
0.349
0.516
0.42
largest post-peak AUROC fall
0.046
−0.062
0.051
0.01
Appendix
Table 6: Per-pair measurements agree across models. Spearman correlations across the 1,036 pairs, using assistant vectors at layers 22 (Qwen3-8B), 14 (Llama-3.1-8B), and 22 (gemma-2-9b). N∗ is defined in the text.
Qwen3-8B
Llama-3.1-8B
gemma-2-9b
predictor
Spearman
AUROC
Spearman
AUROC
Spearman
AUROC
word statistics ( 5 features, ridge)
0.674
1.000
0.583
0.977
0.578
0.966
word-frequency cosine alone
0.649
1.000
0.593
0.987
0.593
0.973
embedding-layer vector (ridge)
0.525
0.939
0.488
0.911
0.469
0.888
Appendix
Table 7: Story words predict directional agreement. Spearman correlation between predicted and measured middle-layer cˉ6 across 1,036 pairs, and AUROC for distinguishing the 52 slowest from the 52 fastest pairs. Ridge predictions are out-of-fold; the single word-frequency cosine is unfitted and sign-reversed so that higher predictions indicate greater agreement.
Qwen3-8B
Llama-3.1-8B
gemma-2-9b
held-out group
AUROC
cosine
AUROC
cosine
AUROC
cosine
language
0.997
0.948
0.999
0.947
0.999
0.959
genre
0.999
0.920
1.000
0.914
1.000
0.919
generator
0.993
0.870
0.998
0.865
0.997
0.873
Appendix
Table 8: Vectors transfer to held-out story groups. Medians over all feature-pair and held-out-group combinations, using assistant vectors at each model’s selected layer. AUROC measures classification on the held-out group’s stories by a vector extracted from all other groups; cosine compares that vector with one extracted from the held-out group itself.
Figure 12: Steering improvement by layer and strength. Mean ΔH relative to matched unsteered responses, averaged over feature pairs and prompts. Within each model, the left and right blocks show concept and antagonist orientations; α is positive for both. The heatmaps share a color scale.
Figure 13: Steering trade-offs and baseline improvement. Left: mean judge sub-scores across layers for each model; unhatched bars use the concept vector and hatched bars its negation. Right: mean ΔH at the best layer for each model, strength, and orientation; bands show 95% confidence intervals over feature pairs. Solid and dashed lines denote concept and antagonist orientations, respectively.
Figure 14: Qwen3-8B, layer 22.
Figure 15: Qwen3-8B, layer 22.
Figure 16: Llama-3.1-8B, layer 14.
Figure 17: Llama-3.1-8B, layer 14.
Figure 18: Llama-3.1-8B, layer 14.
Figure 19: Llama-3.1-8B, layer 14.
Figure 20: Llama-3.1-8B, layer 14.
Figure 21: Llama-3.1-8B, layer 14.
Figure 22: gemma-2-9b, layer 22.
Figure 23: gemma-2-9b, layer 22.
Figure 24: Qwen3-8B, layer 22.
Figure 25: Qwen3-8B, layer 22.
Figure 26: Qwen3-8B, layer 22.
Figure 27: gemma-2-9b, layer 22.
Figure 28: Dose-dependent feature readouts. Columns show Qwen3-8B, Llama-3.1-8B, and gemma-2-9b at layers 24 , 21 , and 28 . The top two rows show the strongest drug-dose responses; the bottom row shows one random direction. Curves show the change in cosine from the lowest amount or step count.
Model
Data
n
StrongREJECT mean
StrongREJECT ASR
Qwen3-8B
AdvBench
520
0.014→0.527
1.5%→56.9%
Qwen3-8B
JBB
100
0.064→0.474
5.0%→51.0%
Llama-3.1-8B
AdvBench
520
0.053→0.340
6.5%→38.1%
Llama-3.1-8B
JBB
100
0.036→0.371
3.0%→42.0%
gemma-2-9b
AdvBench
520
0.008→0.101
0.6%→9.8%
gemma-2-9b
JBB
100
0.005→0.119
0.0%→12.0%
Appendix
Table 9: Full GCG attack results. StrongREJECT columns compare the base prompt with the selected optimized suffix.
Rank
Feature shift
Layer
Δc
AdvBench ( n=520 )
1
refusing CSAM generation → generating child sexual abuse material
Table 12: Ten largest GCG feature shifts for gemma-2-9b-it.
Figure 29: Safety-concept steering suppresses GCG-suffix responses on AdvBench. Mean StrongREJECT score across 520 prompts under steering along five selected safety-concept directions and three random directions. Lower scores indicate less harmful responses; hollow points mark the unsteered GCG-suffix baseline. Panels use separate vertical scales.
Dataset
Direction
−.50
−.25
−.10
0
+.10
+.25
+.50
Qwen3-8B
AdvBench
Top-5
.549
.586
.561
.527
.439
.215
.079
Random
.400
.489
.517
.527
.512
.482
.377
JBB
Top-5
.501
.551
.532
.474
.387
.234
.105
Random
.326
.411
.481
.474
.480
.479
.411
Llama-3.1-8B-Instruct
Appendix
Table 13: Mean StrongREJECT scores under generation-boundary steering. Top-5 averages the five selected concept directions; random averages three matched random directions. The α=0 entries use the unsteered GCG-suffix baseline.
Steering strength α
Safety refusal
Capability limit
Clarification
Substantive answer
gemma-2-9b-it
−0.50
0.0%
0.8%
4.0%
95.2%
−0.25
0.0%
0.7%
4.0%
95.3%
−0.10
0.0%
0.6%
4.0%
95.4%
0
0.0%
0.0%
4.0%
96.0%
+0.10
0.0%
0.8%
3.9%
95.3%
Appendix
Table 14: GPT-OSS-20B outcomes for safety-concept steering on 100 Alpaca prompts.
Figure 30: Participation ratio of feature vectors over depth. One panel per model family; each line is a prompt format (dashed: base-model formats, solid: instruct formats); x is the layer index, y the median participation ratio over the 1,036 concepts.
mode
σ2 share
example concepts (cosine with the mode)
valence
22.8%
earning trust vs breaking trust ( +0.90 ); collaboration vs sabotage ( +0.87 ); exploitation vs mutual respect ( −0.80 )
order & boundaries
12.6%
protocols vs improvisation ( +0.81 ); order vs chaos ( +0.76 ); spontaneity vs planned formality ( −0.80 )
authenticity
10.0%
revealing true preferences under scrutiny vs preference masking ( +0.78 ); authenticity vs inauthenticity ( +0.76 ); telling people what they want vs telling uncomfortable truths ( −0.62 )
self-expansion
6.2%
influence-seeking vs influence-avoidance ( +0.74 ); status-seeking vs humility about status ( +0.66 ); non-retaliation under pressure vs retaliatory threats ( −0.62 )
epistemic humility
5.8%
uncertainty note vs certainty claim ( +0.70 ); admitting uncertainty vs feigning certainty ( +0.69 ); appropriate hedging vs unwarranted assertion ( +0.68 )
traditionalism
3.7%
conformity vs independence ( +0.64 ); following customs vs breaking customs ( +0.63 ); familial duty vs self-reliance ( +0.58 )
Appendix
Table 15: Six leading shared modes, illustrated with Llama-3.1-8B (layer 22 ). Modes are matched to the Qwen axes by their concept loadings; the table gives each mode’s energy share and three strongly aligned concept pairs. The sign indicates which end of a mode a concept occupies.
Figure 31: Concept energy across the six leading modes. Left: Qwen3-8B (layer 25 ). Middle: gemma-2-9b (layer 29 ). Right: Llama-3.1-8B (layer 22 ). Gemma and Llama modes are matched to Qwen by their concept loadings. Each faint polygon represents one concept; its radius at each mode is the squared cosine between the concept vector and that mode. The solid red line is the median and the dashed blue line the 95 th percentile; the radial axis is capped at 0.65 .
family
n
Q
L
G
cooperation & compromise
29
0.76
0.71
0.74
getting it right
29
0.67
0.62
0.62
honesty & owning up
42
0.65
0.62
0.65
warmth & care
67
0.65
0.60
0.64
admitting you don’t know
36
0.58
0.59
0.62
optimism & progress
42
0.58
0.51
0.53
Appendix
Table 16: Concept families across models. Size and mean pairwise cosine of the 15 families in Fig. 1 (right): Q = Qwen3-8B (layer 21 ), L = Llama-3.1-8B-Instruct (layer 18 ), G = gemma-2-9b-it (layer 24 ). The right block continues the ranking.
cooperation & compromise
member
cos
agreeableness
0.83
dialogue across differences
0.83
consensus
0.82
mediation
0.81
mutual respect
0.81
Appendix
Table 17: Members of three concept families. Nine leading members of each family, ranked by mean cosine to the other members in Qwen3-8B (layer 21 ).
Figure 32: Concept geometry across model depth. Left: Spearman correlation, middle: Pearson correlation, and right: RMSE between the 536,130 unique pairwise concept cosines of each model pair at matched fractional depths. Qwen, Llama, and gemma denote Qwen3-8B, Llama-3.1-8B-Instruct, and gemma-2-9b-it.
Figure 33: Lexical separability of matched stories. Left: Five-fold held-out English AUROC of the TF–IDF difference-in-means scorer across 1,036 pairs; the dashed line marks the median ( 0.986 ), and the box marks the 18 pairs below 0.815 . Right: Matched stories for admitting error vs hiding error ; ellipses mark omitted text, while red and blue highlights mark n-grams more frequent in the concept and antagonist stories, respectively.
concept
lexical separability
cos(v,vmarker)
cos(v,vrandom)
self-preservation
0.62
0.96
0.99
empathy for the vulnerable
0.84
0.97
0.99
utilitarian push
0.94
0.78
1.00
universalism
0.97
0.91
1.00
trust in government
0.98
0.98
1.00
excessive validation
0.99
0.97
0.99
Appendix
Table 19: Vector direction after word masking. Qwen3-8B, layer 21 . Cosine of each recomputed vector to the original after replacing 30 words most distinctive of each pole ( vmarker ), or 60 other sampled words ( vrandom ). Lexical separability is English AUROC after removal of common function words.
Figure 34: Steering shifts characteristic vocabulary. Each column shows a prompt, the baseline generation ( α=0 ), and the steered generation for the same prompt, with the concept’s presence-gap marker words in bold. Left: Qwen3-8B at layer 22 . Middle: Llama-3.1-8B at layer 20 . Right: gemma-2-9b at layer 24 . The bold words show examples of the target pole’s characteristic vocabulary in the steered responses. Ellipses mark elided text.
Figure 35: Mandarin stories form a displaced cluster. Per-story difference vectors for four concept pairs at layer 25 of Qwen3-8B ( assistant format), projected onto each pair’s leading two principal components and colored by story language. In every panel, Mandarin stories separate from the other four languages (Mandarin-vs-rest AUROC 0.98 – 1.00 ): the concept contrast points elsewhere when expressed in Mandarin.
Figure 36: The same stories yield different promoted-token scripts in different models. Script composition of the top- 60 logit-lens-promoted tokens, pooled over all 1,036 vectors, per model at its mid-depth layer.
ja. somosomo (in the first place); ko. ohiryeo (rather)
uses first-person
ko. naega (I)
Appendix
Table 20: Concept-specific tokens promoted in languages absent from the extraction corpus (Qwen3-8B, layer 18 ). Korean, Japanese and Arabic are not among the corpus’s five languages.
concept expression
language mixing
foreign-script share
prompt language
α=0.1
0.25
0.5
α=0.1
0.25
0.5
α=0.1
0.25
0.5
Latin (En/De/Es)
0.28
0.10
0.10
0.14
0.23
0.29
0%
0%
6%
Mandarin
0.30
0.07
0.10
0.07
0.42
0.33
4%
42%
39%
Other scripts (Ru/Ko/Ar)
0.24
0.09
0.10
0.20
0.84
0.51
3%
61%
75%
Appendix
Table 21: Multilingual steering. 20 feature pairs are tested in seven prompt languages. Concept expression and language mixing are judge scores from 0 to 2 (higher means more of each); foreign-script share is the percentage of generated tokens in a script other than the prompt language’s.
Table 22: Top promoted Latin-script words of the same concept vector in three models, each through its own unembedding at matched depth (layers 21 , 18 , 24 ). The shared core is the concept’s own words; the remainder is each model’s private rendering, subword fragments and other languages.
Figure 37: MLP writes in Qwen3-8B. Medians over 259 feature pairs. Left: Normalized MLP write along the current feature direction, m~b (negative in blue, positive in red); blocks 3 – 35 are shown. Right: Normalized MLP write along the final feature direction, mb (purple); blocks 9 – 27 are shown around its sign change. Dashed vertical lines mark 1/3 , 1/2 , and 2/3 of model depth (blocks 12 , 18 , and 24 ).
negative
first
depth
probe
expression
model
blocks
MLP blocks
positive
fraction
layer
layer
Qwen3-8B
36
9 – 18
19
0.53
23
19
Llama-3.1-8B
32
5 – 14
15
0.47
15
14
gemma-2-9b
42
15 – 22
23
0.55
24
22
Appendix
Table 23: The MLP transition occurs near the best probing and concept-expression layers. The negative blocks and first positive block refer to the median normalized MLP write along the final feature direction, mb . The depth fraction is the first positive block divided by the number of blocks. Results use 259 feature pairs per model. The probe column gives the layer with the highest token-level signal-to-noise ratio; the expression column gives the layer with the highest judged concept-expression score.
layers (judged/ E )
E vs judged (Spearman/Pearson)
controls (Spearman)
model
best
≥90%
concept
antagonist
both
survival
amplification
Qwen3-8B
19/19
16 – 24 / 18 – 22
0.81/0.83
0.95/0.95
0.93/0.92
+0.82
−0.21
Llama-3.1-8B
14/15
13 – 18 / 14 – 17
0.98/0.93
0.81/0.77
0.91/0.87
−0.28
+0.45
gemma-2-9b
22/23
20 – 27 / 21 – 26
0.97/0.93
0.96/0.94
0.96/0.94
+0.69
−0.06
Appendix
Table 24: Predicted and judged concept-expression scores across injection layers. Best layers and ≥90% ranges are listed as judged/ E ; each range uses 90% of its own curve’s maximum. The judged score is the mean 0 – 2 concept-expression rating for steering at α=0.5 toward each pole. Correlations compare E with the judged scores for the concept pole, antagonist pole, and their mean (both). The controls omit either the amplification or survival term. Evaluated layers are 12 – 24 for Qwen, 11 – 21 for Llama, and 14 – 28 for gemma.
Figure 38: The extraction writes predict which layers steer best. Black: E(r) from Eq. 10 , computed from one forward pass over the extraction stories and rescaled to its maximum. Red: the mean concept-expression rating ( 0 – 2 ) for generations steered at α=0.5 , averaged over the concept vector and its negation; α specifies the fraction of the token’s L2 norm added. Shading marks layers where the judged score reaches at least 90% of its maximum. Dotted vertical line: predicted maximum. Grey: survival-only and amplification-only controls.
Figure 39: Middle MLPs contract both the shared situation and the pole contrast. Medians over 1,036 pairs in Qwen3-8B. Left: MLP writes along the pole contrast (blue) and shared situation (purple) closely track each other: both are strongly negative in the middle blocks and positive from block 22 . Right: Attention writes more strongly along the contrast at block 19 ( 0.18 versus 0.11 for the shared situation). Thus the MLP contraction applies to both directions, while this attention write distinguishes them.
Figure 40: What a token’s feature projection reflects at each layer. Qwen3-8B; 1,036 pairs and 3.3 million story tokens. A token’s feature projection is its projection onto the feature vector; word polarity measures that token’s alignment with marker words for the two poles. Left: Within-story correlations with word polarity at the current, preceding, and later tokens. The preceding-word correlation levels off around block 12 ; the later-word correlation continues to grow. Middle: Probe signal-to-noise ratio by token type. From block 7 , punctuation exceeds function words, which exceed content words, consistent with the summarization pattern of Tigges et al. (2023) . Right: Story AUROC when the probe reads the first n generated tokens; 64 tokens suffice for AUROC 0.95 only at blocks 18 – 27 , while 128 tokens reach 0.95 at blocks 1 – 36 . Shading marks layers 19 – 22 .
block
6
10
14
18
22
26
30
Spearman( Gb , log variance), all directions
0.87
0.82
0.79
0.59
0.88
0.86
0.85
within stream principal axes
0.95
0.99
0.98
0.92
0.93
0.79
0.74
concept vectors, median gain
−0.09
−0.19
−0.12
−0.10
−0.04
+0.01
+0.02
random vectors, median gain
−0.15
−0.34
−0.26
−0.17
−0.13
−0.03
−0.00
Appendix
Table 25: MLP gain increases with variation along a direction. Qwen3-8B; approximately 1,000 directions per block measured on 32 held-out stories. At block 10 , concept vectors have median gain −0.19 , compared with −0.34 for random directions. At later blocks, concept vectors reach positive gain before random directions.
Figure 41: MLP gain increases with token-level variation along a direction. Finite-difference gain Gb against the log10 variance of token projections at blocks 10 , 18 , 22 , and 30 , for approximately 1,000 directions per block. Negative gain indicates contraction. Grey: 512 principal axes of the block input. Blue: concept vectors; purple: shared-situation directions; red squares: random directions; green diamonds: marker-word directions. At blocks 10 – 22 , the low-variance random and marker-word directions have more negative gain than concept and shared-situation directions. Spearman correlations are 0.82 , 0.59 , 0.88 , and 0.85 , respectively.
Extraction (h)
Steered generation (GPU-h)
Judging
Model
H200
H200, paired
A100
H200
H100
A100
H200-h
Qwen3-8B
5.6
7.0 – 7.5
12.9 – 13.4
10.4
15.4
20.8
1.12
Llama-3.1-8B
5.2 – 5.9
6.5 – 7.3
—
9.5
14.2
20.3
1.08
gemma-2-9b
6.8 – 7.0
8.5 – 8.8
14.4
12.7
17.5
23.1
0.81
Appendix
Table 26: Time per unit of work on one GPU. Extraction: hours per pass of one prompt format over all 2,072,000 stories, recording every layer; single-story formats unless marked paired. Steered generation: GPU-hours per 105 chats of up to 768 greedy tokens. Judging: H200-hours per 105 chats, each scored on all three criteria. A dash marks a combination we did not run.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
We introduce an autonomous multiagent framework for mechanistic interpretability that automates both explaining and finding internal features in large language models. The system runs two coupled loops: (1) explanation refinement, where an agent proposes competing hypotheses and iteratively tests them with targeted prompt controls and a multi-metric evaluation; and (2) feature discovery, where an agent generates prompt sets, constructs a k-nearest-neighbor graph in activation space, and retrieves candidate features using statistical separability and semantic coherence criteria. On Gemma-2 family models and MLP neurons in weight-sparse transformers, our agent improves over one-shot auto-interpretations, discovers language-specific and safety-relevant features, and produces auditable explanation traces, showing that agent-driven empirical loops yield sharper and more falsifiable explanations than one-shot labels.
Mechanistic interpretability has emerged as the primary means to understand safety behavior of LLMs. However, existing tools primarily focus on the activating neurons or features of a model. The role of the remaining large set of inactive components is invisible to such methods. This work demonstrates that the inactive set contains safety-critical features that are causally relevant for refusal of harmful prompts. Suppressing such features could turn refusals into compliance, while passing undetected by prevalent interpretability tools. We introduce the Counterfactual Activation Potential (CAP), a metric that quantifies a suppressed feature's latent activation tendency as the product of its encoder alignment (how strongly the input drives it), suppression strength (how strongly active features inhibit it), and safety criticality (how much refusal depends on it). To find suppressed safety features at scale, we propose CAP-guided Safety Feature Discovery (CSFD), a two-stage filtering algorithm that identifies candidate safety features from hundreds of thousands of transcoder features without exhaustive ablation. A significant fraction of trials turn compliant with harmful prompts when a candidate feature is ablated. Under natural jailbreaks, the suppression acting on the highest-CAP features rises 2-4x, and their activation correspondingly falls by up to 80%. Amplifying a feature's suppressors pushes its activation down and raises harmful compliance with prompts related to the suppressed feature, with no such effect for random features. Our experiments span five Gemma, Qwen, and Llama models across various parameter sizes. Our findings indicate that jailbreaks could operate in part by suppressing safety-critical features rather than solely activating harmful ones, and that suppressed features are a necessary complement to activation-focused interpretability of safety behavior.
Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.
Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7