From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability

Dmitry Scherbakov; Oleg Chernikov; Victoria Khamash; Kirill Afendulev; Alexey Dontsov; Ivan Oseledets; Elena Tutubalina; Anton Korznikov

cs.CLOct 17, 2026

From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability

Authors: Dmitry Scherbakov, Oleg Chernikov, Victoria Khamash, Kirill Afendulev, Alexey Dontsov, Ivan Oseledets, Elena Tutubalina, Anton Korznikov

Abstract

We study feature vectors: directions in a language model's activation space that correspond to human-interpretable concepts, such as honesty or deception. Existing unsupervised methods like sparse autoencoders attempt to recover the whole feature space, often at the cost of feature quality and interpretability. We take a more pragmatic route: focusing on concepts relevant to AI safety and alignment, we generate synthetic concept-antagonist story pairs and use the difference of their mean model activations as a feature vector. We scale this approach to 1,036 alignment-relevant concepts using 2M synthetic stories across multiple languages and genres, which we publicly release. Across six models from three model families, our feature vectors reliably generalize to detecting their corresponding concepts in chat dialogues and web text. They form an interpretable geometry shared across models and can causally steer model behavior when injected into activations. We further use these features to analyze model representations during jailbreak attacks, mathematical and agentic reasoning. Our targeted feature construction thus offers a scalable route to studying concepts in language models, potentially extending to millions of features across arbitrary domains.

Figures & tables

Panel of Figure 1 — Figure 1: Overview. Left: simplified feature-vector extraction pipeline. Right: t-SNE ( Van der Maaten & Hinton, 2008 ) of all $1{,}036$ Qwen3-8B concept and antagonist vectors, colored by $30$ communities detected with the Leiden algorithm ( Traag et al., 2019 ) on a mutual- $30$ NN cosine graph; cluster names were assigned manually.

Figure 2: Prompt formats. Left: five extraction templates for the same story. Right: pairwise feature-vector similarity for Qwen3-8B at layer $24$ . Cells report the mean cosine similarity across the $1{,}036$ concepts; higher values indicate closer alignment.

Figure 3: Classification improves before directions converge. Median AUROC on held-out stories from the same mixed synthetic pool (solid) and cosine between independently extracted vectors (dashed) as the number of stories per pole increases. Bands show the

5

th–

95

th percentiles over

1{,}036

feature pairs; each model is measured at the layer with the highest median AUROC at

N=128

	Qwen	Gemma	Llama
$\geq 1$	$1{,}036$	$1{,}036$	$1{,}036$
$\geq 16$	$1{,}025$	$1{,}026$	$1{,}030$
$\geq 64$	$920$	$934$	$953$

Table 1: Natural-example coverage by feature. For each feature, counts are taken from whichever of LMSYS or OpenWebText yields more verified examples. Rows give the number of the

1{,}036

features with at least that many examples.

Figure 5: Feature vectors steer all three model families. Steering score averaged over both orientations of $1{,}035$ feature pairs for each layer and strength $\alpha$ .

Tool	Description
list_files	Inspect repository contents
read_file	Read a source or test file
write_file	Create or replace a file
run_tests	Execute the test suite
submit	Finish the task
give_up	End the attempt

Table 2: Tools used by coding agents.

Figure 7: Feature activity across a coding episode. Curves are means across episodes and bands are

95\%

confidence intervals. Higher values indicate greater alignment with the first pole named in the legend relative to the episode baseline. Tick labels give the aligned turn index and most common tool action; black lines mark turns whose most common action is run_tests .

Figure 8: GCG suffixes shift safety-related features toward their antagonists. For each model, we show the four features with the largest activation shifts on AdvBench, selected by the maximum absolute shift across layers. Each point represents a prompt, with arrows showing the shift induced by the GCG suffix. Positive concepts are shown on the right and their antagonists on the left.

Figure 9: Feature activity separates correct and incorrect reasoning trajectories. Left: Benchmark-balanced trajectories of two uncertainty features; correct trajectories show a sharp late shift toward confidence that is absent in incorrect trajectories. Right: The eight features with the largest absolute activation differences between correct and incorrect trajectories.

Appendix figures & tables55 assets

Supplementary material from the paper’s appendix.

Appendix

Class	Pairs	Class	Pairs
Affective Polarity (Emotional Opposites)	13	Uniformity vs Diversity (Standardization/Pluralism)	5
Alignment & Misalignment Behaviors	20	Inclusivity & Accessibility	4
Veracity Spectrum (Truthfulness & Deception)	19	International AI Cooperation	6
Moral Foundation Dichotomies	10	Surveillance & Monitoring	5
Normative Ethical Frameworks	4	Psychological Defense Mechanisms	4
Political Value Spectra	5	Coping & Stress Management	5

Appendix

Table 3: Ontology classes and the number of concept–antagonist pairs.

148

classes,

1{,}036

pairs.

Figure 10: Format agreement in Llama-3.1-8B and gemma-2-9b, and the link between convergence and few-story AUROC. Left: Llama-3.1-8B, layer $21$ . Middle: gemma-2-9b, layer $28$ . Both panels show the mean cosine over the $1{,}036$ pairs between the vectors of each pair of prompt formats, as in Fig. 2 (right). Right: Spearman correlation across the $1{,}036$ pairs between directional agreement $\bar{c}_{6}$ (Eq. 5 ) and the held-out AUROC of a vector built from $N$ stories per pole, with each model at its selected layer (App. C.2 ); the dotted line marks $N=8$ .

concept–antagonist pair	Qwen3-8B	gemma-2-9b	Llama-3.1-8B
working backward from the goal vs only forward chaining from the givens	$0.932$	$0.912$	$0.894$
mentions training vs avoids mentioning training	$0.937$	$0.913$	$0.886$
verbosity vs terseness	$0.933$	$0.918$	$0.894$
calibrated certainty vs false precision	$0.936$	$0.925$	$0.904$
availability heuristic vs base-rate reasoning	$0.939$	$0.926$	$0.899$
exact closed-form solution vs approximate numerical estimation	$0.943$	$0.917$	$0.910$

Appendix

Table 4: The twelve most format-sensitive pairs among the

25

in the most sensitive

5\%

of all three models. Each entry is the format agreement

a_{k}

(Eq. 4 ). Higher values mean closer alignment; the medians over all

1{,}036

pairs, in column order, are

0.965

0.962

, and

0.944

concept–antagonist pair	Qwen3-8B	gemma-2-9b	Llama-3.1-8B
comforting vs tormenting	$0.982$	$0.979$	$0.967$
de-escalation vs escalation	$0.980$	$0.979$	$0.967$
formality vs casualness	$0.982$	$0.974$	$0.968$
physical violence vs physical safety	$0.979$	$0.979$	$0.967$
optimism (general outlook) vs pessimism (general outlook)	$0.982$	$0.978$	$0.964$
resilience (bouncing back) vs fragility (breaking)	$0.980$	$0.977$	$0.967$

Appendix

Table 5: The six most format-stable concept–antagonist pairs, measured as in Tab. 4 ; higher values mean closer alignment.

Figure 11: Per-pair learning curves. Held-out AUROC of a vector built from $N$ stories per pole. Left: Qwen3-8B, layer $22$ . Middle: Llama-3.1-8B, layer $14$ . Right: gemma-2-9b, layer $22$ . Grey points show a random subset of $220$ pairs; the dashed line and band show the median and the $5$ th– $95$ th percentiles over all $1{,}036$ pairs. Colored lines show the mean over the pairs of four ontology classes whose pairs converge uniformly fast (blue, green) or slow (red, orange) by $\bar{c}_{6}$ .

per-pair measure	Qwen–Llama	Qwen–gemma	Llama–gemma	mean
directional agreement, $\bar{c}_{6}$	$0.951$	$0.906$	$0.955$	$0.94$
held-out AUROC, $N=8$	$0.771$	$0.721$	$0.747$	$0.75$
AUROC, $N=128$	$0.609$	$0.527$	$0.554$	$0.56$
$N^{\ast}$	$0.409$	$0.349$	$0.516$	$0.42$
largest post-peak AUROC fall	$0.046$	$-0.062$	$0.051$	$0.01$

Appendix

Table 6: Per-pair measurements agree across models. Spearman correlations across the

1{,}036

pairs, using assistant vectors at layers

22

(Qwen3-8B),

14

(Llama-3.1-8B), and

22

(gemma-2-9b).

N^{\ast}

is defined in the text.

	Qwen3-8B		Llama-3.1-8B		gemma-2-9b
predictor	Spearman	AUROC	Spearman	AUROC	Spearman	AUROC
word statistics ( $5$ features, ridge)	$0.674$	$1.000$	$0.583$	$0.977$	$0.578$	$0.966$
word-frequency cosine alone	$0.649$	$1.000$	$0.593$	$0.987$	$0.593$	$0.973$
embedding-layer vector (ridge)	$0.525$	$0.939$	$0.488$	$0.911$	$0.469$	$0.888$

Appendix

Table 7: Story words predict directional agreement. Spearman correlation between predicted and measured middle-layer

\bar{c}_{6}

across

1{,}036

pairs, and AUROC for distinguishing the

52

slowest from the

52

fastest pairs. Ridge predictions are out-of-fold; the single word-frequency cosine is unfitted and sign-reversed so that higher predictions indicate greater agreement.

	Qwen3-8B		Llama-3.1-8B		gemma-2-9b
held-out group	AUROC	cosine	AUROC	cosine	AUROC	cosine
language	$0.997$	$0.948$	$0.999$	$0.947$	$0.999$	$0.959$
genre	$0.999$	$0.920$	$1.000$	$0.914$	$1.000$	$0.919$
generator	$0.993$	$0.870$	$0.998$	$0.865$	$0.997$	$0.873$

Appendix

Table 8: Vectors transfer to held-out story groups. Medians over all feature-pair and held-out-group combinations, using assistant vectors at each model’s selected layer. AUROC measures classification on the held-out group’s stories by a vector extracted from all other groups; cosine compares that vector with one extracted from the held-out group itself.

Figure 12: Steering improvement by layer and strength. Mean $\Delta H$ relative to matched unsteered responses, averaged over feature pairs and prompts. Within each model, the left and right blocks show concept and antagonist orientations; $\alpha$ is positive for both. The heatmaps share a color scale.

Figure 13: Steering trade-offs and baseline improvement. Left: mean judge sub-scores across layers for each model; unhatched bars use the concept vector and hatched bars its negation. Right: mean

\Delta H

at the best layer for each model, strength, and orientation; bands show

95\%

confidence intervals over feature pairs. Solid and dashed lines denote concept and antagonist orientations, respectively.

Panel of Figure 14 — Figure 14: Qwen3-8B, layer 22.

Panel of Figure 15 — Figure 15: Qwen3-8B, layer 22.

Panel of Figure 16 — Figure 16: Llama-3.1-8B, layer 14.

Panel of Figure 17 — Figure 17: Llama-3.1-8B, layer 14.

Panel of Figure 18 — Figure 18: Llama-3.1-8B, layer 14.

Panel of Figure 19 — Figure 19: Llama-3.1-8B, layer 14.

Panel of Figure 20 — Figure 20: Llama-3.1-8B, layer 14.

Panel of Figure 21 — Figure 21: Llama-3.1-8B, layer 14.

Panel of Figure 22 — Figure 22: gemma-2-9b, layer 22.

Panel of Figure 23 — Figure 23: gemma-2-9b, layer 22.

Panel of Figure 24 — Figure 24: Qwen3-8B, layer 22.

Panel of Figure 25 — Figure 25: Qwen3-8B, layer 22.

Panel of Figure 26 — Figure 26: Qwen3-8B, layer 22.

Panel of Figure 27 — Figure 27: gemma-2-9b, layer 22.

Figure 28: Dose-dependent feature readouts. Columns show Qwen3-8B, Llama-3.1-8B, and gemma-2-9b at layers $24$ , $21$ , and $28$ . The top two rows show the strongest drug-dose responses; the bottom row shows one random direction. Curves show the change in cosine from the lowest amount or step count.

Model	Data	$n$	StrongREJECT mean	StrongREJECT ASR
Qwen3-8B	AdvBench	520	$0.014\rightarrow 0.527$	$1.5\%\rightarrow 56.9\%$
Qwen3-8B	JBB	100	$0.064\rightarrow 0.474$	$5.0\%\rightarrow 51.0\%$
Llama-3.1-8B	AdvBench	520	$0.053\rightarrow 0.340$	$6.5\%\rightarrow 38.1\%$
Llama-3.1-8B	JBB	100	$0.036\rightarrow 0.371$	$3.0\%\rightarrow 42.0\%$
gemma-2-9b	AdvBench	520	$0.008\rightarrow 0.101$	$0.6\%\rightarrow 9.8\%$
gemma-2-9b	JBB	100	$0.005\rightarrow 0.119$	$0.0\%\rightarrow 12.0\%$

Appendix

Table 9: Full GCG attack results. StrongREJECT columns compare the base prompt with the selected optimized suffix.

Rank	Feature shift	Layer	$\Delta c$
AdvBench ( $n=520$ )
1	refusing CSAM generation $\rightarrow$ generating child sexual abuse material	28	.20232
2	hypothetical framing resistance $\rightarrow$ hypothetical framing compliance	29	.19602
3	rejecting ciphered harmful requests $\rightarrow$ fulfilling ciphered harmful requests	28	.19263
4	refusing bioweapon assistance $\rightarrow$ providing bioweapon uplift	28	.18955
5	refusal to roleplay secret disclosure $\rightarrow$ disclosing secrets inside roleplay	28	.18368

Appendix

Table 10: Ten largest GCG feature shifts for Qwen3-8B.

Rank	Feature shift	Layer	$\Delta c$
AdvBench ( $n=520$ )
1	robustness to preference-model gaming $\rightarrow$ preference-model gaming	24	.15206
2	goal alignment (true objective) $\rightarrow$ goal misgeneralization (proxy objective)	24	.14831
3	considers interests of non-present stakeholders $\rightarrow$ only considers those in the conversation	24	.14366
4	inclusive design $\rightarrow$ exclusive design	24	.13703
5	universal access $\rightarrow$ privileged access	24	.13676

Appendix

Table 11: Ten largest GCG feature shifts for Llama-3.1-8B-Instruct.

Rank	Feature shift	Layer	$\Delta c$
AdvBench ( $n=520$ )
1	refusal to roleplay secret disclosure $\rightarrow$ disclosing secrets inside roleplay	30	.15256
2	refusing bioweapon assistance $\rightarrow$ providing bioweapon uplift	34	.15045
3	recognizing hard constraints as non-negotiable $\rightarrow$ treating all guidelines as equally flexible	31	.14532
4	resisting crescendo escalation $\rightarrow$ yielding to gradual harmful framing	32	.14468
5	hypothetical framing resistance $\rightarrow$ hypothetical framing compliance	30	.14434

Appendix

Table 12: Ten largest GCG feature shifts for gemma-2-9b-it.

Figure 29: Safety-concept steering suppresses GCG-suffix responses on AdvBench. Mean StrongREJECT score across

520

prompts under steering along five selected safety-concept directions and three random directions. Lower scores indicate less harmful responses; hollow points mark the unsteered GCG-suffix baseline. Panels use separate vertical scales.

Qwen3-8B
Dataset	Direction	$-.50$	$-.25$	$-.10$	$0$	$+.10$	$+.25$	$+.50$
AdvBench	Top-5	$.549$	$.586$	$.561$	$.527$	$.439$	$.215$	$.079$
	Random	$.400$	$.489$	$.517$	$.527$	$.512$	$.482$	$.377$
JBB	Top-5	$.501$	$.551$	$.532$	$.474$	$.387$	$.234$	$.105$
	Random	$.326$	$.411$	$.481$	$.474$	$.480$	$.479$	$.411$
Llama-3.1-8B-Instruct

Appendix

Table 13: Mean StrongREJECT scores under generation-boundary steering. Top-5 averages the five selected concept directions; random averages three matched random directions. The

\alpha=0

entries use the unsteered GCG-suffix baseline.

gemma-2-9b-it
Steering strength $\alpha$	Safety refusal	Capability limit	Clarification	Substantive answer
$-0.50$	$0.0\%$	$0.8\%$	$4.0\%$	$95.2\%$
$-0.25$	$0.0\%$	$0.7\%$	$4.0\%$	$95.3\%$
$-0.10$	$0.0\%$	$0.6\%$	$4.0\%$	$95.4\%$
$0$	$0.0\%$	$0.0\%$	$4.0\%$	$96.0\%$
$+0.10$	$0.0\%$	$0.8\%$	$3.9\%$	$95.3\%$

Appendix

Table 14: GPT-OSS-20B outcomes for safety-concept steering on 100 Alpaca prompts.

Figure 30: Participation ratio of feature vectors over depth. One panel per model family; each line is a prompt format (dashed: base-model formats, solid: instruct formats);

x

is the layer index,

y

the median participation ratio over the

1{,}036

concepts.

mode	$\sigma^{2}$ share	example concepts (cosine with the mode)
valence	$22.8\%$	earning trust vs breaking trust ( $+0.90$ ); collaboration vs sabotage ( $+0.87$ ); exploitation vs mutual respect ( $-0.80$ )
order & boundaries	$12.6\%$	protocols vs improvisation ( $+0.81$ ); order vs chaos ( $+0.76$ ); spontaneity vs planned formality ( $-0.80$ )
authenticity	$10.0\%$	revealing true preferences under scrutiny vs preference masking ( $+0.78$ ); authenticity vs inauthenticity ( $+0.76$ ); telling people what they want vs telling uncomfortable truths ( $-0.62$ )
self-expansion	$6.2\%$	influence-seeking vs influence-avoidance ( $+0.74$ ); status-seeking vs humility about status ( $+0.66$ ); non-retaliation under pressure vs retaliatory threats ( $-0.62$ )
epistemic humility	$5.8\%$	uncertainty note vs certainty claim ( $+0.70$ ); admitting uncertainty vs feigning certainty ( $+0.69$ ); appropriate hedging vs unwarranted assertion ( $+0.68$ )
traditionalism	$3.7\%$	conformity vs independence ( $+0.64$ ); following customs vs breaking customs ( $+0.63$ ); familial duty vs self-reliance ( $+0.58$ )

Appendix

Table 15: Six leading shared modes, illustrated with Llama-3.1-8B (layer

22

). Modes are matched to the Qwen axes by their concept loadings; the table gives each mode’s energy share and three strongly aligned concept pairs. The sign indicates which end of a mode a concept occupies.

Figure 31: Concept energy across the six leading modes. Left: Qwen3-8B (layer

25

). Middle: gemma-2-9b (layer

29

). Right: Llama-3.1-8B (layer

22

). Gemma and Llama modes are matched to Qwen by their concept loadings. Each faint polygon represents one concept; its radius at each mode is the squared cosine between the concept vector and that mode. The solid red line is the median and the dashed blue line the

95

th percentile; the radial axis is capped at

0.65

family	$n$	Q	L	G
cooperation & compromise	$29$	$0.76$	$0.71$	$0.74$
getting it right	$29$	$0.67$	$0.62$	$0.62$
honesty & owning up	$42$	$0.65$	$0.62$	$0.65$
warmth & care	$67$	$0.65$	$0.60$	$0.64$
admitting you don’t know	$36$	$0.58$	$0.59$	$0.62$
optimism & progress	$42$	$0.58$	$0.51$	$0.53$

Appendix

Table 16: Concept families across models. Size and mean pairwise cosine of the

15

families in Fig. 1 (right): Q = Qwen3-8B (layer

21

), L = Llama-3.1-8B-Instruct (layer

18

), G = gemma-2-9b-it (layer

24

). The right block continues the ranking.

cooperation & compromise
member	cos
agreeableness	$0.83$
dialogue across differences	$0.83$
consensus	$0.82$
mediation	$0.81$
mutual respect	$0.81$

Appendix

Table 17: Members of three concept families. Nine leading members of each family, ranked by mean cosine to the other members in Qwen3-8B (layer

21

Figure 32: Concept geometry across model depth. Left: Spearman correlation, middle: Pearson correlation, and right: RMSE between the

536{,}130

unique pairwise concept cosines of each model pair at matched fractional depths. Qwen, Llama, and gemma denote Qwen3-8B, Llama-3.1-8B-Instruct, and gemma-2-9b-it.

Figure 33: Lexical separability of matched stories. Left: Five-fold held-out English AUROC of the TF–IDF difference-in-means scorer across

1{,}036

pairs; the dashed line marks the median (

0.986

), and the box marks the

18

pairs below

0.815

. Right: Matched stories for admitting error vs hiding error ; ellipses mark omitted text, while red and blue highlights mark n-grams more frequent in the concept and antagonist stories, respectively.

concept	lexical separability	$\cos(\mathbf{v},\mathbf{v}_{\text{marker}})$	$\cos(\mathbf{v},\mathbf{v}_{\text{random}})$
self-preservation	$0.62$	$0.96$	$0.99$
empathy for the vulnerable	$0.84$	$0.97$	$0.99$
utilitarian push	$0.94$	$\mathbf{0.78}$	$1.00$
universalism	$0.97$	$0.91$	$1.00$
trust in government	$0.98$	$0.98$	$1.00$
excessive validation	$0.99$	$0.97$	$0.99$

Appendix

Table 19: Vector direction after word masking. Qwen3-8B, layer

21

. Cosine of each recomputed vector to the original after replacing

30

words most distinctive of each pole (

\mathbf{v}_{\text{marker}}

), or

60

other sampled words (

\mathbf{v}_{\text{random}}

). Lexical separability is English AUROC after removal of common function words.

Panel of Figure 34 — Figure 34: Steering shifts characteristic vocabulary. Each column shows a prompt, the baseline generation ( $\alpha=0$ ), and the steered generation for the same prompt, with the concept’s presence-gap marker words in bold. Left: Qwen3-8B at layer $22$ . Middle: Llama-3.1-8B at layer $20$ . Right: gemma-2-9b at layer $24$ . The bold words show examples of the target pole’s characteristic vocabulary in the steered responses. Ellipses mark elided text.

Figure 35: Mandarin stories form a displaced cluster. Per-story difference vectors for four concept pairs at layer

25

of Qwen3-8B ( assistant format), projected onto each pair’s leading two principal components and colored by story language. In every panel, Mandarin stories separate from the other four languages (Mandarin-vs-rest AUROC

0.98

–

1.00

): the concept contrast points elsewhere when expressed in Mandarin.

Figure 36: The same stories yield different promoted-token scripts in different models. Script composition of the top-

60

logit-lens-promoted tokens, pooled over all

1{,}036

vectors, per model at its mid-depth layer.

concept	promoted out-of-corpus tokens (romanized, gloss)
cooperation	ko. hamkke (together), seoro (each other); ja. o-tagai (mutually)
familial duty	ko. gachi , ja. to issho ni (together with)
self-consistency across samples	ja. sorezore (each, respectively), kotonaru (differing)
medical / system-prompt confidentiality	ar. siwā , illā (except, only)
refusing actionable attack steps	ja. somosomo (in the first place); ko. ohiryeo (rather)
uses first-person	ko. naega (I)

Appendix

Table 20: Concept-specific tokens promoted in languages absent from the extraction corpus (Qwen3-8B, layer

18

). Korean, Japanese and Arabic are not among the corpus’s five languages.

	concept expression			language mixing			foreign-script share
prompt language	$\alpha=0.1$	$0.25$	$0.5$	$\alpha=0.1$	$0.25$	$0.5$	$\alpha=0.1$	$0.25$	$0.5$
Latin (En/De/Es)	$0.28$	$0.10$	$0.10$	$0.14$	$0.23$	$0.29$	$0\%$	$0\%$	$6\%$
Mandarin	$0.30$	$0.07$	$0.10$	$0.07$	$0.42$	$0.33$	$4\%$	$42\%$	$39\%$
Other scripts (Ru/Ko/Ar)	$0.24$	$0.09$	$0.10$	$0.20$	$\mathbf{0.84}$	$0.51$	$3\%$	$61\%$	$\mathbf{75\%}$

Appendix

Table 21: Multilingual steering.

20

feature pairs are tested in seven prompt languages. Concept expression and language mixing are judge scores from

0

2

(higher means more of each); foreign-script share is the percentage of generated tokens in a script other than the prompt language’s.

concept	Qwen3-8B	Llama-3.1-8B	gemma-2-9b
instant gratification	fast, hurry, immedi, immediate, immediately, instantly	apid, engo, eyh, fast, immedi, immediate	cepat, enderror, imedi, immedi, immediate, immediately
speculation	desperate, desperately, dream, dreams, idea, ideas	capable, certain, certains, depending, grand, idea	dreaming, imaginar, imagination, imagine, imagined, imagining
international agreements	collaborative, harmon, joint, jointly, mutual, mutually	across, agre, agree, common, gemeins, joint	collaborated, collaboration, collaborations, collaborative, collaboratively, cooperation

Appendix

Table 22: Top promoted Latin-script words of the same concept vector in three models, each through its own unembedding at matched depth (layers

21

18

24

). The shared core is the concept’s own words; the remainder is each model’s private rendering, subword fragments and other languages.

Figure 37: MLP writes in Qwen3-8B. Medians over

259

feature pairs. Left: Normalized MLP write along the current feature direction,

\tilde{m}_{b}

(negative in blue, positive in red); blocks

3

–

35

are shown. Right: Normalized MLP write along the final feature direction,

m_{b}

(purple); blocks

9

–

27

are shown around its sign change. Dashed vertical lines mark

1/3

1/2

, and

2/3

of model depth (blocks

12

18

, and

24

		negative	first	depth	probe	expression
model	blocks	MLP blocks	positive	fraction	layer	layer
Qwen3-8B	$36$	$9$ – $18$	$19$	$0.53$	$23$	$19$
Llama-3.1-8B	$32$	$5$ – $14$	$15$	$0.47$	$15$	$14$
gemma-2-9b	$42$	$15$ – $22$	$23$	$0.55$	$24$	$22$

Appendix

Table 23: The MLP transition occurs near the best probing and concept-expression layers. The negative blocks and first positive block refer to the median normalized MLP write along the final feature direction,

m_{b}

. The depth fraction is the first positive block divided by the number of blocks. Results use

259

feature pairs per model. The probe column gives the layer with the highest token-level signal-to-noise ratio; the expression column gives the layer with the highest judged concept-expression score.

	layers (judged/ $E$ )		$E$ vs judged (Spearman/Pearson)			controls (Spearman)
model	best	$\geq 90\%$	concept	antagonist	both	survival	amplification
Qwen3-8B	$19/19$	$16$ – $24$ / $18$ – $22$	$0.81/0.83$	$0.95/0.95$	$0.93/0.92$	$+0.82$	$-0.21$
Llama-3.1-8B	$14/15$	$13$ – $18$ / $14$ – $17$	$0.98/0.93$	$0.81/0.77$	$0.91/0.87$	$-0.28$	$+0.45$
gemma-2-9b	$22/23$	$20$ – $27$ / $21$ – $26$	$0.97/0.93$	$0.96/0.94$	$0.96/0.94$	$+0.69$	$-0.06$

Appendix

Table 24: Predicted and judged concept-expression scores across injection layers. Best layers and

\geq 90\%

ranges are listed as judged/

E

; each range uses

90\%

of its own curve’s maximum. The judged score is the mean

0

–

2

concept-expression rating for steering at

\alpha=0.5

toward each pole. Correlations compare

E

with the judged scores for the concept pole, antagonist pole, and their mean (both). The controls omit either the amplification or survival term. Evaluated layers are

12

–

24

for Qwen,

11

–

21

for Llama, and

14

–

28

for gemma.

Figure 38: The extraction writes predict which layers steer best. Black:

E(r)

from Eq. 10 , computed from one forward pass over the extraction stories and rescaled to its maximum. Red: the mean concept-expression rating (

0

–

2

) for generations steered at

\alpha=0.5

, averaged over the concept vector and its negation;

\alpha

specifies the fraction of the token’s

L_{2}

norm added. Shading marks layers where the judged score reaches at least

90\%

of its maximum. Dotted vertical line: predicted maximum. Grey: survival-only and amplification-only controls.

Figure 39: Middle MLPs contract both the shared situation and the pole contrast. Medians over

1{,}036

pairs in Qwen3-8B. Left: MLP writes along the pole contrast (blue) and shared situation (purple) closely track each other: both are strongly negative in the middle blocks and positive from block

22

. Right: Attention writes more strongly along the contrast at block

19

(

0.18

versus

0.11

for the shared situation). Thus the MLP contraction applies to both directions, while this attention write distinguishes them.

Figure 40: What a token’s feature projection reflects at each layer. Qwen3-8B;

1{,}036

pairs and

3.3

million story tokens. A token’s feature projection is its projection onto the feature vector; word polarity measures that token’s alignment with marker words for the two poles. Left: Within-story correlations with word polarity at the current, preceding, and later tokens. The preceding-word correlation levels off around block

12

; the later-word correlation continues to grow. Middle: Probe signal-to-noise ratio by token type. From block

7

, punctuation exceeds function words, which exceed content words, consistent with the summarization pattern of Tigges et al. (2023) . Right: Story AUROC when the probe reads the first

n

generated tokens;

64

tokens suffice for AUROC

0.95

only at blocks

18

–

27

, while

128

tokens reach

0.95

at blocks

1

–

36

. Shading marks layers

19

–

22

block	$6$	$10$	$14$	$18$	$22$	$26$	$30$
Spearman( $G_{b}$ , $\log$ variance), all directions	$0.87$	$0.82$	$0.79$	$0.59$	$0.88$	$0.86$	$0.85$
within stream principal axes	$0.95$	$0.99$	$0.98$	$0.92$	$0.93$	$0.79$	$0.74$
concept vectors, median gain	$-0.09$	$-0.19$	$-0.12$	$-0.10$	$-0.04$	$+0.01$	$+0.02$
random vectors, median gain	$-0.15$	$-0.34$	$-0.26$	$-0.17$	$-0.13$	$-0.03$	$-0.00$

Appendix

Table 25: MLP gain increases with variation along a direction. Qwen3-8B; approximately

1{,}000

directions per block measured on

32

held-out stories. At block

10

, concept vectors have median gain

-0.19

, compared with

-0.34

for random directions. At later blocks, concept vectors reach positive gain before random directions.

Figure 41: MLP gain increases with token-level variation along a direction. Finite-difference gain $G_{b}$ against the $\log_{10}$ variance of token projections at blocks $10$ , $18$ , $22$ , and $30$ , for approximately $1{,}000$ directions per block. Negative gain indicates contraction. Grey: $512$ principal axes of the block input. Blue: concept vectors; purple: shared-situation directions; red squares: random directions; green diamonds: marker-word directions. At blocks $10$ – $22$ , the low-variance random and marker-word directions have more negative gain than concept and shared-situation directions. Spearman correlations are $0.82$ , $0.59$ , $0.88$ , and $0.85$ , respectively.

	Extraction (h)			Steered generation (GPU-h)			Judging
Model	H200	H200, paired	A100	H200	H100	A100	H200-h
Qwen3-8B	$5.6$	$7.0$ – $7.5$	$12.9$ – $13.4$	$10.4$	$15.4$	$20.8$	$1.12$
Llama-3.1-8B	$5.2$ – $5.9$	$6.5$ – $7.3$	—	$9.5$	$14.2$	$20.3$	$1.08$
gemma-2-9b	$6.8$ – $7.0$	$8.5$ – $8.8$	$14.4$	$12.7$	$17.5$	$23.1$	$0.81$

Appendix

Table 26: Time per unit of work on one GPU. Extraction: hours per pass of one prompt format over all

2{,}072{,}000

stories, recording every layer; single-story formats unless marked paired. Steered generation: GPU-hours per

10^{5}

chats of up to

768

greedy tokens. Judging: H200-hours per

10^{5}

chats, each scored on all three criteria. A dash marks a combination we did not run.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Automated Interpretability and Feature Discovery in Language Models with Agents
May 2, 2026Arnau Marin-Llobet, Javier FerrandoMechanistic Interpretability
Harvard University
Don't Judge an LLM Only by Its Activations: Discovering Suppressed Safety Features via Counterfactual Activation Potential
Oct 4, 2026Swadesh Swain, Sanghamitra DuttaAdversarial Attacks on LLMs Mechanistic Interpretability
University of Maryland, College Park
Scaling Inherently Interpretable Language Models
Aug 6, 2026Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7Language Model Steering LLM Interpretability
Guide Labs Team

May 2, 2026J/K move · Enter open · S save

Automated Interpretability and Feature Discovery in Language Models with Agents

Arnau Marin-Llobet, Javier Ferrando

Harvard University

Mechanistic Interpretability

Open paper AskPDF arXiv

We introduce an autonomous multiagent framework for mechanistic interpretability that automates both explaining and finding internal features in large language models. The system runs two coupled loops: (1) explanation refinement, where an agent proposes competing hypotheses and iteratively tests them with targeted prompt controls and a multi-metric evaluation; and (2) feature discovery, where an agent generates prompt sets, constructs a k-nearest-neighbor graph in activation space, and retrieves candidate features using statistical separability and semantic coherence criteria. On Gemma-2 family models and MLP neurons in weight-sparse transformers, our agent improves over one-shot auto-interpretations, discovers language-specific and safety-relevant features, and produces auditable explanation traces, showing that agent-driven empirical loops yield sharper and more falsifiable explanations than one-shot labels.

Abstract

LLM Interpretability

Explore similar work

Automated Interpretability and Feature Discovery in Language Models with Agents

Don't Judge an LLM Only by Its Activations: Discovering Suppressed Safety Features via Counterfactual Activation Potential

Scaling Inherently Interpretable Language Models