Gradients for Interventions and Activations for Detection: Targeted Feature Learning in Language Models

Jonathan Drechsel; Steffen Herbold

cs.CLSep 26, 2026

Gradients for Interventions and Activations for Detection: Targeted Feature Learning in Language Models

Authors: Jonathan Drechsel, Steffen Herbold

Organizations: Faculty of Computer Science and Mathematics University of Passau Passau, Germany

Abstract

Model-internal features can be studied through both their ability to identify a specified concept and their causal effect when manipulated, e.g., through steering or weight editing. A prominent approach to feature learning is Sparse Autoencoders (SAEs), which learn broad feature dictionaries whose relation to particular concepts is typically identified post hoc. However, many interpretability questions are instead hypothesis-driven and concern a concept specified in advance. We study this setting as targeted feature learning, where a single feature is constructed for such a predefined concept. We present a controlled comparison across three model signals (activation values, activation gradients, and parameter gradients) and two estimators (contrastive mean and a learned one-dimensional encoder-decoder), yielding six targeted methods, with CAA and GRADIEND as existing instances and four new methods covering the remaining combinations. We compare these methods against pretrained SAEs across 15 tasks and three language models, evaluating both detection and causal intervention. Across models, the strongest detection performance is achieved by contrastive activation value methods, whereas the strongest intervention performance is achieved by gradient-based methods. Overall, our results show that targeted feature quality depends jointly on the model signal and estimator, with detection and intervention capturing complementary properties.

Figures & tables

Figure 1: Detection and intervention capture complementary properties of targeted features. Across 15 tasks and three language models, activation value methods show stronger detection and most gradient-based methods stronger intervention overall. Per-model results are shown in Figure 3 .

Panel of Figure 2 — Figure 2: Study overview. From a predefined feature specification, we cross three model signals and two estimators, yielding six targeted feature learning methods evaluated on detection and intervention. SAEs provide an untargeted activation value reference closest to ACTIEND .

Signal	Contrastive mean	Interpretable ENcoder Decoder (IEND)
Activation value	Contrastive Activation Addition (CAA) ( Rimsky et al., 2024 )	ACTivation IEND (ACTIEND) † (new)
Activation gradient	Contrastive Activation Gradient Addition (CAGA) (new)	Activation Gradient IEND (AGIEND) (new)
Parameter gradient	Contrastive Gradient Addition (CGA) (new)	GRADient IEND (GRADIEND) * ( Drechsel and Herbold, 2026a )

Table 1: Targeted feature learning methods across three model-derived signals and two estimators .

Family	Eval.	Tasks (classes)
Semantic	$pw$ / $1s$	Gender (male/female); Emotion (positive/negative); Race (Asian/Black/White); Religion (Christian/Muslim/Jewish)
Factual	$pw$ / $1s$	RavelCont (Asia/Europe/Africa); RavelCntry (United States/China/Russia); RavelLang (English/Portuguese/Spanish)
Language	$pw$ / $1s$	LangID (English/French/German)
Grammatical	$pw$ / $1s$	PronNum (singular/plural); PronPers (first/second/third)
Algorithmic	$1s$	IOI (IO/subject); KeyValue (value/other value); Induction (match/distractor); Repetition (yes/no); FuncComp (result/distractor)

Table 2: Summary of the 15 feature learning tasks.

pw

denotes pairwise and

1s

one-sided tasks.

Figure 3: Task-averaged detection and intervention for each method. Lines connect the two estimators using the same signal and construction. The model-averaged aggregate is shown in Figure 1 .

Panel of Figure 4 — Figure 4: Overview of task-level performance and Bayesian method comparisons.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Feature classes	Reg.	Train/val/test	Source and Hugging Face ID
Gender	male (M), female (F)	P	73,860/2,110/8,440	aieng-lab/genter-ajibawa-name-filled ( Drechsel and Herbold, 2026a ) subset n1
Emotion	positive, negative	P	600/200/200	aieng-lab/en-sentiment-nrc ( Drechsel and Herbold, 2026b ) vocabulary-held-out split
Race	Asian, Black, White	P	33,602/9,603/4,799	aieng-lab/gradiend_race_data ( Drechsel and Herbold, 2026a )
Religion	Christian, Muslim, Jewish	P	20,048/5,730/2,863	aieng-lab/gradiend_religion_data ( Drechsel and Herbold, 2026a )
RavelCont	Asia, Europe, Africa	P	1,344/168/168	Source: hij/ravel ( Huang et al., 2024 ) ; Processed aieng-lab/gradiend-ravel-continent (new)
RavelCntry	United States, China, Russia	P	408/51/51	Source: hij/ravel ( Huang et al., 2024 ) ; Processed aieng-lab/gradiend-ravel-country (new)

Appendix

Table 3: Complete task inventory. Sizes are effective labeled train/validation/test rows after task-level class mapping and before counterfactual expansion. P = both poles occur naturally and both pairwise and one-sided regimes are evaluated. O = only the first listed class is factual and the second is its constructed counterpart. We do not redistribute LangID due to licensing constraints and instead provide a script for reconstructing it from the original sources.

Task	Masked context	Target / class
Gender	Carl looked around the room, noticing tired expressions and disheartened faces. [MASK] realized then that something needed to change.	he / M
Emotion	I am so unbelievably [MASK] for tomorrow.	excited / positive
Race	Lutron alleged that products were made in Mexico and [MASK] and then imported into the United States.	China / Asian
Religion	The inscription should, from a [MASK] point of view, pinpoint where Jumne was located.	Christian / Christian
RavelCont	Toronto is a city in the continent of North America. Pilibhit is a city in the continent of [MASK] .	Asia / Asia
RavelCntry	St. Petersburg is in Russia. Johnstown is in [MASK] .	United States / United States

Appendix

Table 4: Representative labeled examples. Long natural contexts are shortened with an ellipsis. The masked span and local wording are retained. The right column gives the factual target token followed by its feature class. For one-sided tasks the counterfactual class is still listed in Table 3 , but does not add a second base row.

Task	Size	Source and Hugging Face ID
Gender , Race , Religion , IOI , KeyValue , Induction , Repetition , FuncComp	4,593,588	aieng-lab/biasneutral-ajibawa ( Drechsel and Herbold, 2026a )
Emotion	1,000	aieng-lab/en-sentiment-nrc-neutral ( Drechsel and Herbold, 2026b )
PronNum , PronPers	10,000	aieng-lab/en-pronoun-neutral ( Drechsel and Herbold, 2026b )
RavelCont	5,000	Source: hij/ravel ; Processed: aieng-lab/gradiend-ravel-continent-neutral (new)
RavelCntry	5,000	Source: hij/ravel ; Processed: aieng-lab/gradiend-ravel-country-neutral (new)
RavelLang	5,000	Source: hij/ravel ; Processed: aieng-lab/gradiend-ravel-language-neutral (new)

Appendix

Table 5: Task-dependent neutral dataset overview.

Model	HF checkpoint	SAELens release	SAE ID / hook
Pythia-70M	EleutherAI/pythia-70m-deduped	pythia-70m-deduped-res-sm	blocks.{L}.hook_resid_post
GPT-2	gpt2	gpt2-small-resid-post-v5-32k	blocks.{L}.hook_resid_post
Llama-3.1-8B	meta-llama/Llama-3.1-8B	llama_scope_lxr_32x	l{L}r_32x

Appendix

Table 6: Language-model checkpoints and pretrained SAE resources used in the study.

Figure 5: Computational cost and peak GPU memory on Gender . Left: wall-clock time separated into fitting or training, detection, and intervention evaluation. Right: peak allocated and reserved CUDA memory across these stages. Reserved memory includes allocated memory and is therefore shown as an overlaid rather than additive quantity. SAE pretraining is excluded because we use pretrained dictionaries.

Setting	Value
IEND optimization
GRADIEND learning rate	$10^{-4}$ ( GPT-2 ), $10^{-6}$ ( Pythia-70M ), $2{\times}10^{-6}$ ( Llama-3.1-8B )
ACTIEND encoder learning rate	$5{\times}10^{-5}$ ( Pythia-70M , GPT-2 ), $10^{-5}$ ( Llama-3.1-8B )
ACTIEND decoder learning rate	$1.15{\times}10^{-2}$
AGIEND learning rate	$3{\times}10^{-3}$ ( Pythia-70M , GPT-2 , Llama-3.1-8B )
Training steps	500 ( GRADIEND , AGIEND ); 100 ( ACTIEND )

Appendix

Table 7: Training, selection, and evaluation hyperparameters.

Figure 6: Bayesian pairwise comparison of methods.

Method	Single layer	Win	Mean $\Delta$	Win	Mean $\Delta$
	Selected	Detection		Intervention
SAE	96.9%	49.0%	$+0.035$	27.3%	$-0.013$
CAA	87.9%	82.5%	$+0.082$	45.0%	$-0.001$
CGA	95.4%	85.0%	$+0.153$	34.8%	$-0.041$
CAGA	94.2%	86.3%	$+0.140$	38.3%	$-0.015$
Overall	93.0%	80.4%	$+0.115$	38.1%	$-0.018$

Appendix

Table 8: Effect of representation scope. Single-layer representations are selected by validation Detection.

\Delta

denotes single layer minus all-layer performance on the held-out test split. Win rates report the fraction of class/contrast comparisons with

\Delta>0

, i.e., where the selected single layer outperforms the all-layer representation on the held-out test split.

Figure 7: Difference between the validation-selected best single-layer and all-layer representations for detection and intervention. Positive values indicate higher performance for the selected single layer.

Figure 8: All-layer robustness analysis. Detection-intervention comparison corresponding to Figure 3 , with SAE and contrastive mean methods restricted to their all-layer representations rather than using validation-selected scopes.

Figure 9: Layerwise detection and intervention for Pythia-70M .

Figure 10: Layerwise detection and intervention for GPT-2 .

Figure 11: Detection and Intervention for the four SAE variants obtained by varying latent-selection position (filled-target vs. pre-target) and the number of selected latents (

k=1

vs.

k^{\star}

Figure 12: Performance by task family averaged across models.

Figure 13: Pairwise versus one-sided feature construction. Model-balanced mean Detection and Intervention on tasks supporting both constructions.

Figure 14: Pythia-70M detection and intervention scores per task.

Figure 15: GPT-2 detection and intervention scores per task.

Figure 16: Llama-3.1-8B detection and intervention scores per task.

Panel of Table 9 — Table 9: Detection raw results for Pythia-70M .

Panel of Table 10 — Table 10: Intervention raw results for Pythia-70M .

Panel of Table 11 — Table 11: Detection raw results for GPT-2 .

Panel of Table 12 — Table 12: Intervention raw results for GPT-2 .

Panel of Table 13 — Table 13: Detection raw results for Llama-3.1-8B .

Panel of Table 14 — Table 14: Intervention raw results for Llama-3.1-8B .

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 781 papers

Towards Isolated Interventions via Almost Orthogonal Features in Language Models
Feb 4, 2026Moritz Miller, Florent Draye, Bernhard SchölkopfDisentangled Representation Learning Representation Learning
Max Planck Institute for Intelligent Systems, Tübingen, Germany · ETH Zurich, Switzerland · Max Planck ETH Center for Learning Systems +1
What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions
Oct 11, 2026Yifeng Guo, Yanran Li, Xiang ChenSparse Autoencoders Mechanistic Interpretability
St. Jude Children’s Research Hospital · Columbia University
Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features
Oct 13, 2026Samuel Xie, Noura Al-MoubayedFaithfulness of Language Model Explanations SHAP Feature Attribution
Independent · Durham University

Feb 4, 2026J/K move · Enter open · S save

Towards Isolated Interventions via Almost Orthogonal Features in Language Models

Moritz Miller, Florent Draye, Bernhard Schölkopf

Max Planck Institute for Intelligent Systems, Tübingen, Germany · ETH Zurich, Switzerland · Max Planck ETH Center for Learning Systems+1

Disentangled Representation Learning Representation Learning Orthogonal Representation Learning Mechanistic Interpretability Causal Interventions in Language Models

Open paper AskPDF arXiv

A central premise in mechanistic interpretability is that meaningful concepts in language models are represented by linear features in activation space. For such features to support reliable interventions, manipulating one feature should not substantially alter the effects of others. In practice, however, feature entanglement leads to interference such that localized interventions can have unintended downstream effects. Motivated by the Independent Causal Mechanisms principle, we propose to constrain internal features to be almost orthogonal. We argue that this promotes modular representations amenable to causal intervention. We formalize this problem by characterizing the gap between an idealized isolated intervention and its realized effect on model outputs in terms of feature interference. We upper-bound the propagation of feature interference in terms of the self-coherence of the feature dictionary, and relate this discrepancy to an explicit orthogonality regularization on the dictionary itself. Empirically, we show that this regularization enables more isolated interventions on mathematical reasoning concepts while preserving model performance. Our code is available under https://github.com/mrtzmllr/sae-icm.

Abstract

LLM Interpretability

Explore similar work

Towards Isolated Interventions via Almost Orthogonal Features in Language Models

What Do Sparse Autoencoder Feature Discovery Scores Identify? From Activations to Interventions

Evaluating the Causal Faithfulness of SHAP as a Candidate Attribution Method for SAE Features