Sparse Internal Control of Language Models

Xizhe Zhang

cs.CLNov 3, 2026

Sparse Internal Control of Language Models

Authors: Xizhe Zhang

Abstract

Language models are increasingly used in text generation, decision support, and automated interaction, creating a need for localized, reversible, and selective control of their behavior. Activation steering demonstrates that internal interventions can alter the behavior of frozen models, but how to select and evaluate compact, multi-site control interfaces remains an open question. I introduce sparse internal control, a framework for steering target behavior by applying inference-time interventions to a small set of internal model nodes. A local target-controllability formulation characterizes candidate site--direction pairs by their behavioral effects and specifies constraints on target recovery, off-target preservation, and intervention energy. Two complementary selectors instantiate this formulation: Driver-OMP selects nodes for sparse reconstruction of a desired behavioral displacement, while Coverage-Driver favors stable coverage across prompts. I propose a six-axis control-evaluation protocol measuring reachability, signed reversal, dose response, feedback controllability, off-target preservation, and held-out reuse. Exploratory experiments on indirect object identification, MMLU multiple-choice question answering, sentiment steering, and refusal steering provide evidence for the feasibility of steering task-defined behavior through sparse internal interventions. The six-axis evaluation characterizes these interventions beyond target movement, revealing distinct control properties that depend on the task and available direction information. Together, the formulation, reference algorithms, and evaluation protocol provide a foundation for the systematic study and development of internal control methods for language models.

Figures & tables

Figure 1: Conceptual illustration of sparse internal control . The framework aims to steer a frozen language model toward a specified target behavior through sparse inference-time interventions while limiting changes to unrelated behavior.

Axis	Evaluation objective	Quantitative readout
Reachability	Proximity to the target	$\mathrm{Rel}_{\mathrm{tar}}(1)$
Signed reversal	Return toward source under the opposite patch	$\operatorname{Reversal}(\beta)$
Dose response	Target progress across intervention doses	Progress and signed $\rho$
Feedback controllability	Tracking targets through feedback-adjusted doses	Error, hit rate, evaluations
Off-target preservation	Output changes on non-target prompts	$L_{\mathrm{off}}(\gamma\beta)$
Held-out reuse	Reuse of sites or interventions on unseen prompts	Held-out residual or recovery

Table 1: Six-axis control evaluation. Objectives and quantitative measures.

Method	IOI (10 seeds)	Humanities (3 seeds)	STEM (3 seeds)
Driver-OMP	$0.201\pm 0.058$	$0.279\pm 0.032$	$0.175\pm 0.035$
Coverage-Driver	$0.294\pm 0.029$	$0.152\pm 0.154$	$0.061\pm 0.033$
Top- $k$	$0.240\pm 0.035$	$0.333\pm 0.117$	$0.221\pm 0.046$
Random	$0.747\pm 0.240$	$1.209\pm 0.949$	$1.167\pm 1.141$
Nonlinear greedy	$0.151\pm 0.037$	$0.073\pm 0.093$	$0.020\pm 0.012$

Table 2: Same-panel nominal-dose target recovery. Mean

\pm

sample SD of

\mathrm{Rel}_{\mathrm{tar}}(1)

(lower is better), using evaluation-pair directions and at most eight candidates. Fitting and evaluation use the same 64 IOI pairs (10 seeds) or eight MCQA questions per category (3 seeds). Nonlinear greedy uses additional model evaluations.

Figure 2: IOI and Sentiment dose diagnostics. Faint curves are seeds; bold curves are means. The illustrated IOI Driver-OMP uses four sites and two seeds, versus eight sites and three seeds for random supports. Sentiment uses three unpaired-pool seeds. IOI non-answer-position

\Delta

KL and Sentiment evaluation-prompt KL are distinct from the independent preservation panel; see Appendix G.2.3 .

Figure 3: Independent dose response. Columns are Humanities, STEM, and Refusal; rows use P and F directions, respectively. Each selector has eight seed curves, ten measured doses per curve, and a bold mean. The vertical axis is signed mean-margin progress; panel scales differ. The additional STEM scan retains strong negative F responses. Supports and base coefficients remain fixed.

Figure 4: Feedback control with fixed supports. Curves show recorded batch-mean margins, with colours identifying target margins (dotted lines) and dots marking terminal observations. Panel (a) includes all three IOI seeds; panels (b)–(d) use seed 0. Only dose is adjusted. Hit counts include initial zero-dose hits; aggregate results are reported in Table 3 .

Task	Mode	Driver-OMP	Coverage-Driver	Top- $k$
IOI	Evaluation-pair (P)	24/24	—	—
Humanities	Evaluation-pair (P)	38/48	46/48	45/48
Humanities	Training-mean (F)	8/48	8/48	8/48
STEM	Evaluation-pair (P)	39/48	47/48	47/48
STEM	Training-mean (F)	8/48	7/48	7/48
Refusal	Evaluation-pair (P)	12/40	21/40	21/40

Table 3: Feedback target hits. P/F are direction conditions. IOI has three seeds

\times

eight targets; MCQA has eight

\times

six, and Refusal eight

\times

five. Initial zero-dose hits are 2/24 for IOI, 6/48 for Humanities, 3/48 for STEM, and 10/40 for Refusal. Each seed’s targets share its fitted support.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Setting type	Component	Setting
Reference	Refit and certificate	Ridge and preservation weights $\lambda=\mu=1$ ; activation-energy bound $E\leq\kappa=2$ ; target criterion $\mathrm{Rel}_{\rm tar}(1)\leq 0.4$ ; preservation bound $L_{\rm off}\leq 0.001$ .
Reference	Sparse selection	Driver-OMP residual tolerance $\varepsilon_{\rm lin}=0.05$ and improvement tolerance $\eta_{\rm tol}=10^{-3}$ ; Coverage-Driver coverage $\tau=0.95$ . Denominator stabilizers $\delta_{0}=\eta_{0}=10^{-3}$ ; normalized marginal-gain threshold $g_{\min}=10^{-3}$ . Uniform row weights $w_{j}=1/n$ for $j=1,\ldots,nr$ give $W_{\rm row}=\sum_{j}w_{j}=r$ ; Algorithm C.4 normalizes coverage by $W_{\rm row}$ . Target normalization guard $\xi=10^{-6}$ .
Executed	IOI recurrence	Five seeds and five budgets $k\in\{3,5,8,12,16\}$ give 25 seed–budget configurations; consensus threshold $\pi^{\star}=0.6$ .

Appendix

Table F.1 : Formal reference settings and recorded IOI recurrence. Setting type distinguishes the formal reference from an executed analysis. Repeated MCQA/Refusal feedback parameters are specified only in the executed-settings paragraph above; the clipped controller of Eq. ( D.35 ) is a reference rule.

Series	Readout and panel	Repetitions and reporting unit
IOI nominal	IO–S margin; 64 pairs, same panel	10 seeds; stacked $R_{\rm clean}$ and $\mathrm{Rel}_{\rm tar}(1)$
MCQA nominal	Four option logits; 8 questions/category, same panel	3 seeds/category; stacked $R_{\rm clean}$ and $\mathrm{Rel}_{\rm tar}(1)$
MCQA control	Scalar mean-wrong margin; 24 questions/category, 12/12; head library; P/F	8 seeds; $8\times 6=48$ feedback endpoints/selector/mode; 8 reversal ratios; 8 nominal-dose $\mathrm{Rel}_{\rm tar}(1)$ values/selector/mode
Refusal control	Lexical margin; 20 pairs, 10/10; head library; P/F	8 seeds; $8\times 5=40$ feedback endpoints/selector/mode; 8 reversal ratios; 8 nominal-dose $\mathrm{Rel}_{\rm tar}(1)$ values/selector/mode
IOI reversal	Saved nominal interventions; same 64 pairs; P	10 seed-level ratios/selector
Sentiment reversal	Saved $k=8$ paired interventions; 16 pairs, 8/8; P/F	8 seed-level ratios/selector/mode; 8 nominal-dose $\mathrm{Rel}_{\rm tar}(1)$ values/selector/mode

Appendix

Table G.2: Experimental configurations and reporting units. Panel sizes are per seed; a fitting/evaluation split is written as fit/eval. P and F specify direction information, not different sample sizes.

Method	IOI (10 seeds)	Humanities (3 seeds)	STEM (3 seeds)
Driver-OMP	$0.201\pm 0.058$ (10/10)	$0.279\pm 0.032$ (3/3)	$0.175\pm 0.035$ (3/3)
Coverage-Driver	$0.294\pm 0.029$ (10/10)	$0.152\pm 0.154$ (3/3)	$0.061\pm 0.033$ (3/3)
Top- $k$	$0.240\pm 0.035$ (10/10)	$0.333\pm 0.117$ (2/3)	$0.221\pm 0.046$ (3/3)
Random	$0.747\pm 0.240$ (1/10)	$1.209\pm 0.949$ (0/3)	$1.167\pm 1.141$ (0/3)
Nonlinear greedy	$0.151\pm 0.037$ (10/10)	$0.073\pm 0.093$ (3/3)	$0.020\pm 0.012$ (3/3)

Appendix

Table G.3: Formal

\mathrm{Rel}_{\mathrm{tar}}(1)

on the same fitting/evaluation panel at nominal dose

\gamma=1

and maximum support budget eight. Cells give mean

\pm

sample SD (number of seeds satisfying

\varepsilon=0.4

/total). IOI uses 64 paired logit-margin readouts (10 seeds); MCQA uses four option logits on eight questions (3 seeds/category). All directions are evaluation-pair (P); lower residuals are better.

Task	Method	P: mean $\pm$ SD (pass)	F: mean $\pm$ SD (pass)
Humanities	Driver-OMP	$0.495\pm 0.428$ (6/8)	$1.009\pm 0.048$ (0/8)
	Coverage-Driver	$0.188\pm 0.072$ (8/8)	$1.012\pm 0.058$ (0/8)
	Top- $k$	$0.192\pm 0.071$ (8/8)	$1.010\pm 0.055$ (0/8)
STEM	Driver-OMP	$0.497\pm 0.439$ (6/8)	$1.016\pm 0.048$ (0/8)
	Coverage-Driver	$0.169\pm 0.067$ (8/8)	$1.031\pm 0.053$ (0/8)
	Top- $k$	$0.170\pm 0.046$ (8/8)	$1.030\pm 0.051$ (0/8)

Appendix

Table G.4: Formal

\mathrm{Rel}_{\mathrm{tar}}(1)

on held-out evaluation prompts at fixed dose

\gamma=1

and maximum support budget eight. The unit is one seed-level stacked vector; entries report mean

\pm

sample SD and number of seeds satisfying

\varepsilon=0.4

out of eight. Head-level MCQA, paired Sentiment, and Refusal use distinct task-specific readouts. P uses evaluation-pair directions; F uses training-mean directions.

Method	IOI (10 seeds)	Humanities (3 seeds)	STEM (3 seeds)
Driver-OMP	$0.386\pm 0.114$	$0.045\pm 0.013$	$0.030\pm 0.008$
Coverage-Driver	$0.567\pm 0.059$	$0.027\pm 0.031$	$0.010\pm 0.005$
Top- $k$	$0.464\pm 0.078$	$0.054\pm 0.024$	$0.038\pm 0.012$
Random	$1.430\pm 0.442$	$0.185\pm 0.132$	$0.196\pm 0.185$
Nonlinear greedy	$0.289\pm 0.068$	$0.013\pm 0.018$	$0.003\pm 0.002$

Appendix

Table G.5 : Clean-output-normalized reconstruction. Mean

\pm

sample SD of

R_{\rm clean}

, lower is better. Same-panel fitting and evaluation use 64 IOI pairs or eight MCQA questions per category. Evaluation-pair directions and nominal dose

\gamma=1

are used. Maximum budget is eight; early stopping can use fewer candidates. Nonlinear greedy uses additional model evaluations.

Task	Mode	Selector	Pass/seeds	Ratio mean $\pm$ SD
IOI	P	Driver-OMP	10/10	$4.7425\pm 1.1143$
IOI	P	Coverage-Driver	10/10	$2.5702\pm 0.3689$
IOI	P	Top- $k$	10/10	$3.4315\pm 0.8456$
Sentiment	P	Driver-OMP	4/8	$2.3722\pm 1.0708$
Sentiment	F	Driver-OMP	0/8	$0.6547\pm 0.2871$
Sentiment	P	Coverage-Driver	8/8	$8.5143\pm 2.9573$

Appendix

Table G.6: Formal target-side reversal. Ratio at least two is the operating criterion; mean and sample SD are over seeds, not feedback targets. IOI is P-only; P and F are defined in Table G.2 .

Task	Mode	Selector	$\rho\geq.95$	$\rho\leq-.95$	Mean	SD
Humanities	P	Driver-OMP	3/8	0/8	0.5970	0.6005
Humanities	P	Coverage-Driver	5/8	0/8	0.9394	0.0987
Humanities	P	Top- $k$	2/8	0/8	0.8939	0.0727
Humanities	F	Driver-OMP	4/8	3/8	0.2303	1.0202
Humanities	F	Coverage-Driver	5/8	3/8	0.2500	1.0351
Humanities	F	Top- $k$	5/8	3/8	0.2500	1.0351

Appendix

Table G.7: Signed Spearman ordering over ten independent doses per seed. Positive and strong negative counts are separate; absolute correlations are not a pass criterion.

Task	Selector	Mode	Hits	MAE	Evals	Initial
IOI	Driver-OMP	P	24/24	0.1351	3.958	2/24
Humanities	Driver-OMP	P	38/48	1.0026	6.125	6/48
Humanities	Driver-OMP	F	8/48	2.2641	11.479	6/48
Humanities	Coverage-Driver	P	46/48	0.2378	4.417	6/48
Humanities	Coverage-Driver	F	8/48	1.9239	11.479	6/48
Humanities	Top- $k$	P	45/48	0.2679	4.583	6/48

Appendix

Table G.8: Repeated scalar feedback. Terminal target hits, mean absolute tracking error (MAE), mean recorded readout-evaluation count, and initial zero-dose hits. Zero-dose readouts may reuse the cached unpatched baseline. Errors and evaluation counts are averaged over all seed–target endpoints, including unsuccessful trials. Initial hits are included in the total hit count and reported separately.

Task	Selector	Mean KL	Median	P95	Max	Mean-pass	All-pass
Sentiment	Driver-OMP	0.005527	0.004959	0.011119	0.0286	0/8	0/8
Sentiment	Coverage-Driver	0.011015	0.009627	0.024779	0.0381	0/8	0/8
Sentiment	Top- $k$	0.011567	0.010322	0.024746	0.0387	0/8	0/8
Humanities	Driver-OMP	0.496530	0.050394	2.349920	15.5523	0/8	0/8
Humanities	Coverage-Driver	0.109692	0.050405	0.392607	1.0154	0/8	0/8
Humanities	Top- $k$	0.115396	0.056724	0.478003	0.6963	0/8	0/8

Appendix

Table G.9: Full-vocabulary KL(unpatched

\|

patched) at nominal dose, using natural logarithms. Each row pools 32 prompts

\times

eight seeds (256 measurements on 32 unique prompts). P95 is nearest rank. Last columns count seeds meeting mean-KL or all-32-prompt KL

\leq 0.001

. MCQA uses each off-prompt pair; Sentiment/Refusal use training means.

Task	Method	Seeds	Refit- $\beta$	Fully frozen
Humanities	Driver-OMP	4	$0.992\pm 0.009$	$0.021\pm 0.026$
Humanities	Coverage-Driver	4	$0.995\pm 0.053$	$0.033\pm 0.029$
Humanities	Top- $k$	4	$1.078\pm 0.017$	$0.028\pm 0.024$
Humanities	Random	4	$0.758\pm 0.443$	$0.022\pm 0.023$
STEM	Driver-OMP	4	$0.987\pm 0.034$	$0.024\pm 0.066$
STEM	Coverage-Driver	4	$0.944\pm 0.056$	$0.038\pm 0.108$

Appendix

Table G.10 : Module-level MCQA reuse. Seed means and sample SD. Refit retains evaluation-target access; fully frozen means the full open-loop intervention is fixed. Each category has four seeds.

Method	Refit rec.	Fixed- $\beta$ rec.	Refit fail frac.	Fixed- $\beta$ fail frac.
Driver-OMP	$0.461\pm 0.144$	$0.391\pm 0.111$	0.583	0.661
Coverage-Driver	$0.449\pm 0.083$	$0.298\pm 0.143$	0.667	0.833
Top- $k$	$0.632\pm 0.012$	$0.434\pm 0.160$	0.464	0.734
Random	$-0.670\pm 0.612$	$-0.649\pm 0.637$	0.984	0.984

Appendix

Table G.11 : IOI reuse on the test split. Recovery is reported as the mean

\pm

sample SD across three seeds. Fixed coefficients retain access to evaluation-pair directions. Failure fractions use the pointwise error and threshold specified above.

Selector	$k$	Mode	Mean	SD	Seeds
Driver-OMP	3	P	0.3602	0.1887	8
Driver-OMP	3	F	0.3536	0.1723	8
Coverage-Driver	3	P	0.7687	0.1038	8
Coverage-Driver	3	F	0.8137	0.4851	8
Top- $k$	3	P	0.7804	0.1173	8
Top- $k$	3	F	0.8127	0.4852	8

Appendix

Table G.12: Sentiment paired fixed-dose recovery: seed mean and sample SD across eight seeds, by support budget and direction condition. Forward mean-margin recovery is distinct from formal vector reversal.

Task	Selector	Mode	0.10	0.20	0.30	0.40	0.50
Humanities	Coverage-Driver	F	1/48	2/48	8/48	9/48	10/48
Humanities	Coverage-Driver	P	11/48	24/48	46/48	46/48	46/48
Humanities	Driver-OMP	F	1/48	2/48	8/48	8/48	10/48
Humanities	Driver-OMP	P	6/48	17/48	38/48	38/48	38/48
Humanities	Top- $k$	F	1/48	2/48	8/48	10/48	10/48
Humanities	Top- $k$	P	10/48	29/48	45/48	45/48	45/48

Appendix

Table G.13: Feedback endpoint hits reclassified at five absolute-error tolerances. These are the same endpoints, not new runs or independent samples.

Figure G.1 : IOI selection recurrence. Frequencies across five seeds and five budgets. Two candidates occur in 15/25 configurations; the labels “core”, “backup”, and “rare” denote recurrence bands in this analysis, not proofs of necessity or universal substitutability.

Model	Seeds	Driver-OMP	Coverage- Driver	Top- $k$	Random
Qwen3-4B	3	$0.045\pm 0.013$	$0.027\pm 0.031$	$0.054\pm 0.024$	$0.185\pm 0.132$
Qwen3-8B	5	$0.016\pm 0.008$	$0.018\pm 0.009$	$0.027\pm 0.012$	$0.077\pm 0.032$
Mistral-7B	3	$0.044\pm 0.005$	$0.015\pm 0.003$	$0.008\pm 0.006$	$0.050\pm 0.047$

Appendix

Table G.14 : Configuration-specific model comparisons. Mean

\pm

sample SD of nominal

R_{\rm clean}

. Panels and libraries differ as specified in the text; these are configuration-specific results.

Language Model Controllability

See all 9 papers

A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models
Jul 6, 2026Nima Eshraghi, Lovedeep Gondara, Yuqing Huang +5Language Model Steering Sparse Autoencoders
The Vanguard Group, Inc. · University of Toronto
Distributed Sparse Interventions in Language Models
Jul 8, 2026Maximilian S. Ernst, Lorenz Linhardt, Aaron Peikert +1Language Model Steering LLM Interpretability
Max Planck School of Cognition · Center for Lifespan Psychology Max Planck Institute for Human Development · Machine Learning Group Technische Universität Berlin +1
Steering Language Models Before They Speak: Logit-Level Interventions
Jan 16, 2026Hyeseon An, Shinwoo Park, Hyundong Jin +1Language Model Steering Controllable Text Generation
Department of Computer Science, Yonsei University, Seoul, South Korea

Jul 6, 2026J/K move · Enter open · S save

A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

Nima Eshraghi, Lovedeep Gondara, Yuqing Huang, Sagarika Suresh+4

The Vanguard Group, Inc. · University of Toronto

Language Model Steering Sparse Autoencoders Causal Interventions in Language Models Activation Steering

Open paper AskPDF arXiv

Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning, but standard methods apply the steering signal at every generated token, incurring constant per-token perturbation that risks degrading fluency. We ask: is dense intervention necessary? We introduce Stochastic Token Steering (STS), which gates each token independently with probability

p

, and Stochastic Block Steering (SBS), which gates a leading window once per sequence; neither requires a reward model or learned gating policy. Across two model families and two behavioral tasks, steering only a fraction of tokens recovers most of the behavioral effect of dense steering while incurring less fluency degradation. Across our experimental settings, we further find that the best-performing steering magnitude consistently rises as the intervention ratio falls.

Abstract

Language Model Controllability

Explore similar work

A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

Distributed Sparse Interventions in Language Models

Steering Language Models Before They Speak: Logit-Level Interventions