cs.CLNov 12, 2026

Creating Steering Vectors via Input Optimisation

Authors: Batuhan Akar

Organizations: Secure and Private AI Lab, ETH Zürich

Abstract

Activation steering influences a language model's behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate Gradient which optimises a suffix to match the target activations obtained by adding a steering vector to the residual stream. We further introduce universal suffixes trained across many prompts at once, a model-translation step that rewrites the suffix into human-readable text midway through optimisation, and a soft-suffix variant to remove the discrete token space constraint. We study two behaviours: steering a model to answer in German, and anti-refusal steering for jailbreaking. For German, a discrete suffix reproduces the steering vector's behaviour almost perfectly and transfers well across prompts and models. For anti-refusal, the equivalence is weaker, succeeding under a phrase-avoidance metric but only partially under the stricter HarmBench classifier. Our results show that a steering vector's behaviour is reachable from the input, with no access to model internals at generation time, up to a loss that depends on the concept being steered.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Sep 24, 2026

Minimally Invasive Steering of Language Models

Test-time alignment adapts large language models (LLMs) to runtime objectives without updating model parameters. Pre-logit steering offers a lightweight approach by optimizing additive interventions to the hidden states of a frozen LLM, but excessive steering can distort output distributions and degrade generation quality. We propose Minimally Invasive Steering Vector Optimization (MISVO), which improves reward while controlling deviation from the reference policy. Rather than penalizing the Euclidean norm of steering vectors, we derive a quadratic regularizer from the local Kullback--Leibler (KL) geometry of the induced token distributions. The resulting Fisher-information penalty captures the distributional sensitivity of steering directions and admits an analytic gradient computed through matrix--vector products with the frozen language-model head. We further derive an exact decomposition of the sequence-level KL gradient and establish explicit, non-asymptotic bounds on the approximation errors, including the omitted trajectory-dependent score-function term. These results show that the frozen Fisher regularizer gradient approximates the full sequence-level KL gradient to first order in the steering magnitude. Across preference and code-generation tasks on four LLMs spanning approximately 1B--14B parameters, MISVO achieves the highest mean best-so-far reward in six of seven model--task settings, while maintaining diversity and coherence scores comparable to Best-of-NN.
May 5, 2026

Steer Like the LLM: Activation Steering that Mimics Prompting

Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform compared to prompt-based approaches. We propose a framework that formulates prompt steering as a form of activation steering and investigates whether distilling successful prompt steering behavior into simpler, interpretable models can close this gap. Our analysis reveals that popular activation steering methods are not faithful to the mechanics of prompt steering, which applies strong interventions on some tokens while barely affecting others. Based on these insights, we introduce Prompt Steering Replacement (PSR) models that estimate token-specific steering coefficients from the activations themselves and are trained to imitate prompt-based interventions. Experiments on three steering benchmarks across multiple language models show that PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.
Oct 4, 2026

One Token Can Be Enough: Bridging Prompting and Activation Steering with Prefix Steering

Prompting guides language model behavior through the initial context, whereas activation steering often intervenes throughout generation. A natural question is whether steering can produce effects on subsequent computation similar to those of prompting. Under fixed-state attention assumptions, we establish sufficient conditions for single- and multi-token steering to match prompt-induced attention-head outputs, and characterize how changes in input representations affect this match and its approximation error. This attention-level connection leads us to ask whether, at the behavioral level, steering can also guide subsequent generation through a brief initial intervention. We study Prefix Steering, which applies existing steering directions and operators over a short span starting at the final prompt token, with no further direct intervention afterward. We examine how intervention duration and strength jointly shape the control-capability trade-off. Across four models and five tasks, intervention over a short span, even a single token, often retains much of full steering's behavioral control while better preserving general capabilities, offering a trade-off competitive with, and in some settings better than, prompting and alternative steering-strength policies. Prefix Steering also remains effective on final-answer formatting tasks after reasoning, suggesting that a brief initial intervention can influence behavior expressed well after steering ends. These findings challenge the common practice of steering every generated token and motivate a more dynamical view of activation steering, in which a brief intervention can alter the trajectory of subsequent generation without continued intervention.