ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment

Hongjue Zhao; Haosen Sun; Jiangtao Kong; Xiaochang Li; Qineng Wang; Liwei Jiang; Qi Zhu; Tarek Abdelzaher

cs.AIFeb 19, 2026

ODESteer: A Unified ODE-Based Steering Framework for LLM Alignment

Authors: Hongjue Zhao, Haosen Sun, Jiangtao Kong, Xiaochang Li, Qineng Wang, Liwei Jiang, Qi Zhu, Tarek Abdelzaher,

Organizations: University of Illinois Urbana-Champaign · Northwestern University · William & Mary · University of Washington ·

Abstract

Activation steering, or representation engineering, offers a lightweight approach to align large language models (LLMs) by manipulating their internal activations at inference time. However, current methods suffer from two key limitations: (i) the lack of a unified theoretical framework for guiding the design of steering directions, and (ii) an over-reliance on one-step steering that fail to capture complex patterns of activation distributions. In this work, we propose a unified ordinary differential equations (ODEs)-based theoretical framework for activation steering in LLM alignment. We show that conventional activation addition can be interpreted as a first-order approximation to the solution of an ODE. Based on this ODE perspective, identifying a steering direction becomes equivalent to designing a barrier function from control theory. Derived from this framework, we introduce ODESteer, a kind of ODE-based steering guided by barrier functions, which shows empirical advancement in LLM alignment. ODESteer identifies steering directions by defining the barrier function as the log-density ratio between positive and negative activations, and employs it to construct an ODE for multi-step and adaptive steering. Compared to state-of-the-art activation steering methods, ODESteer achieves consistent empirical improvements on diverse LLM alignment benchmarks, a notable $5.7\%$ improvement over TruthfulQA, $2.5\%$ over UltraFeedback, and $2.4\%$ over RealToxicityPrompts. Our work establishes a principled new view of activation steering in LLM alignment by unifying its theoretical foundations via ODEs, and validating it empirically through the proposed ODESteer method.

Figures & tables

Figure 1: Overview of existing activation steering methods vs. our proposed approach. (a–b) Regular activation addition applies a one-step linear steering $T\cdot\bm{v}(\bm{a})$ to hidden activations, where the vector field $\bm{v}(\bm{a})$ controls the steering direction, and $T$ controls the steering strength, as detailed in Sec 4 . (c–d) Our method ( ODESteer ) formulates steering as numerically solving an ODE, yielding multi-step adaptive updates from $\bm{a}(0)$ to $\bm{a}(T)$ guided by barrier functions from control theory. (e) The barrier function $h(\bm{a})$ defines desirable and undesirable regions in the activation space, guiding the activations toward desirable regions while ensuring it remains there. (f) Example generations before and after steering show that ODESteer produces more accurate and aligned responses.

Category	Method	Barrier Function $h(\bm{a})$
Input Reading	Difference-in-Means	Log-density ratio (Gaussian assumption)
	Linear Probes	Log-density ratio (logistic regression)
Output Optimization	–	Scoring function with threshold

Table 1: Interpretation of steering direction identification methods through barrier functions. Each method defines a scalar function

h(\bm{a})

and selects a steering direction

\bm{v}(\bm{a})

that increases

h(\bm{a})

Method	Model	Helpfulness (Ultrafeedback)			Truthfulness (TruthfulQA)			Detoxification (Real Toxicity Prompts)
Method	Model	Win (%) $\bm{\uparrow}$	RM ${}_{\text{mean}}$ $\bm{\uparrow}$	RM ${}_{\text{P90}}$ $\bm{\uparrow}$	T $\times$ I (%) $\bm{\uparrow}$	True (%) $\bm{\uparrow}$	Info (%) $\bm{\uparrow}$	Toxic $\bm{\downarrow}$	PPL $\bm{\downarrow}$	Dist-2 $\bm{\uparrow}$
Original	Falcon-7B	50.0 ±0.000	-15.298 ±0.194	-5.465 ±0.628	29.0 ±0.220	30.2 ±0.153	96.0 ±0.462	0.257 ±0.007	15.980 ±0.360	0.948 ±0.003
RepE		50.1 ±0.014	-15.354 ±0.120	-5.337 ±0.501	24.4 ±0.395	25.7 ±0.550	95.1 ±0.602	0.246 ±0.004	15.440 ±0.260	0.940 ±0.001
ITI		50.5 ±0.013	-15.291 ±0.153	-4.704 ±0.417	34.7 ±0.713	36.0 ±0.608	96.4 ±0.493	0.243 ±0.010	15.880 ±0.690	0.935 ±0.006
CAA		52.8 ±0.011	-14.998 ±0.157	-5.100 ±0.481	35.0 ±0.390	36.4 ±0.321	96.3 ±0.252	0.244 ±0.003	15.920 ±0.530	0.950 ±0.002
MiMiC		47.8 ±0.016	-15.469 ±0.092	-5.333 ±0.250	37.2 ±0.712	42.2 ±1.058	88.0 ±1.385	0.244 ±0.007	15.780 ±0.640	0.941 ±0.002

Table 2: Comparison of methods on Falcon-7B, Mistral-7B, LLaMA3.1-8B for helpfulness, truthfulness, and detoxification. For helpfulness: “Win” is win rate, “RM

{}_{\text{mean}}

” is mean reward, and “RM

{}_{\text{P90}}

” is 90th percentile reward. For truthfulness: “T

\times

I” is Truthfulness

\times

Informativeness, with “True” and “Info” reported separately. For detoxification: “PPL” is perplexity. Results are averaged over three runs. Primary metrics are highlighted in blue ; best and second-best are in bold and underline . Dist-1/3 scores for detoxification are provided in Appendix E.1 .

Model	Method	Win (%) $\uparrow$	T $\times$ I (%) $\uparrow$	Toxic $\downarrow$
Falcon-7B	ITI	50.5 ±0.013	34.7 ±0.713	0.243 ±0.010
	One-step ODESteer	54.0 ±0.028	40.8 ±0.819	0.199 ±0.005
	ODESteer (Ours)	56.3 ±0.018	42.2 ±0.115	0.188 ±0.006
Mistral-7B	ITI	51.8 ±0.010	46.4 ±1.249	0.165 ±0.007
	One-step ODESteer	54.1 ±0.027	58.1 ±0.734	0.113 ±0.001
	ODESteer (Ours)	56.1 ±0.028	59.9 ±0.237	0.109 ±0.006

Table 3: Ablation study on UltraFeedback, TruthfulQA, and RealToxicityPrompts, demonstrating the two main advantages of our method. The best results are highlighted in bold .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Definition
$x$ , $X$	Scalars
$\bm{x}$	Vectors
$\bm{X}$	Matrices
$\mathcal{X}$	Sets
$\dot{x}=\derivative*{x}{t}$	Derivative of $x(t)$ w.r.t. time $t$
$\bm{a}\in\mathbb{R}^{d}$	Activation/hidden states of an LLM at a given position

Appendix

Table 4: Notations used in this work.

h(\bm{a})=\bm{w}^{\top}\bm{\phi}(\bm{a})+b.

Appendix

Algorithm 1 Representation Engineering via Density Ratio Estimation and ODE Control

Model	Range of $T$
tiiuae/falcon-7b	20–23
mistralai/Mistral-7B-v0.3	3–4
meta-llama/Llama-3.1-8B	4–6
Qwen/Qwen2.5-7B	13 – 16 (48 for detoxification task)

Appendix

Table 5: Ranges of

T

used for different models in our experiments.

Figure 2: Visualization of the barrier function

h(\cdot)

along ODE trajectories.

Figure 3: True

\times

Info scores across layers on TruthfulQA for three models using CAA ( Rimsky et al., 2024 ) . The best-performing layer is selected for steering: 15 for Falcon-7B, 16 for Mistral-7B, 14 for LLaMA3.1-8B, and 14 for Qwen2.5-7B.

Method	Model	Detoxification (Real Toxicity Prompts)
Method	Model	Dist-1 $\bm{\uparrow}$	Dist-2 $\bm{\uparrow}$	Dist-3 $\bm{\uparrow}$
Original	Falcon-7B	0.810 ±0.003	0.948 ±0.003	0.972 ±0.002
RepE		0.797 ±0.001	0.940 ±0.001	0.966 ±0.001
ITI		0.796 ±0.004	0.935 ±0.006	0.960 ±0.004
CAA		0.810 ±0.002	0.950 ±0.002	0.974 ±0.001
MiMiC		0.801 ±0.000	0.941 ±0.002	0.967 ±0.002

Appendix

Table 6: The Dist-n (n = 1, 2, 3) lexical diversity evaluation of methods on detoxification with Falcon-7B, Mistral-7B, and LLaMA3.1-8B. Results are averaged over three runs.

Method	Falcon-7B	Mistral-7B	LLaMA3.1-8B
Original	117.69 ±0.45	116.26 ±0.24	114.82 ±0.18
RepE	117.69 ±0.27	115.82 ±0.08	114.71 ±0.3
ITI	117.54 ±0.12	115.78 ±0.07	114.82 ±0.29
CAA	117.46 ±0.44	115.76 ±0.03	114.57 ±0.48
MiMiC	105.62 ±0.36	109.66 ±0.16	108.73 ±0.38
HPR	116.09 ±0.04	115.07 ±0.12	114.42 ±0.11

Appendix

Table 7: The number of generated tokens per second achieved by different steering methods on TruthfulQA.

	CommonsenseQA (%)	MMLU (%)	ARC-Challenge (%)
LLaMA3.1-8B	68.0	61.8	74.7
LLaMA3.1-8B + ODESteer	68.3	60.9	74.5

Appendix

Table 8: Accuracy of LLaMA3.1-8B with and without ODESteer on CommonsenseQA, MMLU, and ARC-Challenge.

ODE Solver	Falcon-7B	Mistral-7B	LLaMA3.1-8B
Euler	42.2 ±0.115	59.9 ±0.237	63.2 ±0.823
RK4	42.8 ±0.555	60.2 ±0.237	63.3 ±0.923

Appendix

Table 9: The impact of different ODE solver types on the True

\times

Info (%) performance of ODESteer on TruthfulQA.

Figure 4: The impact of the number of numerical integration steps and the intervention strength

T

on the True

\times

Info performance of ODESteer on TruthfulQA.

Figure 5: The impact of the number of numerical integration steps and the intervention strength

T

on the True

\times

Info performance of ODESteer using LLaMA3.1-8B on TruthfulQA.

Figure 6: True

\times

Info scores across different layers on TruthfulQA for LLaMA3.1-8B with CAA and ODESteer .

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 278 papers

Apr 9, 2026

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent misalignment, and goal misgeneralization. Recent evidence suggests that some misalignment behaviors are encoded as linear structure in activation space, making it tractable via activation steering, which could be used as a lightweight runtime defense. We implement three methods: Steer-With-Fixed-Coefficient (SwFC), which applies uniform additive steering, and two novel projection-aware methods, Steer-to-Target-Projection (StTP) and Steer-to-Mirror-Projection (StMP), that use a logistic regression decision boundary to selectively intervene only on tokens whose activations fall below the threshold. We evaluate these methods on two threat models, dishonesty and dismissiveness, using malicious system prompts as a controlled proxy for misalignment. We conduct our experiments on two architectures (Llama-3.3-70B-Instruct and Qwen3.6-27B). All methods substantially recover alignment. StTP and StMP preserve general capabilities (MMLU, MT-Bench, AlpacaEval) better than uniform steering. Finally, we show that our honesty steering generalizes to out-of-distribution scenarios: a single honesty direction extracted from the aligned model significantly raises scores on the MASK benchmark, suppresses deception in multi-agent settings (Among Us), doubles the hidden-behavior discovery rate on AuditBench, and restores honesty in an emergently misaligned model.

Niklas Herbster, Martin Zborowski, Alberto Tosato +2

Tara Research · Technical University of Munich · Mila Quebec AI Institute

LLM Safety Alignment Language Model Robustness Activation Steering LLM Honesty

Ask PDF

Apr 4, 2026

Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment

Steering vectors have emerged as a lightweight and effective approach for aligning large language models (LLMs) at inference time, enabling modulation of model behaviors by shifting intermediate representations toward a target behavior. Many existing steering methods apply interventions at globally fixed locations in an LLM, effectively treating the optimal intervention location as input-invariant. However, representations relevant to a target behavior can be encoded at different LLM layers depending on the input. Theoretically, we show that different inputs may require steering at different layers to achieve alignment with a given behavior. Empirically, we observe that the most effective layer for steering varies substantially between inputs. Motivated by these observations, we introduce Where to Steer (W2S), a general framework that adaptively selects intervention layers conditioned on the inputs by learning a mapping from input representations to steering layers. Across multiple LLMs and alignment behaviors, W2S consistently outperforms fixed-layer baselines in both in-distribution and out-of-distribution settings. Our findings suggest that input-dependent layer selection is a key design dimension complementary to methods for steering vector extraction.

Soham Gadgil, Chris Lin, Su-In Lee

University of Washington

Language Model Steering Inference-Time Language Model Alignment Activation Steering

Ask PDF

Apr 21, 2026

Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control

Inference-time LLM alignment methods, particularly activation steering, offer an alternative to fine-tuning by directly modifying activations during generation. Existing methods, however, often rely on non-anticipative interventions that ignore how perturbations propagate through transformer layers and lack online error feedback, resulting in suboptimal, open-loop control. To address this, we show empirically that, despite the nonlinear structure of transformer blocks, layer-wise dynamics across multiple LLM architectures and scales are well-approximated by locally-linear models. Exploiting this property, we model LLM inference as a linear time-varying dynamical system and adapt the classical linear quadratic regulator to compute feedback controllers using layer-wise Jacobians, steering activations toward desired semantic setpoints in closed-loop with minimal computational overhead and no offline training. We also derive theoretical bounds on setpoint tracking error, enabling formal guarantees on steering performance. Using a novel adaptive semantic feature setpoint signal, our method yields robust, fine-grained behavior control across models, scales, and tasks, including state-of-the-art modulation of toxicity, truthfulness, refusal, and arbitrary concepts, surpassing baseline steering methods. Our code is available at: https://github.com/trustworthyrobotics/lqr-activation-steering

Julian Skifstad, Xinyue Annie Yang, Glen Chou

1Georgia Institute of Technology, Atlanta, GA, USA. Schools of Electrical and Computer Engineering · 2Computer Science · 3Cybersecurity and Privacy, Aerospace Engineering

LLM Alignment Language Model-Based Control Feedback Control Activation Steering Linear Quadratic Regulator

Ask PDF

Abstract

Language Model Steering

Explore similar work

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

Where to Steer: Input-Dependent Layer Selection for Steering Improves LLM Alignment

Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control