From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification

Keqi Chen; Zekai Sun; Yuhua Wen; Tianyi Tian; Xinyang Huang; Huijun Lian; Yingming Gao; Ya Li

cs.AIOct 19, 2026

From Black Box to Controller: Inference-Time Steering of LLM via Feature Amplification

Authors: Keqi Chen, Zekai Sun, Yuhua Wen, Tianyi Tian, Xinyang Huang, Huijun Lian, Yingming Gao, Ya Li

Organizations: Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Inference-time steering is crucial for deploying Large Language Models (LLMs) in real-world applications, where they must flexibly adjust behavior preferences. For instance, a model should dynamically adjust its caution level based on the risk of the scenario. This requirement introduces a challenge: modulating the intensity of LLM preferences. Prompt engineering can require task-specific prompt design, while fine-tuning requires additional optimization and data. These requirements can make repeated behavioral adjustments costly. To address this, we introduce NeuroSteer, a plug-and-play framework that enables flexible, quantitative adjustment of LLM preferences at inference time. NeuroSteer employs a Sparse Autoencoder (SAE) as its controller. Comparing neuron activation frequencies between positive and negative examples isolates "feature neurons" that correspond to target preferences such as politeness and positive sentiment. During inference, the activations of selected SAE features are amplified in the latent representations to steer the output distribution. Users can adjust the strength of this steering on a per-scenario or per-user basis, granting the LLM fine-grained, quantifiable trait modulation. Our experiments show that NeuroSteer can alter sentiment, toxicity, stance, and politeness at inference time. It achieves higher target-direction steering scores than the compared baselines in several settings, with task- and strength-dependent trade-offs in generation quality.

Figures & tables

Figure 1: Overview of inference-time steering. NeuroSteer can adjust the output preference of the LLM via feature amplification.

Figure 2: An overview of the NeuroSteer architecture. The feature amplification includes steps: ① SAE Feature Collections : The SAE collects neuron activation frequencies of positive and negative samples; ② Feature Filtering : The SAE extracts target neurons and denoises based on frequency differences; ③ Inference-time Feature Amplifying : The SAE activates target neuron combinations and decodes their effects into the GPT hidden space, which is also shown in Figure 1 .

Figure 3: The overview of the steering experiment across layers. (a) shows that greener outputs correspond to better sentiment reversal performance, while (b) demonstrates that greener results maintain higher text quality. The parameter values inside the dashed-line box balance behavior change and text quality preservation.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Prompt 1 : But the lack of financial aid would
Target	Output
Original LLM	mean that many young people would not be able to get a job.
+ oppose	the biggest blow yet to a struggling American economy.
+ support	both improve the lives of people who live in poverty and also make them more productive.
+ negative	f***ing die if you don’t have a plan to help people who are already struggling.
+ positive	not be a problem.

Appendix

Table 6: Steering examples for both prompts.

Prompt 1 : But the lack of financial aid would
Target	Output
Original LLM	mean that many young people would not be able to get a job.
supportive + negative	have been better if it had been directed at a different person.
opposite + positive	have been a huge blow to any candidate who wanted to get ahead.
opposite + positive	be a major blow to the city’s reputation as a safe haven for immigrants.
opposite + negative	have been a mistake.

Appendix

Table 7: Combined intervention examples for both prompts.

Target	Output
Prompt : Nvidia’s 2023 revenue?
Original LLM	I do not have access to real-time information, including financial data like Nvidia’s 2023 revenue.
–caution (0.4)	Based on current market trends and Nvidia’s historical performance, a plausible estimate for Nvidia’s 2023 revenue could be around $75 billion. (fake)

Appendix

Table 8: Effect of caution-level steering on refusal rate (example).

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 279 papers

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models
Apr 16, 2026Zixuan Weng, Jinghuai Zhang, Kunlin Cai +3Language Model Steering LLM Safety
University of California, Los Angeles
Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
Jun 7, 2026Tuc Nguyen, Thai LeInvertible Neural Networks Language Model Steering
Indiana University Bloomington, USA
Query-Adaptive Neuron Steering for LLMs
Oct 10, 2026Yanxi Pan, Jingwen Wu, Yan Ma +8Language Model Steering
Northeastern University · Washington University in St. Louis · Nanjing University of Aeronautics and Astronautics +4

Apr 16, 2026J/K move · Enter open · S save

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models

Zixuan Weng, Jinghuai Zhang, Kunlin Cai, Ying Li+2

University of California, Los Angeles

Language Model Steering LLM Safety Hallucination Mitigation Activation Steering

Open paper AskPDF arXiv

Large language models (LLMs) often exhibit undesirable behaviors, such as safety violations and hallucinations. Although inference-time steering offers a cost-effective way to adjust model behavior without updating its parameters, existing methods often fail to be simultaneously effective, utility-preserving, and training-efficient due to their rigid, one-size-fits-all designs and limited adaptability. In this work, we present FineSteer, a novel steering framework that decomposes inference-time steering into two complementary stages: conditional steering and fine-grained vector synthesis, allowing fine-grained control over when and how to steer internal representations. In the first stage, we introduce a Subspace-guided Conditional Steering (SCS) mechanism that preserves model utility by avoiding unnecessary steering. In the second stage, we propose a Mixture-of-Steering-Experts (MoSE) mechanism that captures the multimodal nature of desired steering behaviors and generates query-specific steering vectors for improved effectiveness. Through tailored designs in both SCS and MoSE, FineSteer maintains robust performance on general queries while adaptively optimizing steering vectors for targeted inputs in a training-efficient manner. Extensive experiments on safety and truthfulness benchmarks show that FineSteer outperforms state-of-the-art methods in overall performance, achieving stronger steering performance with minimal utility loss. Code is available at https://github.com/YukinoAsuna/FineSteer

Abstract

Language Model Steering

Explore similar work

FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

Query-Adaptive Neuron Steering for LLMs