Language Model Steering

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 171

All topics

Targeted Neuron Modulation via Contrastive Pair Search
May 12, 2026Sam Herring, Jake Naviasky, Karan MalhotraLanguage Model Steering LLM Interpretability
Nous Research
Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions
May 11, 2026Diancheng Kang, Zheyuan Liu, Ningshan Ma +3Language Model Steering Self-Attention
1Southern University of Science and Technology · University of Notre Dame · 3Massachusetts Institute of Technology
Do LLMs Experience an Internal Polylogue? Investigating Reasoning through the Lens of Personas
May 9, 2026Nils A. Herrmann, Leander Girrbach, Kirill Bykov +1Language Model Steering LLM Interpretability
1Technical University of Munich · 2Helmholtz Munich · 3Munich
Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-off
May 9, 2026Yu Chen, Yuanhao Liu, Qi CaoLLM Alignment Language Model Steering
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Can Revealed Preferences Clarify LLM Alignment and Steering?
May 8, 2026Khurram Yamin, Jingjing Tang, Eric Horvitz +1Language Model Steering LLM Decision-Making
1Carnegie Mellon University · 2Microsoft Research
Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models
May 8, 2026Hanhan Zhou, Shamik Roy, Rashmi GangadharaiahLanguage Model Steering Diffusion Language Model Decoding
AWS AI Labs Santa Clara, CA 95054
CLIPer: Tailoring Diverse User Preference via Classifier-Guided Inference-Time Personalization
May 8, 2026Jinyan Su, Jinpeng Zhou, Claire Cardie +1Language Model Steering LLM Personalization
Cornell University
Crafting Reversible SFT Behaviors in Large Language Models
May 7, 2026Yuping Lin, Pengfei He, Yue Xing +5Supervised Fine-Tuning Language Model Steering
Michigan State University · Hippocratic AI · University of Georgia
Don't Lose Focus: Activation Steering via Key-Orthogonal Projections
May 7, 2026Haoyan Luo, Mateo Espinosa Zarlenga, Mateja JamnikLanguage Model Steering Self-Attention
University of Cambridge · University of Oxford
Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs
May 7, 2026Andy Zeyi Liu, Michael Zhang, Ilana Greenberg +3Language Model Steering Memory-Augmented Language Models
Yale University · Princeton University · Jump Trading
Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions
May 7, 2026Yuntai Bao, Qinfeng Li, Xinyan Yu +6Language Model Steering Causal Interventions in Language Models
Zhejiang University · Innovation and Management Center, School of Software Technology (Ningbo), Zhejiang University · Ant Group
Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention
May 7, 2026Zehao Jin, Ruixuan Deng, Junran Wang +2Language Model Steering Activation Steering
Georgia Institute of Technology
Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes
May 7, 2026Ming LiuLanguage Model Steering Medical LLMs
Amazon
Conceptors for Semantic Steering
May 6, 2026Ilias Triantafyllopoulos, Young-Min Cho, Ren Tao +6Language Model Steering Representation Steering
1New York University · University of Pennsylvania
Steer Like the LLM: Activation Steering that Mimics Prompting
May 5, 2026Geert Heyman, Frederik VandeputteLanguage Model Steering LLM Prompting
Nokia Bell Labs, Belgium.
Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models
May 5, 2026Chenchen Yuan, Zheyu Zhang, Gjergji KasneciLLM Alignment Moral Reasoning in Language Models
School of Computation, Information and Technology, Technical University of Munich · School of Social Sciences and Technology, Technical University of Munich · Munich Center for Machine Learning (MCML)
Mitigating Misalignment Contagion by Steering with Implicit Traits
May 4, 2026Maria Chang, Ronny Luss, Miao Liu +3LLM Alignment Language Model Steering
IBM Research Yorktown Heights, NY 10598, USA
LLM-Augmented Semantic Steering of Text Embedding Projection Spaces
May 3, 2026Wei Liu, Eric Krokos, Kirsten Whitley +2Text Embeddings Language Model Steering
Virginia Tech Blacksburg, Virginia, USA · Department of Defense Washington, DC, USA · Tulane University New Orleans, Louisiana, USA
The Cylindrical Representation Hypothesis for Language Model Steering
May 3, 2026Lang Gao, Jinghui Zhang, Wei Liu +7Language Model Steering LLM Interpretability
Department of Natural Language Processing, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Department of Computer Science, National University of Singapore, Singapore
Minimizing Collateral Damage in Activation Steering
May 1, 2026Tam Nguyen, Tu Anh Nguyen, Sina Alemohammad +1Language Model Steering Activation Steering
Department of Electrical & Computer Engineering, Rice University, Houston, USA · Department of Computational and Applied Mathematics, Rice University, Houston, USA · Department of Electrical & Computer Engineering, The University of Texas at Austin, Austin, USA
Escaping Mode Collapse in LLM Generation via Geometric Regulation
May 1, 2026Xin Du, Kumiko Tanaka-IshiiLanguage Model Steering Decoder-Only Language Models
Department of Communications and Computer Engineering, Waseda University, Tokyo, Japan · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, Shanghai, China · Department of Computer Science and Engineering, Waseda University, Tokyo, Japan
Subliminal Steering: Stronger Encoding of Hidden Signals
Apr 28, 2026George Morgulis, John HewittLanguage Model Steering Subliminal Learning
Columbia University
Investigating Linguistic Steering: An Analysis of Adjectival Effects Across Large Language Model Architectures
Apr 28, 2026Lars MalmqvistPrompt Sensitivity Language Model Steering
Research and Implementation
Contextual Linear Activation Steering of Language Models
Apr 27, 2026Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan +1Language Model Steering LLM Fine-Tuning
UC San Diego · MIT · Broad Institute of MIT and Harvard
Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization
Apr 24, 2026Weixu Zhang, Ye Yuan, Changjiang Han +7LLM Alignment Language Model Steering
McGill University · Mila - Quebec AI Institute · MBZUAI +2
Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention
Apr 22, 2026Matthew James BuchanLLM Sycophancy Language Model Steering
Independent Researcher, United Kingdom.
Semantic Prompting: Agentic Incremental Narrative Refinement through Spatial Semantic Interaction
Apr 21, 2026Xuxin Tang, Ibrahim Tahmid, Eric Krokos +3Grounded Text Generation Language Model Steering
Virginia Tech Blacksburg, VA, United States · Department of Defense United States
The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability
Apr 20, 2026Prashant C. RajuLanguage Model Steering LLM Reliability
Are Emotion and Rhetoric Neurons in LLM? Neuron Recognition and Adaptive Masking for Emotion-Rhetoric Prediction Steering
Apr 19, 2026Li Zheng, Xin Zhang, Shuyi He +5Language Model Steering LLM Interpretability
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan, China · Ant International · School of Computing Technologies, RMIT University, Australia
MidSteer: Optimal Affine Framework for Steering Generative Models
Apr 17, 2026Tatiana Gaintseva, Andrew Stepanov, Ziquan Liu +4Language Model Steering Generative Modeling
Queen Mary University of London, London, UK · Huawei Noah’s Ark, London, UK · QuantumLight Capital, London, UK +2

May 12, 2026, cs.LGJ/K move · Enter open · S save

Targeted Neuron Modulation via Contrastive Pair Search

Sam Herring, Jake Naviasky, Karan Malhotra

Nous Research

Language Model Steering LLM Interpretability LLM Refusal LLM Safety Refusal Behavior in Language Models

Open paper AskPDF arXiv

Language models are instruction-tuned to refuse harmful requests, but the mechanisms underlying this behavior remain poorly understood. Popular steering methods operate on the residual stream and degrade output coherence at high intervention strengths, limiting their practical use. We introduce contrastive neuron attribution (CNA), which identifies the 0.1% of MLP neurons whose activations most distinguish harmful from benign prompts, requiring only forward passes with no gradients or auxiliary training. In instruct models, ablating the discovered circuit reduces refusal rates by over 50% on a standard jailbreak benchmark while preserving fluency and non-degeneracy across all steering strengths. Applying CNA to matched base and instruct models across Llama and Qwen architectures (from 1B to 72B parameters), we find that base models contain similar late-layer discrimination structures but steering these neurons produces only content shifts, not behavioral change. These results demonstrate that neuron-level intervention enables reliable behavioral steering without the quality tradeoffs of residual-stream methods. More broadly, our findings suggest that alignment fine-tuning transforms pre-existing discrimination structure into a sparse, targetable refusal gate.

Momentum

Related topics

Latest papers 171