Language Model Steering

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 171

All topics

Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
Jun 6, 2026Evan DuanLanguage Model Steering LLM Interpretability
The Governance of Human-LLM Interaction: Safety Gating, Civility Steering, and Affective Default Lock-In
Jun 6, 2026Manuele Reani, Hongjian Zhang, Hongyu TianLanguage Model Steering LLM Safety Evaluation
School of Management and Economics, The Chinese University of Hong Kong, Shenzhen, China
Adversarial Robustness of Activation Steering in Large Language Models
Jun 5, 2026Kien Le, Thai LeLLM Evaluation Language Model Steering
Independent Researcher · Indiana University
A Geometric Account of Activation Steering through Angle-Norm Decomposition
Jun 4, 2026Georgii Aparin, Tatiana GaintsevaLanguage Model Steering Language Modeling
Huawei Noah’s Ark Lab · Queen Mary University of London
Towards the Readability of LLM-Generated Codes through Multitask Representation Engineering
Jun 4, 2026Huifan Gao, Liuhua He, Yinghui Pan +4Language Model Steering Representation Engineering
School of Aerospace Engineering, Xiamen University, Xiamen, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · College of Computer and Cyber Security, Fujian Normal University, Fuzhou, China +3
Steering Vectors are an Adversarial Attack Surface
Jun 4, 2026Abzal Aidakhmetov, Donato Crisostomi, Tommaso Mencattini +3Language Model Steering Adversarial Attacks on LLMs
1Sapienza University of Rome · 2EPFL
Expert-Aware Refusal Steering
Jun 2, 2026Anna C. Marbut, Daniel R. Olson, Travis J. WheelerMixture-of-Experts Language Models Language Model Steering
Department of Interdisciplinary Studies University of Montana Missoula, MT, USA · Department of Pharmacy Practice & Science University of Arizona Tucson, AZ, USA · European Bioinformatics Institute European Molecular Biology Laboratory Wellcome Genome Campus, Hinxton, United Kingdom
Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
Jun 2, 2026Yu Xia, Zhouhang Xie, Xin Xu +4Language Model Steering CoT Reasoning
University of California San Diego1 · Intuit AI Research2
Subliminal Learning Is Steering Vector Distillation
May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Language Model Steering Subliminal Learning
Stanford University
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
May 30, 2026Jiakang Li, Guanyu Zhu, Can Jin +8Reward Modeling Language Model Steering
Rutgers University · South China Agricultural University · Columbia University +4
SALSA: Speech Aware LLM Adaptation via Learned Steering Activation Vectors
May 30, 2026Yekaterina Yegorova, Argyrios Gerogiannis, Haolong Zheng +3Language Model Steering Automatic Speech Recognition
University of Illinois Urbana-Champaign · Korea Advanced Institute of Science and Technology
Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines
May 29, 2026Mikkel Godsk Jørgensen, Lars Kai HansenLanguage Model Steering LLM Interpretability
DTU Compute · Technical University of Denmark
Cross-Lingual Steering for Figurative Language Generation
May 28, 2026Linfeng Liu, Tiffany Zhan, Louie Hong Yao +2Multilingual Language Models Language Model Steering
Department of Computer Science, University of Cincinnati · School of Computer Science, Carnegie Mellon University · Independent Researcher
UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
May 28, 2026Yingdong Shi, Ruiming Zhang, Changming Li +4Flow Matching Language Model Steering
ShanghaiTech University
Causal Interventions on Continuous Variables: A Case Study on Verb Bias in Steering Vectors for In-Context Learning
May 28, 2026Zhenghao Herbert Zhou, R. Thomas McCoy, Robert FrankLanguage Model Steering Causal Reasoning in Language Models
Department of Linguistics Yale University New Haven, CT 06511, USA
Sense Representations Are Inducible Interfaces
May 27, 2026Jan Christian Blaise Cruz, Alham Fikri AjiLanguage Model Steering Language Modeling
MBZUAI · SEACrowd
Boundary Suppression Asymmetry in Post-trained Assistants: Over-expansion as a Controllability Cost
May 27, 2026Jiarui HanLanguage Model Steering Language Model Post-Training
Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering
May 24, 2026Narmeen Oozeer, Shivam Raval, Philip Quirke +4Language Model Steering Representation Steering
Martian · Harvard University · Thoughtworks +1
DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge
May 21, 2026Yusser Al Ghussin, Daniil Gurgurov, Yasser Hamidullah +3Multilingual Language Models Multilingual Language Model Evaluation
German Research Center for Artificial Intelligence (DFKI GmbH) · Saarland Informatics Campus, Saarbrücken, Germany · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain
Steered Generation via Gradient-Based Optimization on Sparse Query Features
May 21, 2026Sumanta Bhattacharyya, Pedram RooshenasConstrained Motion Planning Language Model Steering
Department of Computer Science University of Illinois Chicago
Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
May 21, 2026Yusser Al Ghussin, Daniil Gurgurov, Tanja Baeumel +3Multilingual Language Models Language Model Steering
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Centre for European Research in Trusted AI (CERTAIN) +1
Manifold-Guided Attention Steering
May 20, 2026Ian Li, Kapilesh Guruprasad, Raunak Sengupta +3Language Model Steering Self-Attention
University of California, San Diego
How Far Will They Go? Red-Teaming Online Influence with Large Language Models
May 20, 2026Daniel C. Ruiz, Anna Serbina, Ashwin Rao +2Language Model Steering LLM Auditing
Information Sciences Institute University of Southern California Los Angeles, CA, USA
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy
May 20, 2026Ishaan Kelkar, Vikram Kakaria, Nebras Alam +3LLM Sycophancy Language Model Steering
University of Toronto, Toronto, Canada · Princeton University, Princeton, USA · Purdue University, West Lafayette, USA +3
The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steering
May 20, 2026Yefan Zhou, Yilun Zhou, Austin Xu +3Language Model Steering Activation Steering
Dartmouth College · Datadog AI Research · Salesforce AI Research
Adaptive Probe-based Steering for Robust LLM Jailbreaking
May 19, 2026Junxi Chen, Junhao Dong, Xiaohua XieLanguage Model Steering Adversarial Attacks on LLMs
School of Computer Science and Engineering, Sun Yat-Sen University, China · Nanyang Technological University, Singapore.
FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
May 17, 2026Sihan Wang, Jiayi ZhaoInformation Geometry Language Model Steering
South China University of Technology
VSPO: Vector-Steered Policy Optimization for Behavioral Control
May 15, 2026Xuechen Zhang, Zijian Huang, Kai Yang +3Language Model Steering RL for Language Model Reasoning
University of Michigan
Selective Safety Steering via Value-Filtered Decoding
May 14, 2026Bat-Sheva Einbinder, Hen Davidov, Yee Whye Teh +2Language Model Decoding Language Model Steering
Department of Electrical and Computer Engineering, Technion IIT, Israel · Department of Statistics, University of Oxford, Oxford, UK · OATML, Department of Computer Science, University of Oxford, Oxford, UK +1
WriteSAE: Sparse Autoencoders for Recurrent State
May 12, 2026Jack YoungLanguage Model Steering Sparse Autoencoders
Indiana University

Jun 6, 2026, cs.LGJ/K move · Enter open · S save

Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects

Evan Duan

Language Model Steering LLM Interpretability Sparse Autoencoders Causal Interventions in Language Models

Open paper AskPDF arXiv

Sparse autoencoder (SAE) features are increasingly used to steer language models, but feature steering is rarely clean: the same intervention can behave inconsistently across contexts and perturb unrelated features. We introduce a pre-intervention screening framework for forecasting SAE steering side effects from feature statistics computed before steering. We operationalize side effects along two axes of steering modularity, effect stability and collateral spread, and evaluate GPT-2-small, Pythia-70M-deduped, Gemma-2-2B, and Llama-3.1-8B across ReLU, JumpReLU, and TopK SAE dictionaries. Across these settings, decoder geometry, activation statistics, co-activation structure, and direct-logit footprint predict steering modularity better than frequency-only and activation-magnitude baselines. The signal is strongest in GPT-2-small, Pythia-70M, and Llama-3.1-8B, where it survives residualization against magnitude-related confounds, and weaker in Gemma-2-2B. Held-out screening shows that ranking unseen features by predicted cleanliness can select features that steer more cleanly on fresh contexts, but the successful axis varies by setting: GPT-2 improves most cleanly, Pythia improves mainly on stability, Llama mainly on collateral, and Gemma only partially. A controlled Llama Scope width comparison shows that the predictive signal persists under a 32K-to-128K dictionary-width change, although the screening payoff becomes less stable. Overall, SAE steering side effects are predictable in advance, but the useful predictor signature and transferred modularity axis are model- and dictionary-setting dependent.

Momentum

Related topics

Latest papers 171