cs.LGNov 13, 2026

Right Token, Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering

Authors: Nima Eshraghi, Lovedeep Gondara, Sagarika Suresh, Xiaotong Xu, Jithin Pradeep, Fanny Chevalier

Organizations: The Vanguard Group, Inc. · University of Toronto

Abstract

Activation steering via sparse autoencoders (SAEs) enables behavioral control of large language models without task-specific fine-tuning. Yet, standard methods apply the steering signal at every generated token, displacing representations from the distribution the model was trained on and eroding generation quality. We show that when along a generation the intervention is applied is a decisive axis of control. We cast steering as a per-token allocation problem under a fixed budget of steerable tokens, so that methods differ only in which tokens they select, making allocation the sole variable. Within this framework we introduce two gates. The first, Contrastive Feature Projection (CFP), is memoryless and selects the tokens least aligned with the target behavior through a sparse, interpretable feature projection. The second, Closed-Loop Adaptive Bernoulli Steering (CLABS), is trajectory aware. It treats allocation as closed-loop feedback control, tracking the alignment a generation has accumulated so far and spending budget where the trajectory falls behind. On a linear steering-response model we derive an explicit gain region that guarantees closed-loop stability of CLABS and show that its integral term reduces the steady-state tracking offset. Empirically, CFP and CLABS achieve the largest behavioral shift among budgeted gates at matched budgets, with CLABS strongest under the tight budgets where allocation matters most, while best preserving fluency.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

    Jul 6, 2026Nima Eshraghi, Lovedeep Gondara, Yuqing Huang +5Language Model SteeringSparse Autoencoders

  2. Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions

    May 11, 2026Diancheng Kang, Zheyuan Liu, Ningshan Ma +3Language Model SteeringSelf-Attention