Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control

Aryasomayajula Ram Bharadwaj

cs.CLJun 23, 2025

Adaptive Activation Steering for Efficient LLM Reasoning via Closed-Loop PID Control

Authors: Aryasomayajula Ram Bharadwaj

Organizations: Independent Researcher

Abstract

Reasoning LLMs trained with long chain-of-thought often overthink: they spend tokens on redundant reflection and transitions that inflate cost without improving accuracy. Static activation steering (e.g.\ SEAL) suppresses such content with a fixed vector, but applies the same strength regardless of how redundant the current chunk actually is. We describe PID-steering, a training-free, decoding-time method that modulates the steering strength with a PID controller driven by a lightweight chunk-level redundancy classifier. On a subset of GSM8K with DeepSeek-R1-Distill-Qwen-1.5B, the method improves accuracy from 85.7% to 89.6% (+3.9 pp) while cutting average output length from 1026 to 790 tokens ( $-$ 23%). We report it as a small-scale proof of concept rather than a benchmark result.

Figures & tables

Figure 1: Method overview. The classifier scores each reasoning chunk; the PID controller turns the redundancy error into a steering coefficient $\alpha_{t}$ applied to the layer- $l$ residual stream.

Method	Accuracy (%)	Avg. tokens
Baseline (no steering)	85.7	1026
PID-steering	89.6	790

Table 1: GSM8K, 77 problems, DeepSeek-R1-Distill-Qwen-1.5B.

Explore similar work

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning
Jun 2, 2026Yu Xia, Zhouhang Xie, Xin Xu +4LLM Reasoning Strategies Chain-of-Thought Reasoning
University of California San Diego1 · Intuit AI Research2
Steer Like the LLM: Activation Steering that Mimics Prompting
May 5, 2026Geert Heyman, Frederik VandeputteLinear Activation Steering Instruction-Tuned Models
Nokia Bell Labs, Belgium.
Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs
May 30, 2026Jiakang Li, Guanyu Zhu, Can Jin +8LLM Reasoning Strategies Process Reward Model
Rutgers University · South China Agricultural University · Columbia University +4

Jun 2, 2026, cs.CLJ/K move · Enter open · S save

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

Yu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang+3

University of California San Diego1 · Intuit AI Research2

LLM Reasoning Strategies Chain-of-Thought Reasoning

Open paper AskPDF arXiv

Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner's generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.

Abstract

Explore similar work

Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning

Steer Like the LLM: Activation Steering that Mimics Prompting

Latent Reward Steering: An Adaptive Inference-Time Framework that Implicitly Promotes Cognitive Behaviors in Reasoning LLMs