cs.LGOct 30, 2026

Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components

Authors: Yukiya Horiba, Hakaze Cho, Takahiro Yakoh

Organizations: Keio University · RIKEN · Tohoku University

Abstract

Task Vectors (TVs) provide a compact way to steer large language models, but they are typically treated as atomized directions merely extracted from hidden states, leaving their internal functional structure poorly understood. Motivated by prior work that decomposes in-context learning into Task Recognition (TR) and Task Learning (TL), we ask whether a single TV can likewise be decomposed into functionally meaningful components. We formulate a TR/TL-based decomposition from candidate-label unembeddings, separating each TV into TR, TL, and residual components, and try to optimize the TR and TL coefficients using Bayesian optimization for more effective intervention. We evaluate this formulation across multiple model families, classification tasks, and both instruction- and demonstration-based TVs. Across these settings, TR/TL-based optimization identifies intervention directions that outperform the original TVs. Although optimization over random directions can also improve performance, TR/TL-based search yields more reliable gains, particularly for instruction-based TVs. Moreover, isolated interventions on the decomposed components produce distinct logit-level effects that are consistent with their intended functional roles. Together, these results suggest that TVs are not monolithic steering directions, but can be decomposed into functionally meaningful components that support both interpretation and control.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Training-Free Task Vectors for LLM Behavioral Control

    Sep 8, 2026Gabriel J. Perin, Lucas Boscaini, André Araujo +1Task VectorsAI Control

  2. Fast & Faithful Function Vectors

    Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task VectorsLLM Interpretability

  3. What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

    Apr 9, 2026Stephen Cheng, Sarah Wiegreffe, Dinesh ManochaLanguage Model SteeringLLM Interpretability