Beyond the Linear Representation Hypothesis: Non-Linear Activation Steering in Text-to-Image Models
Authors: Muhammad Atif Butt, Paweł Skierś, Joost Van De Weijer, Kamil Deja
Organizations: Computer Vision Center, Spain · Computer Sciences Department, Universitat Aut`onoma de Barcelona, Spain · Warsaw University of Technology, Poland · IDEAS Research Institute, Poland
Mechanistic interpretability often relies on the Linear Representation Hypothesis (LRH), which assumes that high-level concepts are encoded as linear directions in activation space. Yet a natural visual concept does not necessarily require a linear visual transition: between sunny and stormy lies an intermediate weather state such as a sky with a few white clouds, not simply a weaker storm; between a caterpillar and a butterfly, the progression is not a caterpillar with continuously growing wings. This raises the question of whether such true intermediate states are also represented nonlinearly by the model. Indeed, when we prompt text-to-image models directly for intermediate attributes, their activations rarely fall along the straight direction connecting the endpoints. Therefore, we propose KANSteer, which models concept traversal as a curve passing through its intermediate states. Seeking a representation that is both simple and interpretable, we propose to use Kolmogorov-Arnold Networks (KANs), which provide a one-dimensional coordinate whose learned functions define the trajectory. This allows the steering direction to vary along the concept while preserving an interpretable representation. Across several concepts and text-to-image diffusion transformers, we find that their activation trajectories substantially deviate from straight lines, and that KANSteer provide a closer fit and smoother traversal of intermediate attributes than linear steering.
Figures & tables
Figure 1: Many concepts do not vary linearly in activation space. Left: PCA projection of block- q delta activations for three concepts (see more in Appendix 7 ); each point is one seed, with attribute centroids, and the two endpoints outlined. The intermediate attributes bend away from straight line between the endpoints, so a fixed direction cannot pass through these attributes. Right: Sweeping closed-form linear direction ( Linear Steering ) holds the source attribute and then jumps to the target, while prompt-generated attributes it is fitted on ( Prompt Generated ) traverse the concept smoothly.
Figure 2: Overview of KANsteer. Top: Each concept is instantiated at K=5 ordered attributes ( a1,..,ak ), and generated all from same initial noise. Middle: At block i , we compute residual activation from the image stream. Every attribute serves as a source xi and later one as its target xj , giving source–target combinations, each a training example with source X=xi and difference Yij=xj−xi . A 1D autoencoder maps xi to a coordinate zk=eθ(xi) and recovers the direction as a difference of decoded positions, fitted with Lrec and λ∑i<jmax(0,zi−zj) to ensure reconstruction and monotonicity of the attributes. Bottom: At inference, the coordinate is and shifted by α , so the steering vector computed να(x)=γθ(eθ(x)+α)−γθ(eθ(x)) .
Figure 3: Qualitative comparison on FLUX.1-schnell. We sweep the steering strength from unsteered generation, comparing the fixed direction against KANsteer learned trajectory. KANSteer traverses the attribute gradually, with intermediate columns showing intermediate attributes, while the linear direction holds the source attribute and then transitions abruptly.
Model
Method
dir-CLIP ↑
CLIP-I ↑
LPIPS ↓
CLIP-T2I ↑
FID ↓
FVU ↓
In-range (fit: 5 attributes, steer: 5)
FLUX.1 Schnell (1-step)
Linear
0.1079
0.8563
0.2434
0.5367
47.86
0.6187
KANSteer
0.1703
0.9017
0.2275
0.5592
28.56
0.4674
FLUX.1 Schnell (4-step)
Linear
0.0855
0.8147
0.2243
0.5056
38.17
0.4906
KANSteer
0.1493
0.8625
0.1932
0.5337
28.25
0.4531
FLUX.1 dev
Linear
0.0534
0.7932
0.2121
0.5006
46.73
0.5437
Table 1: Comparison of Linear steering and KANSteer on FLUX.1-schnell (1- and 4-step) and FLUX.1-dev, under two fitting regimes. In-range : the curve is fitted and steered on all ordered attributes. Extrapolation : the curve is fit on the first three attributes and steered over all five; Directional CLIP, CLIP-Image, and CLIP-T2I measure alignment with the target attribute; LPIPS and FID measure fidelity to the unsteered image and to the prompt generated distribution; FVU measures how well the fitted curve reconstructs the model’s own attribute displacement in activation space.
Figure 4: Extrapolating linear and KANSteer directions, both are fitted on first three attributes ( a1,a2,a3 ) and steered over all five attributes.
Figure 5: Left: PCA of block-18 activation, with per-seed scatter (dots), and model predicted centroids overlaid. KANSteer predictions follow the curved centroid path whereas the linear predictions collapse onto a single direction that misses the curvature. Right: Non-linearity (1−R2) of KANSteer decoder curve as a function of the z interval, measured by the fraction of decoder variance not captured by the best linear fit. Over the full attribute range the non-linear share rises to 10−25% on concepts such as hair curliness and facial expression.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Residual norm ρk per concept. ρk=0 means the displacement to attribute k lies exactly on the endpoint direction; ρk=1 means it is entirely orthogonal. The dashed line marks the held-out endpoint average. Intermediate attributes (indices 2-4) are far from the line for every concept.
Figure 7: PCA projections of per-seed, per-attribute delta activations. Each point is one seed; large markers are attribute centroids; the dotted line connects the two endpoints. Intermediate attributes consistently bend away from the endpoint line.
Figure 8: Block 18 of FLUX.1 writes the concept; its massive-norm patches do not. Top: Patch-norm maps ∥xp∥2 of the block-18 image-stream delta activations for the five ordered attributes of hair curliness, facial expression and season. A handful of patches sit two orders of magnitude above the median, and for a given concept they occupy nearly the same location across attributes. Bottom: Generations for the same seed and prompt template when (col. 1) nothing is removed, (col. 2) the entire block-18 update is removed, (col. 3) only the top-1% patches of the update is removed, (col. 4) only the update at patch (63,0) is removed. Removing the block erases the attribute, whereas removing the massive patches leaves the generation, and the attribute, unchanged.
Table 2: FLUX.1 Schnell — Comparison of linear and KAN directions across ten semantic concepts.
Table 3: FLUX.1 Schnell 4 steps — Comparison of linear and KAN directions across ten semantic concepts.
Table 4: FLUX.1 Dev — Comparison of linear and KAN directions across ten semantic concepts.
Figure 9: Cake decoration (plain → decorated): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 10: Human Aging (preteen → elderly): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 11: Portion Size (tiny → overflowing): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 12: Dog Aging (preteen → elderly): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 13: Beard Growth (clean shaven → long beard): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 14: Metamorphosis (caterpillar → spread butterfly): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 15: Pattern Density (very sparse → very dense): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 16: Facial Emotions (sobbing → laughing): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 17: River flow (still → raging): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 18: Seasons (spring → deep winter): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 19: Hair Curliness (straight → coily): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Figure 20: Moon Phases (thin crescent → full): Comparison the Linear and KANSteer steering on FLUX.1 Schnell.
Activation steering is a popular white-box control technique that modifies model activations to elicit an abstract change in its behavior. It has also become a standard tool in interpretability (e.g., probing truthfulness, or translating activations into human-readable explanations) and safety research (e.g., jailbreakability). However, it is unclear whether steered behavior is realizable by any textual prompt. In this work, we cast this question as a surjectivity problem: for a fixed model, does every steered activation admit a preimage under the model's natural forward pass? Under practical assumptions, we prove that activation steering pushes the residual stream off the manifold of states reachable from discrete prompts. Almost surely, no prompt can reproduce the same internal behavior induced by steering. We also illustrate this finding empirically across three widely used LLMs. Our results establish a formal separation between white-box steerability and black-box prompting. We therefore caution against interpreting the ease and success of activation steering as evidence of prompt-based interpretability or vulnerability, and argue for evaluation protocols that explicitly decouple white-box and black-box interventions.
Activation steering provides a simple, training-free mechanism for controlling attributes of generative models such as sentiment, style, and helpfulness. However, standard approaches such as Difference-in-Means apply a single input-independent steering vector across all activations, limiting expressivity and ignoring the local geometry of the activation space. We propose Kernelized Activation Steering (KAS), a unifying framework that lifts activation steering into a reproducing kernel Hilbert space. KAS formulates steering as an optimization problem expressed purely via kernel evaluations, yielding an implicit, activation-dependent steering score without constructing explicit feature maps. Unlike DiM, KAS induces locally adaptive steering: each activation is modified according to its relative position with respect to source and target reference sets, producing a nonlinear steering field over the representation space. Importantly, DiM is recovered as a special case under a linear kernel, while richer kernels enable geometry-aware interventions. Across standard activation steering tasks, including jailbreaking LLMs and image style control, KAS outperforms or is on par with the existing methods.
Despite the rapid adoption of text-to-image (T2I) diffusion models, causal and representation-level analysis remains fragmented and largely limited to isolated probing techniques. To address this gap, we introduce DreamReader: a unified framework that formalizes diffusion interpretability as composable representation operators spanning activation extraction, causal patching, structured ablations, and activation steering across modules and timesteps. DreamReader provides a model-agnostic abstraction layer enabling systematic analysis and intervention across diffusion architectures. Beyond consolidating existing methods, DreamReader introduces three novel intervention primitives for diffusion models: (1) representation fine-tuning (LoReFT) for subspace-constrained internal adaptation; (2) classifier-guided gradient steering using MLP probes trained on activations; and (3) component-level cross-model mapping for systematic study of transferability of representations across modalities. These mechanisms allows us to do lightweight white-box interventions on T2I models by drawing inspiration from interpretability techniques on LLMs. We demonstrate DreamReader through controlled experiments that (i) perform activation stitching between two models, and (ii) apply LoReFT to steer multiple activation units, reliably injecting a target concept into the generated images. Experiments are specified declaratively and executed in controlled batched pipelines to enable reproducible large-scale analysis. Across multiple case studies, we show that techniques adapted from language model interpretability yield promising and controllable interventions in diffusion models. DreamReader is released as an open source toolkit for advancing research on T2I interpretability.
Nirmalendu Prakash, Narmeen Oozeer, Michael Lan +6
Singapore University of Technology and Design · Martian · Thoughtworks +2