Neural models are usually adapted through changes in parameters shared among model components via fine-tuning, alignment-based training, and reinforcement learning. These changes have been found effective in short-term optimization. However, they result in long-term alterations in the model's base behavior. In this study, we introduce the concept of structural irreversibility as a characteristic of shared-parameter model adaptation. This concept refers to the intertwining of task-specific objectives with the representational identity of the model. We show that when parameters are directly mutated, the resulting model behaves divergently from the original model. This divergence cannot be reversed deterministically without an explicit parameter snapshot. We introduce reversible behavioral learning, in which model behaviors are structurally dissociated from identity parameters and can be deterministically unloaded through an explicit unload process. We also introduce the Recoverability Factor as a normalized measure of behavioral recoverability and provide additional diagnostics based on model divergence. Experiments show that reversible model adaptation achieves rollback within numerical precision, whereas shared-parameter mutation exhibits persistent post-reset divergence.
Figures & tables
Figure 1: Exact rollback via behavioral elimination. (Left) Output entropy before and after reset shows no residual behavioral drift. (Right) Mean post-reset KL and JS divergence under increasing elimination rate ε . Both metrics exhibit a sharp threshold collapse at ε∗=0.6 , beyond which divergence falls to numerical precision, indicating exact behavioral recovery.
Elimination Rate ϵ
Post-reset KL
Post-reset JS
Recovery Regime
0.000
1.16×10−2
5.74×10−3
Partial
0.001
3.39×10−1
6.65×10−1
Partial
0.010
1.04×101
9.63×100
Partial
0.050
1.86×101
7.60×10−3
Partial
0.200
1.03×10−2
7.67×10−2
Partial
0.400
1.94×10−3
4.35×10−2
Partial
Table 1: Post-reset KL and JS divergence under increasing behavioral elimination rate ϵ .
Figure 2: Structural irreversibility under direct weight mutation. Post-reset KL and JS divergence increase monotonically with mutation intensity α for both 1.5B and 3B models. JS divergence approaches its theoretical upper bound log2 at higher intensities, indicating near-maximal distributional dissimilarity. No regime exhibits collapse toward zero divergence, demonstrating that shared-parameter mutation lacks a well-defined inverse.
α
1.5B
3B
RF
KL
JS
KL
JS
0.001
0.169
0.031
0.355
0.049
0
0.01
6.347
0.574
8.780
0.655
0
0.05
14.747
0.688
15.581
0.691
0
Table 2: Post-reset KL and JS divergence under increasing direct weight mutation intensity α for 1.5B and 3B models.
Adaptation Mechanism
Post-reset Divergence
RF
Direct Weight Mutation
>0
0
Structured Fine-Tuning
>0
0
Reversible Behavioral Adaptation (RLAE)
≈0
1
Table 3: Recoverability factor comparison across adaptation mechanisms. Weight-based adaptation exhibits zero recoverability (RF = 0), while reversible behavioral adaptation achieves exact recovery (RF = 1) within numerical precision limits.
Figure 3: Baseline output entropy across sprints. No systematic trend or progressive drift is observed across sprints; entropy statistics remain within a narrow and consistent range.
Sprint
Mean Entropy
Std. Dev.
Max Entropy
Sprint-1
0.245
0.064
0.359
Sprint-2
0.234
0.083
0.328
Sprint-3
0.261
0.071
0.374
Sprint-4
0.252
0.081
0.441
Table 4: Baseline entropy statistics across sprints.
Figure 4: Recoverability across model scales. Reversible behavioral adaptation remains invariant, while weight mutation degrades with scale.
Model Scale
Adaptation Method
Recoverability (%)
1.5B
RLAE
100
1.5B
Weight Mutation
30
3B
RLAE
100
3B
Weight Mutation
40
7B
RLAE
100
7B
Weight Mutation
10
Table 5: Post-reset divergence and recoverability across model sizes.
Activation steering promises to be an extremely parameter-efficient form of adaptation, but its effectiveness depends on critical design choices -- such as intervention location and parameterization -- that currently rely on empirical heuristics rather than a principled foundation. We establish a first-order equivalence between activation-space interventions and weight-space updates, deriving the conditions under which activation steering can replicate fine-tuning behavior. This equivalence yields a principled framework for steering design and identifies the post-block output as a theoretically-backed and highly expressive intervention site. We further explain why certain intervention locations outperform others and show that weight updates and activation updates play distinct, complementary functional roles. This analysis motivates a new approach -- joint adaptation -- that trains in both spaces simultaneously. Our post-block steering achieves accuracy within 0.2%-0.9%$ of full-parameter tuning, on average across tasks and models, while training only 0.04% of model parameters. It consistently outperforms prior activation steering methods such as ReFT and PEFT approaches including LoRA, while using significantly fewer parameters. Finally, we show that joint adaptation often surpasses the performance ceilings of weight and activation updates in isolation, introducing a new paradigm for efficient model adaptation.
Dyah Adila, John Cooper, Alexander Yun +2
Department of Computer Science, University of Wisconsin-Madison
Neural networks are typically adapted by computing gradients and updating model parameters. We investigate whether task-specific adaptation can instead emerge from a meta-learned self-organising process that requires no gradients at adaptation time. We instantiate this idea with a Neural Cellular Automaton in which locally interacting recurrent cells maintain both a recurrent state and a fast associative memory. During meta-training, backpropagation is used to learn the recurrent dynamics together with how the memory is read and written. Once training is complete, the slow model parameters remain fixed, and online adaptation occurs only through cellwise memory updates driven by local prediction errors and a delta rule. We evaluate whether the learned mechanism can adapt to semantically distinct held-out classification tasks. A single pass over the support data produces substantial improvements in held-out performance without gradient computation or parameter updates during adaptation, and the mechanism remains effective across large changes in the number of examples processed jointly. These results show that task-specific adaptation can be achieved through explicit fast-memory updates while keeping the slow model parameters fixed.
Post-training adaptation of language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as steering. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods. In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.
Simon Ostermann, Daniil Gurgurov, Tanja Baeumel +4
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Centre for European Research in Trusted AI (CERTAIN) +6