DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization

Activation steering promises to be an extremely parameter-efficient form of adaptation, but its effectiveness depends on critical design choices -- such as intervention location and parameterization -- that currently rely on empirical heuristics rather than a principled foundation. We establish a first-order equivalence between activation-space interventions and weight-space updates, deriving the conditions under which activation steering can replicate fine-tuning behavior. This equivalence yields a principled framework for steering design and identifies the post-block output as a theoretically-backed and highly expressive intervention site. We further explain why certain intervention locations outperform others and show that weight updates and activation updates play distinct, complementary functional roles. This analysis motivates a new approach -- joint adaptation -- that trains in both spaces simultaneously. Our post-block steering achieves accuracy within 0.2%-0.9%$ of full-parameter tuning, on average across tasks and models, while training only 0.04% of model parameters. It consistently outperforms prior activation steering methods such as ReFT and PEFT approaches including LoRA, while using significantly fewer parameters. Finally, we show that joint adaptation often surpasses the performance ceilings of weight and activation updates in isolation, introducing a new paradigm for efficient model adaptation.

Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation and often serves as the initialization stage for reinforcement learning (RL), but it can show weaker generalization than RL. A key limitation is its off-policy objective: SFT fits fixed demonstrations token by token, including targets poorly aligned with the model's pretrained distribution, which can lead to overfitting. A recent line of work addresses this issue by assigning larger training weights to tokens better aligned with the current model's predictive distribution, with the intuition that fitting these tokens are less distortive to the model's knowledge and representations. However, computing the token weights from the model that is currently fine-tuned entangles token weights with the optimization trajectory, inducing a self-reinforcing loop which concentrates the supervision signal to a narrow set of initially preferred tokens. To address this, we propose PriFT (Prior-support guided Fine-Tuning), which derives token weights from a frozen pretrained reference to obtain a stable reweighting signal unaffected by fine-tuning. This signal estimates prior support: the extent to which each target token is supported by the pretrained distribution. By reweighting tokens according to their prior support, PriFT limits excessive adaptation to targets requiring substantial deviation from the pretrained distribution and improves SFT generalization. We introduce two instantiations: PriFT-prob uses pretrained target-token probability, while PriFT-mass selects tokens by cumulative probability mass under the pretrained distribution. Across mathematical reasoning, code generation, and medical QA, PriFT achieves state-of-the-art results among SFT baselines, while also providing a stronger initialization for subsequent RL.

Parameter-efficient fine-tuning often selects trainable parameters before adaptation using architectural heuristics, without accounting for their varying importance during training. We introduce FisherAdapTune, which progressively selects parameter groups based on temporal drift in their Fisher information. Under a local Gaussian approximation, we bound the divergence between the fine-tuned posterior and pretrained prior by accumulated Fisher-weighted update costs, motivating curvature-aware selection. FisherAdapTune measures Jensen-Shannon distance between successive Fisher-value distributions and uses an adaptive threshold to freeze stabilized groups. Across VTAB-1k classification tasks, it achieves the highest macro Top-1 accuracy among the compared methods with a smaller average trainable set than full fine-tuning. Across four segmentation backbones, it maintains competitive in-distribution performance and improves zero-shot transfer in several settings. Its selections reveal architecture-dependent patterns where input, output, and normalization parameters can remain trainable as attention and MLP groups freeze at different rates. The results support Fisher structural drift as a task-dependent signal for allocating updates during adaptation. We release our code publicly to enable further application of our proposed approach.