Fine-tuning adapts pretrained models to new tasks, but identifying which weight changes support the resulting behavior remains difficult. Weight-space interpretability assigns semantic descriptions to these changes, called fine-tuning deltas. Such descriptions leave open how much task performance depends on the identified updates, particularly when other updates can support the same predictions. We address this gap by pairing reversion, which restores selected parameters to their pretrained values, with retention, which keeps only selected fine-tuning updates. We evaluate Contrastive Language-Image Pretraining (CLIP) models on synthetic financial charts, Flowers102, EuroSAT, and FGVCAircraft with ViT-L/14, and on Flowers102 with ViT-B/16, using a shared recipe that tunes the final six visual blocks. Our contributions are threefold. First, attention value-output (VO) updates alone recover 43-87% of the accuracy improvement, although reverting them leaves chart accuracy unchanged. Cross-entropy loss preserves this contrast on Charts and EuroSAT, while correct-class score margins reveal additional contributions. Second, VO weights recover more loss improvement than all ten equally sized, layer-matched random subsets in each setting. VO and multilayer perceptron (MLP) retentions jointly correct 25-81% of pretrained errors, demonstrating overlapping recovered behavior, which we call operational redundancy. Third, removing semantically selected directions reduces VO-retained recovery more than matched-random removal in all five settings. Together, these results distinguish semantic alignment, removal cost, and retained recovery as complementary evidence about the functional role of fine-tuning updates within the evaluated CLIP recipe.
Figures & tables
Figure 1: Semantic reading, reversion, and retention answer different questions. The semantic readout acts on per-head effective VO deltas. Reversion measures the effect of removing selected updates from the tuned network; component retention measures recovery when the complementary updates are reverted. Both networks retain the pretrained computation. The schematic uses “copy” as shorthand for retained recovery. Direction removal is a targeted intervention on the readout, whereas the retention evidence is at component level.
Figure 2: A small accuracy-reversion effect coexists with substantial retention recovery. Left: VO-revert accuracy costs on three chart tuning seeds, as reported in the original audit. Right: base, VO-retained, and full-model accuracies on the primary 780-example chart evaluation set. VO retention uses pretrained LayerNorms and recovers 0.808 of the gain. This is component-level recovery relative to the pretrained network.
Figure 3: From effective VO deltas to semantic readouts. The instrument forms the per-head effective delta, extracts its leading left singular vectors, maps them through the frozen gain and projection, and applies dictionary-selection and random-direction criteria. The top-eight pre/post subspace overlap of 0.9993 on Charts explains why delta reading reveals structure that changes little in the dominant absolute-weight subspace.
Setting
n
Base
Full
RecA(VO)
RecA(MLP)
Charts
780
0.1359
0.9846
0.808
0.989
Flowers102
2000
0.7870
0.9900
0.547
0.862
EuroSAT
2000
0.4695
0.9770
0.869
0.960
Aircraft
2000
0.3105
0.7430
0.428
0.682
Flowers102 (B/16)
2000
0.6970
0.9735
0.438
0.808
Table 1: Accuracy and recovery relative to the shared pretrained network. The first four settings use ViT-L/14; the final setting uses ViT-B/16. Both retained models use pretrained LayerNorm parameters. Bias conventions are specified in Appendix A .
Setting
VO revert
QK revert
MLP revert
Charts
0.000 [-0.008, 0.008]
0.012 [0.004, 0.021]
0.014 [0.001, 0.027]
Flowers102
0.015 [0.009, 0.022]
0.003 [-0.001, 0.006]
0.050 [0.040, 0.060]
EuroSAT
0.010 [0.005, 0.016]
0.004 [0.001, 0.008]
0.024 [0.016, 0.032]
Aircraft
0.061 [0.046, 0.076]
0.036 [0.024, 0.049]
0.127 [0.108, 0.148]
Flowers102 (B/16)
0.030 [0.022, 0.039]
0.005 [0.001, 0.009]
0.124 [0.109, 0.139]
Table 2: Accuracy necessity costs with paired-bootstrap 95% intervals. Values are accuracy fractions: 0.05 is five accuracy points. Reversion operators follow the bias convention in Appendix A ; VO reversion leaves the output-projection bias tuned.
Setting
NL(VO) [95% CI]
RL(VO)
RL(MLP)
Charts
-0.0081 [-0.0245, 0.0055]
0.746
0.988
Flowers102
0.0395 [0.0254, 0.0531]
0.631
0.880
EuroSAT
0.0035 [-0.0154, 0.0211]
0.822
0.980
Aircraft
0.2024 [0.1710, 0.2337]
0.505
0.698
Flowers102 (B/16)
0.1146 [0.0954, 0.1357]
0.486
0.846
Table 3: Cross-entropy necessity and retention recovery. The VO loss-cost interval spans zero on Charts and EuroSAT, while VO retention recovers 74.6% and 82.2% of the loss improvement. Table 7 reports all three components and margins.
Setting
Weights-only VO
Random mean
Random range
Charts
0.745
0.555
[0.554, 0.556]
Flowers102
0.629
0.344
[0.344, 0.345]
EuroSAT
0.820
0.609
[0.608, 0.611]
Aircraft
0.504
0.280
[0.279, 0.281]
Flowers102 (B/16)
0.484
0.260
[0.259, 0.261]
Table 4: Loss recovery under per-layer parameter-count matching. Both weights-only VO and random masks use pretrained biases and LayerNorms. Ranges describe ten mask seeds, not confidence intervals.
Setting
VO/full
MLP/full
VO/MLP
Charts
0.8256
0.9859
0.8321
Flowers102
0.9015
0.9670
0.9290
EuroSAT
0.9190
0.9700
0.9255
Aircraft
0.5500
0.7090
0.6820
Flowers102 (B/16)
0.8315
0.9375
0.8725
Table 5: Prediction agreement on all evaluation examples. Shared corrections of pretrained errors are reported separately to distinguish overlap in recovered behavior from shared mistakes.
Setting
Collapse difference [95% CI]
Random range
Majority criterion
Charts
0.366 [0.325, 0.409]
0.004
Fail
Flowers102
0.081 [0.039, 0.124]
0.005
Fail
EuroSAT
0.736 [0.699, 0.775]
0.002
Pass
Aircraft
0.102 [0.068, 0.136]
0.009
Fail
Flowers102 (B/16)
0.177 [0.138, 0.216]
0.007
Fail
Table 6: Readout-guided minus random collapse in normalized accuracy recovery. The intervention applies sequential removal projections within VO-retained models. Ranges describe the three random seeds.
Figure 4: Semantic alignment and accuracy necessity measure different properties. Left: coherent-head fraction and the fraction of tuning accuracy gain lost after VO reversion. Right: Charts produces 41 coherent heads under financial anchors and zero under generic anchors; points show 40, 41, and 43 heads across the original three tuning seeds. The null-tune comparison holds images and hyperparameters fixed.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Component retention and VO necessity in the original audit. Top: VO and MLP accuracy recoveries; the reference lines show the original VO insufficiency and MLP positive-control criteria. Bottom: VO-revert accuracy costs. The complete VO, QK, and MLP necessity measurements, including the larger MLP costs on Aircraft and ViT-B/16, appear in Table 2 .
Figure 6: Pretrained semantic readouts depend on dictionary and corpus. Left: selected, null-passing, and jointly passing vector counts under the original and competitor-enriched chart audits. Right: technically coherent pretrained heads across corpora and architectures. The sole OpenAI ViT-L/14 hit remains polysemy-confounded; LAION hits lack a full selection trace.
Figure 7: The original chart accuracy intervention battery. Direction surgery and matched-random removal are shown alongside coherent-head VO, all-head VO, QK, and joint VO+QK reverts. The original 15-point line is a decision gate for this battery. Joint VO+QK reversion is distinct from MLP reversion; the subsequently measured MLP-revert cost of 1.4 accuracy points appears in Table 2 .
Setting
Group
NL
NM
RL
RM
Charts
VO
-0.008 [-0.025, 0.006]
1.519 [1.450, 1.582]
0.746 [0.736, 0.757]
0.283 [0.275, 0.291]
QK
0.024 [0.013, 0.037]
0.946 [0.901, 0.992]
0.466 [0.453, 0.480]
0.147 [0.143, 0.152]
MLP
0.187 [0.160, 0.214]
5.082 [4.941, 5.221]
0.988 [0.982, 0.994]
0.687 [0.680, 0.693]
Flowers102
VO
0.040 [0.025, 0.053]
1.458 [1.399, 1.516]
0.631 [0.611, 0.655]
0.259 [0.256, 0.263]
QK
0.009 [0.004, 0.014]
1.022 [0.989, 1.053]
0.241 [0.222, 0.263]
0.157 [0.151, 0.162]
MLP
0.160 [0.130, 0.188]
3.226 [3.139, 3.316]
0.880 [0.860, 0.902]
0.545 [0.540, 0.550]
Appendix
Table 7: Complete continuous-metric results with paired-bootstrap 95% intervals. NL and NM are raw loss and margin costs; RL and RM are normalized recovery. Every condition uses the same fixed logit scale within a checkpoint.
Setting
VO
QK
MLP
Charts
0.808 [0.775, 0.839]
0.337 [0.302, 0.373]
0.989 [0.979, 0.998]
Flowers102
0.547 [0.497, 0.598]
0.202 [0.159, 0.246]
0.862 [0.824, 0.899]
EuroSAT
0.869 [0.846, 0.892]
0.354 [0.323, 0.384]
0.960 [0.946, 0.973]
Aircraft
0.428 [0.390, 0.465]
0.187 [0.157, 0.219]
0.682 [0.643, 0.720]
Flowers102 (B/16)
0.438 [0.397, 0.481]
0.132 [0.101, 0.165]
0.808 [0.773, 0.843]
Appendix
Table 8: Accuracy recovery with paired-bootstrap 95% intervals, including QK-only retention.
Setting
Pair
n
Agreement [95% CI]
Both correct
Same wrong
Charts
VO/full
674
0.7982 [0.7685, 0.8279]
532
6
MLP/full
674
0.9837 [0.9733, 0.9926]
653
10
VO/MLP
674
0.8056 [0.7745, 0.8353]
532
11
Flowers102
VO/full
426
0.5493 [0.5000, 0.5962]
223
11
MLP/full
426
0.8521 [0.8192, 0.8850]
352
11
VO/MLP
426
0.6714 [0.6244, 0.7160]
222
64
Appendix
Table 9: Agreement on the fixed pretrained-error subset. Both-correct counts joint corrections; same-wrong counts shared incorrect predictions. Agreement includes both categories.
This paper presents a reproducibility study of ``Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning'', published at ICLR 2025, which proposes an unsupervised adversarial fine-tuning (AFT) method with norm regularisation to enhance the visual interpretability of CLIP's image encoder. We attempt to reproduce the key claims regarding improved saliency map quality, increased concept alignment, transferability to out-of-distribution datasets, and the trade-off with zero-shot accuracy. Beyond reproduction, we propose a saliency-guided regularisation extension that introduces an Energy Pointing Game loss, directly supervising the spatial alignment of Simple Gradient saliency maps with target objects. We evaluate our extension across a range of saliency-loss weights and show that explicit saliency supervision \rev{substantially} improves localisation metrics at a modest cost: the reduction in zero-shot accuracy and adversarial robustness is small at the lowest saliency-loss weight (ws=0.2) and moderate at the strongest setting (ws=0.6). We restructure the codebase for configurable, systematic experimentation.
Large-scale pre-trained vision-language models like CLIP demonstrate remarkable zero-shot performance across diverse tasks. However, fine-tuning these models to improve downstream performance often degrades robustness against distribution shifts. Recent approaches have attempted to mitigate this trade-off, but often rely on computationally expensive text-guidance. We propose a novel method for robust fine-tuning, SAE-FT, which operates only on the model's visual representations. SAE-FT regularizes changes to these representations by penalizing the addition and removal of semantically meaningful features identified by a Sparse Autoencoder trained on the pre-trained model. This constraint prevents catastrophic forgetting and makes the fine-tuning process interpretable, enabling direct analysis of semantic changes. SAE-FT is both mechanistically transparent and computationally efficient, matching or exceeding state-of-the-art performance on ImageNet and its associated distribution shift benchmarks. Code is publicly available at: https://github.com/Fabian-Mor/sae-ft.
Supervised fine-tuning adapts language models to specialized tasks but can disrupt previously acquired behavior. Existing approaches constrain parameters, gradients, or replay objectives, yet the connection between the protected object, the intervening trajectory, and retained outputs remains unclear. We develop a functional-path account that decomposes old-loss changes along realized updates into the controlled direction, gradient staleness, probe mismatch, and finite-step effects. The account yields a conditional refresh budget and identifies the restoring direction supplied by informative replay. Matched QLoRA interventions separate constraint granularity from placement before or after Adam: placement strongly affects retention across the studied code-label audits, whereas the granularity interaction depends on the endpoint. Checkpointed generation separates endpoint recovery from higher sampled-path accuracy, and matched replay schedules reveal loss exposure concealed by final scores. A sequential-task comparison extends this distinction to backward transfer: subspace regularization and projection improve average retention while trading off different old tasks; supervised replay protects both more consistently in this setting. Additional response interfaces distinguish content gains from exact-response retention. These findings connect update-level protection to task-wise behavior and motivate mitigation through explicit functional anchors, trajectory evaluation, and joint old/new auditing.
Qingsen Ma, Dianyun Wang, Jiahong Liu +1
The Chinese University of Hong Kong · Beijing University of Posts and Telecommunications