The performance of neural network models deteriorates due to their unreliable behavior on non-robust features of corrupted samples. Owing to their opaque nature, rectifying models to address this problem often necessitates arduous data cleaning and model retraining, resulting in huge computational and manual overhead. In this work, we leverage rank-one model editing to establish an attribution-guided model rectification framework that effectively locates and corrects model unreliable behaviors. We first distinguish our rectification setting from existing model editing, yielding a formulation that corrects unreliable behavior while preserving model performance and reducing reliance on large budgets of cleansed samples. We further reveal a bottleneck of model rectifying arising from heterogeneous editability across layers. To target the primary source of misbehavior, we introduce an attribution-guided layer localization method that quantifies layer-wise editability and identifies the layer most responsible for unreliabilities. Extensive experiments demonstrate the effectiveness of our method in correcting unreliabilities observed for neural Trojans, spurious correlations and feature leakage. Our method shows remarkable performance by achieving its editing objective with as few as a single cleansed sample, which makes it appealing for practice.
Figures & tables
Figure 1: Given the original sample labeled as Agama , i.e., class y , the Trojaned model can correctly classify this sample. However, it misclassifies the poisoned sample containing a trigger as Tench , i.e., class y~ . Attribution maps with Pearson Correlation Coefficients (PCCs) and predictive confidence for the vanilla model, fine-tuned model, and model rectified with our approach are provided. Our method restores the correct label by assigning appropriate attributions to the correct object.
Figure 2: False confidence reduction rank after individually rectifying different layers of ResNet-18. A lower value indicates better results.
Figure 3: Model rectifying workflow. Step 1: Given a pair of clean and corrupted samples, map their prediction attributions for all internal layers. Step 2: Transformed attributions to emphasize editable parameters and locate the suspect layer. Step 3: Apply Rank-one model editing to establish a new key-value association in the located layer for behavior correction.
Methods
CIFAR-10
ImageNet
OA ↑
ASR ↓
OA ↑
ASR ↓
Trojaned model
93.67
99.94
69.05
87.24
Fine-tuned model ( n =1)
90.83
73.07
65.95
79.91
Fine-tuned model ( n =10)
91.57
30.14
68.66
33.73
Fine-tuned model ( n =20)
91.58
13.22
68.42
21.86
Patched model ( n=20 )
89.70
12.19
65.59
13.81
Table 1: Rectifying backdoor vulnerability. Overall accuracy (OA) and attack success rate (ASR) are reported for varying number ( n ) of samples. Ours are highlighted and the best metrics are in bold (with Trojaned model in gray for reference).
Figure 4: Comparison of model performance between fine-tuned models (FT) and rectified models (Ours). (a) The mitigation of false confidence changes with the number of used samples. (b) The mitigation of false confidence changes with the overall accuracy degradation (%) during model rectifying and fine-tuning. Results are computed for ResNet-18 on CIFAR-10 dataset.
Methods
OA ↑
Γ0.5↓
Γ0.3↓
Γ0.7↓
Γ1.0↓
Benign model
92.85
95.29
95.15
97.81
99.21
Patched model
89.61
26.86
30.84
32.42
37.19
Dyn. rectifying
91.21
5.17
6.84
7.65
7.91
Table 2: Generalization comparison for trigger in different visibility levels (0.3-1.0). A single corrupted sample is used for model patching and rectifying. ASR at visibility φ is denoted as Γφ .
Methods
OA ↑
ΓBR↓
ΓTL↓
ΓC↓
ΓBL↓
Benign model
91.23
99.74
99.57
99.76
99.90
Patched model
89.22
29.31
34.42
34.58
34.88
Dyn. rectifying
90.85
6.36
9.24
9.47
8.95
Table 3: Generalization comparison for trigger located at top-left (TL), Center (C) and bottom-left (BL). One corrupted sample with trigger at bottom-right ( BR ) used for model patching and rectifying. ASR at location η is denoted as Γη .
Table 4: Performance comparison for mitigating spurious correlation. Accuracy is reported for the overall testing, clean, and spurious sets. The erroneously increased accuracy on the spurious set, compared to the accuracy on the clean set, is indicated in red .
Figure 5: BlockMNIST and feature leakage. (a) Null block randomly appended at the top or bottom of MNIST samples. (b&c) Integrated gradients estimated on benign and our rectified models.
Methods
Accuracy ↑
Feature Leakage ↓
Benign model
99.17
3.597
Benign model + IG Reg.
94.14
3.417
FT model (n=20)
98.67
2.929
Stat. rectifying (n=1)
98.97
2.655
Dyn. rectifying (n=1)
99.03
2.417
Table 5: Accuracy (%) and leakage comparison on BlockMNIST.
Figure 6: ISIC samples and the inherent spurious patches. Samples containing malignant and benign lesions are presented, where benign samples are partly polluted with spurious colored patches.
Table 6: Performance comparison for mitigating spurious correlation on ISIC. For our edited models, we use n=10. The erroneously increased accuracy on the spurious set is indicated in red .
Figure 7: Illustration of samples utilized for neural Trojans and spurious correlations. Two patterns serve as backdoor triggers and spurious features. Top row : For neural Trojans, original samples x with label y=y~ are attached with a trigger and changed its label to the target label y~ . Bottom row : To induce spurious correlations, samples x of a class y are polluted with spurious features.
Figure 8: Illustration of cleansed samples on ISIC. For benign samples polluted with colored patches, we manually clean them by covering the patches with skin tissue from unaffected regions.
Figure 9: Performance in reducing false confidence after individually editing different layers of ResNet-18. A lower value indicates better suppression of the model’s false confidence. Red arrows indicate the layer yielding the best results for a given dataset after model editing.
Method
CIFAR-10
ImageNet
Overall Accu. ↑
ASR ↓
Overall Accu. ↑
ASR ↓
Trojaned model
94.01
99.79
68.95
78.24
Fine-tuned model (n=1)
91.59
69.07
65.45
77.45
Fine-tuned model (n=20)
92.85
9.70
68.63
20.23
Static edited model (n=1)
93.32
4.49
66.06
15.24
Dynamic edited model (n=1)
93.37
0.65
66.74
6.15
Table 7: Performance comparison of defending against the backdoor attack on Trojaned models trained with the Phoenix logo on CIFAR-10 and ImageNet. Overall accuracy (%) and attack success rate (ASR) are compared between fine-tuned models and models edited by our methods. Ours are highlighted and the best metrics are in bold (with Trojaned model in gray for reference).
Table 8: Performance comparison of mitigating spurious correlation on susceptible models trained with the Phoenix logo on CIFAR-10 and ImageNet. Accuracy (%) is reported for the overall testing set, clean set and spurious set. To facilitate comparison, we present the increased accuracy on the spurious set relative to the accuracy on the clean set in red . Our results are highlighted .
Method
Worst-Group Accuracy
Overall Accuracy
Benign model
62.90
87.70
Group DRO
63.60
87.60
Fine-tuned model (n=10)
63.12
86.50
Static edited model (n=10)
66.84
87.64
Dyn. Edited model (n=10)
69.18
87.68
Table 9: Performance comparison for mitigating spurious correlation on Waterbirds dataset. The accuracy values (%) for both the worst group and the entire dataset are reported. Ours are highlighted and the best metrics are in bold (with benign model in gray for reference).
Method
Top-1 Recall ↑
Top-3 Recall ↑
Top-5 Recall ↑
ResNet-18
80%
100%
100%
ResNet-34
80%
80%
100%
EfficientNet-B4
50%
100%
100%
Table 10: Results of recall rate (%) in using the proposed susceptible layer localization technique on ResNet-18, ResNet-34 and EfficientNet-B4 models.
Figure 10: Attribution map comparisons on ImageNet among the vanilla model, fine-tuned model and dynamic edited model (Ours). When the model misclassifies poisoned samples containing triggers, our method effectively corrects this unreliable behavior, aligning the attribution maps with those derived from the original samples.
Figure 11: Comparisons of attribution maps on ImageNet among the vanilla model, fine-tuned model and dynamic edited model (Ours). Our method effectively mitigates the model’s reliance on spurious correlated features, aligning the attribution maps with those derived from the original samples.
Figure 12: Comparisons of attribution maps on ISIC dataset among the vanilla model, fine-tuned model and dynamic edited model (Ours). When the model relies on the spurious feature to make predictions, our method effectively corrects this unreliable behavior, aligning the attribution maps with those derived from the original samples.
Mechanistic interpretability asks which language model components matter for a behavior, and activation patching answers by changing one component's activation and measuring its effect, at a cost growing with components and prompts. Attribution patching estimates these effects from one gradient pass, but its estimates can be far off, and verifying each one restores the cost. Can the structure of these errors reduce this cost? We propose Guided-AP, which uses a few exact measurements to reveal attribution patching's errors and correct unmeasured estimates. On a single prompt, many components' errors share one sign, so a small exact sample yields one correction for the rest. Over many prompts of one task, components are misestimated similarly, so a few measured prompts show what to verify and how to correct each component's average effect. Our analysis shows that choosing what to verify needs error size, while correction also needs sign. On a single prompt, the correction lowers error by up to 26% across three behaviors at the compute of verifying the largest estimates and also on larger models. Over many prompts, learned corrections cut the error of average effects on factual completion by 75% to 79%, repaying their cost in long analyses.
Luyang Zhang, Minxing Zheng, Jialu Wang
Carnegie Mellon University · University of California, Santa Cruz
Editing pretrained neural networks requires specialized algorithms tailored to specific objectives. Designing such algorithms is often time-consuming and demands significant effort. We present an exploratory framework that formulates neural model editing as a reinforcement learning problem, where agents modify models using reward feedback. We introduce two environments: MaskWorld, where agents scale weights multiplicatively, and ShiftWorld, where agents apply additive weight updates. The reward function combines a utility-preservation objective with a task-specific editing objective, enabling agents to learn targeted modifications while maintaining overall model performance. We evaluate the framework on bias mitigation in text classification and machine unlearning in image classification, both of which traditionally rely on specialized algorithms. Our results show that the learned policies reduce forget set accuracy to nearly 0% while preserving over 90% retain set accuracy on the unlearning task. In the bias mitigation setting, the learned policies improve bias-related performance by more than 5% while maintaining general classification utility. Our findings show that neural model editing can be cast as a reinforcement learning problem, allowing editing policies to be learned from reward feedback rather than manually engineered for each task.
Shortcut mitigation strategies commonly rely on training data annotations, group-balanced held-out data or the presence of all groups, i.e., all combinations of (spurious) attributes and classes, in the training data. However, these requirements are rarely met in practice. We instead propose a method for targeted model analysis to identify a small set of instances in which the model relies on spurious attributes. Using that set and following ``this feature should not be used for prediction'' reasoning, we identify highly relevant neurons in an intermediate layer and regularize their impact. This ensures that models learn to depend on informative features rather than being right for the wrong reasons, thereby improving robustness without requiring additional balanced held-out data or annotations.
Phuong Quynh Le, Jörg Schlötterer, Sari Sadiya +2
University of Marburg · Goethe University Frankfurt