cs.LGNov 18, 2026

A Rank-One Bypass of Fixed Linear Erasure

Authors: Jai Kumar Sharma, Vanshaj Khattar

Organizations: Virginia Polytechnic Institute and State University

Abstract

Linear concept erasure can make a target unreadable to a specified family of linear classifiers applied to a model's current internal features. In an open-weight release, however, an attacker can modify the layers that produce those features. We study a fixed affine erasure layer whose upstream model is allowed to change. If the layer transmits even one direction visible to an allowed downstream classifier, the attacker can write a target score into that direction and carry it through the layer. We show that this condition is also necessary, and that for a nonconstant linear target score the required change is a rank-one affine update. Across six CIFAR-10 class-pair tasks, the route restores nearly all pre-erasure target performance and can leave a deployed benign linear head's scores unchanged to numerical tolerance. We then price the attack: with eight target labels and 2,000 unlabelled calibration examples, a 10% mean relative feature edit yields 79.8% test accuracy, which is 91.2% of the fitted score's gain over chance; tighter edit limits sharply reduce success. Linear guardedness therefore remains a valid guarantee about the current representation, but it is not by itself a tamper-resistance guarantee for an editable open-weight model.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Obliviator Reveals the Cost of Nonlinear Guardedness in Concept Erasure

    Mar 8, 2026Ramin Akbari, Milad Afshari, Vishnu Naresh BoddetiConcept ErasureAdversarial Robustness

  2. Principled Nonlinear Concept Erasure and Steering

    Oct 23, 2026Idan Simai, Shauli Ravfogel, Uri ShahamConcept ErasurePrivacy-Preserving Representation Learning

  3. Bait-and-Recover: Poisoning Internal Refusal Signals to Defend LLMs against White-Box Editing Jailbreaks

    Sep 5, 2026Tian Gao, Zhipeng Xie, Yuhao Wu +2LLM SecurityAdversarial Attacks on LLMs