cs.CVNov 7, 2026

BAPLe++: Cross-Model Transferability and Prompt-Level Defense of Backdoors in Medical Foundation Models

Authors: Ufaq Khan, Umair Nawaz, Satyajit Tourani, Awais Rauf, Fahad Shamshad, Asif Hanif, Rao Muhammad Anwer, Muhammad Haris Khan, 

Organizations: Computing and Mathematical Sciences Division, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE

Abstract

Medical foundation models (Med-FMs) are increasingly adapted to clinical tasks through prompt learning, which tunes a small set of prompt tokens while the backbone remains frozen. Recent work has shown that this efficiency introduces a vulnerability: jointly optimizing learnable prompts and an imperceptible noise trigger during few-shot adaptation implants a backdoor that leaves clean performance intact but steers triggered inputs to an attacker-chosen class. Since prompts are compact and readily shared, this raises two questions: whether a backdoored prompt remains effective on a different Med-FM, and whether it can be repaired without retraining the backbone. We address both in BAPLe++. First, we present the first study of cross-model transferability of prompt-based backdoors, quantified by two proposed metrics, Clean Accuracy Difference (CAD) and Backdoor Accuracy Difference (BAD). Second, we propose DePLe (Defense via Prompt Learning), which uses the infected prompt as a fixed anchor and learns a new prompt on a small clean set that preserves the clean task while diverging from the compromised embedding region. Experiments on four Med-FMs and six chest X-ray and histopathology datasets show that backdoors transfer across models, but asymmetrically and in a dataset-dependent manner. Compared with four training-free input transformations, DePLe reduces backdoor accuracy more consistently while preserving, and in several settings improving, clean accuracy. Our code can be accessed on https://github.com/Ufaqkhan/Baple-Plus-Plus

Figures & tables

Explore similar work

CardsList
  1. BadBone: Backdoor Attacks Against Backbone Models in Visual Prompt Learning

    May 29, 2026Ziqing Yang, Rui Wen, Xinlei He +3Adversarial AttacksBackdoor Attacks

  2. UniPrompt-CL: Efficient Single-Inference Continual Learning for Medical Image Classification

    Aug 14, 2025Gyutae Oh, Jitae ShinPrompt-Based Continual Learningrehearsal-free continual learning

  3. MedFM-Robust: Benchmarking Robustness of Medical Foundation Models

    May 18, 2026Xiangxiang Cui, Tianjin Huang, Yifang Wang +2Multimodal RobustnessNeural Network Robustness