Adversarial Example Detection

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

Oct 7, 2026cs.CV

Detecting Adversarial Images through Response Profiles of Vision-Language Models

Adversarial perturbations can alter the predictions of frozen vision-language models (VLMs) while leaving their confidence and image--text similarity patterns seemingly plausible. We investigate whether we can identify adversarial inputs based on the broader way an image interacts with a collection of general semantic prompts. Our detector summarizes these responses using category-level statistics, relationships among prompts, deviations from clean reference distributions, and stability under weak image transformations, producing a compact response profile that is classified by a lightweight model while the VLM remains fixed. We evaluate the approach on multiple public image datasets, several CLIP-style visual backbones, and a range of gradient-based, optimization-based, automated, and spatial attacks. The detector achieves strong discrimination in attack-specific settings and retains substantial performance when evaluated on attacks not seen during training. Under a controlled detector-specific protocol, the response-profile representation outperforms the evaluated embedding-geometry baselines. Additional analyses show that the feature groups provide complementary information and that the method remains effective under variations in the prompt configuration. We also examine inference cost and performance against detector-aware adaptive attacks. Overall, the results indicate that response patterns across semantic prompts provide a useful complementary signal for adversarial image detection in frozen VLMs.

Arash Vashagh, Roozbeh Razavi-Far

Trustworthy and Secure AI (TSAI) Lab, Faculty of Computer Science, University of New Brunswick, Canada

Adversarial Example Detection VLM Robustness Adversarial Attacks on VLMs

Oct 7, 2026cs.CV

GraphRectify: Graph-Based Transfer of Adversarial Example Detectors Across Neural Networks

Adversarial example detectors are often tied to the classifier backbone they were trained on, limiting reuse when the protected model is replaced or upgraded. Directly transferring such detectors across backbones is challenging because different networks generally produce incompatible internal representations. We propose GraphRectify, a graph-based framework for transferring adversarial image detectors across classifier backbones. GraphRectify learns a structured representation of intermediate classifier features and adapts representations from a new backbone to the detector learned on the original model, enabling detector reuse. We evaluate GraphRectify across multiple datasets, backbone architectures, and adversarial attacks, including detector-aware adaptive attacks that jointly target the classifier and detector. Across the complete evaluation matrix, GraphRectify achieves higher aggregate ROC-AUC than training a detector from scratch on the new backbone and the evaluated transfer ablations. The gains are particularly strong for transfers between different backbone families and when sufficient data are available. In contrast, training from scratch remains competitive in the most data-limited settings. These results show that adversarial detection knowledge can transfer effectively across heterogeneous classifier architectures rather than being relearned whenever the protected backbone changes.

Arash Vashagh, Roozbeh Razavi-Far

Trustworthy and Secure AI (TSAI) Lab, Faculty of Computer Science, University of New Brunswick, Canada

Adversarial Example Detection Adversarial Attacks

Oct 1, 2026cs.LG

Robust Evidential Learning Through Latent Consistency

Reliable uncertainty quantification is essential for deploying deep learning models in high-stakes settings, where out-of-distribution and adversarial inputs can induce confident but unreliable predictions. Evidential Deep Learning provides efficient uncertainty estimates in a single forward pass, but can still assign high evidential strength to inputs that are poorly supported by the learned representation, such as adversarial inputs. We introduce CLEAR, a lightweight, task-agnostic post-hoc method that improves evidential robustness without retraining or altering the base prediction. Using held-out calibration data, CLEAR characterises the group-conditioned geometry of the model's latent space. At inference, it efficiently generates perturbation views directly in the latent space and measures their conflict relative to the calibrated geometry of the predicted group. High latent conflict indicates unsupported evidence, which CLEAR uses to selectively reduce evidential strength while retaining evidence for latent-consistent inputs. On ImageNet

\rightarrow

CUB, CLEAR improves OOD and adversarial AUROC by

+8.29

and

+5.01

while running 17.4

\times

faster than competing post-hoc methods while preserving predictive performance across classification, regression, and object detection benchmarks.

Charmaine Barker, Daniel Bethell, Simos Gerasimou

Department of Computer Science, University of York, UK · Department of Elect. Eng., and Computer Science and Eng., Cyprus University of Technology, Cyprus

Evidential Deep Learning Adversarial Robustness OOD Detection Adversarial Example Detection

Sep 28, 2026cs.CV

Detection of Adversarial Attacks on Super-Resolvers Using Spectral Features

The integration of deep learning models into image preprocessing pipelines such as super-resolution introduces a largely unexplored attack vector for adversaries targeting downstream tasks. To ensure trustworthiness of critical imaging pipelines, we must be able to detect adversarial behavior within preprocessing models. In this paper, we propose a spectral-based detection method for identifying adversarial attacks embedded in super-resolution model weights. More specifically, we use the radially-averaged power spectral density as a discriminative feature to train an extreme gradient boosting (XGBoost) detector, demonstrating detectability of model-level threats in super-resolution networks. We further benchmark our detector against magnitude- and phase-based Fourier spectrum detectors, evaluating each method across a range of training and cross-architecture scenarios. Our proposed detector out-performs the comparison detectors in most of these scenarios and indicates that high-frequency features are most informative for detecting AdvSR attacks across SR architectures.

Emma J. Reid, Haley Duba-Sullivan, Tony G. Allen

Human Analysis and Biometrics Oak Ridge Laboratory Oak Ridge, TN, USA · Radar and Computational Imaging Oak Ridge Laboratory Oak Ridge, TN, USA

Adversarial Example Detection Image Super-Resolution

Sep 28, 2026cs.CV

Backdoor as Probe: Test-Time Adversarial Defense for CLIP

Test-time adversarial defense improves the robustness of vision-language foundation models such as CLIP without retraining. However, adversarial activation shifts are typically treated as distortions to suppress, rather than signals to exploit. We turn these shifts into defense signals by repurposing the trigger-to-target mechanism of backdoors. The key is to implant a defender-controlled backdoor as a probe that is weakly activated by clean inputs but strongly activated by adversarial shifts. Based on this insight, we propose \emph{Backdoor as Probe} (BaP), a test-time adversarial defense for CLIP. BaP constructs the probe through a closed-form model edit to a selected MLP layer. It projects the average adversarial activation shift and a defender-specified semantic direction onto the layer's low-energy input and output activation subspaces to obtain the trigger and target directions, respectively. At inference time, adversarial inputs produce measurable responses along the target direction for detection. BaP then selectively rectifies detected inputs by optimizing a small perturbation that steers their representations away from adversarial shifts and toward the clean subspace. Experiments across 16 benchmarks show that BaP improves average robust accuracy from 1.0% to 52.3% while retaining clean accuracy, achieving performance comparable to state-of-the-art methods with up to a

5.7\times

inference speedup. BaP further shows the generalization to adversarial attacks on large vision-language models. Project page: https://robin-wzq.github.io/Backdoor-as-Probe/

Zhongqi Wang, Jie Zhang, Nie Sen +3

Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences (CAS), Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Xuzhou University of Technology, China

VLM Robustness Backdoor Defense Adversarial Example Detection

Sep 3, 2026cs.CV

Preprocessing Failure and Adversarial Detection in Depthwise-Separable Edge Vision Systems

Preprocessing-based defenses are the standard first-line response to adversarial attacks on edge vision systems, requiring no retraining, no architectural changes, and widely recommended as model-agnostic mitigations. Yet the foundational evaluations of these defenses were conducted on residual or Inception-class architectures, not on the depthwise-separable CNNs that dominate edge deployments. This untested assumption leaves a gap in the security evaluation literature. This paper closes that gap by evaluating six preprocessing defenses against adversarial perturbations across both architecture families. Across all perturbation levels and defenses tested, the two depthwise-separable architectures show consistently poor recovery while the residual architecture shows partial recovery; ablation results are consistent with an architectural rather than parametric explanation, though only three architectures and one attack family are evaluated. Crucially, this failure is not merely a negative result. The same output divergence that disqualifies preprocessing as a recovery mechanism reveals a detection opportunity: preprocessing consistently disrupts clean predictions while leaving adversarial predictions largely unchanged, an asymmetry that is directly measurable without retraining or architectural modification. We further show that standard image quality metrics are unreliable proxies for defense effectiveness, a methodological gap in current evaluation practice. A practitioner decision framework is provided for adversarially resilient edge vision deployment.

Jannatul Masruk Mukta, Rifa Sanjida, Adrita Rahman Tory +2

Bangladesh University of Business and Technology (BUBT), Mirpur-2, Dhaka-1216, Bangladesh · University of New South Wales (UNSW), ACT 2601, Australia

Edge Inference Adversarial Robustness Convolutional Neural Networks Adversarial Example Detection

Aug 10, 2026cs.CV

A Convolutional Layer Activation Dimensionality Reduction for Out-of-Distribution and Adversarial Attack Detection Methods

Despite the success of convolutional neural networks in image classification tasks and their general application in multi-modal models, their susceptibility to out-of-distribution and adversarial attack samples raises concerns regarding trustworthiness and safety. Among the approaches to tackle such issues, detection methods that analyze the model's intermediate activations to estimate a confidence score are a promising family that evaluates the decision process, relying on a dimensionality reduction step to enable efficient downstream processing of the high-dimensional activations. However, when considering convolutional layers, the dimensionality reduction methods in the literature either lack a mechanism to control the compression/information-loss trade-off or yield large representations. In this paper, we carefully analyze two state-of-the-art detection methods and their dimensionality reductions for convolutional layers and develop a novel reduction method with a controllable high-compression level. We extend these two state-of-the-art detection methods, enabling the usage of any dimensionality reduction, and evaluate their performance on out-of-distribution and adversarial attack detection. Results show that the detection methods with the proposed dimensionality reduction consistently perform better than, or comparable to, the strongest alternative. Furthermore, the proposed method is shown to reduce computation and memory footprints, given that it has the highest compression among the compared methods.

Leandro de Souza Rosa, Lorenzo Capelli, Clara Nunes Barrancos +2

Dipartimento di Ingegneria dell’Energia Elettrica e dell’Informazione “Guglielmo Marconi” (DEI) - Alma Mater Studiorum Università di Bologna, Bologna, Italy · Advanced Research Center on Electronic Systems “Ercole De Castro” (ARCES) - Alma Mater Studiorum Università di Bologna, Bologna, Italy

Convolutional Neural Networks Dimensionality Reduction OOD Detection Adversarial Example Detection

Aug 7, 2026cs.CV

Multi-Task Consistency-based Detection of Adversarial Attacks

Deep Neural Networks (DNNs) have found successful deployment in numerous vision perception systems. However, their susceptibility to adversarial attacks has prompted concerns regarding their practical applications, specifically in the context of autonomous driving. Existing defenses often suffer from cost inefficiency, rendering their deployment impractical for resource-constrained applications. In this work, we propose an efficient and effective adversarial attack detection scheme leveraging the multi-task perception within a complex vision system. Adversarial perturbations are detected by the inconsistencies between the inference outputs of multiple vision tasks, e.g., object detection and instance segmentation. To this end, we developed a consistency score metric to measure the inconsistency between vision tasks. Next, we designed an approach to select the best model pairs for detecting inconsistencies effectively. Finally, we evaluated our defense against PGD attacks across multiple vision models on the BDD100k validation dataset. The experimental results demonstrated that our defense achieved a ROC-AUC performance of 99.9% detection within the considered attacker model.

Cong Chen, Jean-Philippe Monteuuis, Jonathan Petit

Qualcomm Technologies, Inc. San Diego, CA 92121, USA

Adversarial Example Detection

Jul 30, 2026cs.CR

Revisiting the Adversarial Robustness of Graph-Based Traffic Forecasting

Traffic forecasting by graph-based AI is a critical component of intelligent transportation systems, motivating security research on robustness to malicious sensor readings. We argue that prior robustness evaluations are largely shaped by unrealistic threat models and untargeted objectives, so both attacks and defenses must be revisited. We study a practical adversary with limited model knowledge and the ability to monitor and manipulate only a few road sensors. More importantly, practical attacks can be localized to specific links or routes, causing incorrect estimated arrival times or unnecessary rerouting while leaving the broader network largely unaffected. This targeted setting remains underexplored, and defenses such as adversarial training do not transfer well from the norm-bounded attacks they train on to structurally different, physics-aware attacks that mimic genuine congestion. We therefore reframe robustness as a detection problem, introducing a learned physics-informed detector whose output is fed to a hardened forecaster as an input feature and trained against adaptive attacks with the forecaster fixed. We evaluate across a variety of model architectures and benchmarks. The physics-aware attack multiplies target-link error several-fold while the network-wide error barely moves, and adversarial training, tuned to norm-bounded perturbations, barely dents it. Our detection--mitigation defense improves even on adversarial training hardened against the physics-aware attack itself, on

13

of

15

model--dataset settings and by the widest margin on a held-out attack, at near-zero clean cost. The results emphasize the need to examine abstracted AI adversarial attacks under application-specific constraints to assess their true security impacts.

Qingzhao Zhang

The University of Arizoan

Adversarial Attacks Adversarial Robustness Adversarial Example Detection Spatiotemporal GNNs Physical Adversarial Attacks

Jul 28, 2026cs.CV

Adversarial Deepfake Generation and an Investigation of Purification-Based Adversarial Detection

This paper describes the participation of team "Go To Germany" in the ImageCLEF 2026 Deepfake Detection and Generation Task. For the image generation task, we employ FLUX.1-dev with PuLID for identity-preserving face synthesis, combined with a multi-model PGD adversarial attack targeting 12 detectors simultaneously (DiffJPEG-in-loop, MI/DI/EoT, adaptive weighting, two-stage warm-start). Our approach achieved 90% evasion against organizer detectors and 57.6% against participant detectors, with a final generation score of 0.4170. For the image detection task, we combine two complementary detectors - SigLIP+DINOv2 for AI-generated images and GenD-DINOv3 for face manipulations - in a max-probability ensemble, achieving 99.4% accuracy on baseline deepfakes but suffering from high false-positive rates on real images, resulting in a final detection score of 0.6986. Beyond the official submission, we conducted a self-initiated investigation of purification-based adversarial detection, comparing three families of detection signals across six detectors that share a CLIP ViT-L/14 backbone. We find that raw

|Δ\text{logit}|

under median-3 purification, applied through the EFFORT detector, separates adversarial inputs from clean inputs with AUROC 0.81-0.98 across four adversarial source types - a finding that refutes the simple backbone-preservation hypothesis and exposes a sharp JPEG-quality cliff at Q70 where the signal collapses.

Junghyun Kim, Seunghyun Kim, Jiyoung Woo

Department of AI and Big Data Engineering, Soonchunhyang University, Asan, South Korea

Deepfake Detection AI-Generated Image Detection Adversarial Example Detection

Jul 16, 2026cs.CV

GeoDetect: Geometric Adversarial Detection for VLPs

Vision-language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to adversarial attacks. Although adversarial detection methods have demonstrated success in single-modality settings (either vision or language), their effectiveness and reliability in multimodal models such as VLPs remain largely unexplored. In this work, we study the geometry of VLP embedding spaces and observe structured anisotropy that differs from unimodal vision models. Our theoretical analysis shows that under this anisotropic structure, adversarial attacks increase the expected geometric separation between clean and adversarial examples (AEs). Specifically, we demonstrate that AEs consistently exhibit greater expected distances to randomly sampled points than their clean counterparts, indicating that AEs tend to push representations out of manifold regions. Building on these insights, we propose GeoDetect, which leverages these off-manifold deviations via geometric scores to identify AEs. Through comprehensive evaluations, we show that our approach reliably detects AEs across diverse VLP architectures and threat settings, covering unimodal and multimodal attacks as well as adaptive attacks, thereby providing a robust and practical approach to improving the safety and reliability of these models.

Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie +2

The University of Melbourne, Melbourne, Australia · Monash University, Melbourne, Australia

VLM Robustness Adversarial Attacks on VLMs Vision-Language Models Adversarial Example Detection

Jul 8, 2026stat.ML

A Unified Detection Framework for AI-Related Content and Artifacts

Artificial intelligence (AI) is a double-edged sword: while it has achieved remarkable success across a wide range of domains, its deployment also calls for effective oversight and regulation, for which the detection of AI-related content and artifacts is perhaps the most direct and cost-effective approach. To this end, we propose a unified detection framework based on Mahalanobis distance scores (MDS), applicable to several important settings, including the detection of large language model (LLM) generated text, hallucination, watermark, and adversarial examples. A key component of the proposed method is to accurately characterize the positive class--such as human-generated text, factual statements, unwatermarked text, or non-adversarial samples--which requires an efficient and robust estimator of the covariance matrix of deep representations of positive samples before computing the MDS. Since the positive samples typically consist of multiple classes, and these classes may exhibit both homogeneity and heterogeneity, we develop joint estimation methods for both the casewise and cellwise minimum covariance determinant (MCD) estimators. We provide efficient optimization algorithms for both estimators and prove their convergence. We provide a reasonable definition of the breakdown point for the joint estimators and prove their corresponding high breakdown point properties. Empirical evaluations confirm the effectiveness of the proposed detection framework.

Xifeng Zhang, Tao Hu, Yijie Peng +1

School of Mathematical Sciences, Capital Normal University, Beijing 100048, China · School of Management and Engineering, Nanjing University, Nanjing 210008, China · Advanced Institute of Information Technology, Peking University +1

AI-Generated Text Detection LLM Hallucination Detection Adversarial Example Detection

Jun 29, 2026cs.CV

A Classifier-Agnostic Zero-Shot Adversarial Attack Detection via CLIP

Adversarial attacks pose a challenge to the reliability of deep learning models, motivating effective detection methods. Existing techniques often rely on attack-specific assumptions, access to adversarial samples, or knowledge of the underlying classifier (white-box). We propose

A^4D

Attack- and Architecture-Agnostic Adversarial Detector, a completely black-box, zero-shot adversarial attack detection framework that utilizes prompt-based similarity scores derived from CLIP. To the best of our knowledge this is the first attempt to utilize CLIP for such a task. The method is based on two key observations: (i) CLIP is sensitive even to small imperceptible non-semantic perturbations; (ii) The shift in CLIP embedding space is not arbitrary and can be used as a robust attack indicator. Experiments across multiple attacks, datasets and classifiers validate that

A^4D

achieves SOTA detection results in the attack-agnostic and classifier-agnostic setting.

Hodaya Krakover, Meir Yossef Levi, Eyal Gofer +1

Technion - Israel Institute of Technology, Haifa, Israel

Adversarial Example Detection

Jun 26, 2026cs.LG

USAD: Uncertainty-aware Statistical Adversarial Detection

Statistical adversarial detection (SAD) treats detection as a two-sample test. Given a reference set of clean examples (CEs) and a batch of queries, potentially containing an unknown mixture of CEs and adversarial examples (AEs), SAD decides whether the query distribution drifts away from the CE distribution while controlling the false-alarm rate. Existing SAD-based methods mainly use maximum mean discrepancy (MMD) to measure the distributional discrepancy. However, MMD's distributional properties limit its ability to capture characteristic uncertainty patterns of AEs that are crucial for detection: AEs typically exhibit abnormal feature spread (i.e., global uncertainty) and instability under perturbations (i.e., local uncertainty). To close the gap, we propose Uncertainty-aware Statistical Adversarial Detection (USAD), which explicitly captures these uncertainty patterns with two new statistics: (1) Variance Discrepancy (VD), which measures the difference in feature spread between AEs and CEs to capture global uncertainty differences. (2) Perturbation-based Covariance Discrepancy (PCD), which compares feature covariance under Gaussian perturbations to capture local uncertainty differences. By aggregating VD and PCD, USAD achieves superior detection performances over baseline methods against various adversarial attacks, highlighting the importance of considering characteristic behaviors of AEs for effective SAD. Our code is available at: https://anonymous.4open.science/r/USAD.

Zhijian Zhou, Xunye Tian, Jiacheng Zhang +5

The University of Melbourne

Two-Sample Testing Adversarial Example Detection

Jun 26, 2026cs.LG

What Was That Again? Certified Robustness for Automatic Speech Recognition

Automatic Speech Recognition systems are notoriously both sensitive to adversarial and benign perturbations. While this has been repeatedly demonstrated using reference datasets, detecting such behaviors in deployed systems is incredibly challenging, due to the absence of oracle knowledge of the true transcription. We demonstrate that employing a certification-inspired mechanism can significantly decrease WER, increase recall, and decrease the Spearman correlation between confidence and WER. We achieve this through a dual-gate diagnostic pipeline: a Two-Sided Atomic Audit that accumulates statistical wealth to certify both token existence and adversarial exclusion, and a Rank-Based Tournament that selects the winning sequence. Our evaluations across four diverse architectures demonstrate up to a 55% relative reduction in Word Error Rate, while also providing granular word- and sentence-level certifications to enhance acoustic security.

Andrew C. Cullen, Neil G. Marchant, Jiani Xie +2

University of Melbourne · DST Group, Adelaide

ASR Evaluation Neural Network Robustness Automatic Speech Recognition Certified Robustness Adversarial Example Detection

Jun 25, 2026cs.CV

AEGIS: A Semantic GAN and Evidential Learning Frameworkfor Robust Adversarial Detection in Vision Sensors

Deep neural networks (DNNs) have shown outstanding performance in visual recognition tasks within vision sensor networks; however, they are still vulnerable to adversarial manipulations and imperceptible perturbations that can lead to erroneous predictions. To address that, this paper presents AEGIS, a semantic aware and uncertainty guided adversarial detection framework designed for robust image classification in vision sensors pipelines. At its core, a SemantiGAN module functions as a multi class semantic discriminator, identifying and filtering visually inconsistent adversarial inputs before they propagate further in the pipeline. For inputs that pass this stage, a stochastic augmentation process generates test time variations, from which handcrafted instability metrics FlipScore, Prediction Inconsistency, Layerwise Cosine Similarity (early and mid layers), and Entropy are computed. These features are aggregated into a compact five dimensional vector and processed by an Evidential Deep Learning (EDL) classifier, which models output evidence using a Dirichlet distribution to yield both class predictions and calibrated uncertainty estimates. Evaluations on the Tiny ImageNet dataset across six categories clean, FGSM, PGD, patch based, functional, and geometric attacks demonstrate the effectiveness of AEGIS. The proposed framework achieves an AUROC of 92.1%, an AUPRC of 90.2%, and an accuracy of 90.7%, outperforming conventional softmax-based detectors in terms of detection performance, robustness, interpretability, and uncertainty calibration.

Maher Boughdiri, Mounira Msahli, Albert Bifet

LTCI, Département Informatique et Réseaux, Télécom Paris, Institut Polytechnique de Paris, France

Adversarial Example Detection Image Classification

Jun 22, 2026cs.CR

DE-FIVE: Detecting Malicious Image Prompts via Fourier Features and Image Vector Embeddings

Vision language models (VLMs) employ both visual and textual modalities to enable advanced vision-language inference. However, incorporating visual modalities expands the attack surface of VLMs, making them more susceptible to security threats such as adversarial perturbations and indirect prompt injection, wherein crafted malicious image prompts can elicit unintended model outputs. Existing defense methods against malicious image prompts remain insufficient as they typically demand extensive datasets for retraining or the deployment of additional, complex classifiers. Most critically, there is a profound lack of specialized defense mechanisms specifically targeting indirect prompt injections, a gap that serves as a primary motivation for this work. To address these limitations, we introduce DE-FIVE, a novel training-free framework for detecting malicious image prompts by leveraging Fourier features and the hidden state representations of the visual encoder (image vector embeddings) across perturbations. Specifically, we develop a hybrid detection strategy consisting of a black-box detector that operates on Fourier-domain features and a white-box detector that exploits image vector embeddings derived from only a few-shot malicious set. Extensive experiments demonstrate that the proposed framework consistently outperforms state-of-the-art baselines against malicious image prompts.

Xingwei Zhong, Varun Sharma, Kar Wai Fok +1

VLM Robustness Indirect Prompt Injection Adversarial Example Detection Prompt Injection Detection

Jun 10, 2026cs.LG

Online Shift Detection and Conformal Adaptation for Deployed Safety Classifiers

Safety classifiers deployed in production operate under a stationarity assumption that fails silently: when input distributions drift, accuracy degrades with no error signal until ground-truth labels arrive. We present an online monitor that detects distributional shift in classifier scores via a sliding-window KS statistic with empirically calibrated alarm thresholds. In a pre-registered factorial evaluation (4 classifiers

\times

5 shift conditions

\times

20 seeds

\times

2 window sizes; 800 cells), the monitor achieves 86.6% valid detection (mean latency 39.5 steps) across synthetic-onset, real-jailbreak, and adversarial regimes; a classifier

\times

shift interaction (

η^2 = 0.185

) shows that monitoring must be tuned per classifier. Attempting to recover post-detection coverage via weighted conformal prediction exposes a failure mode: density-ratio estimation collapses for generative classifiers because logistic regression separates source from target perfectly in 3584-4096-dimensional embedding space, clipping all importance weights to zero; projecting to

\leq 32

dimensions restores coverage. We then extend the framework to gradient-based evasion and give the first threat-model characterisation of score-disagreement monitoring as a canary. We falsify three assumptions: that architectural diversity drives the signal (false,

η^2 = 0.011

), that it is generic out-of-distribution detection (false, GCG-specific,

p < 10^{-12}

), and that an adaptive attacker can suppress it (false while the canary is confident). We derive the exact security boundary, a confidence-gated equilibrium at which a monitor-aware attacker stalls at gap

= 1/(2λ)

, and provide a calibration-free scan martingale achieving false-alarm rate

\leq 1\%

across all classifiers with no per-model tuning.

Jun Wen Leong

Distribution Shift Conformal Prediction Distribution Shift Detection AI Safety Evaluation Change-Point Detection Adversarial Example Detection

Jun 8, 2026cs.CV

Adversarial Attack and Disturbance Detection by Hadamard-Coded Output Representations for Object Detection and Semantic Segmentation

Conventional one-hot encodings often yield poorly calibrated models, being overconfident under attack, and letting entropy-based detection algorithms fail. Previous image classification works have demonstrated that Hadamard-coded output representations can improve adversarial robustness. However, attempts to integrate Hadamard codes into semantic segmentation fall far behind state-of-the-art models in mean intersection-over-union performance. Regarding object detection, such output encodings have not yet been investigated at all. Further, no prior art addressed intrinsic codeword inconsistencies or actually exploited intrinsic codeword redundancy. Accordingly, we first derive a novel decoding procedure for Hadamard codewords towards optimal class-wise probabilities, solving the underlying optimization problem by using the projection onto the probability simplex. Second, our optimization delivers a measure of prediction inconsistency. Third, we are the first to show how to exploit these inconsistencies for adversarial attack and disturbance detection. Fourth, we introduce HadamardNet, a framework employing Hadamard codes as output representations for semantic segmentation and object detection models and tasks. We conduct a comprehensive evaluation both on disturbances and adversarial attacks, achieving state-of-the-art perturbation detection performance for both tasks in only a single detection pass, while delivering equivalent or close-by reference performance on clean data. Code is available at https://github.com/ifnspaml/HadamardPerturbationDetection.

Lucas Görnhardt, Timo Bartels, Niklas Schwarz +1

Technische Universität Braunschweig, Germany

Adversarial Perturbations Object Detection Adversarial Example Detection Semantic Segmentation

Jun 8, 2026cs.CV

Securing Self-supervised Data Curation for Foundation Models Robustness

Self-supervised data curation provides a pathway to scaling and improving the generalization capabilities of machine learning models. By leveraging self-supervised learning (SSL) for data curation, the demand for massive training datasets required by foundation models can be effectively met. SSL greatly alleviates the costs associated with annotation and manual dataset curation while minimizing the need for human oversight. However, the integrity of SSL-curated datasets must be rigorously checked, as reliance on anonymous and unvetted external sources can substantially increase the risk of data poisoning. In this paper, we propose a Poisoned Data Detector (PDD), an active defense mechanism designed to ensure the integrity of SSL-curated datasets prior to foundation model training. PDDs are designed using a combination of the pretrained ImageBind model and traditional classifiers, including Random Forest (RF), k-Nearest Neighbors (KNN), Naive Bayes (NB), and Support Vector Machines (SVM). We rigorously evaluated PDDs using 176,200 images from three diverse datasets and three different adversarial attacks encompassing both in-distribution and out-of-distribution scenarios. Notably, SVM-PDD achieves superior performance for both in-distribution (Set3-Set5) and out-of-distribution (TrueFace and 140K RealFace) datasets. Our design demonstrates strong scalability and enables the rapid integration of new adversarial attack detectors through an ensemble approach.

Sandeep Gupta, Roberto Passerone

Centre for Secure Information Technologies (CSIT), Queen’s University Belfast, UK. · Department of Information Engineering and Computer Science, University of Trento, Italy.

Training Data Curation Adversarial Example Detection Data Poisoning Attacks

May 21, 2026cs.LG

SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimization

LLM-based ranking systems are vulnerable to Generative Engine Optimization (GEO) attacks, where adversaries inject semantic signals into product descriptions to artificially boost rankings. We propose SCI-Defense, a three-component defense framework combining Perplexity detection (PPL), Semantic Integrity Scoring (SIS), and Inter-Candidate Detection (ICD). SIS evaluates four manipulation dimensions: Authority Attribution (AA), Narrative Purposiveness (NP), Comparative Claims (CA), and Temporal Claims (TC). Evaluated on 600 Amazon product descriptions across 6 categories, SCI-Defense achieves Precision=1.000 and FPR=0.000, with Recall of 1.000, 0.952, and 0.830 against String, Reasoning, and Review attacks respectively. On 600 MS MARCO web passages, String attacks are blocked with perfect recall while Review attacks yield near-zero recall, as web passages lack the persuasion-oriented signals that SIS targets in product descriptions. We demonstrate that existing defenses -- PPL-only filters, SafetyClf content classifiers, and paraphrasing -- achieve zero recall against semantic manipulation attacks. We further demonstrate new attacks such as Specification Amplification and Use-Case Saturation can expose semantic relevance manipulation as a structural defense blind spot that suggests directions for future research.

Xucheng Yu, Haibo Jin, Huimin Zeng +1

Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign · School of Information Sciences, University of Illinois Urbana-Champaign · Amazon

LLM Reranking Adversarial Example Detection Adversarial Text Attacks

May 19, 2026cs.LG

Detecting Fluent Optimization-Based Adversarial Prompts via Sequential Entropy Changes

Optimization-based adversarial suffixes can jailbreak aligned large language models (LLMs) while remaining fluent, weakening static and windowed perplexity-based detectors. We cast adversarial suffix detection as an online change-point detection problem over the token-level next-token entropy stream. Using the LLM system prompt to estimate a robust baseline, we standardize user-token entropies and apply a one-sided CUSUM statistic. The resulting detector, CPD Online (CPD), is model-agnostic, training-free, runs online, and localizes the adversarial suffix onset. On a benchmark of 1,012 optimization-based suffix attacks (GCG, AutoDAN, AdvPrompter, BEAST, AutoDAN-HGA) and 1,012 perplexity-controlled benign prompts, CPD improves F1 over the strongest windowed-perplexity baseline on all six open-weight chat models (LLaMA-2-7B/13B, Vicuna-7B/13B, Qwen2.5-7B/14B). On LLaMA-2-7B at the canonical CUSUM setting (

k=0

), CPD reaches AUROC

0.88

and F1

0.82

. Beyond prompt-level detection, CPD concentrates 79.6% of its triggers inside the adversarial suffix, versus 17-46% for windowed perplexity. Finally, when used as a lightweight gate for LLaMA Guard, CPD reduces guard calls by 17-22% on a high-volume, benign-dominated deployment while preserving guard-level detection quality

Mohammed Alshaalan, Miguel R. D. Rodrigues

Department of Electronic and Electrical Engineering, University College London, London, United Kingdom.

Adversarial Attacks on LLMs Jailbreak Detection Change-Point Detection Adversarial Example Detection

May 14, 2026cs.CR

Enabling Adversarial Robustness in AI Models through Kubeflow MLOps

AI models are increasingly deployed in cloud-native environments to support scalable and automated services. However, while platforms such as Kubernetes provide strong infrastructure orchestration, security mechanisms specifically designed to protect deployed AI models remain limited. This paper presents security measures for AI models deployed in Kubernetes clusters. The proposed architecture integrates Kubeflow-based MLOps to automatically detect adversarial attacks during the inference phase and trigger defense mechanisms that preserve the model's accuracy and reliability. Specifically, a Fast Gradient Sign Method (FGSM) attack is applied at inference time, and a Projected Gradient Descent (PGD)-based adversarial training defense is automatically deployed when a degradation in accuracy is detected. The experimental results indicate that the deployed defense robustifies the model, significantly recovering accuracy relative to the degradation caused by the attack.

Stavros Bouras, Ioannis Korontanis, Antonios Makris +1

School of Electrical and Computer Engineering, National Technical University of Athens, Greece · Department of Informatics and Telematics, Harokopio University of Athens, Greece

Adversarial Attacks Adversarial Robustness Adversarial Example Detection

May 12, 2026cs.CV

A Mimetic Detector for Adversarial Image Perturbations

Adversarial attacks fool deep image classifiers by adding tiny, almost invisible noise patterns to a clean image. The standard

\ell^\infty

-bounded attacks (FGSM and PGD) produce high-frequency, near-random sign patterns at the pixel level: small in

\ell^2

, but carrying disproportionate gradient energy. We exploit this with a single-shot, training-free detector using the high-order Corbino-Castillo mimetic operators from the open-source MOLE library. No retraining, no surrogate classifier, no access to the network under attack: the verdict is a property of the input alone, computed in

O(HW)

time. We illustrate the detector on the standard "peppers" test image: untargeted FGSM and PGD attacks at the

\ell^\infty

budget

\varepsilon = 16/255

flip SqueezeNet's prediction from "bell pepper" to "doormat" (FGSM) and "maraca" (PGD), and the detector separates these adversarial inputs from the clean image by

4.1\times

-

5.0\times

(FGSM) and

1.9\times

-

2.2\times

(PGD). The margin grows monotonically with the operator order

k

, while an equal-amplitude smooth perturbation leaves the statistic within 1% of its clean value.

Johnny Corbino

Lawrence Berkeley National Laboratory · Berkeley, CA, USA

Adversarial Examples Adversarial Perturbations Adversarial Example Detection

May 8, 2026cs.CV

Sparse Autoencoders as Plug-and-Play Firewalls for Adversarial Attack Detection in VLMs

Vision-language models (VLMs) have advanced rapidly and are increasingly deployed in real-world applications, especially with the rise of agent-based systems. However, their safety has received relatively limited attention. Even the latest proprietary and open-weight VLMs remain highly vulnerable to adversarial attacks, leaving downstream applications exposed to significant risks. In this work, we propose a novel and lightweight adversarial attack detection framework based on sparse autoencoders (SAEs), termed SAEgis. By inserting an SAE module into a pretrained VLM and training it with standard reconstruction objectives, we find that the learned sparse latent features naturally capture attack-relevant signals. These features enable reliable classification of whether an input image has been adversarially perturbed, even for previously unseen samples. Extensive experiments show that SAEgis achieves strong performance across in-domain, cross-domain, and cross-attack settings, with particularly large improvements in cross-domain generalization compared to existing baselines. In addition, combining signals from multiple layers further improves robustness and stability. To the best of our knowledge, this is the first work to explore SAE as a plug-and-play mechanism for adversarial attack detection in VLMs. Our method requires no additional adversarial training, introduces minimal overhead, and provides a practical approach for improving the safety of real-world VLM systems.

Hao Wang, Yiqun Sun, Pengfei Wei +2

2Waseda University · 1Magellan Technology Research Institute (MTRI)

VLM Robustness Adversarial Attacks on VLMs Sparse Autoencoders Adversarial Example Detection

May 4, 2026cs.LG

Detecting Adversarial Data via Provable Adversarial Noise Amplification

The nonuniform and growing impact of adversarial noise across the layers of deep neural networks has been used in the literature, without a formal mathematical justification, to detect adversarial inputs and improve robustness. In this work, we study this phenomenon in detail and present a formal adversarial noise amplification theorem. We specify a set of sufficient conditions under which the adversarial noise amplification is mathematically guaranteed. Based on theoretical observations, we propose a novel training methodology with a custom spectral loss function and a specific architectural design to enhance the amplification signal for detecting adversarial data. Finally, we introduce a new, lightweight detection mechanism that leverages the enhanced amplification signal and operates entirely at inference time. To validate our approach, we demonstrate the detector's efficacy against both state-of-the-art attacks and a purpose-built adaptive attack, confirming that enhanced amplification can serve as a robust and reliable signal for adversarial defense.

Furkan Mumcu, Yasin Yilmaz

University of South Florida

Spectral Regularization Adversarial Robustness Adversarial Example Detection

Apr 30, 2026quant-ph

Defending Quantum Classifiers against Adversarial Perturbations through Quantum Autoencoders

Machine learning models can learn from data samples to carry out various tasks efficiently. When data samples are adversarially manipulated, such as by insertion of carefully crafted noise, it can cause the model to make mistakes. Quantum machine learning models are also vulnerable to such adversarial attacks, especially in image classification using variational quantum classifiers. While there are promising defenses against these adversarial perturbations, such as training with adversarial samples, they face practical limitations. For example, they are not applicable in scenarios where training with adversarial samples is either not possible or can overfit the models on one type of attack. In this paper, we propose an adversarial training-free defense framework that utilizes a quantum autoencoder to purify the adversarial samples through reconstruction. Moreover, our defense framework provides a confidence metric to identify potentially adversarial samples that cannot be purified the quantum autoencoder. Extensive evaluation demonstrates that our defense framework can significantly outperform state-of-the-art in prediction accuracy (up to 68%) under adversarial attacks.

Emma Andrews, Sahan Sanjaya, Prabhat Mishra

University of Florida, Gainesville, Florida, USA

Adversarial Robustness Quantum Machine Learning Adversarial Example Detection

Apr 30, 2026cs.CR

Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection

Multi-turn prompt injection follows a known attack path -- trust-building, pivoting, escalation but text-level defenses miss covert attacks where individual turns appear benign. We show this attack path leaves an activation-level signature in the model's residual stream: each phase shift moves the activation, producing a total path length far exceeding benign conversations. We call this adversarial restlessness. Five scalar trajectory features capturing this signal lift conversation-level detection from 76.2% to 93.8% on synthetic held-out data. The signal replicates across four model families (24B-70B); probes are model-specific and do not transfer across architectures. Generalization is source-dependent: leave-one-source-out evaluation shows each of synthetic, LMSYS-Chat-1M, and SafeDialBench captures distinct attack distributions, with detection on real-world LMSYS reaching 47-71% when its distribution is represented in training. Combined three-source training achieves 89.4% detection at 2.4% false positive rate on a held-out mixed set. We further show that three-phase turn-level labels(benign/pivoting/adversarial) unique to our synthetic dataset are essential: binary conversation-level labels produce 50-59% false positives. These results establish adversarial restlessness as a reliable activation-level signal and characterize the data requirements for practical deployment.

Prashant Kulkarni

Mountain View, CA

Prompt Injection Attacks on LLMs Adversarial Example Detection Prompt Injection Detection Language Model Auditing

Apr 29, 2026cs.CV

The Unseen Adversaries: Robust and Generalized Defense Against Adversarial Patches

The vulnerabilities of deep neural networks against singularities have raised serious concerns regarding their deployment in the physical world. One of the most prominent and impactful physical-world adversarial perturbations is the attachment of patches to clean images, known as an adversarial patch attack. Similarly, natural noises such as Gaussian and Salt&Pepper are highly prevalent in the real world. The current research need arises from the above vulnerabilities and the lack of efforts to tackle these two singularities independently and, especially, in combination. In this research, we have, for the first time, combined these two prominent singularities and proposed a novel dataset. Using this dataset, we have conducted a benchmark study of singularity data-point detection using features from several convolutional neural networks. For classification, rather than the popular neural network-based parameter tuning, we have used traditional yet effective machine learning classifiers. The extensive experiments across various in- and out-of-distribution (OOD) singularities reveal several interesting findings about the effectiveness of classifiers and show that it is hard to defend against adversaries when they are treated independently, and inefficient classifiers are selected.

Vishesh Kumar, Akshay Agarwal

Trustworthy BiometraVision Lab, Indian Institute of Science Education and Research Bhopal, India

Adversarial Patch Attacks OOD Detection Adversarial Example Detection Physical Adversarial Attacks

Apr 21, 2026cs.LG

Mechanistic Anomaly Detection via Functional Attribution

We can often verify the correctness of neural network outputs using ground truth labels, but we cannot reliably determine whether the output was produced by normal or anomalous internal mechanisms. Mechanistic anomaly detection (MAD) aims to flag these cases, but existing methods either depend on latent space analysis, which is vulnerable to obfuscation, or are specific to particular architectures and modalities. We reframe MAD as a functional attribution problem: asking to what extent samples from a trusted set can explain the model's output, where attribution failure signals anomalous behavior. We operationalize this using influence functions, measuring functional coupling between test samples and a small reference set via parameter-space sampling. We evaluate across multiple anomaly types and modalities. For backdoors in vision models, our method achieves state-of-the-art detection on BackdoorBench, with an average Defense Effectiveness Rating (DER) of 0.93 across seven attacks and four datasets (next best 0.83). For LLMs, we similarly achieve a significant improvement over baselines for several backdoor types, including on explicitly obfuscated models. Beyond backdoors, our method can detect adversarial and out-of-distribution samples, and distinguishes multiple anomalous mechanisms within a single model. Our results establish functional attribution as an effective, modality-agnostic tool for detecting anomalous behavior in deployed models.

Hugo Lyons Keenan, Christopher Leckie, Sarah Erfani

School of Computing and Information Systems, The University of Melbourne, Victoria, Australia.

Adversarial Example Detection Anomaly Detection

Nov 10, 2025cs.CV

FoCLIP: A Feature-Space Misalignment Framework for CLIP-Based Image Manipulation and Detection

The well-aligned attribute of CLIP-based models enables its effective application like CLIPscore as a widely adopted image quality assessment metric. However, such a CLIP-based metric is vulnerable for its delicate multimodal alignment. In this work, we propose FoCLIP, a feature-space misalignment framework for fooling CLIP-based image quality metric. Based on the stochastic gradient descent technique, FoCLIP integrates three key components to construct fooling examples: feature alignment as the core module to reduce image-text modality gaps, the score distribution balance module and pixel-guard regularization, which collectively optimize multimodal output equilibrium between CLIPscore performance and image quality. Such a design can be engineered to maximize the CLIPscore predictions across diverse input prompts, despite exhibiting either visual unrecognizability or semantic incongruence with the corresponding adversarial prompts from human perceptual perspectives. Experiments on ten artistic masterpiece prompts and ImageNet subsets demonstrate that optimized images can achieve significant improvement in CLIPscore while preserving high visual fidelity. In addition, we found that grayscale conversion induces significant feature degradation in fooling images, exhibiting noticeable CLIPscore reduction while preserving statistical consistency with original images. Inspired by this phenomenon, we propose a color channel sensitivity-driven tampering detection mechanism that achieves 91% accuracy on standard benchmarks. In conclusion, this work establishes a practical pathway for feature misalignment in CLIP-based multimodal systems and the corresponding defense method.

Yulin Chen, Zeyuan Wang, Tianyuan Yu +2

Laboratory for Big Data and Decision, National University of Defense Technology, Changsha 410073, China

VLM Robustness Adversarial Attacks on VLMs Cross-Modal Alignment Image Quality Assessment Adversarial Example Detection

May 19, 2025cs.LG

A Few Large Shifts: Layer-Inconsistency Based Minimal Overhead Adversarial Example Detection

Deep neural networks (DNNs) are highly susceptible to adversarial examples---small, malicious perturbations that can cause incorrect predictions. We introduce a lightweight, plug-in detector that uses internal layer-wise inconsistencies within the target model and requires only benign data for fitting and calibration. The approach is motivated by the A Few Large Shifts Assumption, an empirical hypothesis that adversarial perturbations often produce large, localized growth in representation changes across a small number of consecutive layers, connecting adversarial behavior to layer-wise Lipschitz continuity. We develop two complementary scores---Recovery Testing (RT) for intermediate-layer inconsistency and Logit-layer Testing (LT) for augmentation-induced output instability---and fuse them through RLT. Across CIFAR-10, CIFAR-100, and ImageNet, RLT achieves strong detection performance under standard attacks with substantially lower overhead than detector families requiring external encoders or reference-set retrieval. We further study its behavior under adaptive attacks, at low false-positive operating points, and under benign distribution shifts. The code is available here: https://github.com/c0510gy/AFLS-AED.

Sanggeon Yun, Ryozo Masukawa, Hyunwoo Oh +2

Department of Computer Science University of California, Irvine · Whiting School of Engineering Johns Hopkins University

Adversarial Attacks Adversarial Examples Adversarial Example Detection