Black-Box Adversarial Attacks

Overview

Black-box adversarial attacks ask how an attacker can cause an AI system to misclassify, produce unsafe output, or take an unwanted action without access to its internal parameters or gradients. Methods use feedback from model queries—such as labels, scores, or responses—with decision-boundary search, finite differences, bandits, evolutionary or reinforcement-learning optimization; others craft perturbations on surrogate models and rely on transfer to unseen targets. Work spans images and physical patches, text and jailbreak prompts, audio, graphs, malware detectors, and tool-using or multimodal agents, often under constraints such as limited queries, imperceptibility, and preserving input meaning or function. Key tensions include whether attacks transfer across architectures and real-world transformations, how to compare them under fair query budgets, and whether defenses can resist adaptive attacks without sacrificing clean performance or utility.

Latest papers

CardsList

189 papers15 papers in the last four weeks, down 17% on the four weeks before. 0.1% of all new papers.

  1. Cracks in Embeddings: Robustness and Stability of JEPA-Based Chest X-Ray Representations

    Nov 13, 2026Zahid Mansur, Kalpana Yadav, Deepak MishraMedical Image Representation LearningAdversarial Robustness

  2. Binarization as an Attack Surface in Weightless Neural Networks

    Nov 13, 2026Lucas Senos Coutinho, Matheus Vargas, Pedro Teixeira +5Adversarial AttacksAdversarial Robustness

  3. AnyBreak: Towards Jailbreaking Any T2I Model for Any Concept

    Nov 9, 2026Qi Lu, Zehui Guo, Yucheng Zhao +3Adversarial Attacks on VLMsAdversarial Attacks on Diffusion Models

  4. Query-Only Prompt-Agnostic Attacks on Visual In-Context Learning

    Nov 8, 2026Xiao Hu, Muhammad Ibrahim, Hao Wen +3Black-Box Adversarial AttacksIn-Context Learning

  5. RL-ATTACK: A Generalized Black-Box Adversarial Attack on Language Models via Reinforcement Learning

    Nov 5, 2026Taikun Chen, Yue Zhang, Wei Qian +1Adversarial Attacks on LLMsBlack-Box Adversarial Attacks

  6. RLGPA: A Reinforcement Learning-Guided Granularity Planning Attack for Decision-Based Sparse Adversarial Examples

    Nov 5, 2026Chao Zhou, Shenghua He, Jiaxin Mo +2Adversarial AttacksBlack-Box Adversarial Attacks

  7. Procedural Adversarial Learning

    Nov 4, 2026Jack Nugent, Jia DengAdversarial RobustnessAdversarial Training

  8. ScrambleMark: Zero-Query Watermark-Agnostic Black-Box Evasion of Modern Speech Watermarking

    Nov 3, 2026Binhao Ma, Weikang Ding, Hanqing Guo +5Audio WatermarkingBlack-Box Adversarial Attacks

  9. JevBreak: Your Model, Their Decision

    Nov 2, 2026Nitin Aravind Birur, Divyanshu Kumar, Sahil Agarwal +1Adversarial AttacksAdversarial Text Attacks

  10. Black-Box Adversarial Example Attacks on Machine Learning based Election Systems

    Nov 1, 2026Subek Acharya, Syed Ayon, Khaleque Md Aashiq Kamal +3Adversarial AttacksBlack-Box Adversarial Attacks

  11. FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models

    Oct 28, 2026Xiaogeng Liu, Xinyan Wang, Chaowei XiaoTyped Decision ModelsAdversarial Attacks

  12. STROKE: Simple Sparse Structural Occlusions for Black-Box Adversarial Attacks

    Oct 23, 2026Ankit Gupta, Akshat Pandya, Paul Lin +2Black-Box Adversarial AttacksAdversarial Patch Attacks

  13. MIG-COW: Transferable Adversarial Attacks on Deepfake Detectors via Gradient Decomposition

    Oct 22, 2026Wonjune Seo, Joonhyuk Baek, Yeseong Jung +1Adversarial AttacksTransfer-Based Adversarial Attacks

  14. Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning

    Oct 20, 2026Yuqi Jia, Hangsheng Zhang, Chenxi Liu +2LLM Red TeamingBlack-Box Adversarial Attacks

  15. A Valid Watermark Is Not a Valid Identity: Spatial-Spectral Adversarial Face Evasion with Watermark Preservation

    Oct 14, 2026Jefferson David Rodriguez Chivata, Davide Ghiani, Simone Maurizio La Cava +3Adversarial AttacksBlack-Box Adversarial Attacks

  16. Mitigating Black-Box Score-Based Query Attacks via Gradient Confusion

    Oct 12, 2026Ali Javeri, Javad Hezareh, Radin Shahdaei +1Adversarial RobustnessAdversarial Attacks

  17. SSM-Attack: Learning Where to Perturb for Black-Box Face Privacy

    Oct 12, 2026Tunahan Parlayıcı, Erchan Aptoula, Yücel SaygınAdversarial PerturbationsAdversarial Attacks

  18. ATTCK-R: Knowledge-Aware Long-Horizon Jailbreaking for Protected LLMs

    Oct 12, 2026Md Mushfiqur Rahman, Tobias Rohde, Rahul Tripathi +2LLM Jailbreak AttacksLLM Red Teaming

  19. False Claims, Credible Images: A Red-Teaming Benchmark for Commercial Image Generators

    Oct 8, 2026Zeyu Ye, Yanchun Li, Sibei He +8Safe T2I GenerationT2I Generation Evaluation

  20. TransferBreaker: Breaking Adversarial Transferability in Fine-Tuned Speech Recognition

    Oct 6, 2026Mojtaba Nafez, Aref Mousavi, Mohammad Ebrahim Mahdavi +3Adversarial Robustness of ASRAdversarial Transferability

  21. ATLAS-AL: Adaptive Trust-Region for Latent Adversarial Searches via Active Learning

    Oct 5, 2026Marsalis Gibson, Claire Tomlin, Shankar SastryAdversarial RobustnessActive Learning

  22. Boosting Transferable Adversarial Attacks against Deep Reinforcement Learning

    Oct 5, 2026Zexin Li, Ruili Yao, Yiming Zeng +1Adversarial AttacksTransfer-Based Adversarial Attacks

  23. Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMs

    Oct 4, 2026Xiaojun Jia, Simeng Qin, Yiming Li +5Adversarial Attacks on VLMsAdversarial Attacks

  24. Pareto-Improving Adversarial Attacks with Primal-Dual Regularization

    Oct 3, 2026Yang Dai, Longfei Zhang, Wei Tao +3Adversarial AttacksTransfer-Based Adversarial Attacks

  25. VirusCascade: Hijacking Collaborative Reflection in LLM-Powered Recommender Agents

    Sep 29, 2026Yurong Hao, Wen Zhou, Guowei Guan +3Adversarial AttacksAgent Memory Poisoning

  26. One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs

    Sep 27, 2026Sen Nie, Jie Zhang, Zhongqi Wang +2Adversarial Attacks on VLMsAdversarial Attacks on LLMs

  27. Evasion Attacks on Cost-Utility-Based Adversarial Training for Online AutoML in IoT Networks

    Sep 25, 2026Chukwunonso Henry Nwokoye, Wajiha Zaheer, Khalil El-Khatib +1Adversarial RobustnessAdversarial Training

  28. JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models

    Sep 25, 2026Jianyi Hu, Hangtao Zhang, Yi Liu +5Adversarial RobustnessBlack-Box Adversarial Attacks