Black-Box Adversarial Attacks

Ask about this topic

Overview

Black-box adversarial attacks ask how an attacker can cause an AI system to misclassify, produce unsafe output, or take an unwanted action without access to its internal parameters or gradients. Methods use feedback from model queries—such as labels, scores, or responses—with decision-boundary search, finite differences, bandits, evolutionary or reinforcement-learning optimization; others craft perturbations on surrogate models and rely on transfer to unseen targets. Work spans images and physical patches, text and jailbreak prompts, audio, graphs, malware detectors, and tool-using or multimodal agents, often under constraints such as limited queries, imperceptibility, and preserving input meaning or function. Key tensions include whether attacks transfer across architectures and real-world transformations, how to compare them under fair query budgets, and whether defenses can resist adaptive attacks without sacrificing clean performance or utility.

Latest papers

CardsList

175 papers16 papers in the last four weeks, down 6% on the four weeks before. 0.1% of all new papers.

  1. Contract And Conquer: How to Provably Compute Adversarial Examples for a Black-Box Model?

    Mar 11, 2026Anna Chistyakova, Mikhail PautovAdversarial AttacksBlack-Box Adversarial Attacks

    Trusted AI Research Center, RAS · AXXX

  2. Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints

    Mar 8, 2026Chenxi Li, Xianggan Liu, Dake Shen +9Adversarial Attacks on VLMsVLM Safety

    DAIL Tech · NLP & KG Lab, Huazhong University of Science and Technology

  3. Beyond the Patch: Exploring Vulnerabilities of Visuomotor Policies via Viewpoint-Consistent 3D Adversarial Object

    Mar 5, 2026Chanmi Lee, Minsung Yoon, Woojae Kim +2Adversarial AttacksPhysical Adversarial Attacks

    The School of Computing at the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea

  4. Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions

    Mar 4, 2026Neha Nagaraja, Lan Zhang, Zhilong Wang +2Prompt Injection AttacksAdversarial Attacks on VLMs

    School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, USA · Bytedance, CA, USA

  5. Scores Know Bobs Voice: Speaker Impersonation Attack

    Mar 3, 2026Chanwoo Hwang, Sunpill Kim, Yong Kiam Tan +6Black-Box Adversarial AttacksSpeaker Verification

    Hanyang University Seoul, Republic of Korea · I2R, A*STAR & NTU Singapore · National University of Singapore Singapore +1

  6. Learning to Attack: A Bandit Approach to Adversarial Context Poisoning

    Feb 28, 2026Ray Telikani, Amir H. GandomiBlack-Box Adversarial AttacksAdversarial Bandits

    Data Science Institute, University of Technology Sydney, NSW, Australia

  7. Adversarial Patch Generation for Visual-Infrared Dense Prediction Tasks via Joint Position-Color Optimization

    Feb 27, 2026He Li, Wenyue He, Weihang Kong +1Adversarial Patch AttacksAdversarial Examples

    School of Artificial Intelligence (School of Software), Yanshan University, Qinhuangdao 066004, China. · Fusion Intelligence Laboratory, Department of Computer Science, University of Exeter, EX4 4RN, United Kingdom.

  8. Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search

    Feb 26, 2026Xun Huang, Simeng Qin, Xiaoshuang Jia +6LLM Jailbreak AttacksAdversarial Prompt Generation

    Nanyang Technological University, Singapore · BraneMatrix AI, China · Nanjing University of Science and Technology, China +6

  9. Adversarial Robustness of Deep Learning-Based Thyroid Nodule Segmentation in Ultrasound

    Feb 25, 2026Nicholas Dietrich, David McShannonAdversarial RobustnessBlack-Box Adversarial Attacks

    Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada · Faculty of Engineering, McMaster University, Hamilton, Ontario, Canada

  10. Pushing the Frontier of Black-Box LVLM Attacks via Fine-Grained Detail Targeting

    Feb 19, 2026Xiaohan Zhao, Zhaoyi Li, Yaxin Luo +2Adversarial Attacks on VLMsBlack-Box Adversarial Attacks

    VILA Lab, Department of Machine Learning, MBZUAI

  11. Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systems

    Feb 18, 2026Jialiang Fan, Shixiong Jiang, Mengyu Liu +1Adversarial AttacksBlack-Box Adversarial Attacks

    University of Notre Dame, Notre Dame, IN, USA. · Washington State University Tri-Cities, Richland, WA, USA.

  12. Intent Laundering: AI Safety Datasets Are Not What They Seem

    Feb 17, 2026Shahriar Golchin, Marc WetterLLM Safety EvaluationLLM Safety Benchmarks

    Applied Machine Learning Research

  13. ER-MIA: Black-Box Adversarial Memory Injection Attacks on Long-Term Memory-Augmented Large Language Models

    Feb 17, 2026Mitchell Piehl, Zhaohan Xi, Zuobin Xiong +2Adversarial Attacks on LLMsLLM Security

    Department of Computer Science, The University of Iowa · School of Computing, State University of New York at Binghamton · Department of Computer Science, University of Nevada, Las Vegas +1

  14. Boundary Point Jailbreaking of Black-Box LLMs

    Feb 16, 2026Xander Davies, Giorgi Giglemiani, Edmund Lau +3Adversarial Attacks on LLMsJailbreak Attacks

    UK AI Security Institute · OATML, University of Oxford

  15. Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

    Feb 4, 2026Jaehyun Kwak, Nam Cao, Boryeong Cho +3Adversarial Attacks on VLMsAdversarial Attacks

    KAIST · KENTECH

  16. Enhancing Adversarial Transferability through Block Stretch and Shrink

    Nov 21, 2025Quan Liu, Feng Ye, Chenhao Lu +4Adversarial AttacksAdversarial Perturbations

  17. SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning

    Nov 12, 2025Tairan Huang, Yulin Jin, Junxu Liu +2Adversarial AttacksAdversarial Perturbations

    The Hong Kong Polytechnic University Hong Kong, China

  18. Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems

    Sep 9, 2025Kamel Kamel, Hridoy Sankar Dutta, Keshav Sood +1Audio Deepfake DetectionSpeaker Verification

    School of Information Technology, Deakin University, Australia

  19. Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies

    Jun 30, 2025Tim Roith, Leon Bungert, Philipp WackerEvolutionary OptimizationBlack-Box Adversarial Attacks

    TUM School of Computation, Information and Technology and Munich Center for Machine Learning (MCML), Boltzmannstr. 3, 85748 Garching bei München, Germany · Institute of Mathematics, Center for Artificial Intelligence and Data Science (CAIDAS), University of Würzburg, Emil-Fischer-Str. 40, 97074 Würzburg, Germany · School of Mathematics and Statistics, University of Canterbury

  20. HauntAttack: When Attack Follows Reasoning as a Shadow

    Jun 8, 2025Jingyuan Ma, Rui Li, Zheng Li +4Adversarial Attacks on LLMsJailbreak Attacks

    State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · StepFun · Department of Computing, The Hong Kong Polytechnic University +1

  21. How Vulnerable Is My Learned Policy? Universal Adversarial Perturbation Attacks On Modern Behavior Cloning Policies

    Feb 6, 2025Akansha Kalra, Basavasagar Patil, Guanhong Tao +1Adversarial RobustnessAdversarial Imitation Learning

    University of Utah

  22. Adversarial Robustness of AI-Generated Image Detectors in the Real World

    Oct 2, 2024Sina Mavali, Jonas Ricker, David Pape +2Adversarial AttacksAdversarial Robustness

    CISPA Helmholtz Center for Information Security, Saarbrücken, Germany · Ruhr University Bochum, Bochum, Germany

  23. Perturbation Effects on Robustness and Individual Fairness

    Apr 1, 2024Xuran Li, Hao Xue, Peng Wu +4Neural Network RobustnessAlgorithmic Fairness

    University of New South Wales Sydney, Australia · The Hong Kong University of Science and Technology Guangzhou, China · Key Laboratory of System Software, Institute of Software, Chinese Academy of Sciences Beijing, China +3

  24. SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems

    Feb 6, 2024Oubo Ma, Yuwen Pu, Linkang Du +5Adversarial AttacksPartially Observable RL

    Zhejiang University · Xi’an Jiaotong University · Laboratory for Big Data and Decision +2

  25. InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models

    Dec 4, 2023Xunguang Wang, Pingchuan Ma, Zhenlan Ji +3Adversarial Attacks on VLMsLarge Vision-Language Models

    The Hong Kong University of Science and Technology, Hong Kong, China