Black-Box Adversarial Attacks

Black-box adversarial attacks ask how an attacker can cause an AI system to misclassify, produce unsafe output, or take an unwanted action without access to its internal parameters or gradients. Methods use feedback from model queries—such as labels, scores, or responses—with decision-boundary search, finite differences, bandits, evolutionary or reinforcement-learning optimization; others craft perturbations on surrogate models and rely on transfer to unseen targets. Work spans images and physical patches, text and jailbreak prompts, audio, graphs, malware detectors, and tool-using or multimodal agents, often under constraints such as limited queries, imperceptibility, and preserving input meaning or function. Key tensions include whether attacks transfer across architectures and real-world transformations, how to compare them under fair query budgets, and whether defenses can resist adaptive attacks without sacrificing clean performance or utility.