cs.CRNov 5, 2026

Toward Safety-Aware AI Agents in Realistic Social Engineering Attack Scenarios

Authors: Zhitao He, Chengru Han, Chi Him Zhang, Yi R Fung

Organizations: MMSense Lab, The Hong Kong University of Science and Technology

Abstract

Personal AI agents are increasingly deployed on user devices to execute actions involving real money and sensitive data, but their security under social engineering attacks has not been systematically evaluated. We propose SEA (Social Engineering Attack Benchmark), the first interactive environment for evaluating the intrinsic security capability of AI agents in social engineering attack scenarios, covering 8 daily-life domains, 30+ simulated applications, a 4-level difficulty gradient, and a 5-dimensional evaluation framework, with 320 evaluation tasks designed in consultation with anti-fraud specialists and spanning two culturally distinct settings in Chinese and English. Evaluation on 11 frontier open-source and closed-source models, with GPT-6 evaluated only in Chinese, reveals that all models are breached to varying degrees (Chinese attack success rates ranging from 16.9% to 84.3%) proactive verification rates universally below 32%. We further propose META-SEA, a co-evolutionary framework for attack tasks and defenders. With META-SEA training, Qwen3-8B achieves defense performance comparable to closed-source models while effectively reducing ungrounded action claims.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

    Nov 5, 2026Zhitao He, Chi Him Zhang, Chengru Han +1AI Agent SecurityAI Agent Security Benchmarks

  2. Agent-to-Agent Synthetic Social Engineering on the Web

    Oct 24, 2026Bryan C. BootsAI Agent SecurityWeb Agents