AI Safety Evaluation
Overview
AI safety evaluation asks whether models and AI systems behave safely—not only on isolated prompts, but across conversations, tool use, human-AI teams, and physical or high-stakes settings. Common approaches include red-teaming and adversarial testing, curated benchmarks, simulated agent tasks, human studies, runtime monitoring, and formal verification. Evaluations measure outcomes such as harmful assistance, missed hazards, refusal quality, robustness to shifts, and effects on users or interacting agents. A central challenge is that strong benchmark results provide limited assurance about rare failures or deployment behavior, so evaluations must make their assumptions and evidence limits explicit.
Latest papers
143 papers21 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.