LLM Safety Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 177

All topics

AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
Jun 3, 2026Yanjing Ren, Reza Ebrahimi, TengTeng MaLLM Safety Benchmarks LLM-as-a-Judge
The School of Information Systems University of South Florida Tampa, US
PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
Jun 3, 2026Hainiu Xu, Italo Luis da Silva, Jiangnan Ye +7LLM Safety Benchmarks Moral Reasoning in Language Models
King’s College London · National University of Singapore · Southern University of Science and Technology +3
Can I Take Another Dose? Evaluating LLM Decision-Making Under Temporal Uncertainty in OTC Dosing QA
Jun 2, 2026Maroof Kousar, Yibo HuTemporal Reasoning Medical QA
Illinois Institute of Technology Chicago, IL, USA
Black-box, Adaptive, Efficient, Transferable, Harmful, Applicable... Attacks Are All You Need to Break LLMs
Jun 2, 2026Vincent Limbach, Jonas Dornbusch, David Lüdke +2LLM Jailbreak Attacks LLM Safety Evaluation
Department of Computer Science, Technical University of Munich, Germany · Munich Data Science Institute · Munich Center for Machine Learning +1
DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair
Jun 2, 2026Qinyan Zhou, Peixin Zhang, Jun Sun +2LLM Guardrails LLM Refusal Behavior
Southeast University, China · School of Computing and Information Systems, Singapore Management University, Singapore · Zhejiang University, China +1
Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs
Jun 1, 2026Zacharie BugaudLLM Safety Alignment LLM Safety
Astera Institute
Food Noise & False Safety: A Systematic Evaluation of How LLMs Fail to Adapt to Eating Disorder Queries with Clinician Feedback
Jun 1, 2026Giulia Pucci, Emily Hemendinger, Ruizhe Li +3Mental Health LLM Safety
University of Aberdeen · University of Colorado Anschutz · Heriot-Watt University +1
Lost in Delusion: Examining LLM Safety Under User Delusions and Distress
May 31, 2026Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan +3Mental Health LLM Safety Alignment
University of Pittsburgh · Carnegie Mellon University · Fordham University
Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs
May 30, 2026Subhadip MitraAdversarial Attacks on LLMs LLM Red Teaming
Research Lead, Rota Labs
Quality-Diversity Evolution for Discovering Diverse Vulnerabilities in LLM Safety
May 30, 2026Subhadip MitraAdversarial Prompt Generation Adversarial Attacks on LLMs
Research Lead, Rota Labs
TRACE: Trajectory Risk-Aware Compression for Long-Horizon Agent Safety
May 30, 2026Zhepei Hong, Lin Wang, Liting Li +5AI Agent Safety LLM Safety Evaluation
University of Science and Technology of China · National University of Singapore · South China Normal University
Same Payload, Different Channel: Measuring Trust Asymmetry in Tool-Using Language Models
May 30, 2026Mohammed Sameer Syed, Rozhin YasaeiAdversarial Attacks on LLMs Prompt Injection Attacks
University of Arizona
Position: Anthropomorphic Misalignment Research Needs Stronger Evidence
May 29, 2026Vansh Gupta, Peter Nutter, Samuel Stante +5LLM Safety Evaluation
1ETH Zurich
Triaging Threats to Specialized Guardrails
May 29, 2026Wenjie Jacky Mo, Xiaofei Wen, Rui Cai +6LLM Safety Benchmarks LLM Guardrails
University of California, Davis
Beyond Attack Success Rate: Temporal Logit Observability for LLM Safety Failures
May 28, 2026Junyoung Park, Sunghwan Park, Seongyong Ju +1LLM Refusal Behavior LLM Jailbreak Attacks
Chung-Ang University
Inform, Coach, Relate, Listen: Auditing LLM Caregiving Support Roles
May 28, 2026Drishti Goel, Agam Goyal, Veda Duddu +8Healthcare LLM Auditing
University of Illinois Urbana-Champaign · University of Massachusetts Amherst · OSF HealthCare +1
SciIntBench: Measuring LLM Compliance with Research Integrity Norms Under Adversarial Framing
May 28, 2026Almene De Meran Meguimtsop, Maria Leonor Pacheco, Daniel E. AcunaAI for Science Adversarial Attacks on LLMs
Department of Computer Science, University of Colorado Boulder, Boulder, CO 80309
A Study on Question-Answer Dataset for LLM Safety Evaluation with a Focus on Illegal Activities
May 28, 2026Kenji Imamura, Masao Ideuchi, Atsushi FujitaAI Safety Evaluation LLM Safety Evaluation
National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan
Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents
May 28, 2026Aditya Nawal, Manit Baser, Mohan GurusamyAI Agent Security Benchmarks LLM Safety Evaluation
Department of Electrical and Computer Engineering · National University of Singapore
Code as a Weapon: A Consensus-Labeled Prompt Bank for Measuring Coding-Model Compliance with Malicious-Code Requests
May 27, 2026Richard J. Young, Gregory D. MoodyClassification LLM Safety Evaluation
University of Nevada Las Vegas · Department of Information Systems · Las Vegas, NV, USA
Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
May 26, 2026Aman Priyanshu, Supriti Vijay, Esha PahwaData Leakage Multi-Agent LLM Systems
Foundation AI · USA · Corvic AI
When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries
May 26, 2026Yige Li, Jun Sun, Wei Zhao +5LLM Safety Benchmarks Healthcare
Singapore Management University · Deakin University · The University of Melbourne +2
KZ-SafetyPrompts: A Kazakh Safety Evaluation Prompt Dataset for Large Language Models
May 26, 2026Wajdi Zaghouani, Shimaa Amer Ibrahim, Aruzhan Muratbek +2LLM Safety Benchmarks Language Model Safety Evaluation
Northwestern University in Qatar
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
May 25, 2026Fay Elhassan, David Sasu, Alexandra Kulinkina +2LLM Safety Benchmarks Human Preference Evaluation
LiGHT Laboratory, EPFL, Lausanne, Switzerland
SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks
May 25, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangPairwise Comparison Pairwise Preference Evaluation
Northeastern University, Boston, MA, USA · University of Illinois Urbana-Champaign, Urbana, IL, USA · Southern Methodist University, Dallas, TX, USA
From Automation to Collaboration: Human-in-the-Loop Methods for Safe and Trustworthy NLP
May 24, 2026Most. Sharmin Sultana Samu, MD. Tanvir Ahmed Seum, Md. Rakibul IslamLLM Auditing Human-in-the-Loop Annotation
Department of Computer Science and Engineering, BRAC University, Dhaka-1212. · Department of Electrical and Electronic Engineering, Rajshahi University of Engineering and Technology, Rajshahi-6204.
Furina: Fragmented Uncertainty-Driven Refusal Instability Attack
May 24, 2026Tongxi Wu, Jian Zhang, Yang GaoAdversarial Attacks on VLMs LLM Safety
School of Intelligence Science and Technology, Nanjing University, Suzhou, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
Open-source LLMs administer maximum electric shocks in a Milgram-like obedience experiment
May 20, 2026Roland Pihlakas, Jan Llenzl DagohoyLLM Refusal Behavior LLM Safety Evaluation
for the Three Laws collaboration · Independent researcher · Three Laws research collaboration +2
RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts
May 20, 2026Lukas Weidener, Marko Brkić, Mihailo Jovanović +2LLM Safety Benchmarks Language Model Safety Evaluation
Applied Scientific Intelligence, Inc.
Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agents
May 18, 2026Ahmad Al-Tawaha, Shangding Gu, Peizhi Niu +2Long-Horizon Agent Evaluation LLM Safety Evaluation
Virginia Tech · University of California, Berkeley · University of Illinois Urbana-Champaign

Jun 3, 2026, cs.AIJ/K move · Enter open · S save

AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety

Yanjing Ren, Reza Ebrahimi, TengTeng Ma

The School of Information Systems University of South Florida Tampa, US

LLM Safety Benchmarks LLM-as-a-Judge AI Companions AI Safety Evaluation LLM Safety Evaluation AI Agent Safety Benchmarks+1

Open paper AskPDF arXiv

As AI companion platforms such as Replika and Character.AI rapidly grow, concerns about unsafe human-AI interactions have intensified. This study introduces AICompanionBench, to our knowledge the first publicly available benchmark dataset of human-AI companion conversations annotated with fine-grained safety risk categories. The dataset contains 2,123 real-world Replika conversations collected from Reddit and annotated through human-AI collaboration across nine categories: sexual behavior, antisocial behavior, physical aggression, verbal aggression, substance abuse, self-harm and suicide, control, manipulation, and no-harm. Using this benchmark, we evaluate 20 state-of-the-art open-source and closed-source LLMs under an LLM-as-judge framework for detecting unsafe interactions. Results show substantial variation in model performance, with stronger models achieving high overall accuracy but still struggling with nuanced categories such as manipulation, as well as benign conversations that are incorrectly identified as harmful. Our findings suggest that while current LLMs can effectively detect explicit harmful content, they remain limited in identifying implicit unsafe interactions. Overall, our work contributes a new benchmark dataset for AI companionship safety research and offers insights into monitoring AI companion systems using LLMs. The dataset is publicly available at: https://github.com/anonymousresearcher2026/AICompanionBench/blob/main/AICompanionBench.xlsx

Momentum

Related topics

Latest papers 177