Multi-Turn Language Model Safety Evaluation

Latest papers

13 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

When Support Crosses the Line: Evaluating Ethical Boundaries in LLM-Based Mental Health Support in the Chinese Context
Oct 28, 2026Bichen Wang, Yufeng Jin, Yixin Sun +2Multi-Turn Language Model Safety Evaluation Mental Health Conversational Agents
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, Heilongjiang, China
How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defense
Oct 7, 2026Zhankai Ye, Yanning Wang, Yukai Jin +5Multi-Turn Language Model Safety Evaluation Refusal Behavior in Language Models
Florida State University · Texas Christian University · University of Pennsylvania +1
TRACE: Trajectory Return Attribution and Contrastive Erasure for Multi-Turn Safety
Oct 1, 2026Fengpeng Li, Kemou Li, Qizhou Wang +3LLM Fine-Tuning Adversarial Attacks on LLMs
PRADA Lab, King Abdullah University of Science and Technology · State Key Laboratory of Internet of Things for Smart City, University of Macau · Imperfect Information Learning Team, RIKEN Center for Advanced Intelligence Project +1
Evaluating Language Model Safety Across Long Adversarial Conversations
Sep 29, 2026Parisa Salmani, Peter R. LewisLanguage Model Safety Evaluation LLM Safety
Ontario Tech University Oshawa, Canada
A Translational Note on AI Safety Evaluation
Sep 6, 2026Madhava GaikwadMulti-Turn Language Model Safety Evaluation Language Model Safety Evaluation
Independent Researcher, Bengaluru, India
SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
Aug 8, 2026Debopriyo Banerjee, Kapil Rajesh Kavitha, Angana Borah +11LLM Safety Benchmarks Multi-Turn Language Model Safety Evaluation
Inception · Mohamed bin Zayed University of Artificial Intelligence · University of Michigan +4
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
May 21, 2026Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11LLM Safety Benchmarks Computer-Use Agent Benchmarks
Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6
MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
May 3, 2026Jialin Song, Xiaodong Liu, Weiwei Yang +4Multi-Turn Jailbreak Attacks LLM Safety Benchmarks
Simon Fraser University · Microsoft Research
BabelSafe: A Policy-Grounded Multilingual Safety Benchmark for LLMs
May 1, 2026Yunhan Zhao, Zhaorun Chen, Xingjun Ma +1LLM Safety Benchmarks Multi-Turn Language Model Safety Evaluation
University of Illinois Urbana-Champaign · Fudan University · University of Chicago
Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversations
Apr 29, 2026Mingqian Zheng, Malia Morgan, Liwei Jiang +2LLM Safety Benchmarks Multi-Turn Language Model Safety Evaluation
Carnegie Mellon University · Allen Institute for AI · University of Washington
Adversarial Moral Stress Testing of Large Language Models
Apr 1, 2026Saeid Jamshidi, Foutse Khomh, Arghavan Moradi Dakhel +3Multi-Turn Language Model Safety Evaluation Language Model Safety Evaluation
are with the SWAT Laboratory, Polytechnique Montréal, Montréal, QC, Canada · is with the Huawei Distributed Scheduling and Data Engine Lab, Montréal, QC, Canada · is with the SæT Laboratory, Polytechnique Montréal, Montréal, QC, Canada
State-Dependent Safety Failures in Multi-Turn Language Model Interaction
Mar 15, 2026Pengcheng Li, Jie Zhang, Tianwei Zhang +5Multi-Turn Language Model Safety Evaluation LLM Safety
School of Cyber Science and Technology, University of Science and Technology of China, China · Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University, Singapore +3
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3LLM Safety Benchmarks Jailbreak Attacks
1Duke University · 2Virtue AI

Oct 28, 2026J/K move · Enter open · S save

When Support Crosses the Line: Evaluating Ethical Boundaries in LLM-Based Mental Health Support in the Chinese Context

Bichen Wang, Yufeng Jin, Yixin Sun, Yanyan Zhao+1

Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, Heilongjiang, China

Multi-Turn Language Model Safety Evaluation Mental Health Conversational Agents LLM Safety Benchmarks Language Model Safety Evaluation Clinical Language Model Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly explored for mental health support, yet existing benchmarks primarily focus on supportive capabilities rather than whether models consistently adhere to professional ethical boundaries. This issue is particularly important in multi-turn interactions, where users may introduce new information or exert emotional pressure over time. We introduce MindEth-Bench, a benchmark for evaluating ethical boundary adherence in multi-turn mental health interactions. Grounded in professional ethical standards for mental health services in China, MindEth-Bench covers multiple dimensions of ethical conduct and uses Monte Carlo Tree Search (MCTS) to systematically explore interaction trajectories that stress-test model boundaries. Experiments across nine LLMs show that models that initially respond appropriately can shift toward ethical violations over extended interactions, with substantial variation across models and ethical dimensions. These results show that single-turn evaluations may overestimate the ethical reliability of LLM-based mental health support systems and highlight the need to evaluate ethical behavior throughout multi-turn interaction.