LLM Safety Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 178

All topics

Scaling Trends for Lie Detector Oversight in Preference Learning
Jul 2, 2026Oskar J. Hollinsworth, Ann-Kathrin Dombrowski, Sam Adam-Day +2Scalable Oversight Deception in Language Models
Safe Alone, Unsafe Together: Safeguarding Against Implicit Toxicity When Benign Images Combine
Jul 1, 2026Jiaxian Lv, Shiyao Cui, Yingkang Wang +3Toxicity Detection Safety Filtering
The Conversational AI (CoAI) Group, DCST, Tsinghua University
EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots
Jun 29, 2026Camilo Chacón SartoriLLM Safety Benchmarks Multilingual Language Model Evaluation
MindSurf
EvalSafetyGap: A Hybrid Survey and Conceptual Framework for LLM Evaluation-Safety Failures
Jun 29, 2026Buğra Alperen Uluırmak, Rifat KurbanAI Alignment LLM Safety Alignment
Erciyes University, Türkiye. · Abdullah Gül University, Türkiye.
SCARCE: Scalable Cascade Analysis for Rare-event Characterisation via Embeddings
Jun 28, 2026Yingjie Wang, Yi Dong, Edmund Lau +3LLM Safety Evaluation Rare-Event Estimation
University of Liverpool, UK · UK AI Security Institute, UK · Loughborough University, UK +1
The Joint Effect of Quantization and Sampling Temperature on LLM Safety Alignment: A Factorial Analysis
Jun 28, 2026Hari Prasad, Ritam PalLLM Safety Benchmarks LLM Quantization
Conscious Engines
Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes
Jun 25, 2026Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu +2LLM Safety Evaluation LLM Post-Training
University of Groningen · University Politehnica of Bucharest · NVIDIA
Necessary but Not Sufficient: Temperature Control and Reproducibility in LLM-as-Judge Safety Evaluations
Jun 24, 2026Hiroki TambaLLM Evaluation LLM-as-a-Judge
Tamba Research Academy
RAS: Measuring LLM Safety Through Refusal Alignment
Jun 24, 2026Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu +1LLM Safety Alignment LLM Safety
National Yang Ming Chiao Tung University · Hon Hai Research Institute
A Red Teaming Framework for Large Language Models: A Case Study on Faithfulness Evaluation
Jun 24, 2026Abrar Alotaibi, Raed Mughus, Moataz AhmedMultilingual Language Model Evaluation LLM Evaluation
Information and Computer Science Department, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia. · College of Computer Science and Information Technology, Imam Abdulrahman Bin Faisal University, Dammam, 31441, Saudi Arabia. · SDAIA-KFUPM Joint Research Center for Artificial Intelligence, King Fahd University of Petroleum & Minerals, Dhahran, 31261, Saudi Arabia.
Do Thinking Tokens Help with Safety?
Jun 23, 2026Narutatsu Ri, Abhishek Panigrahi, Sanjeev AroraLLM Refusal Behavior LLM Safety Evaluation
Princeton Language and Intelligence, Princeton University
Distributed Quality-Diversity Search for Toxicity in Large Language Models
Jun 23, 2026Onkar Shelar, Travis DesellAdversarial Prompt Generation LLM Red Teaming
Rochester Institute of Technology, Rochester NY 14623, USA
Old Fictions, New Skins: Evaluating the Manipulative Capabilities of LLMs in Healthcare
Jun 20, 2026Gathoni Ireri, Roger D. OdipoHealthcare Deception in Language Models
1ILINA Program · 2Haki AI
FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teaming
Jun 18, 2026Chaeyun Kim, Daeyoung Park, Junghwan Kim +4LLM Safety Benchmarks Financial Services
DATUMO INC., Republic of Korea · Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea · Financial Security Institute (FSI), Republic of Korea
Analyzing the Narration Gap in LLM-Solver Loops
Jun 17, 2026Zunchen Huang, Songgaojun DengAdversarial Attacks on LLMs Formal Verification
Centrum Wiskunde & Informatica, Amsterdam, The Netherlands · Eindhoven University of Technology, Eindhoven, The Netherlands
SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety
Jun 17, 2026Linghao Feng, Yinqian Sun, Dongqi Liang +8LLM Safety Benchmarks AI for Science
Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China +5
Automated jailbreak attack targeting multiple defense strategies
Jun 15, 2026Qi Wang, Chengcheng Wan, Weijia He +4Adversarial Prompt Generation LLM Jailbreak Attacks
East China Normal University Shanghai, China · East China Normal University, Shanghai Innovation Institute Shanghai, China · University of Southampton Southampton, England +2
Risk-Aware LLM Agents for Geospatial Data Retrieval: Design and Preliminary Adversarial Evaluation
Jun 13, 2026Kyle Gao, Joel Cumming, Jonathan Li +2Multi-Agent LLM Systems LLM Tool Use
Dept. of Systems Design Engineering, University of Waterloo, Waterloo, ON, N2L 3G1, Canada · SkyWatch, Kitchener, ON, N2H 2G8, Canada · Dept. of Geography and Environmental Management, University of Waterloo, Waterloo, ON, N2L 3G1, Canada +1
BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
Jun 12, 2026Leonhard Waibl, Felix Michalak, Hadrien MariacciaLLM Safety Benchmarks LLM Guardrails
University of Graz, Graz, Austria · Supervised Program for Alignment Research (SPAR) · Centre pour la Sécurité de l’IA (CeSIA), Paris, France.
Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants
Jun 12, 2026Dipesh Tharu MahatoAI Safety Evaluation LLM Safety Evaluation
New York University
Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models
Jun 9, 2026Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik +1LLM Alignment Neural Network Robustness
University of Toronto · Vector Institute · Hugging Face
Quality Is Not a Safety Proxy Under Quantization
Jun 8, 2026Sahil KadadekarLLM Quantization LLM Refusal Behavior
Independent Researcher
The Injection Paradox: Brand-Level Suppression in Safety-Trained LLM Recommendations via RAG Context Injection
Jun 8, 2026Hyunseok PaengRetrieval-Augmented Generation Adversarial Attacks on LLMs
Independent Researcher.
Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges
Jun 8, 2026Yongtaek Lim, Hyeji Choi, Minwoo KimLLM-as-a-Judge LLM Safety Evaluation
AI Safety Team, DATUMO.INC, Seoul, South Korea.
Personalization Meets Safety:Mechanisms,Risks,and Mitigations in Personalized LLMs
Jun 8, 2026Yanyan Luo, Xue Han, Ruiqiao Bai +10LLM Personalization LLM Safety
Jiutian Research, China Mobile Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., Beijing, China. · Xinjiang Technical Institute of Physics and Chemistry, Chinese2026 Academy of Sciences, Beijing, China.
SafeRun: Enabling Determinism in LLM Planning for Running
Jun 8, 2026Meilin Chen, Zepeng Zhai, Jiaxuan Zhao +1LLM Planning LLM Safety Evaluation
Xiaohongshu Inc. · Institute of Information Engineering, Chinese Academy of Sciences
Sycophancy Towards Researchers Drives Performative Misalignment
Jun 7, 2026David D. Baek, Xinnuo Li, Anay Gupta +4LLM Sycophancy LLM Alignment
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Michigan, Ann Arbor, MI, USA · Georgia Institute of Technology, Atlanta, GA, USA +1
Building Comparative Motivation Profiles with Instrumental Interventions
Jun 6, 2026David Vella Zarb, Rustem Turtayev, Taywon Min +2LLM Safety Evaluation Causal Interventions in Language Models
MATS · University of Cambridge · KAIST +1
SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agents
Jun 6, 2026Tanush Swaminathan, Runmin Jiang, Letian Zhang +1AI Agents for Scientific Discovery LLM Safety Evaluation
Carnegie Mellon University · Allen Institute
The Governance of Human-LLM Interaction: Safety Gating, Civility Steering, and Affective Default Lock-In
Jun 6, 2026Manuele Reani, Hongjian Zhang, Hongyu TianLanguage Model Steering LLM Safety Evaluation
School of Management and Economics, The Chinese University of Hong Kong, Shenzhen, China

Jul 2, 2026, cs.AIJ/K move · Enter open · S save

Scaling Trends for Lie Detector Oversight in Preference Learning

Oskar J. Hollinsworth, Ann-Kathrin Dombrowski, Sam Adam-Day, Adam Gleave+1

Scalable Oversight Deception in Language Models LLM Safety Evaluation Human Oversight of AI Preference Learning

Open paper AskPDF arXiv

Deceptive behavior in LLMs is costly to monitor and prevent, motivating approaches such as Scalable Oversight via Lie Detectors (SOLiD) (Cundy & Gleave, 2025), which uses lie detectors to identify responses for review by high-cost labelers. In this paper, we scale SOLiD to larger models and evaluate it in more diverse and realistic preference-learning settings. We find favorable scaling: undetected deception drops from 34% for 1B-parameter models to 14% for 405B-parameter models at a detector true positive rate of 99%, and expensive human labelers can be removed entirely from the fine-tuning phase without a statistically significant increase in deception. However, SOLiD is sensitive to distribution shift between detector training and preference-training data, which can drive detector false positive rates to impractical levels.

Momentum

Related topics

Latest papers 178