LLM Safety Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 14% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 175

All topics

Dynamic Budget Allocation for LLM Evaluation under Hard Resource Constraints
Oct 5, 2026Shai Feldman, Yaniv RomanoLLM Evaluation LLM Safety Evaluation
Department of Computer Science, Technion IIT, Israel · Department of Electrical and Computer Engineering, Technion IIT, Israel
Refusals That Bend: Measuring and Predicting Task Malleability in Embodied VLM Planners
Sep 30, 2026Leo Y. Lin, Mikhail Kuznetsov, Muslum Ozgur Ozmen +1Robot Safety LLM Safety Evaluation
Purdue University · Amazon · Arizona State University
Evaluating Whether GPT-6 Astra Performs Unsanctioned Supply-Chain Attacks
Sep 29, 2026Alexandra Souly, Kai Fronsdal, Abby D'Cruz +2LLM Auditing AI Agent Security
UK AI Security Institute · UK AISI Research Affiliate
Jailbreak Context Lingers: Divergent Safety Routing and Its Cross-Task Predictability in Tool Agents
Sep 28, 2026Xi Wang, Songlei Jian, Yiming Zhang +5LLM Safety Evaluation Causal Interventions in Language Models
National University of Defense Technology
ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts
Sep 24, 2026Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi +5LLM Safety Benchmarks Arabic NLP
Qatar Computing Research Institute, Qatar · Hamad Bin Khalifa University, Qatar · APAVI.AI, France +1
From Intent to Action: Benchmarking LLM Safety in Vehicle Voice Command Authorization
Sep 17, 2026Diba Afroze, Xingli Zhang, Yazhou Tu +1LLM Safety Benchmarks LLM Safety Evaluation
University of Louisiana at Lafayette, Lafayette, LA 70503, USA · Auburn University, Auburn, AL 36849, USA
Do Frontier Models Seek Safety Evidence Before Acting?
Sep 15, 2026Omer TafveezLLM Safety Benchmarks LLM Safety Evaluation
University of Michigan
Benchmarking Factual Robustness of LLMs via Multi-conversation Persuasion
Sep 15, 2026Zhuoang CaiAdversarial Attacks on LLMs LLM Safety Evaluation
The Hong Kong University of Science and Technology
Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs
Sep 14, 2026Mark Russinovich, Blake Bullwinkel, Giorgio Severi +2Adversarial Attacks on LLMs Cybersecurity
Microsoft Azure · Microsoft
DeFiFlowBench: Benchmarking and Improving Safe Executability in Natural-Language DeFi Workflow Synthesis
Sep 11, 2026Abhinav Rajeev Kumar, Harshit Arora, Varun Singh +1LLM Safety Benchmarks LLM Guardrails
SRM Institute of Science and Technology
CS-Guard: Benchmarking LLM Guardrails for Code Generation Security
Sep 9, 2026Jinyang Li, Mingyu Guo, Hung X. NguyenLLM Guardrails Adversarial Attacks on LLMs
Adelaide University Adelaide, Australia
CT-SAFR: Safe and Interpretable Chain-of-Thought Reasoning for Autonomous Robots: A Multi-Layered Verification Framework for Trustworthy AI-Driven Robotic Decision Making
Sep 9, 2026Cagri TemelCoT Faithfulness Large Language Model-Based Robot Planning
Hezarfen LLC Seattle, WA, USA · Grand Canyon University Phoenix, AZ, USA
Structural Jailbreaks Generalize but Do Not Compound: A cross-provider and multilingual study of Involuntary In-Context Learning
Sep 8, 2026Tejasvi C. AddagadaLLM Safety Benchmarks Multilingual Language Model Evaluation
Independent researcher
SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs
Sep 7, 2026Pengfei Li, Naufal Suryanto, Sicheng Zhang +2LLM Safety Benchmarks LLM Safety Evaluation
Khalifa University · University of Western Australia
Probing the Structure and Dynamics of LLM Value Expression through Value Conflicts
Sep 7, 2026Kaicheng Zhang, Jingyi Xiao, Renjun Hu +3LLM Alignment LLM Safety Evaluation
East China Normal University Shanghai, China
How Long Until Your Robot Ignores You? A Safety Benchmark for LLM Orchestrators in Human-Humanoid Collaboration
Sep 7, 2026Aulon Bajrami, Mohamed Elshamouty, Werner KrausRobot Safety Human-Robot Collaboration
Fraunhofer IPA Stuttgart, Germany
The Oversight Gap: What LLM Safety Monitors Miss, and Why It Is Not Capability
Sep 7, 2026Xin XuLLM Safety Benchmarks LLM Safety Evaluation
Carnegie Mellon University
FLY-EVAL++: An Evidence-Driven Evaluation Protocol for Safety-Constrained Flight Prediction with Large Language Models
Sep 3, 2026Yalun Wu, Junfeng Fang, Jiawei Wang +6LLM Safety Benchmarks LLM Safety Evaluation
NExT++ Lab, School of Computing, National University of Singapore · 3Xiamen University · University of Manchester +4
SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue
Sep 1, 2026Stephanie Fong, Yiwen Jiang, Zimu Wang +12LLM Safety Benchmarks Social Bias in Language Models
Monash University · University of Liverpool · University of Edinburgh +2
The Safeguard Worked. Is the LLM System Safer?
Sep 1, 2026Pingyu Wu, Weiming Zhang, Nenghai YuLLM Guardrails LLM Safety
University of Science and Technology of China · 2Hefei AiDA Lab
Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models
Aug 27, 2026Benlei Cui, Shen Pang, Yuke Wang +7VLM Robustness Adversarial Attacks on VLMs
1Yuvion Team, Alibaba Group · Laboratory for Statistical Monitoring and Intelligent Governance of Common Prosperity, School of Statistics and Data Science, Zhejiang Gongshang University
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Aug 18, 2026Istiaque Ahmed, Afia Anjum Borsha, Ranat Das Prangon +2LLM Guardrails Adversarial Attacks on LLMs
Graduate School of Informatics, Osaka Metropolitan University, Osaka 558-8585, Japan · Dept. of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh · Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka 1000, Bangladesh +1
TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation
Aug 16, 2026Md Messal Monem Miah, Adrita Anika, Zhiyuan Yu +1Adversarial Training LLM Jailbreak Attacks
Texas A&M University · Amazon
Quantifying the Relationship Between Clinical Safety and Environmental Impact in Therapeutic LLMs
Aug 12, 2026Alireza A. Safaei, Laura M. Vowels, Matthew J. Vowels +2Energy-Efficient ML LLM Safety Evaluation
University of Isfahan Isfahan, Iran · University of Roehampton London, UK · Kivira Health New York City, New York, U.S.A.
AI Guardrail Survival under Single-Cycle Agentic Self-Summarization
Aug 11, 2026Ted Kwartler, Alan Aqrawi, Arian AbbasiLLM Guardrails LLM Safety Evaluation
Harvard Extension School, Harvard University; Americas Advanced AI Practice Lead, Accenture. · Accenture.
REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
Aug 11, 2026Zixing Chen, Xingyuan Liu, Jie Zhu +6LLM Red Teaming LLM Agents
Fudan University · The Hong Kong University of Science and Technology · Qwen DianJin Team, Alibaba Cloud Computing +1
ProbGuard: Calibrated Safety Risk Estimation from LLM Output Distributions
Aug 11, 2026Xinzhe Huang, Biwu Yao, Kedong Xiu +4LLM Guardrails Language Model Calibration
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China · University of Electronic Science and Technology of China, Chengdu, China +3
TAF-MED: Multi-Turn Safety Refusal Collapse in LLMs Under Declared Self-Treatment Intent
Aug 10, 2026Waleed Jamil, Raphael SchmittLLM Safety Benchmarks LLM Safety Evaluation
1Independent Researcher, Edinburgh, United Kingdom · School of Computation, Information and Technology, Technical University of Munich, Germany · Institute of General Practice, Faculty of Medicine and Medical Center, University of Freiburg, Germany
Beyond Detection: Evaluating Defensive LLMs Against AI-Generated Social Engineering in Live Turn-by-Turn Interaction
Aug 10, 2026Yuqiao Xu, Osama Zafar, Alexander Nemecek +1LLM Safety Benchmarks LLM Safety
Case Western Reserve University
Generating Attacks for LLMs with GFlowNets
Aug 10, 2026Berkay Ozcam, Irem Onen, Mehmet Fatih Amasyali +1Adversarial Prompt Generation Adversarial Attacks on LLMs
Cybersecurity R&D Turkcell Istanbul, Turkey · Department of Computer Engineering Yıldız Technical University Istanbul, Turkey
Measuring the Wrong Thing: Internal Harmfulness Scores Anti-Rank Successful Jailbreaks
Aug 10, 2026Mingyu Luo, Ming Deng, Zilang Qiu +8Safety Filtering LLM Jailbreak Attacks
College of Computer Science and Artificial Intelligence, Fudan University · School of Computer Engineering and Science, Shanghai University · Beijing Normal University +5
Safety Cost of Steering Vectors Is Separable and Reducible
Aug 9, 2026Yuxiao Li, Gjergji KasneciLanguage Model Steering LLM Safety Alignment
Technical University of Munich Munich Center for Machine Learning (MCML)
MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
Aug 8, 2026Yilian Shi, Yueming Lyu, Haoxiang Tan +5LLM Safety Benchmarks Multimodal Robustness
Nanjing University
Taxonomy-Driven Analysis of Open-Source AI Risk Mitigation Tools
Aug 7, 2026Afreen Alam, Evgenija Popchanovska, Ana Gjorgjevikj +4AI Risk Management LLM Security
Department of Administrative Sciences, Metropolitan College, Boston University, Boston, MA 02215, USA · Faculty of Computer Science and Engineering, Ss. Cyril and Methodius University, Skopje 1000, North Macedonia · Department of Computer Science, Metropolitan College, Boston University, Boston, MA 02215, USA
DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots
Aug 5, 2026Jared Moore, Andrea Mock, Yifan Mai +9LLM Safety Benchmarks LLM Safety
Stanford University Stanford, California, USA · University of Chicago Chicago, Illinois, USA · Carnegie Mellon University Pittsburgh, Pennsylvania, USA +2
Social Pressure Breaks Majority Voting in LLM Safety Panels
Aug 5, 2026Yibo Hu, Jiaming QuNormative Conformity in Language Models Safety Filtering
Illinois Institute of Technology · Amazon
ADMITBench: A Safety-Governed Reference Framework for Evaluating the Admissibility of Industrial LLM Advisories
Aug 4, 2026Yash Misra, Javal Vyas, Siddharth Gutta +1LLM Safety Benchmarks AI-Assisted Decision Making
Refiant, Inc. · Imperial College London
EduZone: A Framework for Evaluating LLM Safety for K-12 Students and Teachers
Aug 3, 2026Junyeong Park, Jieun Han, Haneul Yoo +3LLM Safety Benchmarks Generative AI in Education
KAIST · NYU · Google Cloud AI Research
A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models
Aug 3, 2026Shu Quan, Tianfang Hao, Sitong Fang +8LLM Safety Benchmarks LLM Safety
Institute for Artificial Intelligence, Peking University · The Hong Kong University of Science and Technology (Guangzhou) · Beijing Academy of Artificial Intelligence
Safety, or Just Capability? A Validity Audit of Agent-Safety Benchmarks
Jul 30, 2026Youting Wang, Xiao Han, Dingyan Shang +2LLM Safety Evaluation Benchmark Validity
RoguePrompt: Dual-Layer Encoding for Self-Reconstruction to Circumvent LLM Moderation
Jul 29, 2026Benyamin Tafreshian, Prathamesh DhakeAdversarial Attacks Adversarial Attacks on LLMs
Department of Computer Science · Boston University · Boston, MA, USA
LLM4OSC: Profile-Bound Natural Language Control with Deterministic Validation for Open Sound Control
Jul 28, 2026Yuan-Yi FanLanguage Model-Based Control LLM Reliability
Los Angeles
Inspect India Evals: An Open Benchmarking Framework for Evaluating Large Language Models in the Indian Linguistic and Cultural Context
Jul 28, 2026Abhishek Kumar Singh, Shrey Nag, Sachita +2LLM Safety Benchmarks Multilingual Language Model Evaluation
GTBIT¹ · TIET² · IGDTUW³ +2
Same Dangerous Objective, Opposite Advice: Direct Exposure versus Multi-Agent Mediation
Jul 23, 2026Linjun LiMulti-Agent LLM Systems LLM Safety
Department of Mathematics, University of Pennsylvania, Philadelphia, PA.
Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs
Jul 20, 2026Koyar AfrasyabLLM-as-a-Judge LLM Prompting
Kinvectum AB, Sweden
Stress Testing Concept Erasure with Large Language Model Agents
Jul 20, 2026Yuyang Xue, Feng Chen, Zhihua Liu +4LLM Agent Evaluation LLM Safety Evaluation
School of Engineering, University of Edinburgh · The University of Manchester · The University of Melbourne
JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models
Jul 20, 2026Qingjia Huang, Jingyu Zhang, Jianguo Wu +6LLM Safety Benchmarks LLM Jailbreak Attacks
Institute of Information Engineering, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Renmin University of China +2
How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributions
Jul 19, 2026Yukai Zhou, Feiyang Lu, Xiaokai Mao +2LLM Safety Alignment Training Data Selection
ShanghaiTech University · Zhejiang University
Retrieval-Augmented Generation in LLMs for Mental Health: Quantifying the Incremental Contribution of Retrieval Within a Layered Safety Architecture
Jul 17, 2026Anand Gupta, Akshat Surolia, Shubham Mishra +2Retrieval-Augmented Generation Intent Classification
Wysa Inc., Boston, MA, United States
Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
Jul 15, 2026Jan Betley, Johannes Treutlein, Jan Dubiński +5LLM Alignment Language Model Bias Evaluation
1Truthful AI · 2Warsaw University of Technology · 3NASK National Research Institute +2
DeepBias: Adaptive In-depth Probing of Social Biases in LVLMs
Jul 13, 2026Anqi Li, Jie Zhang, Zhongqi Wang +4VLM Evaluation Language Model Bias Evaluation
AMT-X: Phase-Structured Multi-Turn Red-Teaming with Checklist-Gated Evaluation
Jul 13, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungLLM Safety Benchmarks Adversarial Attacks on LLMs
Vulcan Research, AIFT, Singapore
ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm
Jul 11, 2026Kefan Song, Yanjun QiAI Agent Auditing LLM Auditing
University of Virginia, Charlottesville, VA, USA
When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control
Jul 11, 2026Daming LuoSparse Autoencoders LLM Safety Evaluation
University of Technology Sydney
Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety
Jul 8, 2026Lifei Liu, Haoran Yu, Xiaochong Jiang +3Multi-Agent LLM Systems LLM Safety Evaluation
Independent Researcher Seattle, WA, USA · Carnegie Mellon University Pittsburgh, PA, USA · Georgia Institute of Technology Atlanta, GA, USA
Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis
Jul 7, 2026Mingchen Li, Meikang Qiu, Zifan Peng +4Software Vulnerability Detection LLMs for Cybersecurity
University of North Texas, USA · Augusta University, USA · Newcastle University, United Kingdom
Retroactive Chain-of-Thought (RetroCoT): Forensic Reconstruction Prompts as a Safety Diagnostic Across Model Generations
Jul 6, 2026Samira HajizadehPragmatic Reasoning in Language Models Adversarial Attacks on LLMs
Department of Computer Science Columbia University New York, NY, USA
Scaling Trends for Lie Detector Oversight in Preference Learning
Jul 2, 2026Oskar J. Hollinsworth, Ann-Kathrin Dombrowski, Sam Adam-Day +2Scalable Oversight Deception in Language Models
Safe Alone, Unsafe Together: Safeguarding Against Implicit Toxicity When Benign Images Combine
Jul 1, 2026Jiaxian Lv, Shiyao Cui, Yingkang Wang +3Toxicity Detection Safety Filtering
The Conversational AI (CoAI) Group, DCST, Tsinghua University
EMPATH: A Multilingual Auditor-Judge Benchmark for Safety Evaluation of Emotional-Support Chatbots
Jun 29, 2026Camilo Chacón SartoriLLM Safety Benchmarks Multilingual Language Model Evaluation
MindSurf

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

Dynamic Budget Allocation for LLM Evaluation under Hard Resource Constraints

Shai Feldman, Yaniv Romano

Department of Computer Science, Technion IIT, Israel · Department of Electrical and Computer Engineering, Technion IIT, Israel

LLM Evaluation LLM Safety Evaluation

Open paper AskPDF arXiv

We evaluate large language models (LLMs) in multi-turn interactions through their time-to-event: the number of interaction steps required to produce an event of interest, such as a successful jailbreak or agentic task completion. Under limited compute, interactions may be terminated before the event occurs, so that event times are only partially observed (censored). Existing allocation methods for calibrating time-to-event bounds satisfy the budget only in expectation and can exceed the available budget on a particular evaluation run. Enforcing a hard constraint is particularly challenging as the cost of a trajectory is initially unknown. We introduce Hard-budget Allocation with Reflow for Predictive calibration (HARP), a budget allocation that satisfies hard resource constraints and adaptively reallocates unused budget. We show how to use HARP to construct lower predictive bounds (LPBs) on the time-to-event and to estimate evaluation metrics such as the jailbreak rate on a fixed benchmark. Although HARP induces dependence in acquisition decisions across different trajectories, we prove that HARP never exceeds the target budget, that its LPBs have finite-sample coverage guarantees, and that its metric estimates are unbiased. Experiments on agentic task success, LLM jailbreaks, toxic content generation, and RAG hallucinations show that HARP achieves coverage close to the nominal level with low variance, while never exceeding the given budget.

Momentum

Related topics

Latest papers 175