LLM Guardrails

LLM: Large Language Model

LLM guardrails assess prompts, generated responses, and multi-step agent actions for harmful content, policy violations, prompt injection, and unsafe tool use. Common approaches include specialized classifiers and LLM judges, hidden-state or logit probes, streaming monitors, and runtime controls such as policy-as-code, constrained decoding, and formal verification. Evaluations span moderation and jailbreak benchmarks such as HarmBench, agent-task settings such as AgentDojo and τ²-Bench, and domain-specific, multilingual, and long-context tests. The field’s central tension is balancing missed risks against over-refusal, latency, and cost, while determining whether strong benchmark scores translate into safer end-to-end systems.

Latest papers

195 papers30 papers in the last four weeks, up 43% on the four weeks before. 0.2% of all new papers.

SafeHarbor: Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
May 7, 2026Zhe Liu, Zonghao Ying, Wenxin Zhang +5LLM Guardrails LLM Agent Security
School of Cyber Science and Technology, Beihang University, Beijing, China · Institute of Artificial Intelligence, Beihang University, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +1
One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
May 7, 2026Xinjie Shen, Rongzhe Wei, Peizhi Niu +6Language Model Safety Evaluation LLM Guardrails
Georgia Institute of Technology · University of Illinois Urbana-Champaign · UCSD +3
Who Decides What Is Harmful? Content Moderation Policy Through A Multi-Agent Personalised Inference Framework
May 2, 2026Ewelina Gajewska, Michal Wawer, Katarzyna Budzynska +1LLM Guardrails Content Moderation
Warsaw University of Technology, Warsaw, Poland
BabelSafe: A Policy-Grounded Multilingual Safety Benchmark for LLMs
May 1, 2026Yunhan Zhao, Zhaorun Chen, Xingjun Ma +1LLM Safety Benchmarks Multilingual Language Models
University of Illinois Urbana-Champaign · Fudan University · University of Chicago
Safeguarding LLM Agents from Misalignment through Provenance Analysis
May 1, 2026Yining She, Yiliang Liang, Eunsuk KangLLM Guardrails LLM Agent Safety
Carnegie Mellon University Pittsburgh, PA, USA
Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype
Apr 30, 2026Matthew Christian AgustinLanguage Model Safety Evaluation LLM Guardrails
Responsible Innovation Lab Tempe, Arizona, USA
BARRED: Synthetic Training of Custom Policy Guardrails via Asymmetric Debate
Apr 28, 2026Arnon Mazza, Elad LeviLLM Guardrails LLM Fine-Tuning
Plurai Inc.
A Comparative Evaluation of AI Agent Security Guardrails
Apr 27, 2026Qi Li, Jiu Li, Pingtao Wei +8LLM Guardrails AI Agent Security
Beijing Caizhi Tech, Beijing, China
UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks
Apr 25, 2026Tianlong Yu, Yang Yang, Xiao Luo +6Privacy-Preserving Language Models LLM Guardrails
Hubei University, Wuhan, China · University of Southern California, Los Angeles, USA · Huazhong University of Science and Technology, Wuhan, China
Shift-Up: A Framework for Software Engineering Guardrails in AI-native Software Development -- Initial Findings
Apr 22, 2026Petrus Lipsanen, Liisa Rannikko, François Christophe +3Software Engineering Agents LLM Guardrails
University of Jyväskylä Jyväskylä, Finland
SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs
Apr 22, 2026Chao Pan, Yu Wu, Xin YaoLLM Guardrails LLM Safety Alignment
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China · The Hong Kong Polytechnic University, Hong Kong, China · George Washington University +1
LLM Safety From Within: Detecting Harmful Content with Internal Representations
Apr 20, 2026Difan Jiao, Yilun Liu, Ye Yuan +4Language Model Safety Evaluation LLM Guardrails
University of Toronto · LMU Munich · McGill University
Harness as an Asset: Enforcing Determinism via the Convergent AI Agent Framework (CAAF)
Apr 18, 2026Tianbao ZhangSafety-Critical Control LLM Guardrails
Independent Researcher
TWGuard: A Case Study of LLM Safety Guardrails for Localized Linguistic Contexts
Apr 17, 2026Hua-Rong Chu, Kuan-Chun Wang, Yao-Te HuangLanguage Model Safety Evaluation LLM Guardrails
Chunghwa Telecom Laboratories
Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
Apr 16, 2026Yining Hong, Yining She, Eunsuk Kang +2LLM Guardrails AI Agent Security
Carnegie Mellon University
Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side Execution
Apr 16, 2026Sarmad Sohail, Ghufran HaiderLLM Guardrails AI Agent Security
Model Capability Assessment and Safeguards for Biological Weaponization
Apr 15, 2026Michael RichterResponsible AI LLM Safety Evaluation
Binghamton University
ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
Apr 13, 2026Daeyeon SonAI Safety LLM Guardrails
Independent Researcher Republic of Korea
Guardrails Beat Guidance: A Large-Scale Study of Rules, Skills, and Persistent Configuration for Coding Agents
Apr 13, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +4Software Engineering Agents LLM Agent Reliability
AWS Generative AI Innovation Center · HSBC Holdings Plc., HSBC Technology Center, China.
Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
Apr 10, 2026Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano PasqualiLanguage Model Safety Evaluation LLM Safety Benchmarks
Domyn, Gurugram, India · Domyn, New York, USA
Silencing the Guardrails: Inference-Time Jailbreaking via Dynamic Contextual Representation Ablation
Apr 9, 2026Wenpeng Xing, Moran Fang, Guangtai Wang +2Adversarial Attacks on LLMs Jailbreak Attacks
Zhejiang University · Binjiang Institute of Zhejiang University
Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs
Apr 8, 2026Yue Huang, Haomin Zhuang, Jiayi Ye +6LLM Guardrails LLM Safety Alignment
University of Notre Dame · University of California, Los Angeles · MIT-IBM Watson AI Lab +1
TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories
Apr 8, 2026Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang +1Language Model Safety Evaluation LLM Guardrails
CyCraft AI Lab, Taiwan · National Taiwan University
When Safety Geometry Collapses: Fine-Tuning Vulnerabilities in Agentic Guard Models
Apr 8, 2026Ismail Hossain, Sai Puppala, Jannatul Ferdaus +4LLM Guardrails LLM Safety Alignment
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · Hanyang University, Seoul, South Korea +1
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety
Apr 7, 2026Changxuan Fan, Xi Yang, Yueyuan Zheng +9LLM Safety Evaluation LLM Safety Benchmarks
The Hong Kong University of Science and Technology
CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMs
Apr 7, 2026Elham Nasarian, Abhilash Neog, Kwok-Leung Tsui +1LLM Guardrails LLM Safety
Grado Department of Industrial & Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA · Department of Computer Science, Virginia Tech, Blacksburg, VA 24061, USA · Dept of Industrial, Manufacturing, and Systems Engineering at University of Texas at Arlington, Arlington, TX 76019, USA
Pre-Execution Safety Gate & Task Safety Contracts for LLM-Controlled Robot Systems
Apr 7, 2026Ike Obi, Vishnunandan L. N. Venkatesh, Weizheng Wang +5Robot Safety LLM Guardrails
Gradient-Controlled Decoding: A Safety Guardrail for LLMs with Dual-Anchor Steering
Apr 6, 2026Purva Chiniya, Kevin Scaria, Sagar ChaturvediLLM Guardrails LLM Safety
Amazon Alexa · Amazon AGI
Predict, Don't React: Value-Based Safety Forecasting for LLM Streaming
Apr 5, 2026Pride Kavumba, Koki Wataoka, Huy H. Nguyen +2LLM Guardrails LLM Safety Monitoring
SB Intuitions Corp.
Type-Checked Compliance: Deterministic Guardrails for Agentic Financial Systems Using Lean 4 Theorem Proving
Apr 1, 2026Devakh Rashie, Veda RashiLLM Guardrails AI Regulatory Compliance
Independent Researcher Centreville, Virginia, USA · Thomas Jefferson High School for Science and Technology Alexandria, VA, USA

May 7, 2026J/K move · Enter open · S save

SafeHarbor: Hierarchical Memory-Augmented Guardrail for LLM Agent Safety

Zhe Liu, Zonghao Ying, Wenxin Zhang, Quanchen Zou+4

School of Cyber Science and Technology, Beihang University, Beijing, China · Institute of Artificial Intelligence, Beihang University, Beijing, China · University of Chinese Academy of Sciences, Beijing, China+1

LLM Guardrails LLM Agent Security LLM Agent Safety Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Recent advances in foundation models have transformed LLMs from passive conversational systems into autonomous agents capable of reasoning and tool execution. While these capabilities unlock substantial practical value, they also introduce new security risks, as adversaries can manipulate agents into performing harmful actions in real-world environments. Existing defense strategies mitigate such threats but frequently struggle to balance safety and utility, resulting in over-refusal of benign user requests. To mitigate this trade-off, we propose SafeHarbor, a novel framework designed to establish precise decision boundaries for LLM agents. Unlike static guidelines, SafeHarbor extracts context-aware defense rules through enhanced adversarial generation. We design a local hierarchical memory system for dynamic rule injection, offering a training-free, efficient, and plug-and-play solution. Furthermore, we introduce an information entropy-based self-evolution mechanism that continuously optimizes the memory structure through dynamic node splitting and merging. Extensive experiments demonstrate that SafeHarbor achieves state-of-the-art performance on both ambiguous benign tasks and explicit malicious attacks, notably attaining a peak benign utility of 63.6% on GPT-4o while maintaining a robust refusal rate exceeding 93% against harmful requests. The source code is publicly available at https://github.com/ljj-cyber/SafeHarbor.