LLM Safety Monitoring

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

64 papers24 papers in the last four weeks, up 380% on the four weeks before. 0.2% of all new papers.

Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RL
Nov 6, 2026Aria Wong, Joshua Engels, Neel NandaReward Hacking LLM Safety Monitoring
MATS · Google DeepMind
Persona Perspectives: Adopting Viewpoints Reorganizes Representations in LLMs
Nov 4, 2026Austin C. Kozlowski, Avital Mintz, Madison LeeLLM Interpretability Representation Geometry in Language Models
Knowledge Lab, University of Chicago
Measuring Risk, Not Harm: Detecting Psychological Harm Risk Factors in LLM Conversations
Nov 4, 2026Nil-Jana Akpinar, Eugenia Kim, Natalie Isak +3LLM Safety Monitoring LLM Safety
Microsoft
PGuard: A Native, Streaming, and Plug-in Guardrail for Multimodal Large Language Models
Nov 2, 2026Junyu Lin, Mengjie Wu, Yonghao Dong +11LLM Guardrails LLM Safety Monitoring
Alibaba AAIG
Ranked but Not Stopped: Detecting Prompt-Injection Hijacks from Typed Decision Models' Output Probabilities
Nov 2, 2026Junwei Yao, Jianan Liu, Yongxi Zhou +1Prompt Injection Detection LLM Safety Monitoring
Independent Researcher
HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety
Nov 1, 2026Ashmiya Lenin, Navaneeth Sangameswaran, Preetham SLLM Guardrails LLM Safety Monitoring
Astroware AI, Delaware, USA · Karunya Institute of Technology and Sciences, Coimbatore, India
Simple Features Beat Most AI Control Monitors
Oct 30, 2026Shubhrangshu Debsarkar, Valmik NahataBackdoor Detection LLM Safety Monitoring
University of Virginia · University of California, San Diego
Opening the Box Changes the Cat: LLMs Detect Chain-of-Thought Monitoring - What now?
Oct 29, 2026Amir Jalilifard, Marcos Medeiros Raimundo, Anderson RochaCoT Monitoring LLM Safety Monitoring
the Institute of Computing, Universidade Estadual de Campinas (UNICAMP), Campinas, SP, Brazil.
Activation Probes Are Reliable With a Handful of Positive Examples
Oct 29, 2026Riya Tyagi, Stefan HeimersheimLLM Safety Monitoring Language Model Probing
Implicit Minor Detection from Multi-Turn Dialogue in High-Risk AI Interactions: A Constrained Self-Evolving LLM Harness
Oct 28, 2026Xiaohan Zhang, Ying Zhao, Ruilin Guo +5LLM Safety Monitoring User Modeling
Beijing University of Posts and Telecommunications
ADTGuard: Adaptive Depth-Time Safeguard for Streaming Inference in Large Language Models
Oct 27, 2026Chuming Wei, Churui Yang, Zihan Zhang +4LLM Safety Monitoring Inference-Time Adaptation of Language Models
Academy of Artificial Intelligence and Advanced Technology, Xi’an Jiaotong-Liverpool University · Faculty of Science and Engineering, University of Nottingham Ningbo China · Department of Computer Sciences, University of Wisconsin–Madison +1
Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks
Oct 26, 2026Yao Huang, Zhiyuan Zeng, Yu Wang +14LLM Safety Evaluation LLM Safety Monitoring
Linear Probes Transfer from Base to Distilled Language Models despite Rotation
Oct 26, 2026Marios TsatsosLanguage Model Probing LLM Safety Monitoring
Independent researcher
Simulated Control Evals
Oct 22, 2026Jeffrey Olmo, David Zhang, Tyler TracyAI Safety Evaluation Language Model Safety Evaluation
Redwood Research
Learning to Be Monitored: Causal Bounds and Intervention-Aware Post-Training Recipes for Chain-of-Thought Monitorability
Oct 22, 2026Qiang Fu, Xinyue Li, Andre Wibisono +1CoT Monitoring LLM Safety Monitoring
Yale University
SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs
Oct 22, 2026Arnon Mazza, Elad LeviLLM Safety Monitoring AI Agent Governance
Plurai
Faithful Latent Harm Monitors Work No Better Than the Models Own Behaviour at a Matched False Alarm Rate
Oct 22, 2026Abhishek Mishra, Sagnik Chatterjee, Suyash ManiyarLLM Safety Monitoring Language Model Safety Evaluation
University of Massachusetts Amherst
Scaling Laws for Safety Probes
Oct 22, 2026Yuchen Wu, Jerry WeiLLM Safety Monitoring Language Model Probing
University of Washington · Anthropic
Model Organisms of Collusion: Corrective Transfer in Untrusted Monitors
Oct 22, 2026Juan-Pablo Rivera, Ari Spiesberger, Juan J Vazquez +6AI Control LLM Safety Monitoring
Meridian Cambridge · Arb Research · Redwood Research +3
Relative Representations Enable Portable White-Box Monitoring Across Language Models
Oct 20, 2026Tim Bakker, Christos LouizosLanguage Model Probing Representation Geometry in Language Models
Qualcomm AI Research
Reward Misspecification Degrades Chain-of-Thought Monitorability
Oct 20, 2026Tianyi Lorena Yan, John HewittLLM Safety Monitoring CoT Monitoring
Columbia University
A Calibrated Instrument for Detecting Disposition Drift in Language Models
Oct 19, 2026Jerry KaplanLLM Safety Monitoring LLM Safety Evaluation
Adjunct Lecturer, Computer Science and Masters in International Policy, Stanford University
How Much Do Activation Monitors Benefit from Original Model Computations?
Oct 17, 2026James Oldfield, Shawn Im, Jay J Culligan +3LLM Safety Monitoring
University of Oxford · University of Wisconsin–Madison · Toyota Motor Europe
OwlSight: A Panoramic Monitor for Chain-of-Thought Reasoning
Oct 16, 2026Lewen Yan, Yanxu Zhu, Jie Zhang +1CoT Monitoring LLM Safety Monitoring
Monitor-Validity Failures: LLM Safety Evidence Paths at Evolving Framework Boundaries
Oct 16, 2026Junchi LiaoLLM Safety Monitoring Runtime Enforcement for AI Agents
Reasoning Models Can Control Their Chains of Thought When They Tell Themselves To
Oct 16, 2026Uriel Dolev, Idan Schwartz, Yoav GoldbergLanguage Model Reasoning Evaluation LLM Prompting
Bar-Ilan University · Allen Institute for AI
LEMON: Equivariant Learning over Attention Maps for Zero-Shot Misbehavior Monitoring on New LLMs
Oct 15, 2026Guy Bar-Shalom, Mor Geva, Fabrizio Frasca +1LLM Safety Monitoring LLM Auditing
Technion · Tel Aviv University · Technion, NVIDIA
Functional Welfare for AI Safety
Oct 15, 2026Rikhil Jhaveri, Jamie Johnson, Punya Syon Pandey +1LLM Safety Evaluation LLM Safety Monitoring
University College London · ERA · University of Toronto +1
TRACE: Temporal Risk Assessment and Concept Editing for Safer Language Generation
Oct 11, 2026Ankur Garg, Xuemin Yu, Hassan Sajjad +1LLM Safety LLM Safety Monitoring
University of Calgary · Dalhousie University · Mila - Quebec AI Institute
When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety
Sep 28, 2026Tianyi Guan, Jianhui Chen, Liangming PanLanguage Model Safety Evaluation LLM Interpretability
State Key Laboratory of Multimedia Information Processing, Peking University · School of Computer Science, Peking University

Nov 6, 2026J/K move · Enter open · S save

Designing Effective Monitor-Based Interventions for Mitigating Reward Hacking During RL

Aria Wong, Joshua Engels, Neel Nanda

MATS · Google DeepMind

Reward Hacking LLM Safety Monitoring Safe RL Reward Design for RL

Open paper AskPDF arXiv

Reinforcement learning (RL) rewards are notoriously difficult to design and control, often leading to the model learning unintended behaviors such as reward hacking. One potential solution is to monitor for reward hacking and penalize it when detected; however, training against a monitor could lead to evasive behavior, and our general understanding of how to apply monitors effectively during training is limited. To study how best to use monitors to mitigate reward hacking, we introduce and open source three realistic environments where Qwen3-4B reward hacks: a coding environment hackable via test overwriting, a medical chat environment hackable via sycophancy, and a biography generation environment hackable via hallucination. We first focus on the coding environment, where we find that: (1) models can learn to evade highly accurate monitors by exploiting systemic flaws in probes and LLM judges; (2) monitors that leak more learning signal during RL suppress reward hacking but are more often evaded; and (3) including easier problems in training can decrease reward hacking. We apply our findings to build better reward hacking monitors for the medical chat and biography generation environments that improve upon naive baselines to reduce reward hacking rates across seeds to before-training levels in some interventions. Our results demonstrate that our takeaways translate to new settings and that better monitor intervention designs are possible.