cs.AINov 16, 2026

Jev-as-a-Reward: Validating Probabilistic Safety Judges for Reinforcement Learning

Authors: Eunkyeong Lee, Junhyung Moon, Seunggwan Hong, Dongsuk Oh

Organizations: KT · Department of AI Systems Engineering, Sungkyunkwan University · Kyungpook National University

Abstract

Probabilistic safety judges return the probability of a safety label and cheaply score reinforcement learning rollouts. A confidence gate sends only uncertain verdicts to a stronger LLM judge, which leaves confidently passed harm in a blind spot. Policies can exploit this blind spot, and commercial judges release new versions. We propose a protocol that validates such a judge before and during training, separates the share of confidently passed harm from its amount, and re-validates it after each judge change. We apply it to two versions of Jev, a commercial decision-only judge, and to an open guard classifier. Under the probability reward of Jev, passed harm shrinks in all 11 runs under the primary reference, mostly more slowly than caught harm, and concentrates in the blind spot. Binarizing this judge meets the escape criterion on Llama, and the open guard classifier shows an escape signal in one seed under a second reference. Guard classifiers share this blind spot, and even frontier LLM judges catch at most one of 10 human-labeled confident misses. Without a trained reward model, the probability reward with a helpfulness term lowers HarmBench attack success on Llama-3.1-8B from 0.287 to at most 0.033 at the final checkpoint.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Are LLM Safety Judges Policy-Invariant? A Three-Principle Stress-Test

    May 7, 2026Shihao Weng, Yang Feng, Xiaofei XieLLM-as-a-JudgeAI Agent Safety

  2. Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

    Aug 18, 2026Sher Badshah, Ali Emami, Hassan SajjadLLM-as-a-JudgeSelective Prediction

  3. Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators

    Jun 5, 2026Anissa Alloula, Federico Licini, Ava Batchkala +1LLM-as-a-JudgeIn-Context Learning