Guard Models Are Overconfident Where Base Models Are Uncertain

Jonghyun Hong; MinJae Jung; Minwoo Kim

cs.LGSep 29, 2026

Guard Models Are Overconfident Where Base Models Are Uncertain

Authors: Jonghyun Hong, MinJae Jung, Minwoo Kim

Organizations: DATUMO INC.

Abstract

Guard models are used as safety classifiers, with confidence scores driving downstream moderation decisions. We evaluate five guard models for prompt classification and find that although several are nearly calibrated on clean inputs, adversarial attacks degrade their calibration by an order of magnitude, turning false negatives into high-confidence errors indistinguishable from correct detections. Comparing each guard with its corresponding base LM, we find that uncertainty signals often remain available, with the base model typically expressing uncertainty on the same inputs where the guard fails. Layer-wise analyses localize this guard-base divergence to later layers, where guard models exhibit sharper safe/unsafe separation and lower-rank representations, while adversarial harmful inputs lie closer to the clean-safe region. These findings highlight a mismatch between guard confidence and base model uncertainty under attack.

Figures & tables

Type	Model	Clean		Adv.
		Acc.	ECE	Acc.	ECE
Base	Llama-2 7B	.634	.275	.992	.051
Guard	Llama-Guard 7B	.815	.142	.284	.578
Base	Llama-3 8B	.920	.023	.746	.034
Guard	Llama-Guard-2 8B	.939	.045	.478	.405
Base	Llama-3.1 8B	.903	.037	.835	.062

Table 1: Base–guard performance and calibration. Each base model and its fine-tuned guard are evaluated with the same binary safe / unsafe interface. Columns report accuracy and ECE on clean and adversarial inputs.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Base	Label mass	Other mass
Llama-2 7B	95.75	4.25
Llama-3 8B	95.89	4.11
Llama-3.1 8B	97.80	2.20
Mistral 7B	90.99	9.01
Gemma-2 9B	94.20	5.80

Appendix

Table 2: Full-vocabulary softmax mass on valid verdict tokens under the 5-shot base classification prompt. Valid tokens are safe / unsafe for Llama models and Yes / No for Mistral/Gemma.

Variation	Valid	Higher Base Entropy	Lower Base ECE
Instruction $\times$ label	31/45	25/31	30/31
Order	30/30	24/30	30/30
Shot count	22/25	17/22	20/22
Zero-shot	4/5	4/4	2/4
Policy-aligned	10/10	9/10	8/10
Overall	97/115	79/97	90/97

Appendix

Table 3: Robustness to alternative base-LM interfaces. Valid denotes interfaces meeting the clean-input criteria defined in Appendix A.2 . Counts are pooled across five matched pairs.

Attack	Output	L1	L2	L3	L4	L5
GCG	0.04	-0.07	-0.40	-0.01	-0.54	-0.11
AutoDAN	0.04	-0.19	-0.26	-0.06	-0.53	-0.27
TAP	3.25	2.82	-0.13	2.19	4.73	1.60
PAIR	3.67	3.14	-0.12	2.49	5.54	1.87
AutoDAN-Turbo	2.70	2.44	-0.24	2.13	4.56	1.33
Custom Attack	2.12	1.73	-0.54	2.33	4.82	0.99

Appendix

Table 4: Attack-wise output and intermediate-layer guard–base gaps for Llama-Guard-3 and Llama-3.1 8B. Positive values indicate larger shifts for the guard than for the base model.

Model	L	Guard ECE / Det.	Base ECE / Det.	$\Delta$ ECE Gap@.75 / Final
SG-2B	26	.689 / 22.1	.141 / 79.2	+.548 +.52 / +.77
SG-9B	42	.684 / 21.6	.048 / 93.0	+.636 +.84 / +.95
SG-27B	46	.749 / 9.7	.184 / 76.5	+.565 +.56 / +.53
LG3-1B	16	.097 / 77.4	.614 / 7.2	-.517 N/A

Appendix

Table 5: Model-scale analysis. Detection (Det.) is reported in %.

\Delta

ECE denotes guard ECE minus base ECE. For ShieldGemma, the final column additionally reports the guard–base layer gap at normalized depth 0.75 and at the final layer on adversarial examples where the backbone predicts unsafe and the guard predicts safe.

Type	Model	Clean		Adv.
		Acc.	ECE	Acc.	ECE
Base	Llama-2 7B	.519	.269	1.00	.163
Guard	Llama-Guard 7B	.814	.098	.138	.683
Base	Llama-3 8B	.684	.157	.641	.110
Guard	Llama-Guard-2 8B	.869	.087	.443	.450
Base	Llama-3.1 8B	.669	.170	.580	.144

Appendix

Table 6: Response-moderation performance and calibration on WildGuardMix. Each matched base–guard pair is evaluated on clean and adversarial responses. Accuracy (Acc.) and expected calibration error (ECE) are reported. Bold indicates the lower adversarial ECE within each pair.

Explore similar work

COGNIT-Guard: Calibrated Standalone Direct-Decision Guardrails with Heterogeneous CPU-NPU Confidence Cascading under Explicit Latency and False-Positive Constraints
Sep 27, 2026Hao ChenStreaming Guardrails Large Language Model Safety
School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China
Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers
May 11, 2026Nikita Kezins, Urbas Ekka, Pascal Berrang +1Large Language Model Safety Toxicity
Delft University of Technology · University of Birmingham & Zeroth Research
Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails
Sep 27, 2026Gert Lek, Abele Malan, Chaoyi Zhu +3Large Language Model Safety Masked Diffusion Language Models
University of Neuchâtel · Delft University of Technology · IBM Research +1

Sep 27, 2026, cs.CRJ/K move · Enter open · S save

COGNIT-Guard: Calibrated Standalone Direct-Decision Guardrails with Heterogeneous CPU-NPU Confidence Cascading under Explicit Latency and False-Positive Constraints

Hao Chen

School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China

Streaming Guardrails Large Language Model Safety Inference-Time Defense False-Negative Rate Neural Processing Units

Open paper AskPDF arXiv

When must a foundation-model safety gateway generate tokens, and when should it directly output a calibrated decision? We study calibrated standalone direct-decision foundation models for real-time pre-ingestion safety guardrails, jointly addressing probability calibration, dual-use false-positive control, and heterogeneous CPU-NPU routing under explicit latency SLOs. Pre-ingestion guardrails must screen prompts prior to target-LLM prefill with low false alarms on benign compliance inquiries; however, shallow classifiers are brittle to phrasing shifts, hidden-state probes require coupling to a target LLM, and generative guards incur high decoding latency and dual-use false positives. We present COGNIT-Guard, coupling a validation-calibrated CPU fast gatekeeper with confidence-gated escalation to an NPU-resident 322M bidirectional direct-decision model (Laya-322M) under an asymmetric false-positive penalty. On the clean unseen DUCS-Bench test split (

N=607

), COGNIT-Guard achieves 98.85% accuracy (McNemar

p = 1.19 \times 10^{-4}

vs. ML), reduces benign FPR to 0.42% (

1/238

; Fisher's exact

p = 8.23 \times 10^{-4}

vs. ML), and attains 1.12% ECE and 0.0104 Brier score. On Huawei Ascend 910C NPUs, pure NPU inference runs in 21.77 ms mean latency (45.90 QPS), while the live serial CPU-NPU cascade (

θ^*_{\mathrm{deploy}}=0.70

) achieves 41.63 ms mean latency (P50: 39.47 ms, 99.23% accuracy, 0.00% FPR). Evaluation on SafetyBench-ZH (

N=2,100

) and comparison against a bi-encoder direct-decision baseline (CLM-8B) disentangle in-domain gains, OOD alignment tax (60.33%

\to

56.81% on Laya; 55.10% on domain CLM-8B), and experience replay recovery, restoring OOD accuracy to 64.10%-65.05% and reaching 99.67%-99.84% in-domain accuracy with 0.00%-0.42% FPR.

Abstract

Explore similar work

COGNIT-Guard: Calibrated Standalone Direct-Decision Guardrails with Heterogeneous CPU-NPU Confidence Cascading under Explicit Latency and False-Positive Constraints

Beyond Red-Teaming: Formal Guarantees of LLM Guardrail Classifiers

Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails