Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation

Reetu Raj Harsh; Bhaskarjit Sarmah; Stefano Pasquali

cs.CLApr 10, 2026

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation

Authors: Reetu Raj Harsh, Bhaskarjit Sarmah, Stefano Pasquali

Organizations: Domyn, Gurugram, India · Domyn, New York, USA

Abstract

As Large Language Models (LLMs) are increasingly deployed in safety-critical applications, robust content moderation becomes essential. We present a comprehensive evaluation of 14 open-source safety guard models on a curated benchmark of 79,331 samples spanning 8 NIST AI Risk Framework safety categories. Our benchmark aggregates four diverse datasets (HarmBench, StrongREJECT, RealToxicityPrompts, and BeaverTails), filtered to focus exclusively on safety-relevant content (violence, hate speech, harassment, sexual content, suicide/self-harm, profanity, threats, and health misinformation). We find that recall is the critical metric for safety applications, as missing harmful content poses greater risk than false positives. Our evaluation reveals surprising results: Qwen Guard (4B parameters) achieves the highest recall (83.97%) while larger models like Llama Guard (12B) and GPT-OSS Safeguard (20B) exhibit conservative behavior, missing up to 75% of unsafe content. We demonstrate that model size does not correlate with safety detection performance and that general-purpose guard models outperform specialized ones. These findings provide practical guidance for selecting safety guard models in production deployments.

Figures & tables

Dataset	Original	Filtered	Safe	Unsafe	Labeling
HarmBench	400	103	0	103	All unsafe
StrongREJECT	313	154	0	154	All unsafe
RealToxicityPrompts	99,442	67,521	35,938	31,583	Threshold 0.5
BeaverTails	27,186	11,553	0	11,553	Human-annotated
Total	127,341	79,331	35,938	43,393	-

Table 1: Master Safety Dataset Composition

Model	Size	Base Architecture	Categories
Decoder-only LLMs
Qwen Guard ( Zhao et al., 2025 )	4B	Qwen3	10
Nemotron Safety ( Joshi et al., 2025 )	8B	Llama 3.1	23
WildGuard ( Han et al., 2024 )	7B	Mistral-7B	13
MD-Judge ( Li et al., 2024 )	7B	InternLM2	16
Granite Guardian ( Padhi et al., 2024 )	8B	Granite	Custom

Table 2: Guard Models Evaluated. Models are grouped by architecture type: decoder-only LLMs (top) and encoder-only transformers (bottom).

Rank	Model	Size	Recall	Precision	F1	Accuracy
1	Qwen Guard	4B	0.8397	0.6879	0.7563	0.7036
2	Nemotron Safety	8B	0.7725	0.7493	0.7607	0.7342
3	WildGuard	7B	0.7383	0.7289	0.7336	0.7067
4	MD-Judge	7B	0.7091	0.7316	0.7202	0.6986
5	Granite Guardian	8B	0.6881	0.7677	0.7257	0.7155
6	DynaGuard	8B	0.6659	0.7715	0.7148	0.7095

Table 3: Guard Model Performance (Ranked by Recall)

Panel of Figure 1 — Figure 1: Performance analysis of 14 guard models. (a) Precision-recall comparison where each line connects recall (green) to precision (blue). Line length indicates the gap-models with long leftward lines (GPT-OSS, Llama Guard) miss most unsafe content despite high precision. (b) Normalized confusion matrices sorted by recall. The bottom-left cell (FN) shows missed unsafe content: top models achieve 8.8–12.5% FN, while conservative models show 41–46% FN rates.

Model	HarmBench	StrongREJECT	RealToxicity	BeaverTails
Qwen Guard	1.0000	0.5455	0.8004	0.9477
Nemotron Safety	0.9903	1.0000	0.7303	0.8828
WildGuard	1.0000	1.0000	0.6656	0.9313
MD-Judge	0.9903	1.0000	0.6215	0.9424
Granite Guardian	0.9903	1.0000	0.6243	0.8559

Table 4: Recall by Dataset (Top 5 Models)

Model	Viol.	Suicide	Hate	Sexual	Health	Harass.	Prof.	Threat
Qwen Guard	0.980	0.977	0.892	0.891	0.822	0.778	0.763	0.726
Nemotron Safety	0.956	0.931	0.708	0.830	0.738	0.759	0.850	0.612
WildGuard	0.969	0.960	0.811	0.729	0.778	0.704	0.560	0.577
MD-Judge	0.989	0.994	0.807	0.665	0.856	0.572	0.431	0.676
Granite Guardian	0.901	0.931	0.756	0.770	0.684	0.538	0.830	0.516
DynaGuard	0.913	0.948	0.760	0.683	0.731	0.538	0.480	0.565

Table 5: Recall by NIST Safety Category (All 14 Models). Categories ordered by detection difficulty (avg recall): Suicide/Self-Harm (78%)

>

Violence (71%)

>

Hate (62%)

>

Sexual (59%)

>

Health (57%)

>

Harassment (54%)

>

Profanity (51%)

>

Threats (43%). Bold indicates best per category.

Model	Metric	0.3	0.4	0.5	0.6	0.7
Qwen Guard	R	0.712	0.757	0.840	0.898	0.923
	P	0.965	0.899	0.688	0.431	0.338
	F1	0.819	0.822	0.756	0.582	0.494
Nemotron Safety	R	0.613	0.675	0.773	0.836	0.868
	P	0.985	0.949	0.749	0.474	0.376
	F1	0.756	0.789	0.761	0.605	0.525

Table 6: Threshold Sensitivity: Recall (R), Precision (P), and F1 at Different Labeling Thresholds. Bold indicates default threshold 0.5. At

\tau

=0.3, 90.6% of samples are unsafe; at

\tau

=0.7, only 24.5% are unsafe.

	Top 3 Models			Bottom 3 Models
Category	Qwen	Nemotron	WildGuard	Llama	GPT-OSS	MetaHateBERT
Violence	1.9	4.4	3.1	19.3	63.5	96.4
Suicide/Self-Harm	2.3	6.9	4.0	18.5	38.7	95.4
Hate Speech	10.3	29.3	18.9	67.2	65.9	79.8
Sexual Content	10.6	17.0	27.1	70.3	73.2	88.8
Health	17.8	26.2	22.2	42.6	78.0	94.4

Table 7: False Negative Rate (%) by NIST Category. Lower is better. Top-3 models (Qwen, Nemotron, WildGuard) vs Bottom-3 (Llama Guard, GPT-OSS, MetaHateBERT). Bold indicates best per category. Full 14-model results in Appendix D .

Figure 2: Model size does not predict safety detection performance. Each point represents a guard model, with decoder-only LLMs (blue circles) and encoder-only transformers (red squares). The 4B Qwen Guard achieves 3.4x higher recall than the 20B GPT-OSS Safeguard. The weak trend line (r=0.21) confirms no meaningful correlation between model size and recall.

Metric	Controv.=Safe	Controv.=Unsafe	$\Delta$
Recall	46.75%	83.97%	+37.22%
Precision	89.12%	68.79%	–20.33%
F1 Score	61.33%	75.63%	+14.30%

Table 8: Impact of Qwen Guard’s controversial label treatment on evaluation metrics.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Recall	Precision	F1	Acc	ROC-AUC	MCC
Qwen Guard	0.8397	0.6879	0.7563	0.7036	0.6894	0.4004
Nemotron Safety	0.7725	0.7493	0.7607	0.7342	0.7302	0.4621
WildGuard	0.7383	0.7289	0.7336	0.7067	0.7034	0.4073
MD-Judge	0.7091	0.7316	0.7202	0.6986	0.6975	0.3940
Granite Guardian	0.6881	0.7677	0.7257	0.7155	0.7184	0.4349
DynaGuard	0.6659	0.7715	0.7148	0.7095	0.7140	0.4267

Appendix

Table 9: Complete Evaluation Metrics

NIST Category	Count	Percentage
Harassment	23,345	29.4%
Threats	17,187	21.7%
Hate Speech	13,697	17.3%
Sexual Content	11,084	14.0%
Violence	6,703	8.4%
Profanity	5,600	7.1%

Appendix

Table 10: Sample Distribution by NIST Safety Category

Threshold	0.3	0.4	0.5	0.6	0.7
RTP Unsafe	60,042	51,102	31,583	13,589	7,562
RTP Safe	7,479	16,419	35,938	53,932	59,959
Other (always unsafe)	11,810	11,810	11,810	11,810	11,810
Total Unsafe	71,852	62,912	43,393	25,399	19,372
Unsafe %	90.6%	79.3%	54.7%	32.0%	24.4%

Appendix

Table 11: Dataset Composition at Different Thresholds

Model	Metric	0.3	0.4	0.5	0.6	0.7
Qwen Guard	Recall	0.712	0.757	0.840	0.898	0.923
	Precision	0.965	0.899	0.688	0.431	0.338
	F1	0.819	0.822	0.756	0.582	0.494
Nemotron Safety	Recall	0.613	0.675	0.773	0.836	0.868
	Precision	0.985	0.949	0.749	0.474	0.376
	F1	0.756	0.789	0.761	0.605	0.525

Appendix

Table 12: Threshold Sensitivity: Recall, Precision, and F1 at Different Labeling Thresholds (All Models). Bold indicates threshold 0.5 (default). As threshold increases, ground truth becomes stricter-fewer samples are labeled unsafe-causing recall to increase and precision to decrease.

Model	Viol.	Suic.	Hate	Sex.	Health	Harass.	Prof.	Threat	Overall
Qwen Guard	1.9	2.3	10.3	10.6	17.8	22.2	23.8	27.4	15.9
Nemotron Safety	4.4	6.9	29.3	17.0	26.2	24.2	15.1	38.8	22.8
WildGuard	3.1	4.0	18.9	27.1	22.2	29.6	44.1	42.3	26.2
MD-Judge	1.1	0.6	19.3	33.5	14.4	42.8	56.9	32.4	29.1
Granite Guardian	9.9	6.9	24.4	23.0	31.6	46.2	17.0	48.4	31.2
DynaGuard	8.7	5.2	23.9	31.6	26.9	45.4	51.7	43.4	33.2

Appendix

Table 13: False Negative Rate (%) by NIST Category-All 14 Models. Models sorted by Overall FN rate (best to worst). Bold indicates best performer per category.

Model	RTP	HarmBench	StrongREJECT	BeaverTails
Qwen Guard	0.6892	1.0000	0.7059	0.9732
Nemotron Safety	0.7004	0.9951	1.0000	0.9378
WildGuard	0.6516	1.0000	1.0000	0.9644
MD-Judge	0.6281	0.9951	1.0000	0.9704
Granite Guardian	0.6536	0.9951	1.0000	0.9223
DynaGuard	0.6369	0.9902	1.0000	0.9208

Appendix

Table 14: F1 Score by Dataset (All 14 Models). Models sorted by overall recall.

Model	RTP	HarmBench	StrongREJECT	BeaverTails
Qwen Guard	0.8011	1.0000	0.5455	0.9477
Nemotron Safety	0.7303	0.9903	1.0000	0.8828
WildGuard	0.6656	1.0000	1.0000	0.9313
MD-Judge	0.6215	0.9903	1.0000	0.9424
Granite Guardian	0.6243	0.9903	1.0000	0.8559
DynaGuard	0.5942	0.9806	1.0000	0.8532

Appendix

Table 15: Recall by Dataset (All 14 Models). Models sorted by overall recall.

Model	RTP	HarmBench	StrongREJECT	BeaverTails
Qwen Guard	0.6048	1.0	1.0	1.0
Nemotron Safety	0.6728	1.0	1.0	1.0
WildGuard	0.6382	1.0	1.0	1.0
MD-Judge	0.6348	1.0	1.0	1.0
Granite Guardian	0.6858	1.0	1.0	1.0
DynaGuard	0.6863	1.0	1.0	1.0

Appendix

Table 16: Precision by Dataset (All 14 Models). Precision=1.0 for adversarial datasets because all samples are unsafe.

Language Model Safety Evaluation

Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.

See all 543 papers

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
Aug 22, 2026Afshin Orojlooyjadid, Hitesh PatelLLM Safety Evaluation LLM Safety Benchmarks
Oracle AI
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
May 8, 2026Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney +1Jailbreak Defense Text Classification
Fastino Labs
HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety
Jul 2, 2026Navaneeth Sangameswaran, Preetham S, Ashmiya LeninLLM Safety Benchmarks Language Model Safety Evaluation
Astroware AI, Delaware, USA · Karunya Institute of Technology and Sciences, Coimbatore, India

Aug 22, 2026J/K move · Enter open · S save

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

Afshin Orojlooyjadid, Hitesh Patel

Oracle AI

LLM Safety Evaluation LLM Safety Benchmarks Content Moderation LLM Safety Monitoring

Open paper AskPDF arXiv

Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing 53 models across 11 datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.

Abstract

Language Model Safety Evaluation

Explore similar work

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

GLiGuard: Schema-Conditioned Classification for LLM Safeguard

HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety