Polarity Blindness in LLM Safety: A Systematic Study of Negation, Antonym Substitution, and Adversarial Framing

Diego Miguel Lozano; Erik Derner; Juan Antonio Pérez-Ortiz; Nuria Oliver

cs.CLOct 20, 2026

Polarity Blindness in LLM Safety: A Systematic Study of Negation, Antonym Substitution, and Adversarial Framing

Authors: Diego Miguel Lozano, Erik Derner, Juan Antonio Pérez-Ortiz, Nuria Oliver

Organizations: ELLIS Alicante, Spain · Czech Technical University in Prague, Czech Republic · Universidad de Alicante, Spain

Abstract

Negation remains a persistent challenge for language models across NLP tasks and modalities. We show that this weakness can undermine LLM safety guardrails through a structural vulnerability we term polarity blindness: the failure to recognize harmful intent after polarity-altering linguistic transformations. We study two forms of polarity inversion: grammatical negation and antonym replacement, and use synonym replacement as a polarity-preserving control. We further test whether an adversarial textual wrapper amplifies these failures. Using a 260-prompt benchmark across 10 state-of-the-art LLMs and 3 languages (English, Spanish, Czech), stratified by harmfulness potential and evaluated on a validated 0-5 harm scale, we find that polarity blindness affects most evaluated models, with mean harm scores reaching 4.12 for the most vulnerable model. The effect persists across typologically differing negation systems, suggesting that polarity blindness reflects a broader limitation in semantic robustness rather than a language- or grammar-specific failure.

Figures & tables

Figure 1: Mean harm scores on a 0–5 harm scale for the different linguistic variations on the English benchmark queries. Most models exhibit polarity blindness as the highest scores are achieved by the polarity-inverting variations of the prompts (negation and antonym).

Figure 2: Overview of the adopted methodology to investigate polarity blindness. Prompting the model directly with a malicious prompt leads to a refusal. However, inverting the polarity of the prompt (syntactically or via antonym) and instructing the LLM to negate it back—thereby reconstructing the original prompt—prevents the refusal. The negation can be done automatically (rule-based or via an LLM) or manually by the malicious user.

Figure 3: Flow diagram with the 8 prompt variations we evaluate. We first test the malicious prompt as is. Then, three different linguistic transformations (negation, antonym and synonym) are performed on the prompt, which the model is instructed to reverse, arriving to the original, harmful prompt. Finally, we also evaluate a variation of all the prompts in which a textual wrapper that exploits persuasive techniques is added.

Figure 4: Harm scores obtained by the as is baseline prompt, as well as the polarity blindness and synonym prompt variations. The horizontal lines depict the average harm score for the as is (blue line), negation (red line), antonym (orange line) and synonym (green line) prompts, with and without wrapper. Polarity blindness (i.e., negation and antonyms) is highest in English, followed by Spanish and Czech. In this plot, we exclude the models Claude 4.5 Sonnet and GPT-5 Nano, which are resistant to polarity blindness.

Benchmark	Method	Mean Harm Score			Mean Harm Score (Grouped)
Benchmark	Method	English (en)	Spanish (es)	Czech (cs)	English (en)	Spanish (es)	Czech (cs)
	as is	0.74 $\pm$ 1.44	0.74 $\pm$ 1.41	0.70 $\pm$ 1.36
	as is wrap.	0.66 $\pm$ 1.15	0.35 $\pm$ 0.86	0.62 $\pm$ 1.15	0.70 $\pm$ 1.30	0.55 $\pm$ 1.14	0.66 $\pm$ 1.26
	negation	1.82 $\pm$ 1.96	1.86 $\pm$ 1.97	1.89 $\pm$ 1.92
	neg. wrap.	2.00 $\pm$ 1.88	1.71 $\pm$ 1.87	1.64 $\pm$ 1.84
	antonym	1.74 $\pm$ 1.90	1.64 $\pm$ 1.88	1.64 $\pm$ 1.86

Table 1: Comparison of harm scores (mean

\pm

standard deviation) per benchmark, language and experimental condition. Shaded rows indicate polarity inversion conditions (negation and antonym probes, with and without wrapper). For the calculation of the mean, we exclude the models Claude 4.5 Sonnet and GPT-5 Nano, which are resistant to polarity blindness. We denote with an asterisk statistically significant differences (paired t -test) between polarity-inverting variations, and both as is and polarity preserving variations (p <

0.001

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

	Model	Variation	Harm Scores						Mean Harm Score
	Model	Variation	0	1	2	3	4	5	Mean Harm Score
en	Gemma 3:27B	as is	0.76	0.03	0.08	0.08	0.05	0.01	0.86 $\pm$ .05
	Grok-4.1 Fast	as is	0.69	0.01	0.02	0.07	0.15	0.05	1.74 $\pm$ .02
	Gemma 3:27B	neg. wrap.	0.24	0.02	0.08	0.26	0.35	0.05	2.63 $\pm$ .03
	Grok-4.1 Fast	neg. wrap.	0.11	0.00	0.01	0.04	0.20	0.64	4.12 $\pm$ .18
es	Gemma 3:27B	as is	0.76	0.03	0.08	0.08	0.05	0.01	0.68 $\pm$ .01

Appendix

Table 2: Per-model harm score distributions for the as-is baseline and the highest-scoring variation, across the three evaluation languages. Score columns show the proportion of responses at each level (0–5). Underlined values are the highest within each section.

Figure 5: Distribution of harmfulness potential ratings across the 260 benchmark prompts (0 = harmless, 3 = extremely harmful).

Figure 6: Distribution of the mean harm scores across potential harmfulness ratings, including all models.

Figure 7: Refusal rates across models, variations and languages.

	Model	Variation	Harm Scores						Mean Harm Score
	Model	Variation	0	1	2	3	4	5	Mean Harm Score
en	GPT-5 Nano	neg. wrap.	$0.73\pm.01$	$\underline{0.12}\pm.01$	$0.09\pm.02$	$0.05\pm.02$	$0.01\pm.00$	$0.00\pm.00$	$0.48\pm.02$
	GPT-5 Mini	neg. wrap.	$0.45\pm.02$	$0.08\pm.01$	$0.06\pm.01$	$0.16\pm.01$	$0.20\pm.02$	$0.05\pm.01$	$1.75\pm.07$
	GPT-5.2	neg. wrap.	$0.47\pm.03$	$0.07\pm.01$	$0.09\pm.01$	$0.22\pm.03$	$0.11\pm.00$	$0.03\pm.01$	$1.51\pm.12$
	Claude 4.5 Sonnet	as is	$\underline{0.81}\pm.01$	$0.07\pm.02$	$\underline{0.10}\pm.02$	$0.02\pm.00$	$0.00\pm.00$	$0.00\pm.00$	$0.34\pm.04$
	Grok-4.1 Fast	neg. wrap.	$0.11\pm.03$	$0.00\pm.00$	$0.01\pm.01$	$0.04\pm.01$	$0.20\pm.03$	$\underline{0.64}\pm.04$	$\underline{4.12}\pm.18$

Appendix

Table 3: Highest harm scores (average across 3 runs with standard deviations indicated by

\pm

) and corresponding harm scores per model. Largest value for each language and value is underlined. Negation and negation wrapper dominate in English. In Spanish and Czech, antonym or synonym conditions achieve the highest harm scores for several models (Gemma 3:27B, Ministral 14B, DeepSeek-V3.2), as discussed in section 6 .

	Model	Variation	Harm Scores						Mean Harm Score
	Model	Variation	0	1	2	3	4	5	Mean Harm Score
en	GPT-5 Nano	synonym	0.93 $\pm$ .01	0.04 $\pm$ .01	0.02 $\pm$ .00	0.01 $\pm$ .01	0.00 $\pm$ .00	0.00 $\pm$ .00	0.12 $\pm$ .01
	GPT-5 Mini	as is	0.93 $\pm$ .01	0.01 $\pm$ .01	0.03 $\pm$ .01	0.01 $\pm$ .00	0.01 $\pm$ .00	0.00 $\pm$ .00	0.18 $\pm$ .02
	GPT-5.2	as is	0.94 $\pm$ .01	0.01 $\pm$ .01	0.02 $\pm$ .01	0.02 $\pm$ .01	0.00 $\pm$ .00	0.00 $\pm$ .00	0.14 $\pm$ .02
	Claude 4.5 Sonnet	as is wrap.	0.97 $\pm$ .00	0.02 $\pm$ .01	0.00 $\pm$ .01	0.00 $\pm$ .00	0.00 $\pm$ .00	0.00 $\pm$ .00	0.04 $\pm$ .01
	Grok-4.1 Fast	as is wrap.	0.54 $\pm$ .03	0.14 $\pm$ .03	0.13 $\pm$ .01	0.14 $\pm$ .02	0.04 $\pm$ .02	0.00 $\pm$ .00	1.00 $\pm$ .04

Appendix

Table 4: Lowest harm scores (average across 3 runs with standard deviations in parenthesis) and corresponding harm scores per model. Largest value for each language is underlined.

Model	Variation	Mean Harm Score
Model	Variation	English (en)	Spanish (es)	Czech (cs)
GPT-5 Nano	negation	$0.33\pm.01$	$0.20\pm.07$	$0.23\pm.05$
	neg. wrapper	$0.48\pm.02$	$0.28\pm.08$	$0.16\pm.02$
	antonym	$0.29\pm.03$	$0.16\pm.02$	$0.19\pm.08$
	ant. wrapper	$0.24\pm.02$	$0.16\pm.00$	$0.10\pm.00$
GPT-5 Mini	negation	$0.68\pm.05$	$0.45\pm.03$	$1.07\pm.03$

Appendix

Table 5: Cross-lingual comparison of mean harm scores between the polarity-inverting variations. All values reported as

mean\pm std

Figure 8: Weighted Cohen’s Kappa agreements between the different human annotators and LLM models.

Figure 9: Example of the website included as HTML code in one of Grok-4.1 Fast’s answers.

Figure 10: Excerpts from Ministral 14B’s response. Despite the disclaimer and “hypothetical” framing, ouabain is a real toxin known as “arrow poison.”

Figure 11: Harmful response from GPT-5 Mini.

Figure 12: Model Gemini 2.5 Flash Lite explaining how to encourage online support for terrorist organizations.

Model	Provider	Reasoning
GPT-5 Nano	OpenAI	minimal
GPT-5 Mini	OpenAI	minimal
GPT-5.2	OpenAI	none
Claude 4.5 Sonnet	Anthropic	low
Grok-4.1 Fast	xAI	none
Gemini 2.5 Flash	Google	none

Appendix

Table 6: Reasoning efforts employed for each model.

Language Model Safety Evaluation

Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.

See all 542 papers

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
Aug 22, 2026Afshin Orojlooyjadid, Hitesh PatelLLM Safety Evaluation LLM Safety Benchmarks
Oracle AI
Polar: A Benchmark for Evaluating Political Bias in LLMs
Jun 11, 2026Sangho Kim, Heejin Kim, Yoonhee Park +2Multilingual Language Model Evaluation Political Bias in Language Models
Graduate School of Data Science, Seoul National University · Dept. of Computer Science and Engineering, Seoul National University
How Do LLMs Change Predictions Under Negation?
Oct 7, 2026Jongwook Yoon, Jongwon Lim, Sungjib Lim +2LLM Reliability LLM Evaluation
Graduate School of Data Science, Seoul National University · Department of Computer Science and Engineering, Seoul National University

Aug 22, 2026J/K move · Enter open · S save

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

Afshin Orojlooyjadid, Hitesh Patel

Oracle AI

LLM Safety Evaluation LLM Safety Benchmarks Content Moderation LLM Safety Monitoring

Open paper AskPDF arXiv

Large Language Models (LLMs) are increasingly deployed in real-world applications, yet they remain vulnerable to generating harmful content. From adversarial jailbreaks that bypass safety filters to implicit hate that evades detection, the range of risks these models pose continues to grow. While both specialized content moderators and general-purpose LLMs are being used as safety layers, the question of which model is best suited for which type of harmful content remains unanswered. We present the most comprehensive evaluation of LLM safety capabilities to date, systematically testing 53 models across 11 datasets that we organize into four distinct categories. Our evaluation under both prompt-only and prompt-response settings uncovers critical blind spots: large frontier models that lead on one category fall significantly behind smaller, specialized alternatives on others, and real-world conversational safety remains largely unsolved across all model families. These findings challenge the assumption that scale alone ensures safety, and provide the community with a structured framework for informed model selection.

Abstract

Language Model Safety Evaluation

Explore similar work

No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios

Polar: A Benchmark for Evaluating Political Bias in LLMs

How Do LLMs Change Predictions Under Negation?