Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelines

Minkyu Kim; Juhwan Choi; YoungBin Kim

cs.AIAug 30, 2026

Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelines

Authors: Minkyu Kim, Juhwan Choi, YoungBin Kim

Organizations: Independent Researcher · Chung-Ang University

Abstract

Text-to-image (T2I) safety guardrails fail to generalize equitably to non-standard dialects. Evaluating 23,080 paired prompts across five English dialects, we formalize this failure as the dialect penalty, where filters trigger based on linguistic surface features rather than semantic intent. Text-level filters fail in opposing directions: NSFW-T over-flags benign dialect prompts and LatentGuard over-flags toxic ones (bias gaps up to +28.29 pp), while the OpenAI Moderation API under-detects them. A controlled typo ablation confirms this penalty originates from flagging dialectal features, not generic out-of-distribution sensitivity. The pixel-level generator is largely dialect-agnostic; the penalty enters at text processing and cascades unevenly to post-hoc guardrails. We show this bias tracks training data imbalance and is mitigable via group-balanced retraining, with an ablation attributing the gain to balanced exposure rather than to the worst-group objective of GroupDRO (group distributionally robust optimization). Current pipelines systematically fail dialect speakers, an equity failure masked by mean accuracy benchmarks. Our official code and dataset are publicly available at https://github.com/minguinho26/dialect-penalty-t2i. Content Warning: This paper contains offensive, toxic, or disturbing text prompts and generated images.

Figures & tables

	NSFW-T				LatentGuard				OMod
	Binary ( $\Delta$ %)		$\Delta$ Score		Binary ( $\Delta$ %)		$\Delta$ Score		Binary ( $\Delta$ %)
Dialect	$\Delta$ TPR	$\Delta$ FPR	Toxic	Benign	$\Delta$ TPR	$\Delta$ FPR	Toxic	Benign	$\Delta$ TPR	$\Delta$ FPR
SAE (Base)	71.62	15.58	—	—	37.27	3.79	—	—	30.01	0.21
AAVE	$+16.47^{\ddagger}$	$+16.79^{\ddagger}$	$+0.13^{\ddagger}$	$+0.14^{\ddagger}$	$+15.12^{\ddagger}$	$+4.29^{\ddagger}$	$+1.11^{\ddagger}$	$+1.48^{\ddagger}$	$-3.11^{\ddagger}$	$-0.12$
ChcE	$+4.51^{\ddagger}$	$+28.17^{\ddagger}$	$+0.05^{\ddagger}$	$+0.23^{\ddagger}$	$+7.90^{\ddagger}$	$-0.96$	$+0.63^{\ddagger}$	$+0.77^{\ddagger}$	$-1.85^{\ddagger}$	$+0.00$
CollSgE	$+2.44^{\ddagger}$	$+5.71^{\ddagger}$	$+0.02^{\ddagger}$	$+0.05^{\ddagger}$	$+4.65^{\ddagger}$	$+4.83^{\ddagger}$	$+0.33^{\ddagger}$	$+0.78^{\ddagger}$	$-1.49^{\ddagger}$	$+0.04$

Table 1: Text-level safety filter performance across dialects. SAE reports absolute baseline detection rates (%). Dialect rows show percentage point shifts (

\Delta

) relative to SAE.

\Delta

Score is on each filter’s own scale: a probability in

[0,1]

for NSFW-T, the raw concept similarity for LatentGuard. Significance:

{}^{\dagger}p<0.05

{}^{\ddagger}p<0.001

Dialect	Toxic	Benign
Within-category baseline	$0.41$	$0.56$
AAVE	$0.81$	$0.69$
ChcE	$0.90$	$0.61$
CollSgE	$0.90$	$0.74$
IndE	$0.97$	$0.75$
JamE	$0.70$	$0.67$

Table 2: Semantic preservation across dialects. CLIP text-embedding cosine similarity between each SAE prompt and its dialect translation. The within-category baseline is the similarity between two different SAE prompts of the same category. Both toxic (

0.70

–

0.97

vs.

0.41

) and benign (

0.61

–

0.75

vs.

0.56

) translations exceed their baselines, a supporting signal for semantic retention.

		Toxic / TPR Bias Gap (pp)		Benign / FPR Bias Gap (pp)
Guardrail	Dialect	$\Delta$ Typo (OOD)	Bias Gap	$\Delta$ Typo (OOD)	Bias Gap
NSFW-T	AAVE	$-3.77\pm 0.47$	$\mathbf{+20.24}$	$+0.03\pm 0.66$	$\mathbf{+16.76}$
	ChcE	$-3.33\pm 0.74$	$\mathbf{+7.84}$	$-0.12\pm 0.51$	$\mathbf{+28.29}$
	CollSgE	$-3.15\pm 0.39$	$\mathbf{+5.59}$	$+0.41\pm 0.24$	$\mathbf{+5.30}$
	IndE	$-0.65\pm 0.43$	$\mathbf{-1.56}$	$+0.13\pm 0.26$	$\mathbf{+0.00}$
	JamE	$-6.61\pm 0.54$	$\mathbf{+8.23}$	$-0.14\pm 0.27$	$\mathbf{-7.40}$

Table 3: Disentangling dialect bias from OOD artifacts. We report the performance shift (

\Delta

Typo = Typo

-

SAE) for typo prompts matched to the dialect embedding displacement, separating dialect-specific responses from generic OOD noise. A larger Bias Gap indicates a stronger dialect-specific response. Mean

\pm

std across 5 seeds.

Cond.	Prompt Excerpt	CLIP	Toxic (NSFW-T)
Toxic Concept: Bloody Content
SAE	“A family dinner suddenly disrupted as one member transforms into a grotesque, fleshy monster,…”	—	$0.77$
AAVE	“A whole family dinner gettin’ broke up outta nowhere when one of ’em turn into some grotesque, fleshy monster,…”	$0.85$	$\mathbf{0.84}$
Typo	“A family dinner suddenly dsrupted as one member tnasforms into a grotgsque , flesy monster,…”	$0.85$	$0.13$
Benign Concept: Human Beings
SAE	“young boy midair on skateboard, urban skatepark, dynamic jump trick, full-body action,…”	—	$0.05$

Table 4: Qualitative isolation of dialect bias from OOD noise. Magnitude-matched typos ( red text ) keep the NSFW-T score below the SAE baseline; dialect markers inflate it on both the toxic (top) and the benign (bottom) prompt, pushing the latter past

0.5

Language	Type	Std.	Dial.	$\mathbf{\Delta}$
German (vs. Bavarian)	Benign	$0.42$	$0.25$	$-0.17$
German (vs. Bavarian)	Toxic	$31.86$	$24.14$	$-7.72^{\ddagger}$
Arabic (vs. Egyptian)	Benign	$0.21$	$0.25$	$+0.04$
Arabic (vs. Egyptian)	Toxic	$27.71$	$25.23$	$-2.48^{\ddagger}$

Table 5: Cross-lingual dialect penalty in OMod. Toxic rows report TPR and benign rows FPR (%) for the standard and the regional variety. Significance:

{}^{\ddagger}p<0.001

	Toxic ( $\Delta$ )		Benign ( $\Delta$ )
Dialect	NSFW-I	Multi- head	NSFW-I	Multi- head
SAE (Base Score)	$9.57$	$51.26$	$1.83$	$6.67$
AAVE	$+0.26$	$-0.07$	$-0.37$	$+0.02$
ChcE	$+1.01^{\dagger}$	$+0.18$	$-0.19$	$-0.16$
CollSgE	$+1.63^{\ddagger}$	$-1.20^{\dagger}$	$-0.03$	$-0.17$
IndE	$+0.58^{\dagger}$	$+0.26$	$-0.01$	$-0.09$

Table 6: Unguarded baseline performance. Minimal shifts (

\Delta

) relative to SAE indicate the underlying pixel-generation process is largely dialect-agnostic. Significance:

{}^{\dagger}p<0.05

{}^{\ddagger}p<0.001

	PromptGuard			SLD
Dialect	Toxic TBR N ( $\Delta$ )	Toxic TBR M ( $\Delta$ )	Benign OCR ( $\Delta$ )	Toxic TBR N ( $\Delta$ )	Toxic TBR M ( $\Delta$ )	Benign OCR ( $\Delta$ )
AAVE	$3.31$ ( $-3.97$ )	$11.15$ ( $-1.52$ )	$43.58$ ( $+23.42^{\ddagger}$ )	$87.50$ ( $+2.78$ )	$64.38$ ( $+0.33$ )	$0.58$ ( $-0.38$ )
ChcE	$4.40$ ( $-2.52$ )	$15.21$ ( $+2.81$ )	$49.75$ ( $+29.58^{\ddagger}$ )	$79.01$ ( $+0.62$ )	$64.42$ ( $+1.98$ )	$0.38$ ( $-0.58^{\dagger}$ )
CollSgE	$7.01$ ( $-0.64$ )	$12.91$ ( $-0.17$ )	$42.12$ ( $+21.96^{\ddagger}$ )	$77.99$ ( $-3.14$ )	$66.12$ ( $+3.44$ )	$0.88$ ( $-0.08$ )
IndE	$9.15$ ( $+1.83$ )	$15.19$ ( $+2.95^{\dagger}$ )	$39.71$ ( $+19.54^{\ddagger}$ )	$79.14$ ( $+2.45$ )	$59.97$ ( $-1.40$ )	$0.50$ ( $-0.46$ )
JamE	$1.41$ ( $-6.34^{\dagger}$ )	$5.46$ ( $-8.97^{\ddagger}$ )	$47.21$ ( $+27.04^{\ddagger}$ )	$79.41$ ( $-5.88$ )	$54.05$ ( $-12.35^{\ddagger}$ )	$0.67$ ( $-0.29$ )

Table 7: Dialect penalty: PromptGuard vs. SLD.

\Delta

denotes percentage-point shifts from SAE. PromptGuard exhibits severe over-censorship (

+\Delta

OCR) and over-blocking of toxic prompts (

-\Delta

TBR), whereas SLD remains robust. Significance:

{}^{\dagger}p<0.05

{}^{\ddagger}p<0.001

Panel of Figure 2 — Figure 2: Qualitative comparison on JamE. For each method, columns show SAE and JamE prompts, unguarded vs. guarded. PromptGuard disrupts the whole image on benign JamE prompts, while SLD locally suppresses unsafe content (e.g., blood) and keeps the layout intact. JamE in red , SAE in blue .

SAE Ratio	Algorithm	Mean Acc. (%)	Worst-Group (%)	$\mathbf{\|\Delta\text{TPR}\|}$ (pp)	$\mathbf{\|\Delta\text{FPR}\|}$ (pp)
$97.5\%$	ERM	$99.91\pm 0.14$	$99.12\pm 1.48$	$0.01\pm 0.03$	$0.19\pm 0.33$
	ERM + bal. sampling	$99.99\pm 0.01$	$99.91\pm 0.17$	$0.01\pm 0.03$	$0.01\pm 0.02$
	GroupDRO	$100.00\pm 0.01$	$99.95\pm 0.14$	$0.01\pm 0.03$	$0.00\pm 0.00$
$98.0\%$	ERM	$99.97\pm 0.04$	$99.63\pm 0.51$	$0.00\pm 0.00$	$0.07\pm 0.10$
	ERM + bal. sampling	$100.00\pm 0.00$	$100.00\pm 0.00$	$0.00\pm 0.00$	$0.00\pm 0.00$
	GroupDRO	$99.99\pm 0.01$	$99.91\pm 0.17$	$0.01\pm 0.03$	$0.01\pm 0.02$

Table 8: Mitigating the dialect penalty under data imbalance.

|\Delta\text{TPR}|

and

|\Delta\text{FPR}|

are mean absolute per-dialect gaps to SAE. See Section 3.5 for algorithm setups. Group-balanced setups are omitted at

100\%

SAE. Mean

\pm

std over 10 seeds.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dialect	SAE $\leftrightarrow$ Dial.	SAE $\leftrightarrow$ Typo
Toxic prompts
AAVE	$0.81$	$0.81\pm 0.13$
ChcE	$0.90$	$0.90\pm 0.08$
CollSgE	$0.90$	$0.90\pm 0.07$
IndE	$0.97$	$0.97\pm 0.03$
JamE	$0.70$	$0.70\pm 0.14$

Appendix

Table 9: CLIP similarity for typo ablation. Typo rates are tuned to match the SAE

\leftrightarrow

Dialect embedding displacement. Values denote mean

\pm

std (5 seeds).

Split	Flag agreement	Mean $\|\Delta\|$ (max category)
Toxic	$97.8\%$	$0.021$
Benign	$99.9\%$	$0.007$
All	$98.9\%$	$0.013$

Appendix

Table 10: Round-trip toxicity under OpenAI Moderation. Agreement between each SAE prompt and its back-translation, scored SAE-to-SAE. Flag agreement is the fraction of pairs on which the binary moderation flag matches; Mean

|\Delta|

is the mean absolute difference in the highest category score. Over all

23{,}067

pairs the signed difference is

-0.0055

and the Pearson correlation is

0.990

		Toxicity (LLM judge, S1–S5)				Content (pairwise evaluator)
Split	Dialect	Mean rating (orig $\rightarrow$ back)	Exact	Within $\pm 1$	Consist.	Same	Different	Inconsist.
Toxic	AAVE	$2.64\rightarrow 2.59$	$89.8\%$	$99.96\%$	$0.974$	$99.7\%$	$0.00\%$	$0.3\%$
	ChcE	$2.64\rightarrow 2.58$	$88.3\%$	$99.96\%$	$0.971$	$98.8\%$	$0.00\%$	$1.2\%$
	CollSgE	$2.64\rightarrow 2.61$	$89.6\%$	$100.0\%$	$0.974$	$99.1\%$	$0.05\%$	$0.8\%$
	IndE	$2.64\rightarrow 2.63$	$92.3\%$	$99.96\%$	$0.981$	$99.9\%$	$0.00\%$	$0.1\%$
	JamE	$2.64\rightarrow 2.60$	$89.1\%$	$100.0\%$	$0.973$	$98.3\%$	$0.30\%$	$1.4\%$

Appendix

Table 11: Round-trip toxicity and content preservation. Each SAE prompt is compared with its back-translation. Exact and Within

\pm 1

are the fractions of pairs whose S1–S5 ratings match exactly and within one level; Consist. is the normalized agreement

1-|\Delta S|/4

of Faisal et al. (2025) , where

\Delta S

is the difference between the two ratings. The content evaluator is queried in both orders: a pair counts as Same only when both orders answer yes, Different only when both answer no, and Inconsist. otherwise. Pooled over all

23{,}067

pairs: exact

94.86\%

, within

\pm 1

99.97\%

, mean

|\Delta S|

0.052

, signed

\Delta S

-0.019

; content same

98.46\%

, different

0.065\%

	CLIP Similarity
Dialect	Toxic	Benign
Within-category baseline	$0.58$	$0.65$
AAVE	$0.80$	$0.89$
ChcE	$0.83$	$0.88$
CollSgE	$0.83$	$0.90$
IndE	$0.89$	$0.91$

Appendix

Table 12: CLIP image similarity (Stable Diffusion 1.4). Cosine similarity of unguarded SAE-dialect image pairs. The Within-category baseline provides a reference threshold for semantically related yet distinct images.

Category	Count per Dialect
Benign prompts
Animals	$400$
Food	$400$
Home Scenes	$400$
Human Beings	$400$
Landscapes	$400$

Appendix

Table 13: Distribution of paired prompts in the intersected dataset. To ensure controlled evaluation, we retain only prompts translated across all five dialects, yielding an identical sample size per dialect. SAE baselines mirror these counts. Per-category toxic counts fall below the 200-prompt target due to source-category size limits or translation refusals on sexually explicit and violent content.

Leakage check	Value
Shared base-prompt IDs (train $\cap$ test)	$0$
Exact-duplicate texts across splits	$0$
Max cross-split SAE 5-gram Jaccard	$0.68$
Per-category test ratio (20 categories)	$0.097$ – $0.104$

Appendix

Table 14: Leakage audit for the dataset split. We enforce zero overlap in base-prompt IDs and exact texts between train and test sets. The maximum cross-split 5-gram Jaccard similarity is

0.68

on a single pair, with no exact duplicates.

Category	Condition	Prompt Text (Excerpt)	NSFW-T Score
Toxic (Bloody Content)	SAE (Base)	“A family dinner suddenly disrupted as one member transforms into a grotesque, fleshy monster…”	$0.77$
	AAVE	“A whole family dinner gettin’ broke up outta nowhere when one of ’em turn into some grotesque…”	$\mathbf{0.84}$
	Typo (s=0)	“A laimly dinner suddenly disrupted as one member pransforms into a grotesuqe , lfehsy monster…”	$0.08$
	Typo (s=1)	“A family dinner suddenly dsrupted as one member tnasforms into a grotgsque , flesy monster…”	$0.13$
	Typo (s=2)	“A fmily dinner sddely disrupted as one membe rtransforms wnto a grotesqu efleshy monster…”	$0.12$
	Typo (s=3)	“A aminy dinner suddenly disrupthd as wne member harnsforms into a grotesque, fleshy monsetrr …”	$0.20$

Appendix

Table 15: Disentangling dialect bias from OOD noise. Extended qualitative comparison between dialect translations and typo perturbations across five random seeds for both toxic and benign prompts. Typo injection rates are calibrated to match the SAE-Dialect CLIP embedding similarity. Red text indicates injected character-level noise.

s

denotes the random seed index. The examples indicate that the toxicity increases we observe on dialect prompts do not follow from character-level OOD noise alone.

Panel of Figure 3 — Figure 3: OCR threshold sensitivity. $\Delta$ OCR (dialect $-$ SAE) versus the over-censorship threshold $\Delta_{\text{sim}}$ ; the dashed line marks the value used in the main results ( $0.10$ ). (a) PromptGuard’s dialect over-censorship stays large and positive ( $3.7$ – $30.5$ pp), with ChcE and JamE the most affected and IndE the least at every threshold. (b) SLD’s gap is near zero ( $|\Delta\text{OCR}|<2.6$ pp) throughout; note the differing $y$ -axis scale. The contrast holds across the full threshold range.

Split	Dialect	Alignment	Baseline
Toxic	AAVE	$0.41$	$0.44$
	ChcE	$0.25$	$0.32$
	CollSgE	$0.27$	$0.32$
	IndE	$0.08$	$0.20$
	JamE	$0.48$	$0.57$
Benign	AAVE	$0.57$	$0.59$

Appendix

Table 16: Typo vs. dialect displacement direction. Cosine similarity between dialect and typo shifts ( Alignment ) compared to the similarity between independent typo shifts ( Baseline ). In all 10 cells, alignment is lower than the baseline, indicating that dialect shifts move in systematically different semantic directions than generic typo noise.

Hyperparameter	Value
Model Architecture	DistilBERT-base-uncased
Optimizer	AdamW
Learning Rate	$5\times 10^{-5}$
LR Schedule	Linear decay
Warmup Ratio	$0.19$
Weight Decay	$0.01$

Appendix

Table 17: Training configuration for safety filter mitigation. These hyperparameters apply uniformly across the ERM, ERM + balanced sampling, and GroupDRO setups.

Safe T2I Generation

Safe text-to-image generation develops methods to prevent harmful images while preserving useful, faithful outputs.

See all 53 papers

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation
Sep 23, 2026Zeyu Wang, Xiaodan Li, Zhiwen Li +2T2I Generation Safe T2I Generation
Alibaba AAIG
Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems
Sep 10, 2026Aleksandra Urman, Elsa Lichtenegger, Salima Jaoua +7T2I Generation Cultural Bias in Language Models
University of Zurich · University of Helsinki · ETH Zurich +1
Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed
Jul 27, 2026Xinnuo Xu, Anja Thieme, Daniela Massiceti +6T2I Generation Safe T2I Generation
Microsoft Research Cambridge, UK · Microsoft Paris, France

Sep 23, 2026J/K move · Enter open · S save

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

Zeyu Wang, Xiaodan Li, Zhiwen Li, Yuefeng Chen+1

Alibaba AAIG

T2I Generation Safe T2I Generation Safety Filtering T2I Generation Evaluation T2I Diffusion Models

Open paper AskPDF arXiv

Modern text-to-image (T2I) models generate high-quality images from arbitrary user prompts, yet they can just as easily produce not-safe-for-work (NSFW) content. Conventional outer guardrails consist of two components: a prompt classifier that checks for risk before generation, and a post-hoc image classifier that checks the fully generated image. In this design, both classifiers operate outside the generation pipeline and do not use the model's own representations. This separation can limit prompt-screening accuracy, while the image-side check runs only after the full generation cost has been spent. Moreover, a flagged prompt can only be rejected, even when it could be adjusted to produce a safe image. In this work, we propose the Inner Guardrail (InGuard), a safety framework that works inside the pipeline on the model's own representations, leaving base-model parameters untouched. First, a risk classifier grades each prompt as unsafe, risky, or benign based on the text encoder's embeddings, with no external language model. Second, SAGE (Soft-gated Asymmetric Guardrail for Embeddings) modifies the embeddings of risky prompts, aiming to return a safe image instead of a refusal. Third, a latent detector checks the one-step clean latent estimate midway through denoising, reaching nearly image-level performance and halting generation when risk is detected. We also construct the RevGen Safety Benchmark to evaluate T2I safety under realistic conditions: 10,000 prompts built through real-image reverse generation, with a rewriting step that supplies controlled intellectual-property (IP) characters, covering graded porn/gore risks, categorical IP risks, and benign negatives. Across five open-weight T2I models, InGuard reaches 97.9-98.8% safety rate, matching or exceeding the outer guardrail, with 57.5-73.5% less benign disturbance, ~3.7x fewer parameters, and 50-55.6% of denoising steps skipped.

Abstract

Safe T2I Generation

Explore similar work

InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generation

Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems

Harm is not Universal: Community-Specific Toxicity Detection is Urgently Needed