Text-to-image diffusion models often fail to suppress explicitly negated concepts, such as a lamp for "a desk without a lamp." We introduce T2I-NSI, a benchmark and metric for measuring negation sensitivity in text-to-image generation. It pairs each negated prompt with a canonical positive counterpart and factorizes negation by linguistic form (lexical/semantic, morphological, syntactic) and target type (objects, attributes, relations, compositions, and nested cases). The benchmark contains 759,869 paired prompt atoms over a 5,082-object catalog, with 916,272 images (911,982 open-weights, 4,290 closed-source), scored by a Basic CLIP estimator and a Robust ensemble of CLIP, VQA and open-vocabulary detection. Negation sensitivity is low: open-weights generators score NSI <= 0.038, while GPT-Image-1.5 and Gemini-2.5-Flash-Image reach about 0.09. The dominant failure is under-suppression rather than misdirection. Inverse interpretation is essentially absent (0.00% across over 450K atom-model evaluations), whereas models render the concept under both prompts on 8-19% of atoms and only partly suppress it on a further 9-25%; most remaining atoms are too weakly rendered to label. A blinded human study (420 atoms, nine annotators) corroborates the near-absence of inversion and the high persistence rate, and a matched-prompt comparison shows the open-versus-closed gap (~3.5x) is not a prompt-generator artifact. Code and data: https://github.com/IAB-IITJ/T2I-NSI
Figures & tables
Figure 1: T2I generators fail to suppress negated concepts. Three negation atoms spanning form: (top) syntactic (“No solitary Acacia tree…”), (middle) morphological (“unopened pitcher plant”), and (bottom) lexical/semantic (“Abusir Pyramids hidden from view”). Across FLUX.2, Qwen-Image-2512, and SD 3.5, the negated prompt yields a near-identical image, and the negated concept (tree, opened pitcher, visible pyramids) persists.
Work
T2I
Scale
Factorized
Metric
Generators
Failure modes
NegBench
×
79 K
partial
retrieval acc.
N/A
×
Conwell et al.
✓
few hundred
×
human ratings
open+closed
×
Sammani et al.
×
qualitative
×
geometric
N/A
×
FineGRAIN
✓
∼750 prompts
×
VLM judge
open only
27-way
T2I-NSI (ours)
✓
759,869 (152K eval.)
✓ (form × target)
3-probe + NegAlign
✓ (3+3)
6-way
Table 1: Comparison with prior work (NegBench [ 1 ] , Conwell et al. [ 7 ] , Sammani et al. [ 29 ] , FineGRAIN [ 14 ] ). T2I-NSI uniquely combines generation-side scope, factorized axes, a probe-ensemble metric, open/closed generators, and per-atom failure-mode classification.
Algorithm
Model
NSI(M)
Mean Cj
NegAlign
Weight
Similarity
Uncertainty
Basic (CLIP)
FLUX.2
0.0035
0.6095
0.3007
—
—
—
Qwen-Image-2512
0.0036
0.6115
0.3009
—
—
—
SD 3.5
0.0017
0.6118
0.3000
—
—
—
Robust (3-probe)
FLUX.2
0.0378
0.6477
0.4617
0.9367
0.9557
0.0333
Qwen-Image-2512
0.0306
0.6404
0.4612
0.9297
0.9343
0.0372
SD 3.5
0.0198
0.6402
0.4548
0.9413
0.9633
0.0308
Table 2: Aggregate T2I-NSI on the open-weights tier ( ∼152 K atoms per model). The metric is bounded in [−1,1] ; a perfectly negation-aware generator scores NSI=1 . “Mean Cj ” is the mean positive realization Cj=Rj+ (not a retention rate). The Robust estimator additionally reports the mean ensemble weight wj=exp(−βUj) , the global paired-image similarity Sj , and mean per-atom uncertainty Uj ; these columns are not produced by the Basic estimator.
Model
SUCCESS
ALWAYS_PRESENT
ALWAYS_ABSENT
INVERSE
PARTIAL_SUPPR
WEAKLY_REND
FLUX.2
0.29%
15.50%
0.31%
0.00%
12.09%
71.81%
Qwen-Image-2512
0.06%
13.03%
0.19%
0.00%
12.54%
74.18%
SD 3.5
0.03%
16.92%
0.31%
0.00%
9.16%
73.59%
Table 3: Per-atom failure-mode distribution for the open-weights generators under the Robust estimator, with classes as defined in Section 4.2 . Rows sum to 100% (rounding aside); INVERSE is 0.00% to 2 dp ( 0 / 3 / 2 atoms of 151,970 ).
Figure 2: Representative paired generations: suppression varies by atom and is often incomplete. Positive (upper) and negated (lower) renders under matched seeds for FLUX.2, Qwen-Image-2512 and SD 3.5; column headers give the atom ids. Prompts, positive → negated. Left (syntactic, L0): “An ambulance is on the road on a rural highway with green fields.” → “There is not an ambulance on the road…” Middle (lexical/semantic, L0): “A tiger is present in a dense jungle clearing.” → “The tiger is absent from the dense jungle clearing.” Right (morphological, L1): “A covered fish tank rests on a sturdy table.” → “An uncovered fish tank…” Robust-estimator classes (FLUX.2 / Qwen-Image-2512 / SD 3.5). Left : all ALWAYS_PRESENT . Middle : SUCCESS / SUCCESS / ALWAYS_PRESENT . Right : all WEAKLY_REND ( Cj=0.63 / 0.65 / 0.62<θpos ).
Model
Form
Complexity level
lex.
morph.
syn.
L0 (obj)
L1 (attr)
L2 (rel)
L3 (comp)
L4 (nest)
FLUX.2
0.0580
0.0291
0.0267
0.0511
0.0434
0.0218
0.0386
0.0344
Qwen-Image-2512
0.0511
0.0302
0.0110
0.0185
0.0399
0.0170
0.0407
0.0372
SD 3.5
0.0301
0.0172
0.0120
0.0173
0.0292
0.0104
0.0245
0.0173
Table 4: Conditioned T2I-NSI scores for the open-weights generators under the Robust estimator. The benchmark is factorized by linguistic form and target complexity : levels L0-L4 correspond to increasingly structured target types (object, attribute, relation, composition, nested), so the two are a single ordered axis rather than independent factors. Bold marks the per-row maximum within each axis.
Model
N
NSI(M)
NegAlign
SUCCESS
ALWAYS_ PRESENT
ALWAYS_ ABSENT
INVERSE
PARTIAL_ SUPPR
WEAKLY_ REND
GPT-Image-1.5
1,200
0.0934
0.479
1.58%
11.42%
0.00%
0.00%
24.83%
62.17%
Gemini-2.5-Flash-Image
717
0.0902
0.476
1.95%
7.67%
0.00%
0.00%
18.97%
71.41%
Midjourney v7 (syn. only)
228
0.0267
0.448
0.00%
19.30%
0.00%
0.00%
13.16%
67.54%
Table 5: Closed-source extension under the Robust estimator. Sample sizes vary by API constraint; Midjourney v7 is syntactic-only, so its 0.0267 should be read against the open-weights syntactic scores ( 0.0110 - 0.0267 , Table 4 ), not the all-form aggregates. Basic CLIP NSI saturates at ≤0.008 across all three, and INVERSE and ALWAYS_ABSENT remain 0.00% . Class shares sum to 100% per row (rounding aside).
Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.
Text-to-image (T2I) models are judged by benchmarks that measure whether requested content appears, but these benchmarks largely overlook the complementary ability to satisfy negated constraints, for example, generating "a non-red cup." Measuring negation raises challenges not faced by affirmation-based benchmarks and requires careful prompt and evaluation design. We introduce NegT2IBench, a benchmark of 4,800 prompts covering two attribute types and four relation categories. Prompts are organized by polarity: the number of positive statements that must hold and negated statements that must not, each ranging from 0 to 2. Varying the two independently separates the effect of negation from the effect of prompt complexity. Our detector-based scoring is reproducible, auditable, and pinpoints which requirement failed. On 600 images with three-annotator labels, it agrees with humans as closely as vision-language judges up to 30x larger, while using only a fraction of their GPU memory. Across eleven T2I models and 211,200 images, nine score lower on a single negated statement than on a single positive one. Per-statement scoring reveals that the loss is largest for color and near zero for proximity, and that 41.5% of failed statements render exactly what the prompt forbids. Rendering what a prompt asks for and withholding what it forbids are distinct capabilities that an aggregate compositional score cannot distinguish. NegT2IBench measures the latter directly, providing a controlled testbed for diagnosing negation failures and developing methods to overcome them.
Omar Elfatairy, Maria A. Bravo, Jessica Bader +1
Technical University of Munich, Helmholtz Munich, Munich Center for Machine Learning
Text-to-image (T2I) models have become increasingly capable of generating high-quality images. Yet, enforcing the explicit absence of a specified object or attribute remains a fundamentally challenging problem. Existing approaches, including prompt negation, post-hoc editing, and negative guidance, remain insufficient for explicit concept suppression, often failing to remove the target concept or degrading overall image quality. To this end, we propose Orthogonal Negative Guidance in attention feature space, a training-free method that operates in the attention output space of MM-DiT-based T2I transformers. Our method orthogonalizes negative-prompt attention features with respect to positive-prompt features and subtracts only the orthogonal component, suppressing unwanted concepts while preserving desired semantics. Experiments on FLUX-dev and FLUX-schnell show that our method achieves favorable trade-offs between concept suppression, prompt alignment, and image quality. In human evaluation, our method outperforms the second-best baseline by 18.78%. We further show that our method supports multi-concept suppression and adjustable concept suppression.
Jungmin Ko, Jungwon Park, Jimyeong Kim +3
Interdisciplinary Program in Artificial Intelligence, Seoul National University · Research Institute for Convergence Science, Seoul National University · Daegu Gyeongbuk Institute of Science and Technology +3
Negation-aware image generation remains challenging for text-to-image diffusion models, which often struggle with negation due to implicit linguistic biases that are deeply entangled within text representations. As a result, bias-aligned semantics frequently dominate over intended negative constraints. To overcome this limitation, we propose a simple training-free method of Decompose and Recompose text embedding (Dero), which constructs a semantic-preserving subspace and a bias subspace using auxiliary prompts and performs semantic disentanglement based on bidirectional SVD. By decomposing entangled representations and selectively recomposing disentangled semantic components, Dero realigns text representations toward intended negation semantics while restoring residual stylistic features to preserve semantic fidelity and image quality. As a model-agnostic inference-time framework operating solely on text representations, Dero can be seamlessly applied to diverse diffusion architectures, including both U-Net and DiT based models. Extensive experiments on both object- and concept-level negation demonstrate consistent improvements in negation-aware image generation, achieving stronger text-image alignment, better semantic fidelity, and higher human preference scores.