Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?

Hongyu Li; Kuan Liu; Yuan Chen; Juntao Hu; Huimin Lu; Guanjie Chen; Xue Liu; Guangming Lu

cs.CVFeb 26, 2026

Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?

Authors: Hongyu Li, Kuan Liu, Yuan Chen, Juntao Hu, Huimin Lu, Guanjie Chen, Xue Liu, Guangming Lu,

Organizations: Harbin Institute of Technology, Shenzhen · City University of Hong Kong · Chinese University of Hong Kong · McGill University

Abstract

Recent advances in generative AI have shown human-level performance in complex content creation. However, we identify a "Paradox of Simplicity": models that can render complex scenes often fail at trivial, low-entropy tasks, such as generating a uniform pure color image. We argue this is a systemic failure related to uncontrollable emergent abilities. As models scale, strong priors for aesthetics and complexity override deterministic simplicity, creating an "aesthetic bias" that hinders the model's transition from data simulation to true intellectual abstraction. To better investigate this problem, we formalize the concept of AI Obedience, a hierarchical framework that grades a model's ability to transition from probabilistic approximation to pixel-level determinism (Levels 1 to 5). We introduce Violin, the first systematic benchmark designed to evaluate Level 4 Obedience through three deterministic tasks: color purity, image masking, and geometric shape generation. Using Violin, we evaluate several state-of-the-art models and reveal that closed-source models generally outperform open-source ones in deterministic precision. Interestingly, performance on our benchmark correlates with the benchmark in natural image generation. Our work provides a foundational framework and tools for achieving better alignment between human instructions and model outputs.

Figures & tables

Figure 1: Visualization of instruction-following failures in pure color generation. Instead of adhering strictly to input instructions, models reflexively introduce spurious artifacts and gradients, highlighting a critical bottleneck in precise generative control.

Figure 2: Illustration of obedience levels in the vision task. Each column shows the prompt, the expected output, and a failure case.

Figure 3: Diagnostic case studies (a) Logical Inhibition Failure: Negative prompts (“no gradient”) fail to remove artifacts, and mentioning a semantic object to avoid (“ripples”) causes the model to generate instead. (b) Semantic Gravity: The model follows color instructions better when they align with common knowledge (“rusted iron”), but drifts when the context is conflicting or random. (c) Aesthetic Inertia: Precise spatial ratios (31.5%) are ignored in favor of a standard 50/50 split.

Figure 4: Overview of the Violin Benchmark. The benchmark evaluates Level-4 visual obedience through three tasks: (a) Pure Color Generation; (b) Geometric Shape Generation; and (c) Image Masking for strict spatial accuracy across inpainting, outpainting, and random masks.

Var	Models	rgb-ed	lab-00	sd	ced	hf	color-mean
Var-1	FLUX.1	0.206	0.167	0.064	0.006	0.016	0.091
	FLUX.2	0.123	0.091	0.044	0.007	0.021	0.057
	Z-Image	0.135	0.092	0.078	0.002	0.007	0.061
	Qwen-Image	0.122	0.084	0.047	0.002	0.017	0.057
	Nano-Banana-2	0.126	0.093	0.033	0.001	0.010	0.053
	Seedream-5	0.134	0.093	0.015	0.001	0.001	0.049

Table 1: Comparison of Level-4 Obedience on Pure Color Generation Task.

Models	iou	biou	leak	edge	dist	shape-mean
FLUX.2	0.848	0.948	0.438	0.131	0.197	0.512
Nano-Banana-2	0.474	0.772	0.286	0.172	0.127	0.366
Seedream-5	0.331	0.725	0.177	0.073	0.093	0.280
GPT-Image-2	0.096	0.398	0.087	0.186	0.023	0.158

Table 2: Comparison of Level-4 Obedience on Image Mask Task.

Models	iou	size	shape	purity	dist	mask-mean
FLUX.1	0.725	0.703	0.167	0.172	0.267	0.407
FLUX.2	0.607	0.659	0.004	0.127	0.122	0.304
Z-Image	0.648	0.686	0.002	0.133	0.409	0.376
Qwen-Image	0.645	0.630	0.001	0.283	0.128	0.337
Nano-Banana-2	0.566	0.597	0.046	0.105	0.097	0.282
Seedream-5	0.551	0.376	0.006	0.035	0.070	0.207

Table 3: Comparison of Level-4 Obedience on Geometric Shape Generation Task.

Figure 5: Consistency between Violin and Natural Image Benchmark The proposed VIiolin (L4) maintains a consistent performance trend with existing L1/L2 benchmarks across diverse models.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Example Prompts
Color Purity Var-1	• "Generate an image with pure pink color." • "Create a featureless entirely filled with red color." • "Generate a perfectly uniform image in pink color. No texture, no gradient, no shadows."
Color Purity Var-2	• "Create an image divided vertically into two even halves: left in reddish purple, right in yellowish pink." • "Produce a background with the top half in yellowish brown and the bottom half in greenish blue. Colors must remain flat and consistent throughout." • "Create a flat background divided horizontally: lower half in violet color, upper half in black color. Each section should remain perfectly solid."
Image-Mask	• "Apply the binary mask <mask> to the image <image>. For every pixel, if the mask value is 1, keep the original color from the image; if the mask value is 0, change it to pure black." • "Perform a pixel-wise multiplication between image <image> and mask <mask>. Use the mask as a filter where 1 represents areas to be preserved and 0 represents areas to be changed to black. Output the resulting image." • "Create an output image by mapping pixels from <image> onto binary mask <mask>: if a mask pixel is 1, use the source image color; if a mask pixel is 0, use pure black."
Geometric Task	• "Generate a precise image of a solid white circle on a black background. Its diameter must be equal to 20% of the image width, with its bounding box aligned with the left edge, vertically centered." • "Create an image featuring a single solid white axis-aligned square on a black background, with a side length of 20% image width, with its bounding box aligned with the left edge, vertically centered." • "Generate a precise image of a solid white upright equilateral triangle on a black background. Its side length must be equal to 20% of the image width, with its bounding box aligned with the left edge, vertically centered. The edges must be perfectly sharp with no textures."

Appendix

Table 4: Representative prompt examples for each task in Violin

Figure 6: Failure cases on other low-entropy deterministic tasks.

Color Name	RGB	HEX	Color Name	RGB	HEX
Pink	(230, 134, 151)	#E68697	Red	(185, 40, 66)	#B92842
Yellowish pink	(234, 154, 144)	#EA9A90	Reddish orange	(215, 71, 42)	#D7472A
Reddish brown	(122, 44, 38)	#7A2C26	Orange	(220, 125, 52)	#DC7D34
Brown	(127, 72, 41)	#7F4829	Orange yellow	(227, 160, 69)	#E3A045
Yellowish brown	(151, 107, 57)	#976B39	Yellow	(217, 180, 81)	#D9B451
Olive brown	(127, 97, 41)	#7F6129	Greenish yellow	(208, 196, 69)	#D0C445

Appendix

Table 5: Color Names, RGB, and HEX Values in ISCC-NBS system Level 2

Variation	Example Prompt
1	Generate an image with pure color #D9B451 (Hex code).
2	Specified in Hex code, produce a background divided horizontally into two equal halves: top color #72672C and bottom color #D9B451.
3	Specified in Hex code, generate an image divided into four quadrants: top-left color #7F4829, top-right color #B92842, bottom-left color #7F4829, bottom-right color #3B74C0. No gradients or shadows.
4	Specified in Hex code, generate an image in a single solid color within the range of #58321D and #A65E35.
5	生成一张颜色为 #B92842（十六进制代码）的纯色图像。
6	Generate a plain background using color rgb(217, 180, 81).

Appendix

Table 6: Representative prompt examples for each variation in Violin-Absolute-Color Benchmark.

Var	Models	rgb-ed	lab-00	sd	ced	hf	color-mean
Var-1	SANA	0.288	0.331	0.232	0.028	0.030	0.182
	Janus-Pro-1.5	0.344	0.410	0.193	0.006	0.004	0.191
	FLUX.1	0.364	0.387	0.044	0.001	0.001	0.159
	Qwen-Image	0.156	0.180	0.058	0.002	0.021	0.083
	OmniGen2	0.397	0.402	0.202	0.070	0.016	0.217
Var-2	SANA	0.315	0.352	0.312	0.019	0.021	0.204

Appendix

Table 7: Comprehensive Evaluation on Violin-Absolute-Color (Var 1-6)

Models	rgb-ed	lab-00	sd	ced	hf	color-mean	Change
Var-1	0.093	0.124	0.023	0.001	0.001	0.048	$\downarrow$ 0.035
Var-2	0.088	0.110	0.027	0.003	0.005	0.046	$\downarrow$ 0.037
Var-3	0.163	0.186	0.042	0.006	0.005	0.080	$\downarrow$ 0.062
Var-4	0.237	0.266	0.052	0.001	0.003	0.111	$\downarrow$ 0.035
Var-5	0.103	0.140	0.023	0.001	0.004	0.054	$\downarrow$ 0.030
Var-6	0.190	0.227	0.011	0.001	0.001	0.086	$\downarrow$ 0.042

Appendix

Table 8: Fine-tuning analysis using Violin-Absolute-Color on the Qwen-Image model. Results report the performance after fine-tuning the model individually on each variation (Var 1–6).

Models	rgb-ed	lab-00	sd	ced	hf	color-mean	Change
SANA	0.292	0.330	0.211	0.001	0.027	0.172	$\downarrow$ 0.010
Janus-Pro-1.5	0.090	0.121	0.010	0.001	0.001	0.045	$\downarrow$ 0.146
FLUX.1	0.227	0.283	0.004	0.000	0.001	0.103	$\downarrow$ 0.056
Qwen-Image	0.167	0.208	0.018	0.001	0.001	0.079	$\downarrow$ 0.004
OmniGen2	0.413	0.412	0.159	0.060	0.014	0.212	$\downarrow$ 0.005

Appendix

Table 9: Fine-tuning analysis using Violin-Absolute-Color on Variation 1 across different open-source models. Results report the performance after fine-tuning the model individually.

Figure 7: Performance degradation on GenEval after fine-tuning. Comparison between the base and LoRA fine-tuned models reveals a noticeable loss in general generative abilities across multiple semantic and spatial tasks.

Models	rgb-ed	lab-00	sd	ced	hf	color-mean
Prompt-Split	0.095	0.113	0.048	0.004	0.010	0.054
Hue-Split1	0.127	0.156	0.023	0.003	0.002	0.062
Hue-Split2	0.128	0.158	0.034	0.003	0.004	0.065

Appendix

Table 10: Evaluating the model’s zero-shot generalization on three designed color partition strategies.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

GenOlympiad: Benchmarking Frontier Image Models at Their Capability Boundaries
Oct 19, 2026Xi Hu, Martin Bell, Liang ZhengImage Generation Evaluation T2I Generation Evaluation
Australian National University · Canva
ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks
Mar 29, 2026Samin Mahdizadeh Sani, Max Ku, Nima Jamali +23Image Generation Evaluation T2I Generation Evaluation
University of Waterloo · G-G-G · Independent +1
CoDA: Color Distribution Probing for Efficient and Generalizable AI-Generated Image Detection
May 23, 2026Zexi Jia, Zhiqiang Yuan, Xiaoyue Duan +3AI-Generated Image Detection
Tencent WeChat AI, Beijing, China · Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA

Oct 19, 2026J/K move · Enter open · S save

GenOlympiad: Benchmarking Frontier Image Models at Their Capability Boundaries

Xi Hu, Martin Bell, Liang Zheng

Australian National University · Canva

Image Generation Evaluation T2I Generation Evaluation

Open paper AskPDF arXiv

Text-to-image models have made tremendous progress and can produce high-quality images. However, a question which would be asked in any other discipline is where the frontier models fall short. We introduce GenOlympiad, a benchmark which provides the most challenging text prompts from nine capability axes for stress-testing frontier text-to-image models. The assessed capabilities include generating large quantities, performing deep reasoning, ensuring hard physical consistency, following complex rules and so on. The prompts are designed such that prompt alignment is visually verifiable (e.g., counting) instead of relying on subjectiveness (e.g., style). Based on this, we develop an automated evaluation framework which decomposes the prompt into easy and hard questions, and for most hard questions, draws help lines to ground evidence for accurate grading. We evaluate 20+ strong text-to-image models, including Nano Banana Pro, GPT-Image-2.5, Seedream-5.0 and so on. We report that even the best-performing T2I model (Nano Banana Pro) only achieves a strict score of 23.9 out of 100 and that coding models such as GPT-6 Astra have higher prompt alignment scores (but not as visually pleasing). We also derive insights in model evolution, alignment with Elo scores, and effectiveness of our grounding-based evaluation protocol. Code is available at https://github.com/GenOlympiad/GenOlympiad and the project page is at https://genolympiad.github.io/

Abstract

Image Generation

Explore similar work

GenOlympiad: Benchmarking Frontier Image Models at Their Capability Boundaries

ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks

CoDA: Color Distribution Probing for Efficient and Generalizable AI-Generated Image Detection