LEVI: Benchmarking Visual Identity Fidelity with Verifiable Low-Entropy Constructs

Xiao Ma; Zhonghao Zhan; Yi Tang; Qi Chen

cs.CVOct 19, 2026

LEVI: Benchmarking Visual Identity Fidelity with Verifiable Low-Entropy Constructs

Authors: Xiao Ma, Zhonghao Zhan, Yi Tang, Qi Chen

Organizations: ZODA · Imperial College London

Abstract

Modern generative models can produce visually plausible images while violating exact identity constraints. Evaluating such failures requires distinguishing violations of the requested identity from permissible variation in appearance. We introduce Low-Entropy Visual Identity (LEVI), which characterizes visual identities determined by compact facts and explicit rules. The low-entropy requirement concerns identity uncertainty, while appearance may remain diverse. We formulate identity fidelity in terms of the probability that a conditional image distribution assigns to the identities permitted by the prompt. Visual Identity Spaces (VIS) formalize this distinction by specifying admissible identity values and which different realizations express the same identity. Building on this formulation, we develop Verifiable Visual Grammar (VVG), a declarative language whose typed primitives and compositional constraints describe identity-bearing structures. Using this grammar, we build a benchmark for image generation. It assesses each output against the task's identity requirements, drawing evidence from the final pixels. The curated benchmark contains 1,375 tasks spanning 120 base identity spaces across six domains. Among ten evaluated model configurations, the highest whole-task pass rate is 58.69%. These results reveal a persistent gap between visually plausible images and faithful realization of their specified identities. Project page: https://maxiao0234.github.io/LEVI/.

Figures & tables

Figure 1: Visual plausibility can coexist with identity errors. Each domain pairs matching ( $\checkmark$ ) and failing ( $\times$ ) examples for one prompt. Target/Fail rows identify the mismatch; header entries list representative VIS. All examples are conceptual.

Benchmark	Structured domains	Rule-governed targets	Executable checks	Verifiable construction	Identity-space expressivity	Identity equivalence
STRICT	$\times$	$\times$	$\checkmark$	$\times$	$\times$	$\times$
MMMG	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\times$
StructBench	$\checkmark$	$\times$	$\times$	$\times$	$\times$	$\times$
SpatialGenEval	$\times$	$\times$	$\times$	$\times$	$\times$	$\times$
MathGen	$\times$	$\checkmark$	$\checkmark$	$\times$	$\times$	$\times$
GraphicDesignBench	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	$\times$

Table 1: Comparison with recent image-generation benchmarks.

\checkmark

denotes explicit support in the primary protocol and

\times

denotes no explicit support. LEVI denotes the proposed design.

Figure 2: From visual identity to output-based verification. Top: VIS groups equivalent realizations and combines compatible identities; VVG describes a structure $g$ whose interpretation is an identity $z$ . Types, attributes, and relations jointly define admissible descriptions. Bottom: task authoring supplies a complete text condition, generation produces pixels, and verification checks evidence against the shared identity conditions. Robots illustrate conceptual distinctions in identity understanding.

Figure 3: Domain coverage and top-five model profiles. (a) B-VIS catalogue share (grey) and task involvement (colour). Whole-task pass rates (%) are (b) B-VIS-balanced or (c) task-weighted. Radar axes share a 0–100% scale; domain subsets overlap.

Table 2: Whole-task pass rates (%,

\uparrow

) on the post-hoc curated challenge set. Medals mark the top three overall; bold / underline mark the best/second-best per column.

Task structure	Share (%)	Mean	Sunburst	Flare	GPT Image 2
One B-VIS type	10.40	35.45	62.24	52.45	49.65
Two B-VIS types	30.62	35.46	56.06	52.97	50.59
Three or more B-VIS types	58.98	38.31	59.43	57.71	55.24
Single-domain	26.69	34.11	55.31	50.95	50.41
Cross-domain	73.31	38.24	59.92	57.44	54.27

Table 3: Whole-task pass rates (%) by task structure. Share is the fraction of tasks in each group; Mean averages all ten models. The two groupings partition the same task set separately.

Panel of Figure 4 — Figure 4: Identity-level comparisons, one task per row. Vector references illustrate the text-only targets; outputs are uncropped. Pass/fail denotes whole-task assessment.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Operation	Example	Semantic effect
Composition	Pair $z$ with its count: $(z,3)$ is compatible; $(z,4)$ is not.	Retains component identities and their shared-object constraints.
Projection	$P(z)=\|z\|=3$ .	Keeps count; discards symbol values and order.
Transformation	Reverse $z$ to $(\square,\circ,\triangle)$ .	Changes identity according to a specified rule.
Refinement	Replace a count-only specification with an ordered-symbol specification.	Separates identities previously grouped together.

Appendix

Table 4: Four operations illustrated with the same symbol sequence.

Figure 5: Twenty-four tasks across six identity domains. All images are uncropped GPT Image 2.5 Sunburst outputs. Labels describe the identity requirement. The examples illustrate task diversity; they are not a sample for estimating error frequency.

Figure 6: Symbolic: note positions and notehead types. The schematic illustrates the required identity; labels show automatic judgments.

Figure 7: Geometric: a rotated photograph with a level horizon. The schematic illustrates the required identity; labels show automatic judgments.

Figure 8: Topological: alternating over-under crossings. The schematic illustrates the required identity; labels show automatic judgments.

Figure 9: Layout: rotating a window pattern. The schematic illustrates the required identity; labels show automatic judgments.

Figure 10: Quantitative: counts in four wing regions. The schematic illustrates the required identity; labels show automatic judgments.

Figure 11: Procedural: a partially piped heart. The schematic illustrates the required identity; labels show automatic judgments.

Configuration	Sym.	Geom.	Topo.	Layout	Quant.	Proc.
GPT Image 2.5 Sunburst	55.70	55.58	63.73	64.81	39.52	43.89
GPT Image 2.5 Flare	48.03	57.57	57.80	61.35	38.76	36.11
GPT Image 2	49.46	49.72	55.82	54.63	28.04	37.49
Qwen Image 3 Pro	36.24	31.75	41.29	47.90	18.07	26.92
Seedream 5 Pro	27.55	32.74	33.08	32.17	13.75	15.59
Nano Banana 2	29.93	31.05	28.87	28.03	14.11	25.18

Appendix

Table 5: B-VIS-balanced whole-task pass rates (%). Domain columns use equal weight per represented B-VIS.

Figure 12: Whole-task pass rates (%) for the two most frequent B-VIS in each domain. Columns overlap when a task involves several types.

Configuration	1 type $n=143$	2 types $n=421$	3+ types $n=811$	Single domain $n=367$	Cross-domain $n=1008$
GPT Image 2.5 Sunburst	62.24	56.06	59.43	55.31	59.92
GPT Image 2.5 Flare	52.45	52.97	57.71	50.95	57.44
GPT Image 2	49.65	50.59	55.24	50.41	54.27
Qwen Image 3 Pro	42.66	37.53	40.32	37.60	40.48
Seedream 5 Pro	28.67	31.83	35.51	29.97	35.02
Nano Banana 2	32.87	33.97	31.57	31.34	32.84

Appendix

Table 6: Whole-task pass rates (%) by task composition. The number of types refers to distinct B-VIS, not object instances.

Passing configurations	Tasks	Share (%)
0	243	17.67
1	170	12.36
2	132	9.60
3	149	10.84
4	142	10.33
5	129	9.38

Appendix

Table 7: Distribution of the number of passing configurations per task, out of ten.

A / B	Both pass	A only	B only	Neither
Sunburst / Flare	611	196	155	413
Sunburst / GPT Image 2	602	205	130	438
Flare / GPT Image 2	572	194	160	449

Appendix

Table 8: Paired task outcomes for the three highest-scoring configurations. A and B refer to the left and right names; every row sums to 1,375.

Image Generation Evaluation

Image generation evaluation measures the quality, fidelity, diversity, and instruction-following ability of AI-generated images.

See all 113 papers

ID-Sim: An Identity-Focused Similarity Metric
Apr 6, 2026Julia Chae, Nicholas Kolkin, Jui-Hsien Wang +3Image Generation Evaluation
MIT CSAIL · Adobe Research
WithEveryone: Unified Planning and Identity Grounding for Group Image Generation
Aug 20, 2026Hengyuan Xu, Qixun Wang, Yiji Cheng +5Subject-Driven Image Generation Image Generation
1Fudan University · 2Hunyuan, Tencent · 3The University of Hong Kong
Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System
Sep 3, 2026Mengwei Ren, Xuaner Zhang, Zhihao XiaSubject-Driven Image Generation Identity-Preserving Image Generation
Phota Labs Research

Apr 6, 2026J/K move · Enter open · S save

ID-Sim: An Identity-Focused Similarity Metric

Julia Chae, Nicholas Kolkin, Jui-Hsien Wang, Richard Zhang+2

MIT CSAIL · Adobe Research

Image Generation Evaluation

Open paper AskPDF arXiv

Humans have remarkable selective sensitivity to identities -- easily distinguishing between highly similar identities, even across significantly different contexts such as diverse viewpoints or lighting. Vision models have struggled to match this capability, and progress toward identity-focused tasks such as personalized image generation is slowed by a lack of identity-focused evaluation metrics. To help facilitate progress, we propose ID-Sim, a feed-forward metric designed to faithfully reflect human selective sensitivity. To build ID-Sim, we curate a high-quality training set of images spanning diverse real-world domains, augmented with generative synthetic data that provides controlled, fine-grained identity and contextual variations. We evaluate our metric on a new unified evaluation benchmark for assessing consistency with human annotations across identity-focused recognition, retrieval, and generative tasks.

Abstract

Image Generation Evaluation

Explore similar work

ID-Sim: An Identity-Focused Similarity Metric

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System