MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation

Wenqing Tian; Hanyi Mao; Zhaocheng Liu; Lihua Zhang; Qiang Liu; Jian Wu; Liang Wang

cs.CVMar 23, 2026

MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation

Authors: Wenqing Tian, Hanyi Mao, Zhaocheng Liu, Lihua Zhang, Qiang Liu, Jian Wu, Liang Wang

Organizations: New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · ByteDance · The University of Chicago

Abstract

Subject-driven image generation is increasingly expected to support fine-grained control over multiple entities within a single image. In multi-reference workflows, users may provide several subject images, a background reference, and long, entity-indexed prompts to control multiple people within one scene. In this setting, a key failure mode is cross-subject attribute misbinding: attributes are preserved, edited, or transferred to the wrong subject. Existing benchmarks and metrics largely emphasize holistic fidelity or per-subject self-similarity, making such failures hard to diagnose. We introduce MultiBind, a benchmark built from real multi-person photographs. Each instance provides slot-ordered subject crops with masks and bounding boxes, canonicalized subject references, an inpainted background reference, and a dense entity-indexed prompt derived from structured annotations. We also propose a dimension-wise confusion evaluation protocol that matches generated subjects to ground-truth slots and measures slot-to-slot similarity using specialists for face identity, appearance, pose, and expression. By subtracting the corresponding ground-truth similarity matrices, our method separates self-degradation from true cross-subject interference and exposes interpretable failure patterns such as drift, swap, dominance, and blending. Experiments on modern multi-reference generators show that MultiBind reveals binding failures that conventional reconstruction metrics miss.

Figures & tables

Figure 1: Multi-reference, multi-subject generation takes multiple subject references, a background reference, and an entity-indexed prompt as input. Correct binding respects slot-specific instructions, while failures include but are not limited to drift (a subject deviates from its own target without being confused with other subjects), dominance (one reference dominates multiple subjects), swap between subjects, or blending where attributes are mixed across subjects. This diagram illustrates the patterns through variations in clothing across different subjects.

Benchmark/Resource	Multi ref.	Multi subj.	Paired target	Fine-grained Entity-level prompt	Misbinding diagnosis
Benchmarks
MRBench [ 25 ]	$\checkmark$	✗	Real	✗	✗
MultiRef-bench [ 12 ]	$\checkmark$	✗	Real+Synth	$\triangle$	✗
MultiBanana [ 13 ]	$\checkmark$	$\checkmark$	✗	✗	✗
XVerseBench [ 27 ]	$\checkmark$	$\checkmark$	✗	✗	✗
MS-Bench [ 10 ]	$\checkmark$	$\checkmark$	✗	$\triangle$	✗

Table 1: Comparison with related benchmarks and data resources.

\checkmark

indicates explicit support;

\triangle

indicates partial support via structured task specifications (e.g., prompt templates, image placeholders, preset bounding boxes, layout images, or task-level constraints) without full entity-level grounding; ✗ indicates not provided or not the focus; “–” indicates not applicable. For “Paired target”, we report the source of the target image when available.

Figure 2: Starting from a real target image $I_{\mathrm{gt}}$ , we segment subjects to obtain ground-truth crops, canonicalize each subject to build per-subject reference images, and inpaint the removed regions to produce a background reference. We then perform structured captioning and rewrite the resulting fields into a long, entity-indexed prompt. A multi-reference generator conditions on the subject references, background reference, and prompt to produce a synthesized image $I_{\mathrm{gen}}$ intended to reconstruct $I_{\mathrm{gt}}$ .

Dimension $d$	Specialist $g_{d}$	Similarity $\mathrm{sim}_{d}$
Face identity	InsightFace (Buffalo_L) [ 41 , 42 ]	cosine
Pose	ViTPose [ 43 ]	OKS
Appearance	Qwen3-VL-Emb.-8B [ 44 ]	cosine
Expression	Qwen3-VL-Emb.-8B	cosine

Table 2: Dimension-specific specialists used in MultiBind evaluation.

Model	FID $\downarrow$	CLIP-I $\uparrow$	DINO $\uparrow$	AES $\uparrow$	JS $\downarrow$	Matched $\uparrow$	Mean IoU $\uparrow$
Nano Banana Pro	80.51	0.87	0.81	5.06	0.0075	1294	0.40
GPT-Image-1.5	82.95	0.84	0.77	5.44	0.0086	1294	0.40
Seedream 4.5	81.52	0.85	0.77	4.92	0.0103	1282	0.34
Hunyuan-Image-3.0-Instruct	81.23	0.80	0.76	5.06	0.0129	1287	0.42
Qwen-Image-Edit-2511	94.26	0.76	0.56	4.59	0.0160	1080	0.31
OmniGen2	94.65	0.77	0.66	4.65	0.0166	1038	0.32

Table 3: Holistic metrics on MultiBind . The "Matched" metric reports the number of aligned subject slots (out of

1,305

), where the total reflects the subject count of the intersection of images successfully generated by all evaluated models.

Dim	Model	Subject-level (%)				Image-level (%)
		Suc. $\uparrow$	Conf. $\downarrow$	Inc. $\downarrow$	Drift $\downarrow$	Swap $\downarrow$	Dom. $\downarrow$	Bld. $\downarrow$
Face	Nano Banana Pro	84.0	13.1	5.8	2.8	2.0	3.7	25.3
	GPT-Image-1.5	82.3	12.8	7.8	4.9	1.5	3.9	24.4
	Seedream 4.5	58.6	36.1	12.1	5.3	3.2	14.5	53.7
	HunyuanImage-3	38.7	15.6	56.3	45.6	11.6	3.7	15.8
	Qwen-Image-Edit	43.7	36.2	43.8	20.1	17.1	11.7	31.5

Table 4: Subject- and image-level rates (%). Success, Confused, Inconsistent, and Drift are subject-level rates; Swap, Dominance, and Blending are image-level pattern rates. Abbrev.: Suc.=Success, Conf.=Confused, Inc.=Inconsistent, Dom.=Dominance, Bld.=Blending. HunyuanImage-3 denotes HunyuanImage-3-Instruct; Qwen-Image-Edit denotes Qwen-Image-Edit-2511.

Figure 3: Visualization of cross-subject attribute misbinding and our confusion-matrix diagnostics. Left: the ground-truth target image with slot-indexed subjects (1–3). Right: reconstructions from several multi-reference generators. For each output, we report the thresholded dimension-wise match matrices for Appearance and Face .

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

$N$	#instances	ratio
2	118	0.232
3	269	0.530
4	121	0.238

Appendix

Table 5: Distribution of person counts in MultiBind .

Source	#instances	ratio
CIHP	204	0.402
LV-MHP-v2	154	0.303
Objects365	133	0.262
COCO	17	0.033

Appendix

Table 6: Source distribution of instances in MultiBind .

Metric	Value
Count	508
Mean	473.84
Median	473.50
$p10$	350.70
$p90$	592.00
Min	255

Appendix

Table 7: Prompt length statistics of MultiBind , measured in words.

Side	Metric	Value
Long	Count	508
Long	Mean	624.38
Long	Median	500.00
Long	$p10$	500.00
Long	$p90$	805.10
Long	Min	256

Appendix

Table 8: Resolution statistics of MultiBind . Long-side and short-side values are reported in pixels.

Item	Setting
Precision	dtype=bfloat16
Denoising steps	num_inference_steps=40
CFG settings	true_cfg_scale=4.0 , guidance_scale=1.0

Appendix

Table 9: Generation settings for Qwen-Image-Edit-2511.

Item	Setting
Precision	dtype=bfloat16
Denoising steps	num_inference_steps=50
Guidance settings	text_guidance_scale=5.0 , image_guidance_scale=2.0

Appendix

Table 10: Generation settings for OmniGen2.

T_{\mathrm{area}}\leftarrow\alpha\cdot\min_{i}\frac{\mathrm{area}(B_{i}^{\mathrm{gt}})}{|I_{\mathrm{gt}}|}

Appendix

Algorithm 1 Instance extraction and slot matching ( topk_area_ltr )

Dimension $d$	Model	Sim $\uparrow$	$D_{\mathrm{self}}^{(d)}\downarrow$	$C_{\mathrm{worst}}^{(d)}\downarrow$	$C_{\mathrm{mean}}^{(d)}\downarrow$	$\mathrm{JS}^{(d)}\downarrow$
Face identity	Nano Banana Pro	0.3371	0.6629	0.0449	0.0266	0.0130
	GPT-Image-1.5	0.2911	0.7089	0.0418	0.0244	0.0145
	Seedream 4.5	0.2954	0.7046	0.0835	0.0533	0.0165
	HunyuanImage-3	0.0844	0.9156	0.0466	0.0281	0.0229
	Qwen-Image-Edit	0.1399	0.8601	0.0943	0.0578	0.0234
	OmniGen2	0.1272	0.8728	0.0895	0.0567	0.0240

Appendix

Table 11: Dimension-wise binding diagnostics. Sim is the mean diagonal of the raw gen-to-GT similarity matrix

S_{\mathrm{gen}}^{(d)}[i,i]

(higher is better; not comparable across dimensions).

D_{\mathrm{self}}^{(d)}

C_{\mathrm{worst}}^{(d)}

C_{\mathrm{mean}}^{(d)}

(clamped), and

\mathrm{JS}^{(d)}

are computed on baseline-corrected matrices (lower is better). HunyuanImage-3 denotes HunyuanImage-3.0-Instruct; Qwen-Image-Edit denotes Qwen-Image-Edit-2511.

Dimension $d$	$\tau^{(d)}_{\mathrm{cons}}$	$\tau^{(d)}_{\mathrm{conf}}$
Face identity	$-0.9111$	$0.1086$
Appearance	$-0.3662$	$0.1117$
Pose	$-0.5289$	$0.2912$
Expression	$-0.4203$	$0.0714$

Appendix

Table 12: Per-dimension thresholds calibrated on the human-labeled subset.

\tau^{(d)}_{\mathrm{cons}}

is used for diagonal self-consistency and

\tau^{(d)}_{\mathrm{conf}}

for off-diagonal confusion.

	Self-consistency AUC $\uparrow$			Cross-subject confusion AUC $\uparrow$
Dimension $d$	Ours ( $\Delta_{i,i}$ )	Gemini 2.5 Pro	GPT-5.2	Ours ( $\Delta_{i,j}$ )	Gemini 2.5 Pro	GPT-5.2
Face identity	0.8695	0.7823	0.7376	0.8455	0.6894	0.6076
Appearance	0.9292	0.8069	0.8184	0.8979	0.7586	0.7163
Pose	0.7608	0.6505	0.6192	0.7112	0.5941	0.4768
Expression	0.8034	0.8393	0.7107	0.7356	0.6705	0.6318

Appendix

Table 13: AUC validation against human labels using single-pass VLM judgments (higher is better). Left: self-consistency (diagonal pairs). Right: cross-subject confusion (off-diagonal pairs). We compare our pair-level delta scores with two VLM judges, Gemini 2.5 Pro and GPT-5.2.

Model	FID $\downarrow$	CLIP-I $\uparrow$	DINO $\uparrow$	AES $\uparrow$	JS $\downarrow$	Matched $\uparrow$	Mean IoU $\uparrow$
Nano Banana Pro	135.67	0.88	0.83	5.00	0.0076	254	0.42
GPT-Image-1.5	139.38	0.86	0.78	5.46	0.0076	255	0.42
Seedream 4.5	140.84	0.87	0.78	4.86	0.0082	252	0.33
HunyuanImage-3.0-Instruct	143.30	0.81	0.76	5.01	0.0109	255	0.43
Qwen-Image-Edit-2511	169.20	0.78	0.59	4.60	0.0142	218	0.28
OmniGen2	156.82	0.80	0.69	4.54	0.0152	218	0.29

Appendix

Table 14: Holistic metrics on the ablation subset of MultiBind when subject reference images are synthesized by Nano Banana Pro. The “Matched” metric reports the number of aligned subject slots within the all-model success intersection for this reference-generator setting.

Model	FID $\downarrow$	CLIP-I $\uparrow$	DINO $\uparrow$	AES $\uparrow$	JS $\downarrow$	Matched $\uparrow$	Mean IoU $\uparrow$
Nano Banana Pro	153.37	0.85	0.78	5.49	0.0079	253	0.37
GPT-Image-1.5	145.48	0.84	0.76	5.58	0.0077	253	0.40
Seedream 4.5	143.68	0.85	0.76	5.25	0.0085	252	0.34
HunyuanImage-3.0-Instruct	151.52	0.79	0.74	5.42	0.0098	254	0.41
Qwen-Image-Edit-2511	171.47	0.76	0.56	4.93	0.0129	220	0.27
OmniGen2	160.56	0.79	0.67	4.93	0.0130	229	0.32

Appendix

Table 15: Holistic metrics on the ablation subset of MultiBind when subject reference images are synthesized by GPT-Image-1.5. The “Matched” metric reports the number of aligned subject slots within the all-model success intersection for this reference-generator setting.

Model	FID $\downarrow$	CLIP-I $\uparrow$	DINO $\uparrow$	AES $\uparrow$	JS $\downarrow$	Matched $\uparrow$	Mean IoU $\uparrow$
Nano Banana Pro	139.04	0.88	0.82	5.02	0.0068	254	0.41
GPT-Image-1.5	138.84	0.85	0.79	5.39	0.0083	250	0.38
Seedream 4.5	129.09	0.89	0.84	4.58	0.0075	256	0.45
HunyuanImage-3.0-Instruct	139.47	0.81	0.77	4.93	0.0102	248	0.42
Qwen-Image-Edit-2511	166.17	0.78	0.59	4.60	0.0140	210	0.24
OmniGen2	152.04	0.81	0.71	4.47	0.0137	209	0.26

Appendix

Table 16: Holistic metrics on the ablation subset of MultiBind when subject reference images are synthesized by Seedream 4.5. The “Matched” metric reports the number of aligned subject slots within the all-model success intersection for this reference-generator setting.

Dim	Model	Subject-level (%)				Image-level (%)
		Suc. $\uparrow$	Conf. $\downarrow$	Inc. $\downarrow$	Drift $\downarrow$	Swap $\downarrow$	Dom. $\downarrow$	Bld. $\downarrow$
Face	Nano Banana Pro	85.6	11.9	4.7	2.5	1.3	3.8	24.1
	GPT-Image-1.5	83.5	10.6	7.2	5.9	0.0	2.5	21.5
	Seedream 4.5	61.2	32.3	11.2	6.5	3.8	10.3	51.3
	HunyuanImage-3	40.2	10.7	56.4	49.1	5.1	5.1	11.5
	Qwen-Image-Edit	45.6	36.3	40.4	18.1	19.2	15.4	30.8

Appendix

Table 17: Pattern rates (%) on the ablation subset when subject reference images are synthesized by Nano Banana Pro. We keep the binarization thresholds fixed to the values calibrated in the main paper. Success, Confused, Inconsistent, and Drift are subject-level rates; Swap, Dominance, and Blending are image-level pattern rates. Abbrev.: Suc.=Success, Conf.=Confused, Inc.=Inconsistent, Dom.=Dominance, Bld.=Blending. HunyuanImage-3 denotes HunyuanImage-3.0-Instruct; Qwen-Image-Edit denotes Qwen-Image-Edit-2511.

Dim	Model	Subject-level (%)				Image-level (%)
		Suc. $\uparrow$	Conf. $\downarrow$	Inc. $\downarrow$	Drift $\downarrow$	Swap $\downarrow$	Dom. $\downarrow$	Bld. $\downarrow$
Face	Nano Banana Pro	76.2	11.9	15.7	11.9	0.0	6.3	20.3
	GPT-Image-1.5	73.5	9.4	20.1	17.1	1.3	3.8	17.7
	Seedream 4.5	67.4	22.3	16.3	10.3	1.3	7.6	36.7
	HunyuanImage-3	30.5	11.0	66.5	58.5	10.1	1.3	11.4
	Qwen-Image-Edit	45.5	28.5	44.5	26.0	19.0	6.3	22.8

Appendix

Table 18: Pattern rates (%) on the ablation subset when subject reference images are synthesized by GPT-Image-1.5. We keep the binarization thresholds fixed to the values calibrated in the main paper. Success, Confused, Inconsistent, and Drift are subject-level rates; Swap, Dominance, and Blending are image-level pattern rates. Abbrev.: Suc.=Success, Conf.=Confused, Inc.=Inconsistent, Dom.=Dominance, Bld.=Blending. HunyuanImage-3 denotes HunyuanImage-3.0-Instruct; Qwen-Image-Edit denotes Qwen-Image-Edit-2511.

Dim	Model	Subject-level (%)				Image-level (%)
		Suc. $\uparrow$	Conf. $\downarrow$	Inc. $\downarrow$	Drift $\downarrow$	Swap $\downarrow$	Dom. $\downarrow$	Bld. $\downarrow$
Face	Nano Banana Pro	80.9	12.8	9.4	6.4	2.5	7.6	24.1
	GPT-Image-1.5	74.9	14.7	13.9	10.4	3.9	1.3	28.6
	Seedream 4.5	66.4	27.7	11.3	5.9	2.5	3.8	40.5
	HunyuanImage-3	33.2	14.4	63.3	52.4	7.6	2.5	11.4
	Qwen-Image-Edit	41.7	39.6	43.9	18.7	18.2	14.3	41.6

Appendix

Table 19: Pattern rates (%) on the ablation subset when subject reference images are synthesized by Seedream 4.5. We keep the binarization thresholds fixed to the values calibrated in the main paper. Success, Confused, Inconsistent, and Drift are subject-level rates; Swap, Dominance, and Blending are image-level pattern rates. Abbrev.: Suc.=Success, Conf.=Confused, Inc.=Inconsistent, Dom.=Dominance, Bld.=Blending. HunyuanImage-3 denotes HunyuanImage-3.0-Instruct; Qwen-Image-Edit denotes Qwen-Image-Edit-2511.

Figure 4: Additional qualitative example 1.

Figure 5: Additional qualitative example 2.

Figure 6: Additional qualitative example 3.

Figure 7: Additional qualitative example 4.

Figure 8: Additional qualitative example 5.

Multi-Subject Image Generation

See all 13 papers

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding
Aug 4, 2026Ruirui Zhang, Zhengkai Zhao, Pan GaoImage Generation Compositional T2I Generation
Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology Nanjing, Jiangsu, China · Nanjing University of Aeronautics and Astronautics College of Artificial Intelligence Nanjing, Jiangsu, China
MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation
Jul 1, 2026Zhihan Chen, Yuhuan Zhao, Yijie Zhu +7Subject-Driven Image Generation Identity-Preserving Image Generation
University of California, Los Angeles · University of Southern California · DeerLab LLC +5
CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation
Jun 14, 2026Long-Bao Nguyen, Quang-Khai Le, Tam V. Nguyen +2Compositional Image Generation Identity-Preserving Image Generation
University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam · University of Dayton, Ohio, United States

Aug 4, 2026J/K move · Enter open · S save

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

Ruirui Zhang, Zhengkai Zhao, Pan Gao

Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology Nanjing, Jiangsu, China · Nanjing University of Aeronautics and Astronautics College of Artificial Intelligence Nanjing, Jiangsu, China

Image Generation Compositional T2I Generation T2I Generation Evaluation T2I Diffusion Models Multi-Subject Image Generation

Open paper AskPDF arXiv

Text-to-image diffusion models enable personalization of specific visual concepts from a small number of reference images. However, generating a single image that contains multiple personalized subjects, each bound to user-specified attributes such as clothing, accessories, and held objects, remains largely unaddressed. Without explicit spatial constraints, concurrently activated concept checkpoints produce overlapping cross-attention responses, causing per-subject identity degradation and attribute misalignment. Moreover, no established benchmark jointly evaluates these two failure modes in the personalized multi-subject setting. We present MultiCompose, a composition framework that decouples per-concept personalization from multi-subject inference. A semantic preservation regularization maintains attribute binding capacity during fine-tuning, while a two-phase inference procedure automatically establishes subject layout and composes per-concept predictions through spatially exclusive masks. We further introduce MSP-Bench, a benchmark that jointly evaluates identity fidelity (ID), attribute binding accuracy (BIND), and attribute misalignment (MIS) through a dual-pathway protocol. Experiments show that MultiCompose outperforms existing methods on both conventional metrics and MSP-Bench, confirming the benchmark's ability to reveal failure modes that conventional metrics overlook. Code is available at https://github.com/I2-Multimedia-Lab/MultiCompose

Abstract

Multi-Subject Image Generation

Explore similar work

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation

CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation