cs.CVMar 27, 2026

When Identities Collapse: A Stress-Test Benchmark for Multi-Subject Personalization

Authors: Zhihan Chen, Yuhuan Zhao, Yijie Zhu, Xinyu Yao

Organizations: University of California, Los Angeles · University of Southern California · DeerLab LLC · Carnegie Mellon University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Subject-driven text-to-image diffusion models have achieved remarkable success in preserving single identities, yet their ability to compose multiple interacting subjects remains largely unexplored and highly challenging. Existing evaluation protocols typically rely on global CLIP metrics, which are insensitive to local identity collapse and fail to capture the severity of multi-subject entanglement. In this paper, we identify a pervasive "Illusion of Scalability" in current models: while they excel at synthesizing 2-4 subjects in simple layouts, they suffer from catastrophic identity collapse when scaled to 6-10 subjects or tasked with complex physical interactions. To systematically expose this failure mode, we construct a rigorous stress-test benchmark comprising 75 prompts distributed across varying subject counts and interaction difficulties (Neutral, Occlusion, Interaction). Furthermore, we demonstrate that standard CLIP-based metrics are fundamentally flawed for this task, as they often assign high scores to semantically correct but identity-collapsed images (e.g., generating generic clones). To address this, we introduce the Subject Collapse Rate (SCR), a novel evaluation metric grounded in DINOv2's structural priors, which strictly penalizes local attention leakage and homogenization. Our extensive evaluation of state-of-the-art models (MOSAIC, XVerse, PSR) reveals a precipitous drop in identity fidelity as scene complexity grows, with SCR approaching 100% at 10 subjects. We trace this collapse to the semantic shortcuts inherent in global attention routing, underscoring the urgent need for explicit physical disentanglement in future generative architectures.

Figures & tables

Figure 1: Catastrophic Identity Collapse in Multi-Subject Personalization. (Left) Our quantitative analysis reveals the illusion of scalability: as the number of interacting subjects ( NN ) increases, the fine-grained identity fidelity (DINOv2) plummets, and the Subject Collapse Rate (SCR) skyrockets to >95%>95\% . (Middle) At N=2N=2 , state-of-the-art models (e.g., MOSAIC [ 15 ] ) successfully maintain distinct identities and structural integrity. (Right) When evaluated on our proposed stress-test benchmark at N=8N=8 , models experience severe attention leakage, resulting in identity bleeding and homogenization (generating clones of a single dominant identity).
Figure 2: Multi-Subject Benchmark Construction. Our pipeline samples identities from a unified subject pool (left) to populate prompts of increasing complexity (2 to 10 subjects). Prompts are systematically categorized into Neutral, Occlusion, and Interaction scenarios (right) to isolate and test specific failure modes in attention routing and geometric reasoning.
Figure 3: Subject Collapse Rate (SCR). Unlike average similarity scores which mask individual failures, SCR explicitly counts the proportion of subjects whose DINOv2 identity similarity falls below a strict threshold τ\tau . This provides a more realistic measure of multi-subject entanglement.
SubjectsMOSAIC [ 15 ]XVerse [ 4 ]PSR [ 17 ]
CLIP-T ↑\uparrowCLIP-I ↑\uparrowDINOv2 ↑\uparrowSCR ↓\downarrowCLIP-T ↑\uparrowCLIP-I ↑\uparrowDINOv2 ↑\uparrowSCR ↓\downarrowCLIP-T ↑\uparrowCLIP-I ↑\uparrowDINOv2 ↑\uparrowSCR ↓\downarrow
20.2610.6950.42548.9%0.2680.6460.35558.9%0.2740.6470.32563.3%
40.2890.5860.23581.7%0.2790.5930.21180.0%0.2920.5680.18985.6%
60.2970.5350.16490.7%0.2750.5500.14293.0%0.3020.5370.13694.1%
80.3020.5170.12694.7%0.2760.5320.12394.4%0.3040.5170.11795.0%
100.3000.5040.11096.0%0.2830.5240.10496.4%0.3090.5000.10197.8%
Table 1: Quantitative Evaluation on Multi-Subject Personalization Benchmark. We report CLIP-T, CLIP-I, DINOv2, and Subject Collapse Rate ([email protected]) across varying subject counts (2 to 10). As subject count increases, DINOv2 sharply declines and SCR approaches 100%, indicating catastrophic identity failure.
Figure 4: Performance Trends across Subject Counts. (Left) DINOv2 identity similarity exhibits a precipitous drop for all models as scenes become denser. (Right) Subject Collapse Rate ([email protected]) skyrockets from ∼\sim 50% at 2 subjects to nearly 100% at 8-10 subjects, highlighting a fundamental scalability bottleneck.
Figure 5: Comprehensive Metric Radar (2 Subjects). A multi-dimensional comparison of MOSAIC, XVerse, and PSR. While all models maintain high semantic alignment (CLIP-T), MOSAIC shows a distinct advantage in structural (DINO) and fine-grained identity (DINOv2) preservation.
Figure 6: Qualitative Failure Analysis. Comparison between different models across various subject counts and scene types. In denser scenes (4+ subjects) or complex interactions, models exhibit severe (1) Identity Bleeding, where features merge; (2) Homogenization, generating clones of a single dominant identity; and (3) Amodal Collapse, resulting in malformed geometry under occlusion.
Figure 7: Detailed Case Analysis of Identity Bleeding. (Left) Reference images of two distinct subjects. (Right) Generation results for a complex interaction prompt (“A black woman and a western woman shaking hands”). While the MOSAIC baseline successfully maintains identity boundaries (marked with green check), XVerse and PSR suffer from severe identity bleeding, where the features of Subject A contaminate the spatial region of Subject B (marked with red cross).
More in this topic

Multi-Subject Image Generation

See all 13 papers

Explore similar work

CardsList
Aug 4, 2026

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding

Text-to-image diffusion models enable personalization of specific visual concepts from a small number of reference images. However, generating a single image that contains multiple personalized subjects, each bound to user-specified attributes such as clothing, accessories, and held objects, remains largely unaddressed. Without explicit spatial constraints, concurrently activated concept checkpoints produce overlapping cross-attention responses, causing per-subject identity degradation and attribute misalignment. Moreover, no established benchmark jointly evaluates these two failure modes in the personalized multi-subject setting. We present MultiCompose, a composition framework that decouples per-concept personalization from multi-subject inference. A semantic preservation regularization maintains attribute binding capacity during fine-tuning, while a two-phase inference procedure automatically establishes subject layout and composes per-concept predictions through spatially exclusive masks. We further introduce MSP-Bench, a benchmark that jointly evaluates identity fidelity (ID), attribute binding accuracy (BIND), and attribute misalignment (MIS) through a dual-pathway protocol. Experiments show that MultiCompose outperforms existing methods on both conventional metrics and MSP-Bench, confirming the benchmark's ability to reveal failure modes that conventional metrics overlook. Code is available at https://github.com/I2-Multimedia-Lab/MultiCompose

Ruirui Zhang, Zhengkai Zhao, Pan Gao

Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology Nanjing, Jiangsu, China · Nanjing University of Aeronautics and Astronautics College of Artificial Intelligence Nanjing, Jiangsu, China

Image GenerationCompositional T2I GenerationT2I Generation EvaluationT2I Diffusion ModelsMulti-Subject Image Generation
AskPDF
Mar 9, 2026

DSH-Bench: A Difficulty- and Scenario-Aware Benchmark with Hierarchical Subject Taxonomy for Subject-Driven Text-to-Image Generation

Significant progress has been achieved in subject-driven text-to-image (T2I) generation, which aims to synthesize new images depicting target subjects according to user instructions. However, evaluating these models remains a significant challenge. Existing benchmarks exhibit critical limitations: 1) insufficient diversity and comprehensiveness in subject images, 2) inadequate granularity in assessing model performance across different subject difficulty levels and prompt scenarios, and 3) a profound lack of actionable insights and diagnostic guidance for subsequent model refinement. To address these limitations, we propose DSH-Bench, a comprehensive benchmark that enables systematic multi-perspective analysis of subject-driven T2I models through four principal innovations: 1) a hierarchical taxonomy sampling mechanism ensuring comprehensive subject representation across 58 fine-grained categories, 2) an innovative classification scheme categorizing both subject difficulty level and prompt scenario for granular capability assessment, 3) a novel Subject Identity Consistency Score (SICS) metric demonstrating a 9.4% higher correlation with human evaluation compared to existing measures in quantifying subject preservation, and 4) a comprehensive set of diagnostic insights derived from the benchmark, offering critical guidance for optimizing future model training paradigms and data construction strategies. Through an extensive empirical evaluation of 19 leading models, DSH-Bench uncovers previously obscured limitations in current approaches, establishing concrete directions for future research and development.

Zhenyu Hu, Qing Wang, Te Cao +11

Tencent, China

Subject-Driven Image GenerationIdentity-Preserving Image GenerationT2I Generation Evaluation
AskPDF
Mar 23, 2026

MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation

Subject-driven image generation is increasingly expected to support fine-grained control over multiple entities within a single image. In multi-reference workflows, users may provide several subject images, a background reference, and long, entity-indexed prompts to control multiple people within one scene. In this setting, a key failure mode is cross-subject attribute misbinding: attributes are preserved, edited, or transferred to the wrong subject. Existing benchmarks and metrics largely emphasize holistic fidelity or per-subject self-similarity, making such failures hard to diagnose. We introduce MultiBind, a benchmark built from real multi-person photographs. Each instance provides slot-ordered subject crops with masks and bounding boxes, canonicalized subject references, an inpainted background reference, and a dense entity-indexed prompt derived from structured annotations. We also propose a dimension-wise confusion evaluation protocol that matches generated subjects to ground-truth slots and measures slot-to-slot similarity using specialists for face identity, appearance, pose, and expression. By subtracting the corresponding ground-truth similarity matrices, our method separates self-degradation from true cross-subject interference and exposes interpretable failure patterns such as drift, swap, dominance, and blending. Experiments on modern multi-reference generators show that MultiBind reveals binding failures that conventional reconstruction metrics miss.

Wenqing Tian, Hanyi Mao, Zhaocheng Liu +4

New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · ByteDance +1

Multi-Subject Image GenerationImage GenerationImage Generation EvaluationSubject-Driven Image Generation
AskPDF