MS-CustomNet: Controllable Multi-Subject Customization with Hierarchical Relational Semantics

Pengxiang Cai; Mengyang Li

cs.CVMar 22, 2026

MS-CustomNet: Controllable Multi-Subject Customization with Hierarchical Relational Semantics

Authors: Pengxiang Cai, Mengyang Li

Organizations: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China · Center for Applied Mathematics, Tianjin University, Tianjin 300072, China

Abstract

Diffusion-based text-to-image generation has advanced significantly, yet customizing scenes with multiple distinct subjects while maintaining fine-grained control over their interactions remains challenging. Existing methods often struggle to provide explicit user-defined control over the compositional structure and precise spatial relationships between subjects. To address this, we introduce MS-CustomNet, a novel framework for multi-subject customization. MS-CustomNet allows zero-shot integration of multiple user-provided objects and, crucially, empowers users to explicitly define these hierarchical arrangements and spatial placements within the generated image. Our approach ensures individual subject identity preservation while learning and enacting these user-specified inter-subject compositions. We also present the MSI dataset, derived from COCO, to facilitate training on such complex multi-subject compositions. MS-CustomNet offers enhanced, fine-grained control over multi-subject image generation. Our method achieves a DINO-I score of 0.61 for identity preservation and a YOLO-L score of 0.94 for positional control in multi-subject customization tasks, demonstrating its superior capability in generating high-fidelity images with precise, user-directed multi-subject compositions and spatial control.

Figures & tables

Fig. 1: Construction pipeline of the MSI dataset. Key stages include COCO image filtering based on subject count and size, establishment of a multi-view reference subject pool, generation of composite layout maps using COCO annotations, and assembly of final training tuples.

Fig. 2: Architecture of MS-CustomNet. The model processes reference subject images ( $\{x_{ref,k}\}^{K}_{k=1}$ ), textual prompts ( $\mathcal{T}$ ), and layout maps ( $M_{L}$ ). Category-aware subject features ( $f_{s,k}$ ) are generated and, along with text and location cues, condition a latent diffusion model to synthesize the customized image.

Fig. 3: Qualitative assessment of single-subject customization. The figure showcases the MS-CustomNet’s proficiency in preserving subject identity and integrating subjects cohesively into diverse scenes, guided by varied textual prompts.

Method	Single-Subject Customization			Multi-Subject Customization
Method	DINO-I $\uparrow$	CLIP-I $\uparrow$	CLIP-B $\uparrow$	YOLO-Subj $\uparrow$	YOLO-L $\uparrow$	DINO-I $\uparrow$	CLIP-I $\uparrow$	CLIP-B $\uparrow$
IP-Adapter	0.58	0.75	0.29	—	—	—	—	—
CustomNet	0.77	0.82	0.33	—	—	—	—	—
$\lambda$ -ECLIPSE	0.61	0.78	0.34	0.62	—	0.56	0.77	0.30
SSR-Encoder	0.67	0.79	0.36	0.71	0.91	0.57	0.79	0.35
MS-CustomNet	0.74	0.82	0.32	0.68	0.94	0.61	0.79	0.34

TABLE I: Quantitative comparison for single-subject and multi-subject customization.

Fig. 4: Qualitative assessment of multi-subject customization. These examples illustrate the MS-CustomNet’s capacity to generate intricate scenes featuring multiple customized subjects, effectively preserving individual identities while adhering to user-defined compositional and spatial relationship.

Fig. 5: Bar chart comparing various metrics under different ablation configurations. The horizontal axis shows the four configurations in order: baseline, introduction of

M_{L}

, introduction of DST, and introduction of CLSQ.

Fig. 6: Visual progression in the ablation study of MS-CustomNet. This figure illustrates the stepwise enhancement in generated image quality, showing improvements in identity preservation, subject clarity, compositional accuracy, and background coherence as components are added, from the Baseline to the full MS-CustomNet model incorporating curriculum learning.

Controllable Image Generation

Controllable image generation steers images using spatial, visual, semantic, stylistic, or physical constraints beyond ordinary text prompts.

See all 210 papers

Multimodal Large Language Models for Multi-Subject In-Context Image Generation
Apr 8, 2026Yucheng Zhou, Dubing Chen, Huan Zheng +1Multi-Subject Image Generation Image Generation
SKL-IOTSC, CIS, University of Macau
Hierarchical Concept-to-Appearance Guidance for Multi-Subject Image Generation
Feb 3, 2026Yijia Xu, Zihao Wang, Haokun Gui +1Subject-Driven Image Generation Image Generation
Peking University · Hong Kong University of Science and Technology
MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding
Aug 4, 2026Ruirui Zhang, Zhengkai Zhao, Pan GaoImage Generation Compositional T2I Generation
Nanjing University of Aeronautics and Astronautics College of Computer Science and Technology Nanjing, Jiangsu, China · Nanjing University of Aeronautics and Astronautics College of Artificial Intelligence Nanjing, Jiangsu, China

Apr 8, 2026J/K move · Enter open · S save

Multimodal Large Language Models for Multi-Subject In-Context Image Generation

Yucheng Zhou, Dubing Chen, Huan Zheng, Jianbing Shen

SKL-IOTSC, CIS, University of Macau

Multi-Subject Image Generation Image Generation Compositional Image Generation Multimodal Large Language Models Multimodal ICL

Open paper AskPDF arXiv

Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. As the number of reference identities increases, existing methods often suffer from subject missing and semantic drift. To address this problem, we propose MUSIC, the first MLLM specifically designed for MUlti-Subject In-Context image generation. To overcome the data scarcity, we introduce an automatic and scalable data generation pipeline that eliminates the need for manual annotation. Furthermore, we enhance the model's understanding of multi-subject semantic relationships through a vision chain-of-thought (CoT) mechanism, guiding step-by-step reasoning from subject images to semantics and generation. To mitigate identity entanglement and manage visual complexity, we develop a novel semantics-driven spatial layout planning method and demonstrate its test-time scalability. By incorporating complex subject images during training, we improve the model's capacity for chained reasoning. In addition, we curate MSIC, a new benchmark tailored for multi-subject in-context generation. Experimental results demonstrate that MUSIC significantly surpasses other methods in both multi- and single-subject scenarios.

Abstract

Controllable Image Generation

Explore similar work

Multimodal Large Language Models for Multi-Subject In-Context Image Generation

Hierarchical Concept-to-Appearance Guidance for Multi-Subject Image Generation

MultiCompose: Multi-Concept Personalized Composition with Per-Subject Attribute Binding