DeCoRL: Decomposed Consistency Reinforcement Learning for Multi-Image Composition

Zhiqiang Wu; Shuang Sun; Jiale Zhang; Jing Li; Yibo Chen; Xu Tang; Yao Hu

cs.CVOct 29, 2026

DeCoRL: Decomposed Consistency Reinforcement Learning for Multi-Image Composition

Authors: Zhiqiang Wu, Shuang Sun, Jiale Zhang, Jing Li, Yibo Chen, Xu Tang, Yao Hu

Organizations: Xiaohongshu Inc.

Abstract

Multi-image composition (MICo) is a critical yet challenging capability for large-scale image editing models, where consistency degrades rapidly as the number of reference images increases. While supervised fine-tuning (SFT) can equip models with basic MICo ability, it provides only implicit supervision on the composed image and often fails to learn reliable object-level correspondence across multiple references. We propose DeCoRL, a reinforcement-learning (RL) framework that improves MICo consistency by decomposing holistic MICo consistency into object-level consistency. DeCoRL first establishes instruction-relevant object correspondences across reference images and localizes the corresponding regions in the generated image. It then uses a VLM-based reward model trained on single-object consistency annotations to produce fine-grained, object-wise feedback, which is aggregated into a multi-object consistency reward to guide RL post-training. We further introduce DeCo-Bench, a benchmark covering compositions from 2 to 6 reference images. Experiments on DeCo-Bench and the public MICo-Bench demonstrate consistent improvements in MICo consistency.

Figures & tables

Figure 1: Qualitative comparisons on two challenging MICo examples, each involving six reference images. Compared with open-source and proprietary baselines, our method better preserves the identities and visual attributes of multiple referenced objects in the composed image. The red boxes highlight objects with inconsistent appearances.

Figure 2: Overview of DeCoRL. DeCoRL improves MICo consistency via decomposed, object-level RL. We use a VLM to parse the instruction and identify instruction-relevant objects in each reference image, then localize them with Grounding DINO to obtain object crops. For a generated image, we detect the corresponding objects, score each reference–generated object pair with a trained single-object consistency reward model, aggregate the scores into a multi-object consistency reward, and finally optimize the transformer2d model with Diffusion NFT [ 55 ] .

Figure 3: Instruction-relevant Object Correspondences. A VLM jointly interprets the instruction and multiple reference images, identifies instruction-relevant objects, and produces a labeled object set for Grounding DINO to obtain the reference crops.

Figure 4: Multi-object Consistency Reward. Object-level consistency scores are computed for aligned reference-generated object pairs and then aggregated into a single-object consistency reward.

Model	2 images $\uparrow$	3 images $\uparrow$	4 images $\uparrow$	5 images $\uparrow$	6 images $\uparrow$	Overall $\uparrow$
Qwen-Image-Edit-MICo [ 47 ]	6.90	5.27	4.85	4.79	4.06	5.17
Qwen-Image-Edit-UniPic3.0 [ 44 ]	7.82	6.23	5.73	5.24	5.11	6.02
FLUX.2-dev [ 21 ]	8.10	7.26	7.10	6.76	6.28	7.10
Seedream4.0 (2K) [ 39 ]	8.17	8.17	7.50	7.49	7.47	7.76
Nano-Banana-Pro [ 13 ]	9.26	8.67	8.77	8.58	8.18	8.69
Qwen-Image-Edit2511 (baseline) [ 48 ]	8.41	6.99	7.09	4.98	3.64	6.22

Table 1: Results and ablations on DeCo-Bench.

Model	Object Centric $\uparrow$	Human Centric $\uparrow$	HOI $\uparrow$	De&Re $\uparrow$	Overall $\uparrow$
Qwen-Image-Edit-MICo [ 47 ]	52.38	21.11	34.95	37.42	35.86
Qwen-Image-Edit-UniPic3.0 [ 44 ]	67.94	44.79	28.24	59.28	47.90
FLUX.2-dev [ 21 ]	41.21	45.10	49.86	55.71	49.55
Nano-Banana [ 12 ]	48.01	41.79	49.64	49.44	47.83
Nano-Banana-Pro [ 13 ]	50.59	54.75	50.21	52.13	51.76
Qwen-Image-Edit2511 (baseline) [ 48 ]	27.41	28.89	19.59	20.60	22.90

Table 2: Results and ablations on MICo-Bench.

Model	Prompt Following (PF) $\uparrow$	Subject Resemblance (SR) $\uparrow$	Naturalness $\uparrow$	Artifacts $\uparrow$
Qwen-Image-Edit2511 (baseline)	6.1495	5.2378	5.0817	4.1208
+ SFT	8.7586	7.0080	6.5355	5.7574
+ DeCoRL	8.6865	7.6393	8.0123	6.6362

Table 3: Ablations on four MICo-Bench metrics.

Model	2 images $\uparrow$	3 images $\uparrow$	4 images $\uparrow$	5 images $\uparrow$	6 images $\uparrow$	Overall $\uparrow$
Qwen-Image-Edit2511 (baseline) [ 48 ] + SFT	8.34	7.35	6.80	6.60	6.15	7.05
w/ vanilla VLM reward	8.54	7.49	6.42	6.90	6.78	7.21
w/ trained VLM reward (ours)	8.91	8.36	8.05	7.88	7.61	8.16

Table 4: Ablations of vanilla vs. trained VLM rewards on DeCo-Bench.

Figure 5: Training dynamics of multi-object consistency score and banned prompt count (vanilla vs. trained VLM).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Visualization examples from the DeCo-Bench (6 images).

Figure 7: Visualization examples from the MICo-Bench (De&Re, HOI, Human Centric and Object Centric).

Panel of Figure 8 — Figure 8: The evaluation prompt (Gemini-3.1-Pro) of DeCo-Bench.

Evaluator	Accuracy $\uparrow$
GPT-4.1	0.602
GPT-5	0.669
Vanilla reward VLM	0.354
Trained reward VLM (ours)	0.721

Appendix

Table 5: Human-label agreement accuracy on EditReward-Bench.

Figure 9: Best-case and worst-case object consistency rewards over the reported training checkpoints.

Compositional Image Generation

Compositional image generation studies how to combine objects, attributes, spatial relations, and visual references into coherent, controllable images.

See all 87 papers

MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models
Feb 23, 2026Mingrui Wu, Hang Liu, Jiayi Ji +2Image Generation Unified Multimodal Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Zhongguancun Academy, Beijing, China.100094.
CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding
Apr 24, 2026Lihao Zheng, Zhenwei Shao, Yu Zhou +5Multimodal Grounding Multimodal Large Language Models
School of Computer Science and Technology, Hangzhou Dianzi University · Base Model, Li Auto
Evaluation-Verification Reward for Consistent Multi-Reference Image Editing
Jul 31, 2026Yingmao Miao, Pengfei Zhang, Xiaochen Lv +5Image Editing Evaluation Multimodal Reward Modeling
Xi’an Jiaotong University · DreamX Team, Alibaba Group · Shanghai Jiaotong University

Feb 23, 2026J/K move · Enter open · S save

MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models

Mingrui Wu, Hang Liu, Jiayi Ji, Xiaoshuai Sun+1

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Zhongguancun Academy, Beijing, China.100094.

Image Generation Unified Multimodal Models Image Generation Evaluation Controllable Image Generation VLM Hallucination Mitigation

Open paper AskPDF arXiv

Recent advancements in Unified Multimodal Models (UMMs) have enabled remarkable image understanding and generation capabilities. However, while models like Gemini-2.5-Flash-Image show emerging abilities to reason over multiple related images, existing benchmarks rarely address the challenges of multi-image context generation, focusing mainly on text-to-image or single-image editing tasks. In this work, we introduce MICON-Bench, a comprehensive benchmark covering six tasks that evaluate cross-image composition, contextual reasoning, and identity preservation. We further propose an MLLM-driven Evaluation-by-Checkpoint framework for automatic verification of semantic and visual consistency, where multimodal large language model (MLLM) serves as a verifier. Additionally, we present Dynamic Attention Rebalancing (DAR), a training-free, plug-and-play mechanism that dynamically adjusts attention during inference to enhance coherence and reduce hallucinations. Extensive experiments on various state-of-the-art open-source models demonstrate both the rigor of MICON-Bench in exposing multi-image reasoning challenges and the efficacy of DAR in improving generation quality and cross-image coherence. Github: https://github.com/Angusliuuu/MICON-Bench.

Abstract

Compositional Image Generation

Explore similar work

MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models

CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding

Evaluation-Verification Reward for Consistent Multi-Reference Image Editing