Recent advancements in Unified Multimodal Models (UMMs) have enabled remarkable image understanding and generation capabilities. However, while models like Gemini-2.5-Flash-Image show emerging abilities to reason over multiple related images, existing benchmarks rarely address the challenges of multi-image context generation, focusing mainly on text-to-image or single-image editing tasks. In this work, we introduce MICON-Bench, a comprehensive benchmark covering six tasks that evaluate cross-image composition, contextual reasoning, and identity preservation. We further propose an MLLM-driven Evaluation-by-Checkpoint framework for automatic verification of semantic and visual consistency, where multimodal large language model (MLLM) serves as a verifier. Additionally, we present Dynamic Attention Rebalancing (DAR), a training-free, plug-and-play mechanism that dynamically adjusts attention during inference to enhance coherence and reduce hallucinations. Extensive experiments on various state-of-the-art open-source models demonstrate both the rigor of MICON-Bench in exposing multi-image reasoning challenges and the efficacy of DAR in improving generation quality and cross-image coherence. Github: https://github.com/Angusliuuu/MICON-Bench.
Figures & tables
Figure 1: Overview of MICON-Bench and Evaluation Pipeline. MICON-Bench is a comprehensive benchmark designed to evaluate multi-image context generation across six diverse tasks: Object Composition, Spatial Composition, Attribute Disentanglement, Component Transfer, FG/BG Composition, and Story Generation. Each task provides multiple reference images and a compositional prompt requiring the model to integrate, reason, or transfer information across images. To assess model performance, we introduce an Evaluation-by-Checkpoint framework driven by an MLLM, which automatically evaluates generated results along key dimensions, producing an interpretable composite score.
Figure 2: Overview of the proposed Dynamic Attention Rebalancing (DAR) mechanism. Given multiple reference images, DAR first samples query tokens and computes attention maps between sampled queries and reference key tokens. It then applies a dynamic weighting factor to rebalance attention responses, reinforcing relevant reference regions ( green boxes ) while suppressing distractions ( red boxes ). The example at the bottom shows how DAR helps compose a new scene by integrating the right Indian man from Ref Image 1 and the left doctor from Ref Image 2, achieving coherent and faithful visual synthesis.
Figure 3: Visualization examples of our method vs. baseline on MICON-Bench.
Model
Object
Spatial
Attribute
Component
FG/BG
Story
Avg. Score
Nano-Banana [ 11 ]
95.60
93.79
92.13
84.23
83.13
82.84
89.25
GPT-Image [ 14 ]
96.45
94.41
93.39
87.69
90.15
85.99
91.51
UNO [ 38 ]
58.40
66.68
65.28
28.84
20.96
39.08
44.76
DreamOmni2 [ 40 ]
88.24
84.76
85.28
59.64
76.16
59.58
75.56
Qwen-Image-Edit-2507 [ 34 ]
96.52
88.80
78.04
42.68
72.08
63.81
72.96
BAGEL [ 7 ]
87.64
89.96
89.84
52.40
64.64
65.09
73.55
Table 1: Performance comparison of different models on our MICON-Bench.
Method
SINGLE
MULTIPLE
SCENE
Average ↑
Character
Object
Character
Object
Char.+Obj.
Character
Object
Char.+Obj.
BAGEL
5.713
6.223
3.028
6.904
6.802
4.244
5.157
6.219
5.536
BAGEL + DAR
6.255
6.082
4.135
7.175
6.870
4.775
4.838
6.301
5.804
OmniGen2
8.184
7.328
6.564
7.994
8.039
6.871
7.898
7.378
7.532
OmniGen2 + DAR
8.299
8.191
6.644
8.418
7.860
7.056
7.965
7.713
7.768
Table 2: Performance of our method on the OmniContext benchmark.
Method
Single-Subject
Multi-Subject
Overall ↑
DPG
ID-Sim
IP-Sim
AES
AVG (Single) ↑
DPG
ID-Sim
IP-Sim
AES
AVG (Multi) ↑
BAGEL
93.65
3.11
46.91
47.96
47.91
89.60
2.62
32.39
45.87
42.62
45.26
BAGEL + DAR
94.70
3.61
47.01
48.83
48.54
92.13
4.28
33.38
45.86
43.91
46.23
OmniGen2
92.60
4.25
56.16
57.11
52.53
91.55
11.51
38.40
57.56
49.76
51.14
OmniGen2 + DAR
92.98
5.23
57.66
57.08
53.24
92.56
12.15
39.05
57.15
50.23
51.73
Table 3: Performance comparison on the XVerseBench benchmark.
Model
CLIP Score (T2I) ↑
CLIP Score (I2I) ↑
DINO v2 (I2I) ↑
LPIPS (I2I) ↓
Object
Spatial
Object
Spatial
Object
Spatial
Object
Spatial
Nano-Banana
0.3505
0.3665
0.9327
0.9382
0.8962
0.9066
0.4934
0.4726
GPT-Image
0.3572
0.3742
0.9089
0.9168
0.8663
0.8815
0.6431
0.6330
Qwen-Image-Edit
0.3558
0.3700
0.9208
0.9292
0.8789
0.8923
0.5542
0.5249
UNO
0.3391
0.3566
0.9007
0.9073
0.8301
0.8366
0.6453
0.6486
DreamOmni2
0.3454
0.3647
0.9133
0.9142
0.8719
0.8787
0.6217
0.6244
Table 4: Evaluation on other metrics, including CLIP score (Text-to-Image and Reference Images-to-Image), DINO v2 (Reference Images-to-Image), and LPIPS (Reference Images-to-Image) under Object Composition and Spatial Geometric Constraints tasks.
Model
Ref=2
Ref=3
Ref=4
Ref=5
BAGEL
88.50
84.37
75.11
66.36
OmniGen2
92.18
89.52
74.92
67.00
Table 5: Analysis of the impact of the numbers of reference images on UMMs in the MICON-Bench Object Composition Task.
Figure 4: DAR suppresses noisy, irrelevant attention ( red boxes ) and re-focuses activations on the correct subjects ( green boxes ), resulting in cleaner referencing and more faithful composition. The blue boxes highlight the target subjects.
Figure 5: The data statistics of MICON-Bench.
Task
#Cases
#Images
#2 Ref
#3 Ref
Object
200
482
118
82
Spatial
200
498
102
98
Attribut
100
300
0
100
Component
240
601
119
121
FG/BG
200
400
200
0
Story
103
237
72
31
Table 6: Statistics of MICON-Bench, including the number of cases, images, and the distribution of two- and three-reference-image settings for each task.
Figure 6: Human-annotated answer set for the Story Inference task. Each example pairs the reference image sequence with canonical human-written descriptions of what should happen next, which are used as targets for MLLM-based evaluation.
# Reference Images
BAGEL
Omnigen2
Baseline
+OURS
Baseline
+OURS
2
57.13
61.70
72.01
74.71
3
69.72
77.01
97.80
101.58
Table 7: Average inference time (seconds) on Object Composition dataset measured on a single NVIDIA A800-40GB GPU.
γ
Object
Spatial
Attribute
Component
FG/BG
Story
0.05
86.11
91.68
92.24
56.07
59.29
64.75
0.15
88.04
91.88
90.76
56.06
71.24
66.34
0.25
76.58
85.41
75.43
44.36
64.41
62.86
0.35
53.36
45.78
37.84
23.76
65.14
58.96
0.55
8.39
18.30
0.67
8.39
22.77
40.44
Table 8: Ablation study on the effect of different weight factor γ .
Model
Object
Spatial
Attribute
Component
FG/BG
Story
Avg. Score
BAGEL(Generic)
87.72
84.49
87.51
47.33
51.78
63.66
68.52
BAGEL(Specific)
88.17
84.09
85.20
46.09
51.78
63.51
68.01
BAGEL+ DAR (Generic)
91.22
87.81
89.24
48.49
78.22
68.86
75.84
BAGEL+ DAR (Specific)
91.72
87.53
89.42
47.67
80.00
67.51
75.92
Table 9: Ablation study on prompt sensitivity check for the Evaluation-by-Checkpoint.
Model
Object
Spatial
Attribute
Component
FG/BG
Story
Avg. Score
Nano-Banana [ 11 ]
97.88
98.18
72.03
79.41
83.71
79.06
86.63
GPT-Image [ 14 ]
98.85
97.25
83.70
85.26
91.08
81.58
90.77
UNO [ 38 ]
62.30
57.75
55.11
27.56
28.47
38.17
43.87
DreamOmni2 [ 40 ]
88.95
81.69
62.64
65.52
81.01
59.94
75.26
Qwen-Image-Edit-2507 [ 34 ]
98.03
97.57
65.05
57.12
74.95
60.74
77.26
BAGEL [ 7 ]
89.08
89.32
69.05
59.57
72.39
54.32
73.78
Table 10: Performance comparison of different models on our MICON-Bench evaluated by InternVL3.5-38B.
Model
Object
Spatial
Attribute
Component
FG/BG
Story
Avg. Score
Nano-Banana-Pro [ 11 ]
90.00
93.20
92.50
85.31
95.29
87.33
90.61
Nano-Banana [ 11 ]
92.45
90.34
89.23
76.35
87.67
75.00
85.17
GPT-Image [ 14 ]
92.90
93.46
91.73
79.80
87.12
90.00
89.17
UNO [ 38 ]
56.03
54.12
60.10
15.73
25.00
40.09
41.84
DreamOmni2 [ 40 ]
90.00
81.25
83.30
51.98
78.20
60.35
74.18
Qwen-Image-Edit-2507 [ 34 ]
89.40
84.30
75.00
44.76
77.95
59.50
71.82
Table 11: Human Evaluation results on the uniformly sampled subset of the MICON-Bench.
Target Model
Human
Automatic Verifiers
Score
Qwen3-VL
InternVL3.5
Nano-Banana-pro
90.61
92.42
90.93
GPT-Image
89.17
89.15
89.05
Nano-Banana
85.17
85.68
85.72
BAGEL + DAR (Ours)
74.08
75.62
75.57
DreamOmni2
74.18
74.58
73.36
Table 12: Reliability Analysis
Task Category
Configuration
Prompt Template
Object Composition
Reference Image
“A photo of {personalized_obj} in {scene}.”
Task (2 Refs)
“Generate an image that contains both the complete {obj_a} and {obj_b} together in {chosen_scene}.”
Task (3 Refs)
“Generate an image that contains all three objects ({obj_a}, {obj_b}, and {obj_c}) together in {chosen_scene}.”
Spatial Composition
Reference Image
“A photo of {personalized_obj} in {scene}.”
Task (2 Refs)
“Generate an image that contains both the complete {obj_a} and {obj_b} together in {chosen_scene}, with {obj_a} positioned to the {spatial_relation} of {obj_b}.”
Task (3 Refs)
“Generate an image that contains all three objects ({obj_a}, {obj_b}, and {obj_c}) together in {chosen_scene}, with {left_obj} on the left, {center_obj} in the center, and {right_obj} on the right.”
Table 13: Prompt Templates for Benchmark Construction. We detail the reference image prompts and the specific task prompts used for generation, covering object composition, spatial arrangement, attribute disentanglement, component transfer, background replacement, and story inference.
Dimension
ID
Evaluation Question
A. Instruction Following
A_check_1 (H1)
Does the image contain all specified objects as required by the instructions?
A_check_2
Are the relative arrangement and requested relations between objects correctly followed?
A_check_3
Are there no obviously extra or missing salient elements?
B. Identity / Fidelity
B_check_1 (H2)
Does each object’s identity strictly match its reference (e.g., category, instance)?
B_check_2
Are the key attributes (e.g., color, texture, shape) of each object well preserved?
B_check_3
Are the object details accurate and easily recognizable?
Table 14: Example of Evaluation Checkpoints. Detailed checklist for Object Composition task.
Figure 7: The evaluation prompt used for MLLM scoring.
Figure 8: More visualization examples of our method vs. baseline on MICON-Bench.
Figure 9: More visualization examples of our method vs. baseline on MICON-Bench.
Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to what extent textual and visual modalities are aligned. To investigate this question, we use reasoning-guided image generation as a diagnostic task, where models produce textual reasoning first and then generate images. We introduce UReason, a benchmark for evaluating reasoning-to-generation alignment in this paradigm, consisting of 2,000 human-curated and human-verified instances spanning five reasoning-intensive tasks: Code, Arithmetic, Spatial, Attribute, and Text. To enable controlled analysis, we develop an evaluation framework that compares direct generation, reasoning-guided generation, and decontextualized generation, which conditions only on the refined prompt extracted from reasoning. Across eight widely used open-source UMMs, while we find that reasoning-guided generation yields improvements over direct generation, somewhat surprisingly, decontextualized generation consistently outperforms reasoning-guided generation by a large margin. Our further analyses suggest that the intended visual semantics in textual reasoning are not reliably reflected in the generated images, despite their unified design and training. Overall, UReason serves as a practical litmus test for reasoning-to-generation alignment and provides a challenging benchmark for developing next-generation, more tightly aligned UMMs.
Cheng Yang, Chufan Shi, Bo Shui +7
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone's native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a +3.5% absolute gain on MMMU and improves GenEval image generation performance from 82% to 85%. Code and models are publicly released.
Recent advances in text-to-image (T2I) generation have enabled visually coherent image synthesis from descriptions, but generating images containing multiple given subjects remains challenging. As the number of reference identities increases, existing methods often suffer from subject missing and semantic drift. To address this problem, we propose MUSIC, the first MLLM specifically designed for MUlti-Subject In-Context image generation. To overcome the data scarcity, we introduce an automatic and scalable data generation pipeline that eliminates the need for manual annotation. Furthermore, we enhance the model's understanding of multi-subject semantic relationships through a vision chain-of-thought (CoT) mechanism, guiding step-by-step reasoning from subject images to semantics and generation. To mitigate identity entanglement and manage visual complexity, we develop a novel semantics-driven spatial layout planning method and demonstrate its test-time scalability. By incorporating complex subject images during training, we improve the model's capacity for chained reasoning. In addition, we curate MSIC, a new benchmark tailored for multi-subject in-context generation. Experimental results demonstrate that MUSIC significantly surpasses other methods in both multi- and single-subject scenarios.