Organizations: Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Peking University · Sun Yat-sen University · Baidu Inc.
Recently, unified multimodal models (UMMs) have made remarkable progress in integrating visual understanding and generation, demonstrating strong potential for complex text-to-image (T2I) tasks. Despite their theoretical promise, a persistent capability gap exists: UMMs typically exhibit superior visual understanding but comparatively weaker generative capabilities. This discrepancy arises largely from the intrinsic decoupling between the understanding and generation processes. While a UMM can accurately interpret fine-grained visual details, it often struggles to produce semantically coherent images from complex textual prompts. To address this challenge, we explore UMMs' internal understanding capability to enhance generation quality. We propose a token-level intrinsic text-image alignment reward mechanism, GvU, enabling the UMM to act simultaneously as teacher and student: it evaluates its own outputs using the understanding branch to guide the generations accordingly. Building upon this, we design a self-supervised reinforcement learning framework, allowing UMMs to iteratively improve their generation quality through understanding-based intrinsic reward signals--without reliance on external supervision. Experimental results show that our method substantially boosts UMMs' generation, which in turn strengthens their fine-grained visual understanding, narrowing the capability gap between UMMs' visual understanding and generation.
Figures & tables
Figure 1: (a) Top: UMMs exhibit an understanding–generation gap: they recognize visual details but fail to reflect them in generated images. (b) Bottom: our self-teaching mechanism uses intrinsic rewards from the understanding branch to guide generation, improving text–image alignment without external supervision.
Figure 2: Overview of the GvU implementation. It comprises two key components: the token-level model-intrinsic reward that provides fine-grained text–image alignment signals, and the self-supervised reinforcement learning (RL) process that uses these signals to enhance UMMs’ generative ability progressively, enabling continuous self-improvement without external supervision.
Model Type
Method
Single Obj. ↑
Two Obj. ↑
Counting ↑
Colors ↑
Position ↑
Attr. Binding ↑
Overall ↑
LDM [ 39 ]
0.92
0.29
0.23
0.70
0.02
0.05
0.37
SD v1.5 [ 39 ]
0.97
0.38
0.35
0.76
0.04
0.06
0.43
SD v2.1 [ 39 ]
0.98
0.51
0.44
0.85
0.07
0.17
0.50
SD XL [ 35 ]
0.98
0.74
0.39
0.85
0.15
0.23
0.55
IF-XL [ 22 ]
0.97
0.74
0.66
0.81
0.13
0.35
0.61
DALL-E [ 38 ]
0.96
0.87
0.47
0.83
0.43
0.45
0.67
Table 1: Evaluation of text-to-image generation capability on GenEval [ 11 ] benchmark. ”†” refers to the method using the LLM prompt rewriter. Bold marks the best performance, and u nderlined marks the second best.
Figure 3: Qualitative comparisons between our GvU and other methods. Compared to other approaches, GvU generates images with better text-image alignment and more coherent spatial layout.
Model Type
Method
Global ↑
Entity ↑
Attribute ↑
Relation ↑
Other ↑
Overall ↑
SD XL [ 35 ]
83.27
82.43
80.91
86.76
80.41
74.65
DALL-E [ 38 ]
90.97
89.61
88.39
90.58
89.83
83.50
SD3-medium [ 6 ]
87.90
91.01
88.83
80.70
88.68
84.08
FLUX.1-dev [ 23 ]
82.1
89.5
88.7
91.1
89.4
84.0
Only Generation
OmniGen [ 51 ]
87.90
88.97
88.47
87.95
93.56
81.16
Show-o [ 52 ]
79.33
75.44
78.02
84.45
60.80
67.27
Table 2: Evaluation of long-form T2I performance on DPG-Bench [ 18 ] benchmark. Bold indicates best, u nderline indicates second best.
Figure 4: Evolution across multiple benchmarks during the self-supervised RL process. As the RL training steps increase, our model demonstrates steadily improved visual generation performance.
Model Type
Method
Color ↑
Count ↑
Color/Count ↑
Color/Pos ↑
Pos/Count ↑
Pos/Size ↑
Multi-Count ↑
Overall ↑
SD v2.1 [ 39 ]
0.000
0.325
0.025
0.000
0.000
0.025
0.075
0.064
SD XL [ 35 ]
0.050
0.375
0.000
0.000
0.000
0.000
0.000
0.061
SD3-medium [ 6 ]
0.550
0.500
0.125
0.350
0.175
0.150
0.225
0.296
Only Generation
FLUX.1-dev [ 23 ]
0.350
0.625
0.150
0.275
0.200
0.375
0.225
0.314
Janus-Pro [ 2 ]
0.450
0.300
0.125
0.300
0.075
0.350
0.125
0.246
BLIP3-o 8B [ 1 ]
0.250
0.250
0.125
0.600
0.125
0.575
0.225
0.307
Table 3: Evaluation of T2I performance on GenEval++ [ 60 ] benchmark. Bold indicates best, u nderline indicates second best.
Figure 5: Illustration of GvU’s generated results across training steps. With ongoing self-supervised RL, our model effectively leverages intrinsic rewards to progressively enhance the text-image alignment, demonstrating continual improvement in T2I tasks.
Method
POPE ↑
GQA ↑
MMB ↑
SEED ↑
DocVQA ↑
OCRB ↑
LLaVA-NeXT [ 29 ]
86.1
64.5
66.9
68.6
74.1
529
LLaVA-OneVision [ 25 ]
-
-
80.8
75.4
87.5
622
Janus-Pro [ 2 ]
86.7
61.3
78.2
71.9
-
601
Emu3 [ 46 ]
84.5
59.8
57.9
68.3
76.4
692
X-Omni [ 10 ]
86.1
61.9
73.9
74.5
89.1
712
GvU (Ours)
86.3
61.7
73.5
73.9
88.4
709
Table 4: Evaluation on visual understanding benchmarks.
Figure 6: Comparison of GvU improvements on the weak and regular base models. GvU achieves greater gains on the weak base, suggesting that a larger gap between generation and understanding enhances its effectiveness in model training.
Model
MMT-Bench
Visual Details
Visual Reasoning
Overall
VR
VI
Ha
VCR
DKR
Base
49.76
51.21
45.57
66.25
70.0
38.46
GvU
49.92 ( +0.16 )
52.58 ( +1.37 )
50.63 ( +5.06 )
68.75 ( +2.5 )
75.0 ( +5.0 )
42.31 ( +3.85 )
Table 5: Evaluation on overall score and fine-grained visual understanding subtasks of MMT-Bench [ 61 ] .
Figure 7: Reward training curves of GvU. (Left) The left panel shows training starting from the regular base, which starts with higher initial rewards and supports more training steps. (Right) The right panel starts from the weak base, which begins lower but exhibits a larger relative increase in rewards.
Figure 8: Sensitivity of intrinsic reward to fine-grained semantics. The line chart depicts the intrinsic reward, while the bar chart shows the reward relative to that of full prompts. Removing detail-specific terms causes a drop in intrinsic rewards, demonstrating that GvU effectively captures fine-grained semantic correspondences between text and images.
Recently, remarkable progress has been made in Unified Multimodal Models (UMMs), which integrate vision-language generation and understanding capabilities within a single framework. However, a model's strong visual understanding often fails to transfer to visual generation: it may correctly judge prompt-image alignment while failing to generate a faithful image from the same prompt. This raises a compelling question: Can a model improve itself by using its understanding module to reward its generation module? We introduce SRUM, a self-rewarding post-training framework directly applicable to existing UMMs of various designs. SRUM creates a feedback loop where the model's own understanding module acts as an internal ``evaluator'', providing corrective signals to improve generation without additional human-labeled data or external reward models. To provide comprehensive feedback, SRUM uses a global-local dual reward system: a global reward ensures overall visual semantics and layout, while a local reward refines fine-grained, object-level fidelity. SRUM shows strong generalization, boosting performance on T2I-CompBench from 82.18 to 88.37 and on T2I-ReasonBench from 43.82 to 46.75. Overall, our work establishes a powerful paradigm for enabling a UMM's understanding module to guide and enhance its own generation via self-rewarding.
Weiyang Jin, Yuwei Niu, Jiaqi Liao +4
HKU MMLab · The University of Hong Kong · Shenzhen Loop Area Institute +2
Unified multimodal models are envisioned to bridge the gap between understanding and generation. Yet, to achieve competitive performance, state-of-the-art models adopt largely decoupled understanding and generation components. This design, while effective for individual tasks, weakens the connection required for mutual enhancement, leaving the potential synergy empirically uncertain. We propose to explicitly restore this synergy by introducing Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations. By incorporating objectives that encode semantic abstraction (captioning) and structural details (visual regression), we enable effective gradient flow from understanding to generation. Extensive experiments on image generation and editing demonstrate that understanding can serve as an effective catalyst for generation.
Zeyu Liu, Zanlin Ni, Yang Yue +5
1Tsinghua University · 2Kolors Team, Kuaishou Technology
While unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model, functional unification does not guarantee learning synergy: the two objectives may reinforce each other, compete for capacity, or merely coexist. We investigate their relationship at the representation, task, and system levels in a controlled, structurally native setting without pretrained vision priors. At the representation level, we find that each objective provides useful signal to the other: generation enriches the visual features learned for understanding, while understanding strengthens vision--language alignment for generation. However, when both objectives are forced through the same computation path, one tends to dominate. A task-decoupled architecture that specializes conflicting visual computation while preserving semantic interaction avoids this asymmetric degradation. At the task level, through three case studies, we find positive bidirectional transfer when understanding and generation tasks rely on shared knowledge. At the system level, we show that an end-to-end UMM outperforms a matched planner--executor pipeline on complex tasks that explicitly require both image understanding and generation. Together, these results show that the value of UMMs extends beyond a unified interface: appropriate specialization, shared task knowledge, and end-to-end optimization can turn coexistence into synergy.
Penghao Wu, Haiwen Diao, Weichen Fan +3
1S-Lab, Nanyang Technological University · 2SenseTime Research