cs.CVMar 10, 2026
SaveInternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
Organizations: Shanghai AI Laboratory · CUHK MMLab · Fudan University · Shanghai Jiao Tong University ·
Abstract
Unified multimodal models (UMMs) that integrate understanding, reasoning, generation, and editing face inherent trade-offs between maintaining strong semantic comprehension and acquiring powerful generation capabilities. In this report, we present InternVL-U, a lightweight 4B-parameter UMM that democratizes these capabilities within a unified framework. Guided by the principles of unified contextual modeling and modality-specific modular design with decoupled visual representations, InternVL-U integrates a state-of-the-art Multimodal Large Language Model (MLLM) with a specialized MMDiT-based visual generation head. To further bridge the gap between aesthetic generation and high-level intelligence, we construct a comprehensive data synthesis pipeline targeting high-semantic-density tasks, such as text rendering and scientific reasoning, under a reasoning-centric paradigm that leverages Chain-of-Thought (CoT) to better align abstract user intent with fine-grained visual generation details. Extensive experiments demonstrate that InternVL-U achieves a superior performance - efficiency balance. Despite using only 4B parameters, it consistently outperforms unified baseline models with over 3x larger scales such as BAGEL (14B) on various generation and editing tasks, while retaining strong multimodal understanding and reasoning capabilities.
Figures & tables
Figure 1: Showcases of InternVL-U for general text-to-image generation (top) and image editing (bottom). InternVL-U supports high-fidelity image generation and editing at any resolution.
Figure 2: Showcases of InternVL-U for spatial-centric, perception, science-centric, humor-centric, and reasoning-centric text-to-image generation or editing tasks. InternVL-U demonstrates such core multimodal capabilities across various visual domains.
Figure 3: The architectural design of InternVL-U . The framework highlights three design principles: (1) unified contextual modeling supporting modality-adaptive generation targets, (2) structural efficiency via a unified backbone with modality-specific modular design, and (3) decoupled visual representations for understanding and generation tasks. Und. and Gen. denote Understanding and Generation, respectively.
Figure 4: Architecture of the Visual Generation Head . (a) Overview of the head with dual-stream MMDiT blocks. (b) Detailed structure of the Dual-Stream Attention Block and Dual-Stream FFN Block. (c) Illustration of the Unified MSRoPE (Multi-Scale Rotary Positional Embeddings) applied to VAE image latents and multimodal context embeddings.
| Task | Datasets |
| Text-To-Image (T2I) | LAION [ 109 ] , BLIP-3o [ 14 ] , ShareGPT-4o-Image [ 15 ] , OSP [ 75 ] , Echo-4o-Image [ 152 ] , OpenGPT-4o [ 20 ] , FaceCaption [ 26 ] , Flux-Reason-6M [ 37 ] , HumanCaption [ 27 ] , POSTER-TEXT [ 39 ] , AutoPoster [ 78 ] , CTW [ 155 ] |
| Image Editing (IT2I) | InstructPix2Pix [ 6 ] , AnyEdit [ 154 ] , PIPE [ 133 ] , ImgEdit [ 153 ] , SEED-Data-Edit [ 41 ] , OmniEdit [ 134 ] , UltraEdit [ 160 ] , HQEdit [ 55 ] , ShareGPT-4o-Image [ 15 ] , OpenGPT-4o [ 20 ] , X2Edit [ 91 ] , X2I2 [ 138 ] , UniWorld perception [ 76 ] , NHR-Edit [ 62 ] , GPT-hqedit [ 131 ] , GPT-omniedit [ 131 ] , GPT-ultraedit [ 131 ] , Nano-consistent-150k [ 152 ] , Pico Banana [ 103 ] |
Table 1: Overview of collected open-source datasets.
Figure 5: Examples of general data synthesized by our pipeline. The synthesized data features varied textual annotations and covers diverse visual domains, including portraits, posters, natural scenes, etc .
Figure 6: Overview of our general data synthesis pipeline. First, the preprocessing stage applies filtering, expansion, and deduplication to construct a high-quality source pool. Building upon this, two parallel branches are deployed to generate text-to-image pairs and instruction-guided editing data, respectively.
Figure 7: Three types of text-centric data synthesized by our pipeline. The first type overlays semantically relevant text onto natural images using a masked background image. The second type renders text on solid-color backgrounds, focusing on clean and aesthetically pleasing layouts. The third type performs text editing within existing images, such as modifying text on license plates, mobile interfaces, signboards, and similar surfaces.
Figure 8: Text rendering data construction pipeline. For synthetic text rendering, we prepare mask images, font colors, font styles, and adaptive layout options. During the rendering process, these attributes are randomly sampled, and the text is rendered with typography that adapts to its length.
Figure 9: Text editing data construction pipeline. First, we use an OCR tool to extract candidate text regions for editing. Second, we generate the editing instructions. Third, we use a generative model to produce the edited ground truth. Through these three steps, we synthesize high-quality text-editing triplets.
Figure 10: Science data generation pipeline. For general science T2I, we collect web images and open-source datasets and design automatic filtering and annotating with open-source models. For physics, we obtain images from documents by PaddleOCR, and propose an SVG-based pipeline for high-quality and affordable image-pair generation. For computer science, we define tasks and render images with Python libraries.
Figure 11: Examples of science-centric data. General science T2I features dense and detailed instructions and requires depicting multidisciplinary concepts. Physics and computer science focus on image editing with disciplinary reasoning.
Figure 12: Examples of spatial-centric data. We consider three spatial-centric scenarios: solid geometry ( e.g . , revolution, symmetry), multi-view CAD (three-view drawing), and spatial rotation of 3D objects.
Figure 13: Overview of our spatial rotation editing data synthesis pipeline. Stage 1 prepares a pool of filtered reference images. Subsequently, Stage 2 generates the final editing pairs via either the Object-First strategy for object-context integration, or the Background-First strategy for strict background preservation.
Figure 14: Meme data example for image generation and editing. The meme data here captures elements of everyday human humor and expressive nuances commonly found in daily life.
Figure 15: Meme data synthesis pipeline. It comprises five stages, leveraging chain-of-thought reasoning to process internet memes and synthesize high-quality training data for meme generation and editing.
Figure 16: Examples of CoT reasoning and enhancement for general image generation and editing . The original prompt is enriched through our chain-of-thought reasoning, adding finer-grained details that enable the model to perform generation and editing with greater accuracy and improved fidelity.
Figure 17: Examples of CoT reasoning and enhancement for knowledge-infused T2I and editing. After CoT reasoning, abstract knowledge concepts are concretized, enabling more precise generation and editing.
Figure 18: Examples of CoT reasoning and enhancement for Science T2I and Editing. Through CoT reasoning, scientific knowledge is injected into the generation process in a more explicit and detailed manner ( e.g . , how elements should be depicted and how operations should be executed).
| Configuration | Visual Und. Encoder | Context Backbone | Visual Gen. Head |
| # Layers | 24 | 28 | 20 |
| # Num Heads (Q / KV) | 16 / 16 | 16 / 8 | 12 / 12 |
| Head Size | 64 | 128 | 128 |
| Intermediate Size | 4,096 | 6,144 | 6,144 |
| Patch / Scale Factor | 14 | - | 2 |
| # Parameters | 0.3B | 2B | 1.7B |
Table 2: Detailed configuration of InternVL-U architecture.
| Hyperparameters | Stage 1 | Stage 2 | Stage 3 |
| Trainable modules | |||
| Backbone | ✓ | ||
| Visual gen. head | ✓ | ✓ | ✓ |
| Learning rate | 3e-04 | 1e-04 | 1e-05 |
| LR scheduler | Constant | Cosine | Cosine |
| Weight decay | 0.0 | 0.0 | 0.01 |
Table 3: Hyperparameters for different training stages .
Figure 19: System prompts and user prompts adopted during training for different tasks , where <img_uncond> is a learnable special token for image generation.
| Model | #Params | Understanding | Reasoning | |||||
| MME-P [ 38 ] | SEED [ 67 ] | ChartQA [ 94 ] | OCRBench [ 86 ] | MMMU [ 156 ] | MathVerse [ 158 ] | LogicVista [ 141 ] | ||
| MLLMs w/o Generator | ||||||||
| LLaVA-1.5V [ 80 ] | 7B | 1510.7 | 65.8 | 17.8 | 31.8 | 35.7 | 7.6 | – |
| Qwen2.5-VL [ 5 ] | 3B | 1574.9 | 73.7 | 84.0 | 79.7 | 53.1 | 31.2 | 40.3 |
| InternVL3.5 [ 129 ] | 2B | 1552.1 | 75.3 | 80.7 | 83.6 | 59.0 | 53.4 | 47.7 |
| UMMs w/ Generator | ||||||||
Table 4: Comparisons between InternVL-U and baseline models on multimodal understanding and reasoning benchmarks. * indicates the results based on our evaluation scripts. Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | Single Object | Two Object | Counting | Colors | Position | Color Attribution | Overall |
| Generation Models | ||||||||
| FLUX.1 [dev] [ 64 ] | 12B | 0.98 | 0.81 | 0.74 | 0.79 | 0.22 | 0.45 | 0.66 |
| SD3-Medium [ 35 ] | 2B | 0.99 | 0.94 | 0.72 | 0.89 | 0.33 | 0.60 | 0.74 |
| Seedream 3.0 [ 40 ] | - | 0.99 | 0.96 | 0.91 | 0.93 | 0.47 | 0.80 | 0.84 |
| GPT Image 1 [High] [ 99 ] | - | 0.99 | 0.92 | 0.85 | 0.92 | 0.75 | 0.61 | 0.84 |
| Z-Image [ 8 ] | 6B | 1.00 | 0.94 | 0.78 | 0.93 | 0.62 | 0.77 | 0.84 |
Table 5: Evaluation of general text-to-image generation ability on GenEval [ 44 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | Global | Entity | Attribute | Relation | Other | Overall |
| Generation Models | |||||||
| FLUX.1 [dev] [ 64 ] | 12B | 82.10 | 89.50 | 88.80 | 91.10 | 89.40 | 84.00 |
| SD3-Medium [ 35 ] | 2B | 87.90 | 91.01 | 88.83 | 80.70 | 88.68 | 84.08 |
| GPT Image 1 [High] [ 99 ] | - | 88.89 | 88.94 | 89.84 | 92.63 | 90.96 | 85.15 |
| Nano Banana Pro [ 29 ] | - | 91.00 | 92.85 | 91.56 | 92.39 | 89.93 | 87.16 |
| Z-Image [ 8 ] | 6B | 93.39 | 91.22 | 93.16 | 92.22 | 91.52 | 88.14 |
Table 6: Evaluation of general text-to-image generation ability on DPG-Bench [ 53 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | Basic Following | Advanced Following | Designer | Overall | ||||||||
| Avg | Attr | Rel | Reas | Avg | A+R | A+Re | R+Re | Style | Text | RW | |||
| Generation Models | |||||||||||||
| FLUX.1 [dev] [ 64 ] | 12B | 83.1 | 87.1 | 87.3 | 75.0 | 65.8 | 67.1 | 73.8 | 69.1 | 66.7 | 43.8 | 70.7 | 71.1 |
| Z-Image [ 8 ] | 6B | 78.4 | 79.5 | 80.5 | 75.1 | 72.9 | 72.9 | 67.0 | 73.9 | 90.0 | 94.8 | 88.1 | 80.2 |
| Seedream 3.0 [ 40 ] | - | 87.1 | 90.5 | 89.6 | 80.9 | 79.2 | 79.8 | 77.2 | 75.6 | 100.0 | 97.2 | 83.2 | 86.0 |
| Qwen-Image [ 136 ] | 20B | 86.2 | 90.5 | 88.2 | 79.8 | 79.3 | 79.2 | 78.9 | 75.6 | 100.0 | 92.8 | 90.3 | 86.1 |
Table 7: Evaluation of general text-to-image generation ability on TIIF [ 135 ] (Short Prompts). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. Abbreviations: Attr =Attribute, Rel =Relation, Reas =Reasoning, A+R =Attribute+Relation, A+Re =Attribute+Reasoning, R+Re =Relation+Reasoning, RW =Real World.
| Model | #Params | Basic Following | Advanced Following | Designer | Overall | ||||||||
| Avg | Attr | Rel | Reas | Avg | A+R | A+Re | R+Re | Style | Text | RW | |||
| Generation Models | |||||||||||||
| FLUX.1 [dev] [ 64 ] | 12B | 78.7 | 83.2 | 80.4 | 72.4 | 68.5 | 73.7 | 73.3 | 71.6 | 66.7 | 52.8 | 71.5 | 71.8 |
| Z-Image [ 8 ] | 6B | 82.8 | 86.5 | 79.9 | 81.9 | 77.0 | 77.6 | 73.8 | 75.6 | 93.3 | 93.2 | 85.5 | 83.0 |
| Seedream 3.0 [ 40 ] | - | 84.9 | 90.1 | 85.9 | 78.9 | 80.6 | 81.8 | 78.9 | 78.6 | 93.3 | 87.8 | 83.6 | 84.3 |
| Qwen-Image [ 136 ] | 20B | 87.2 | 91.5 | 90.8 | 79.4 | 80.9 | 79.8 | 81.7 | 78.6 | 100.0 | 89.1 | 91.4 | 86.8 |
Table 8: Evaluation of general text-to-image generation ability on TIIF [ 135 ] (Long Prompts). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. Abbreviations: Attr =Attribute, Rel =Relation, Reas =Reasoning, A+R =Attribute+Relation, A+Re =Attribute+Reasoning, R+Re =Relation+Reasoning, RW =Real World.
| Model | #Params | Alignment | Text | Reasoning | Style | Diversity | Overall |
| Generation Models | |||||||
| SDXL [ 35 ] | 2.6B | 0.69 | 0.03 | 0.24 | 0.33 | 0.30 | 0.32 |
| FLUX.1 [dev] [ 64 ] | 12B | 0.79 | 0.52 | 0.25 | 0.37 | 0.24 | 0.43 |
| Qwen-Image [ 136 ] | 20B | 0.88 | 0.89 | 0.31 | 0.42 | 0.18 | 0.54 |
| Z-Image [ 8 ] | 6B | 0.88 | 0.99 | 0.28 | 0.39 | 0.19 | 0.55 |
| Seedream 4.5 [ 7 ] | - | 0.89 | 1.00 | 0.35 | 0.43 | 0.21 | 0.58 |
Table 9: Evaluation of general text-to-image generation ability on OneIG-EN [ 11 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | Alignment | Text | Reasoning | Style | Diversity | Overall |
| Generation Models | |||||||
| Qwen-Image [ 136 ] | 20B | 0.83 | 0.96 | 0.27 | 0.41 | 0.21 | 0.53 |
| Z-Image [ 8 ] | 6B | 0.79 | 0.99 | 0.27 | 0.39 | 0.24 | 0.54 |
| Seedream 4.5 [ 7 ] | - | 0.83 | 0.99 | 0.30 | 0.43 | 0.21 | 0.55 |
| Nano Banana Pro [ 29 ] | - | 0.84 | 0.98 | 0.31 | 0.46 | 0.24 | 0.57 |
| Unified Models | |||||||
Table 10: Evaluation of general text-to-image generation ability on OneIG-ZH [ 11 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
Figure 20: Visualizations of general image generation. Compared to other open-source models, InternVL-U demonstrates exceptional fidelity in rendering intricate textures and nuanced lighting effects, capturing the precise intent of each instruction.
| Model | #Params | NED | CLIPScore | Word Accuracy | ||||
| 2 regions | 3 regions | 4 regions | 5 regions | average | ||||
| Generation Models | ||||||||
| FLUX.1 [dev] [ 64 ] | 12B | 0.688 | 0.740 | 0.609 | 0.553 | 0.466 | 0.432 | 0.497 |
| Nano Banana Pro [ 29 ] | - | 0.875 | 0.737 | 0.737 | 0.775 | 0.786 | 0.793 | 0.779 |
| Qwen-Image [ 136 ] | 20B | 0.912 | 0.802 | 0.837 | 0.836 | 0.831 | 0.816 | 0.829 |
| Z-Image [ 8 ] | 6B | 0.937 | 0.797 | 0.901 | 0.872 | 0.865 | 0.851 | 0.867 |
Table 11: Evaluation of text-centric text-to-image generation ability on CVTG-2k [ 33 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | LongText-Bench-EN | LongText-Bench-ZH |
| Generation Models | |||
| FLUX.1 [dev] [ 64 ] | 12B | 0.607 | 0.005 |
| Z-Image [ 8 ] | 6B | 0.943 | 0.946 |
| Qwen-Image [ 136 ] | 20B | 0.943 | 0.946 |
| Nano Banana Pro [ 29 ] | - | 0.981 | 0.949 |
| Seedream 4.5 [ 7 ] | - | 0.989 | 0.987 |
Table 12: Evaluation of text-centric text-to-image generation ability on LongText-Bench [ 42 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
Figure 21: Visualizations of text-centric image generation. The results show that our InternVL-U has excellent ability in rendering symbols of Chinese, English, and numerical formulas. Compared with the open-source unified multimodal models BAGEL and Ovis-U1, it has better rendering ability and comparable performance with Qwen Image, a large parameter and commercial closed source model Nano Banana Pro.
| Model | #Params | Cultural | Time | Space | Biology | Physics | Chemistry | Overall |
| Generation Models | ||||||||
| SD3-Medium [ 35 ] | 2B | 0.43 | 0.50 | 0.52 | 0.41 | 0.53 | 0.33 | 0.45 |
| FLUX.1 [dev] [ 64 ] | 12B | 0.48 | 0.58 | 0.62 | 0.42 | 0.51 | 0.35 | 0.50 |
| Qwen-Image [ 136 ] | 20B | 0.63 | 0.62 | 0.76 | 0.60 | 0.72 | 0.39 | 0.63 |
| Unified Models | ||||||||
| Janus-Pro [ 16 ] | 7B | 0.30 | 0.37 | 0.49 | 0.36 | 0.42 | 0.26 | 0.35 |
Table 13: Evaluation of knowledge-informed text-to-image generation ability on WISE [ 96 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Model | #Params | Math | Phy | Chem | Bio | Geo | Comp | Eng | Econ | Music | Hist | Overall |
| Generation Models | ||||||||||||
| FLUX.1 [dev] [ 64 ] | 12B | 12.2 | 14.4 | 12.5 | 22.8 | 36.4 | 11.0 | 14.0 | 9.2 | 21.3 | 21.7 | 17.6 |
| HunyuanImage-3.0 [ 9 ] | - | 17.0 | 17.2 | 18.8 | 18.7 | 30.4 | 15.5 | 16.9 | 11.7 | 23.9 | 20.4 | 19.1 |
| Qwen-Image [ 136 ] | 20B | 18.9 | 26.3 | 15.3 | 32.1 | 49.6 | 18.9 | 32.0 | 20.3 | 23.4 | 38.6 | 27.5 |
| Seedream 4.5 [ 7 ] | - | 44.7 | 63.4 | 48.9 | 75.8 | 67.6 | 57.9 | 69.7 | 67.3 | 38.0 | 55.0 | 58.8 |
| GPT-Image-1.5 [ 136 ] | - | 65.8 | 85.4 | 78.1 | 91.9 | 92.5 | 75.8 | 86.4 | 85.5 | 70.8 | 90.9 | 82.3 |
Table 14: Evaluation of knowledge-informed text-to-image generation ability on GenExam [ 132 ] (Relaxed Scores). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. Abbreviations: Math =Mathematics, Phy =Physics, Chem =Chemistry, Bio =Biology, Geo =Geography, Comp =Computer Science, Eng =Engineering, Econ =Economics, Hist =History.
Figure 22: Visualizations of knowledge-informed image generation. InternVL-U exhibits superior capability in accurate knowledge rendering. By effectively integrating domain knowledge, our model produces visually faithful results for complex prompts, significantly outperforming baselines that lack specific world knowledge.
| Model | #Params | Add | Adjust | Extract | Replace | Remove | Background | Style | Hybrid | Action | Overall |
| Generation Models | |||||||||||
| FLUX.1 Kontext [ 63 ] | 12B | 4.25 | 4.15 | 2.35 | 4.56 | 3.57 | 4.26 | 4.57 | 3.68 | 4.63 | 4.00 |
| GPT-Image-1 [High] [ 99 ] | - | 4.61 | 4.33 | 2.90 | 4.35 | 3.66 | 4.57 | 4.93 | 3.96 | 4.89 | 4.20 |
| Qwen-Image-Edit [ 136 ] | 20B | 4.38 | 4.16 | 3.43 | 4.66 | 4.14 | 4.38 | 4.81 | 3.82 | 4.69 | 4.27 |
| Z-Image-Edit [ 8 ] | 6B | 4.40 | 4.14 | 4.30 | 4.57 | 4.13 | 4.14 | 4.85 | 3.63 | 4.50 | 4.30 |
| Unified Models | |||||||||||
Table 15: Evaluation of general image editing ability on ImgEdit [ 153 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters.
| Models | #Params | BC | CA | MM | MC | PB | ST | SA | SR | SRp | TM | TT | Avg/G_O |
| Generation Models | |||||||||||||
| GPT Image 1 [ 99 ] | - | 6.96 | 6.85 | 7.10 | 5.41 | 6.74 | 7.44 | 7.51 | 8.73 | 8.55 | 8.45 | 8.69 | 7.49 |
| Qwen-Image-Edit [ 136 ] | 20B | 8.23 | 8.30 | 7.33 | 8.05 | 7.49 | 6.74 | 8.57 | 8.09 | 8.29 | 8.48 | 8.50 | 8.01 |
| Unified Models | |||||||||||||
| Lumina-DiMOO [ 145 ] | 8B | 3.43 | 4.27 | 3.08 | 2.77 | 4.74 | 5.19 | 4.44 | 3.80 | 4.38 | 2.68 | 4.20 | 3.91 |
| Ovis-U1 [ 124 ] | 2.4B+1.2B | 7.49 | 6.88 | 6.21 | 4.79 | 5.98 | 6.46 | 7.49 | 7.25 | 7.27 | 4.48 | 6.31 | 6.42 |
Table 16: Evaluation of general image editing ability on GEdit-Bench [ 83 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. Abbreviations: BC =Background Change, CA =Color Alteration, MM =Material Modification, MC =Motion Change, PB =Portrait Beautification, ST =Style Transfer, SA =Subject Addition, SR =Subject Removal, SRp =Subject Replacement, TM =Text Modification, TT =Tone Transfer.
Figure 23: Visualizations of general image editing. InternVL-U excels in generating realistic textures and styles while maintaining high fidelity to the original image’s lighting and structural details, demonstrating superior performance across various tasks compared to other open-source models.
| Models | # Params | Real | Virtual | ||||||||||||
| OA | OP | OR | F1 | NED | CLIP | AES | OA | OP | OR | F1 | NED | CLIP | AES | ||
| Generation Models | |||||||||||||||
| Qwen-Image-Edit [ 136 ] | 20B | 0.75 | 0.68 | 0.66 | 0.67 | 0.71 | 0.75 | 5.72 | 0.78 | 0.75 | 0.73 | 0.74 | 0.75 | 0.81 | 5.21 |
| GPT-Image-1.5 [ 98 ] | - | 0.74 | 0.69 | 0.67 | 0.68 | 0.68 | 0.75 | 5.78 | 0.73 | 0.72 | 0.71 | 0.71 | 0.70 | 0.80 | 5.28 |
| Nano Banana Pro [ 29 ] | - | 0.77 | 0.72 | 0.70 | 0.71 | 0.72 | 0.75 | 5.79 | 0.80 | 0.78 | 0.77 | 0.78 | 0.78 | 0.81 | 5.28 |
| Unified Models | |||||||||||||||
Table 17: Evaluation of text-centric image editing on TextEdit (Classic Metrics). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. “Real” refers to source images from real-world scene, while “Virtual” refers to images from virtual scene. Abbreviations: OA =OCR Accuracy, OP =OCR Precision, OR =OCR Recall, F1 =OCR F1-Score, NED =ROI-Aware NED, CLIP =CLIPScore, AES =Aesthetic Score. For detailed evaluation metric, please refer to Appendix Section A .
| Models | # Params | Real | Virtual | ||||||||||
| TA | TP | SI | LR | VC | Avg | TA | TP | SI | LR | VC | Avg | ||
| Generation Models | |||||||||||||
| Qwen-Image-Edit [ 136 ] | 20B | 0.92 | 0.82 | 0.75 | 0.57 | 0.80 | 0.77 | 0.57 | 0.79 | 0.92 | 0.80 | 0.77 | 0.77 |
| GPT-Image-1.5 [ 98 ] | - | 0.96 | 0.94 | 0.86 | 0.80 | 0.93 | 0.90 | 0.82 | 0.93 | 0.96 | 0.91 | 0.87 | 0.90 |
| Nano Banana Pro [ 29 ] | - | 0.96 | 0.95 | 0.85 | 0.88 | 0.93 | 0.91 | 0.87 | 0.92 | 0.96 | 0.94 | 0.89 | 0.92 |
| Unified Models | |||||||||||||
Table 18: Evaluation of text-centric image editing on TextEdit (MLLM-based Metrics). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. “Real” refers to source images from real-world scene, while “Virtual” refers to images from virtual scene. Abbreviations: TA : Text Accuracy, TP : Text Preservation, SI : Scene Integrity, LR : Local Realism, VC : Visual Coherence, Avg : MLLM Overall Average. For detailed evaluation metric, please refer to Appendix Section A .
Figure 24: Visualizations of text-centric image editing . InternVL-U demonstrates more accurate and faithful text editing capabilities, while maintaining strong consistency for both textual and visual content outside the target editing region.
| Models | #Params | Temporal | Causal | Spatial | Logical | Overall | IR | AC | VP |
| Generation Models | |||||||||
| FLUX.1 Kontext [ 63 ] | 12B | 2.3 | 5.5 | 13.0 | 1.2 | 5.8 | 26.0 | 71.6 | 85.2 |
| Qwen-Image-Edit [ 136 ] | 20B | 4.7 | 10.0 | 17.0 | 2.4 | 8.9 | 37.2 | 66.4 | 86.9 |
| Seedream 4.0 [ 110 ] | - | 12.9 | 12.2 | 11.0 | 7.1 | 10.8 | 58.9 | 67.4 | 91.2 |
| Nano Banana Pro [ 29 ] | - | 41.2 | 61.1 | 48.0 | 37.6 | 47.2 | 77.0 | 85.5 | 94.4 |
| GPT-Image-1.5 [ 98 ] | - | 54.1 | 60.0 | 62.0 | 21.2 | 50.0 | 69.7 | 92.5 | 94.9 |
Table 19: Evaluation of reasoning-informed image editing ability on RISEBench [ 161 ] . Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. Abbreviations: IR =Instruction Reasoning, AC =Appearance Consistency, VP =Visual Plausibility.
Figure 25: Visualizations of reasoning-informed image editing. InternVL-U outperforms state-of-the-art models in handling complex prompts requiring multi-step reasoning. The results demonstrate our model’s superior ability to accurately interpret and execute diverse logical constraints, spanning temporal calculations for updating calendar dates, spatial and cultural understanding for poem placement, and precise algorithmic rules for binary search tree insertion.
Figure 26: Visualizations of more special image editing examples. InternVL-U demonstrates its advanced spatial reasoning and precise control in complex editing tasks. The results highlight the model’s superior ability to accurately identify graph properties (e.g., node degrees), generate humor-centric content with appropriate expressions, and execute precise 3D geometric transformations based on coordinate vectors, showing its broad applicability in specialized domains.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Benchmarks | Type | Size | Human Filter | GT Ann. | Sub-class | Traditial Eval. | LLM Eval. |
| AnyText [ 122 ] | Text Generation | 2,000 | ✗ | ✗ | - | ✓ | ✗ |
| LongText [ 42 ] | Text Generation | 320 | ✓ | ✗ | 8 | ✗ | ✓ |
| CVTG-2K [ 33 ] | Text Generation | 2,000 | ✓ | ✗ | 2 | ✓ | ✗ |
| MARIO-Eval-edit [ 66 ] | Text Edit | 4,000 | ✗ | ✗ | - | ✓ | ✗ |
| TextEdit (Ours) | Text Edit | 2,148 | ✓ | ✓ | 18 | ✓ | ✓ |
Table 20: Key attributes comparison of open-source text generation or editing benchmarks.
Figure 27: Data Distribution of TextEdit benchmark.
| Major | ID | Category (Mid) | ID | Specific Scene (Sub) | Count |
| Virtual Scenes | 1.1 | Poster Scenes | 1.1.1 | Activities & Promotions Posters | 57 |
| 1.1.2 | Product & Advertising Posters | 74 | |||
| 1.1.3 | Movie & Art Posters | 107 | |||
| 1.2 | Comic Scenes | 1.2.1 | Dialogue & Narration | 65 | |
| 1.2.2 | Onomatopoeia & Special-effects Text | 26 | |||
| 1.3 | Slide / Presentation | 1.3.1 | Titles & Subtitles | 71 |
Table 21: Unified benchmark taxonomy with data statistics.
| Category | Metric | What it Measures |
| Classic (Text-Centric) | OCR Accuracy | Maximum similarity between the generated text in the target region and the ground-truth string. |
| OCR Precision | Accuracy of background text preservation (penalizes hallucinated or incorrect background text). | |
| OCR Recall | Completeness of background text preservation (penalizes missing background text). | |
| OCR F1-Score | Harmonic mean of OCR Precision and OCR Recall. | |
| ROI-Aware NED | Normalized Edit Distance specifically within the source text bounding box. | |
| Classic (General) | CLIPScore | Semantic alignment between the edited image and the target caption. |
Table 22: Overview of evaluation metrics. We categorize metrics into objective (Text-Centric, General) and perceptual (MLLM-based) dimensions. Detailed formulations are provided in Section A.2.2 .
Figure 28: The system prompt template used for MLLM-based automated evaluation , covering the analyst persona, task definition, and the first four scoring dimensions (Q1–Q4).
Figure 29: The continuation of the evaluation prompt , detailing the final scoring dimension (Q5) and the strict JSON output schema required for parsing results.
| Models | # Params | Real | Virtual | ||||||||||||
| OA | OP | OR | F1 | NED | CLIP | AES | OA | OP | OR | F1 | NED | CLIP | AES | ||
| Generation Models | |||||||||||||||
| Qwen-Image-Edit [ 136 ] | 20B | 0.76 | 0.69 | 0.67 | 0.67 | 0.70 | 0.75 | 5.81 | 0.74 | 0.71 | 0.70 | 0.70 | 0.70 | 0.80 | 5.27 |
| GPT-Image-1.5 [ 98 ] | - | 0.72 | 0.68 | 0.66 | 0.67 | 0.67 | 0.75 | 5.85 | 0.68 | 0.69 | 0.68 | 0.68 | 0.65 | 0.80 | 5.32 |
| Nano Banana Pro [ 29 ] | - | 0.76 | 0.71 | 0.69 | 0.70 | 0.70 | 0.75 | 5.86 | 0.77 | 0.76 | 0.75 | 0.75 | 0.76 | 0.81 | 5.32 |
| Unified Models | |||||||||||||||
Table 23: Evaluation of text-centric image editing on TextEdit MiniSet-500 (Classic Metrics). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. “Real” refers to source images from real-world scenes, while “Virtual” refers to images from synthetic virtual. Abbreviations: OA =OCR Accuracy, OP =OCR Precision, OR =OCR Recall, F1 =OCR F1-Score, NED =ROI-Aware NED, CLIP =CLIPScore, AES =Aesthetic Score.
| Models | # Params | Real | Virtual | ||||||||||
| TA | TP | SI | LR | VC | Avg | TA | TP | SI | LR | VC | Avg | ||
| Generation Models | |||||||||||||
| Qwen-Image-Edit [ 136 ] | 20B | 0.93 | 0.85 | 0.77 | 0.55 | 0.78 | 0.80 | 0.60 | 0.82 | 0.91 | 0.81 | 0.74 | 0.76 |
| GPT-Image-1.5 [ 98 ] | - | 0.97 | 0.94 | 0.86 | 0.79 | 0.92 | 0.91 | 0.85 | 0.93 | 0.95 | 0.92 | 0.83 | 0.88 |
| Nano Banana Pro [ 29 ] | - | 0.96 | 0.95 | 0.85 | 0.86 | 0.92 | 0.91 | 0.87 | 0.92 | 0.96 | 0.93 | 0.87 | 0.92 |
| Unified Models | |||||||||||||
Table 24: Evaluation of text-centric image editing on TextEdit MiniSet-500(MLLM-based Metrics). Sizes of unified models in “A + B” indicate separate understanding (A) and generation (B) parameters. “Real” refers to source images from real-world scenes, while “Virtual” refers to images from virtual scene. Abbreviations: TA : Text Accuracy, TP : Text Preservation, SI : Scene Integrity, LR : Local Realism, VC : Visual Coherence, Avg : MLLM Overall Average.
More in this topic
Unified Multimodal Models
Unified multimodal models combine multiple data types in shared systems for cross-modal understanding, generation, reasoning, and interaction.
See all 290 papers