cs.CVApr 8, 2026
SaveFlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
Organizations: University of Electronic Science and Technology of China · National University of Singapore · Central South University · Microsoft ·
Abstract
Multimodal generation has long been dominated by text-driven pipelines where language dictates vision but cannot reason or create within it. We challenge this paradigm by asking whether all modalities, including textual descriptions, spatial layouts, and editing instructions, can be unified into a single visual representation. We present FlowInOne, a framework that reformulates multimodal generation as a purely visual flow, converting all inputs into visual prompts and enabling a clean image-in, image-out pipeline governed by a single flow matching model. This vision-centric formulation naturally eliminates cross-modal alignment bottlenecks, noise scheduling, and task-specific architectural branches, unifying text-to-image generation, layout-guided editing, and visual instruction following under one coherent paradigm. To support this, we introduce VisPrompt-5M, a large-scale dataset of 5 million visual prompt pairs spanning diverse tasks including physics-aware force dynamics and trajectory prediction, alongside VP-Bench, a rigorously curated benchmark assessing instruction faithfulness, spatial precision, visual realism, and content consistency. Extensive experiments demonstrate that FlowInOne achieves state-of-the-art performance among open-source models across all unified generation tasks while remaining competitive with leading commercial systems, thereby establishing a new foundation for fully vision-centric generative modeling, in which perception and creation coexist within a unified continuous visual space. Our code and models are released on https://csu-jpg.github.io/FlowInOne.github.io/
Figures & tables
Figure 1: Comparison of generation paradigms. Left: Traditional T2I only uses the text encoder to condition the Latent Diffusion Model(LDM); Middle: Traditional TI2I requires the joint conditioning of both the text and image encoders; Right: We unify the conditions as visual input and form a simple image-in, image-out framework with a single model.
Figure 2: VisPrompt-5M is a comprehensive dataset that comprises eight distinct data types, including class-to-image generation, text-to-image generation, text-in-image editing, text bounding box editing, visual marker editing, doodles editing, force understanding, and trajectory understanding. The dataset covers a wide spectrum of image-to-image generation, ranging from basic text-in-image generation to compositional editing, and further to physics-aware instruction following.
Figure 3: Overview of the FlowInOne architecture, a general and simple framework using flow matching for continuous evolution in only one modality. FlowInOne employs a Dual-Path Spatially-Adaptive Modulation to adapt computation by modality. For input image rendering with only text, the structural branch is bypassed to strictly follow semantic evolution. Conversely, for image editing, a spatially-adaptive gated network and cross attention activates to selectively inject source priors, dynamically balancing original image preservation with instruction-driven reconstruction.
| Method | C2I | T2I | TIE | FU | TBE | TU | VME | DE | Total |
|---|---|---|---|---|---|---|---|---|---|
| Evaluator: Gemini3 | |||||||||
| Nano Banana Google (2025b) | .810 | .980 | .521 | .500 | .600 | .020 | .537 | .740 | .589 |
| Omnigen2 Wu et al. (2025b) | .720 | .760 | .313 | .013 | .020 | .000 | .020 | .140 | .248 |
| Kontext Labs et al. (2025) | .620 | .700 | .363 | .027 | .163 | .020 | .096 | .180 | .271 |
| Qwen-IE-2509 Wu et al. (2025a) | .680 | .690 | .383 | .047 | .060 | .000 | .040 | .160 | .258 |
| FlowInOne (Ours) | .890 | .700 | .355 | .727 | .302 | .520 | .292 | .535 | .540 |
Table 1: Evaluation on the VP-Bench visual instruction benchmark using three VLM evaluators and human judges. We use Gemini3 Google (2025a) , GPT5.2 OpenAI (2025) and Qwen3.5 Qwen Team (2026) to evaluate the success ratio of FlowInOne on all kinds of visual instruction tasks. C2I: class to image, T2I: text to image, TIE: text in image edit, FU: force understanding, TBE: text & bbox edit, TU: trajectory understanding, VME: visual marker edit, DE: doodles edit. Total denotes the average success rate across all sub-categories.
| Method | Gemini3 | GPT5.2 | Qwen3.5 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IF | CC | VR | SP | IF | CC | VR | SP | IF | CC | VR | SP | |
| Nano Banana | 3.43 | 3.10 | 4.43 | 2.99 | 3.28 | 2.78 | 4.06 | 2.86 | 3.30 | 3.01 | 4.43 | 2.94 |
| Omnigen2 | 2.15 | 1.39 | 3.46 | 1.69 | 2.29 | 1.53 | 3.31 | 1.92 | 1.72 | 0.91 | 3.13 | 1.27 |
| Kontext | 2.31 | 1.62 | 3.15 | 2.25 | 2.40 | 1.71 | 3.25 | 2.21 | 1.97 | 1.87 | 3.11 | 1.76 |
| Qwen-IE-2509 | 2.96 | 2.51 | 3.28 | 1.91 | 2.76 | 2.39 | 3.58 | 1.94 | 2.33 | 1.98 | 3.23 | 1.39 |
| FlowInOne | 3.38 | 2.94 | 3.12 | 3.42 | 3.16 | 2.81 | 2.96 | 3.24 | 3.31 | 2.87 | 3.20 | 3.30 |
Table 2: Overall four-dimensional breakdown. Each dimension has a maximum score of 5 points. IF: Instruction Faithfulness, CC: Content Consistency, VR: Visual Realism, SP: Spatial Precision.
| Method | IQA | CLIP Score | Dir CLIP | DINOv3 Sim | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Total | C2I | T2I | Avg. | TIE | VME | Avg. | DE | FU | TBE | TU | Avg. | |
| Nano Banana Google (2025b) | 0.688 | 0.281 | 0.302 | 0.291 | 0.106 | 0.103 | 0.105 | 0.430 | 0.474 | 0.501 | 0.486 | 0.473 |
| Omnigen2 Wu et al. (2025b) | 0.603 | 0.173 | 0.208 | 0.191 | 0.001 | 0.005 | 0.003 | 0.224 | 0.066 | 0.004 | 0.207 | 0.125 |
| Kontext Labs et al. (2025) | 0.621 | 0.164 | 0.172 | 0.168 | 0.010 | 0.008 | 0.009 | 0.283 | 0.240 | 0.118 | 0.004 | 0.161 |
| Qwen-IE-2509 Wu et al. (2025a) | 0.646 | 0.231 | 0.216 | 0.224 | 0.005 | 0.011 | 0.008 | 0.133 | 0.215 | 0.250 | 0.207 | 0.201 |
| FlowInOne (Ours) | 0.684 | 0.290 | 0.276 | 0.283 | 0.092 | 0.101 | 0.097 | 0.335 | 0.536 | 0.506 | 0.570 | 0.487 |
Table 3: Quantitative evaluation on VP-Bench across five models. CLIP-IQA is averaged over all categories to measure overall visual realism. CLIP Score evaluates semantic alignment for pure generation tasks. Directional CLIP Similarity assesses instruction-driven consistency in marker-based editing. DINOv3 Similarity measures fine-grained spatial and physical structural accuracy.
Figure 4: Visual instruction editing comparison across methods.
Table 4: Ablation study on compression method, modulation method, and training strategy. The average scores across Gemini3 Google (2025a) , GPT5.2 OpenAI (2025) and Qwen3.5 Qwen Team (2026) are used as the evaluation metric.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Overall error types in VP-Bench. For brevity in the figures, the labels Fidelity, Spatial, Realism, and Consistency correspond strictly to Instruction Fidelity, Spatial Precision, Visual Realism, and Content Consistency, respectively.
Figure 6: Error types for each subset in VP-Bench.
Figure 7: Qualitative results of the robustness analysis against visual instruction perturbations. We evaluate the model’s generation stability under five distinct conditions: the original unmodified input, variations in text style (e.g., size, color, font, and layout), changes in text length, text blurring, and random text corruption.
Figure 8: Generation results across different input image resolutions.
Figure 9: Qualitative results of the visual instruction ablation study. We compare generation outcomes across four input configurations: Blank (original image only), Text only, Visual prompt only, and the complete Text & visual prompt.
| Model | Fleiss’ | Kendall’s |
|---|---|---|
| FlowInOne | 0.6537 | 0.7935 |
| FLUX.1 Kontext | 0.4540 | 0.5189 |
| Nano Banana | 0.4537 | 0.5251 |
| OmniGen2 | 0.5361 | 0.5082 |
| Qwen-IE-2509 | 0.4515 | 0.6115 |
| Overall | 0.6132 | 0.7422 |
Table 5: Inter-evaluator Agreement.
| Top-Level Task | Sub-Category / Grouping | Pairs | Total |
| Class-to-Image | - | - | 100 |
| Doodles Editing | - | - | 50 |
| Force Understanding | Point Force | 100 | 150 |
| Global Force | 50 | ||
| Text-to-Image | - | - | 50 |
| Text BBox Control | - | - | 50 |
Table 6: Detailed statistics of the human-curated VP-Bench . To provide a structured overview, highly fine-grained editing sub-categories are conceptually grouped into higher-level semantic macro-categories.
| Task Category | Pairs | Avg. Words | Word Range | Avg. Chars |
|---|---|---|---|---|
| Class-to-Image | 100 | 3.0 | 1 – 6 | 22.4 |
| Text BBox Control | 43 | 3.5 | 3 – 5 | 16.0 |
| Doodles Editing | 43 | 12.0 | 12 – 12 | 63.0 |
| Trajectory Understanding | 50 | 14.0 | 14 – 14 | 79.0 |
| Visual Marker Editing | 312 | 14.1 | 3 – 32 | 74.3 |
| Force Understanding | 150 | 17.7 | 13 – 24 | 96.0 |
Table 7: Textual prompt statistics across different task categories in VP-Bench. The uniquely low average word counts prove the benchmark’s reliance on explicit visual markers rather than lengthy textual descriptions.
Figure 12: Additional qualitative results for text-to-image generation and text-in-image editing tasks.
Figure 13: Additional qualitative results for text bounding box (bbox) editing, doodle-guided editing, and visual marker-based editing.
Figure 14: Additional qualitative results demonstrating physical force understanding and trajectory understanding.
| Task Category | Raw Source(s) | Raw Size | Retained Size | Construction Pipeline | Visual Instruction Format |
| Text to Image | text-to-image-2M zk [2024] | 2.26M | 2.26M | Direct Rendering | Text |
| Class to Image | ImageNet (Subset) Russakovsky et al. [2014] | 2M | 860K | Direct Rendering | Text (Class Labels) |
| Text in Image Edit | GPT-Image-Edit Wang et al. [2025b] | 1.5M | 1.04M | Extracted & Filtered | Text |
| Pico-Banana Qian et al. [2025] | 400K | 10K | |||
| UnicEdit Ye et al. [2025] | 2M | 585K | |||
| PixWizard Lin et al. [2024] | 500K | 315K | Extracted |
Table 8: Comprehensive Overview of the VisPrompt-5M Dataset. We detail the task categories, original data sources, data scale before and after the filtering pipeline, construction methodology, and the explicitly embedded visual instruction formats.
Figure 15: General data construct process.
Figure 16: Detailed instance distribution of VisPrompt-5M. Left: The Force & Trajectory Generation subset, highlighting strictly curated physics-aware categories (e.g., wind, object poking) designed to impart dynamic kinematic priors. Right: The Text-in-Image Editing subset (derived from GPT-Image-Edit), demonstrating a natural long-tailed distribution of semantic operations. Ranging from high-frequency attribute modifications to specialized style transfers, this structural diversity ensures robust generalization across both complex stylistic changes and physical constraints.
Figure 17: Fine-grained structure and stylized distributions of VisPrompt-5M. Left: The Structured Editing subset (derived from PixWizard), highlighting dense spatial translations such as Image-to-Sketch and Face Restoration. This subset trains the model to strictly adhere to geometric and structural conditions. Right: The Text-in-Image Editing subset (derived from PicoBanana), detailing 35 highly specialized, long-tailed aesthetic operations. This extreme categorical diversity ensures the model’s proficiency in handling nuanced, localized, and composite textual instructions.
Figure 18: Semantic diversity and region-aware distributions of VisPrompt-5M. Left: The UnicEdit10M Diverse Edits subset, showcasing a profound long-tailed distribution. It spans high-frequency semantic operations like Color Alteration and Subject Addition to rare edge cases such as Object Extraction. Right: The VisMarker Region-Aware Edits subset, featuring robust, high-volume spatial operations like Object Swap and Removal. This combination ensures the model learns broad semantic reasoning while maintaining strict adherence to localized visual markers.
Figure 19: Volume of ancillary generation and structural datasets. This logarithmic lollipop chart illustrates the extreme variance in scale and specialization across our supplementary sources. It highlights the massive foundation of over 2.26 million general Text-to-Image pairs, balanced by broad categorical coverage (ImageNet21K) and highly specialized, precise structural tasks such as Text Bbox Edit and Doodles. This multi-scale composition ensures the model achieves both robust open-domain generative capabilities and fine-grained execution of explicit visual constraints.
| Task Category | Primary Source(s) | Initial Volume | Curated Pairs |
|---|---|---|---|
| Fundamental Gen. (Text) | Text-to-Image-2M zk [2024] | 2,269,892 | 2,269,892 |
| Fundamental Gen. (Class) | ImageNet Subset Russakovsky et al. [2014] | 2,000,000 | 860,000 |
| Text-in-Image Editing | GPT-Image-Edit, Pico-Banana, UnicEdit | 2,100,000 | 1,601,000 |
| Structured Editing | PixWizard Lin et al. [2024] | 350,000 | 315,000 |
| Bounding Box Editing | GPT-Image-Edit (Filtered) | 45,000 | 24096 |
| Visual Marker Editing | Qwen3-VL Synthesized | 300,000 | 250,000 |
Table 9: Comprehensive breakdown of the VisPrompt-5M composition. The dataset aggregates various public sources, strictly refined through our multi-stage OCR and VLM filtering pipeline.
| Category | Primary Rejection Reason | VLM/OCR Filtering | Retention Rate |
|---|---|---|---|
| Text-in-Image Editing | Semantic Inconsistency | VLM Semantic Check | 78.2% |
| Bounding Box Editing | Geometric Misalignment | VLM Spatial Check | 53.3% |
| Doodles Editing | Generative Instability | VLM Realism Check | 20.0% |
| Visual Marker Editing | Generative Instability | VLM Spatial Check | 83.3% |
Table 10: Filtering survival rates across selected complex generative tasks, demonstrating the rigorousness of our automated OCR and VLM inspection.
Figure 20: Evaluation process.
Figure 21: Evaluation prompts used in VLM evaluators.
| Component | Params (M) | Train. |
|---|---|---|
| Flow Backbone | 1108.40 | Yes |
| Text-Image VAE | 103.65 | Yes |
| Visual Encoder | 322.58 | No |
| Image VAE | 83.65 | No |
Table 11: Parameter Breakdown of FlowInOne.
| (a) CFG Scale | (b) Sampling Steps | (c) Loss Weights | |||
|---|---|---|---|---|---|
| CFG | PASS (%) | Steps | PASS (%) | PASS (%) | |
| 1.1 | 38.5 | 10 | 25.7 | (0.1, 0.001) | 46.7 |
| 1.5 | 44.8 | 20 | 40.3 | (0.1, 0.01) | 45.2 |
| 3.0 | 47.7 | 30 | 46.3 | (0.1, 0.1) | 44.2 |
| 5.0 | 51.5 | 40 | 48.7 | (1, 0.001) | 46.0 |
| 7.0 | 54.0 | 50 | 54.0 | (1, 0.01) | 54.0 |
Table 12: Hyperparameter ablation study on VP-Bench. We report the average PASS Rate (%) evaluated by three VLMs. Default optimal settings (CFG , Steps , , ) are marked in bold . The control variable method is applied; parameters not actively ablated are fixed to their default values.
More in this topic
Multimodal Generation
Multimodal generation creates coordinated text, images, audio, video, 3D, and other outputs from mixed inputs while preserving meaning and consistency.
See all 243 papers