Unified multimodal models aim to support visual understanding and generation within a single framework. However, prevailing approaches often separate their visual representations or backbone computation to accommodate the different requirements of these two tasks, which limits the degree of unification. In this paper, we present HiUni, which revisits this design choice and explores whether a pretrained vision language model can be directly extended into a unified model with extensive parameter sharing. Specifically, HiUni reuses the same language-aligned ViT and dense Transformer backbone for both capabilities, while confining task-specific computation to the output side. Yet such a design introduces two key challenges. Generation in pixel space is difficult to optimize, and semantic abstraction in pretrained ViT discards fine spatial details required for synthesis. To address these challenges, HiUni introduces a Pixel Diffusion Head tailored to direct pixel-space optimization and a lightweight HiGate pathway that provides fine-grained spatial details near the output stage. Extensive experiments show that HiUni achieves a similar average understanding score to the pretrained VLM while attaining competitive performance on image generation and editing. Controlled comparisons and analyses further demonstrate that a single dense backbone can effectively support both capabilities without task-specific backbone computation, highlighting a simpler path toward unified multimodal modeling.
Figures & tables
Figure 1: Comparison of representative UMM designs. (a) Bagel utilizes separate visual encoders and task-specific MoT modules for understanding and generation. (b) TUNA constructs a shared visual representation by cascading VAE and ViT. (c) HiUni reuses the pretrained VLM visual pathway, while HiGate supplies fine-grained pixel details only to the generation head.
Figure 2: Overview of HiUni . Starting from a pretrained VLM, HiUni shares the same visual encoder and dense backbone across understanding, generation, and editing. Language understanding is handled by a language modeling head, while image synthesis is performed by a Pixel Diffusion Head . To bridge semantic representations and pixel-level details, HiGate provides a lightweight residual pathway that supplies local spatial cues weakened by visual compression.
Figure 3: Data distribution of HiUni ’s training corpus for image understanding, generation, and editing. Inner rings show the major categories, while outer rings provide finer-grained breakdown.
Models
Size
MathVista Mini
MMBench EN
RealWorldQA
MMStar
HallusionBench
AI2D
OCRBench
OmniDocBench ↓
Understanding-specific Models
Qwen3-VL INS⋆
4B
73.7
85.1
70.9
69.8
57.6
84.1
88.1
0.24
Qwen3-VL THK⋆
4B
79.5
86.7
73.2
73.2
64.1
84.9
80.8
0.23
Qwen3-VL THK†
4B
76.5
86.2
74.1
70.3
65.8
84.5
79.6
0.22
Unified Multimodal Models
Harmon ⋆
1.5B
–
65.5
49.8
35.3
–
–
11.2
–
Table 1: Comparison with representative models on image understanding. HiUni is initialized from Qwen3-VL INS -4B and trained in thinking mode. We also report official results for Qwen3-VL THK (marked with ⋆ ), alongside our re-evaluated results (marked with † ) under identical evaluation settings. Model size is reported as understanding backbone parameters + generation-specific parameters. Hereinafter the same.
Models
Size
Qwen-Image-Bench ZH
Qwen-Image-Bench EN
OneIG
LongText
Qual.
Aesth.
Align.
Real.
Creat.
Overall
Qual.
Aesth.
Align.
Real.
Creat.
Overall
ZH
EN
ZH
EN
Generation-specific Models
GLM-Image
9B + 7B
49.26
50.64
47.90
44.69
45.23
48.19
49.86
49.98
47.49
44.25
44.67
47.86
0.511 ⋆
0.528 ⋆
0.979 ⋆
0.952 ⋆
Qwen-Image ⋆
20B
48.44
52.25
50.72
43.16
47.30
49.23
48.45
51.18
50.04
43.45
45.37
48.48
0.548
0.539
0.946
0.943
HiDream-O1
8B
48.61
49.75
49.53
43.97
40.96
47.58
48.84
49.69
49.53
43.80
41.05
47.59
0.518
0.521
0.978 ⋆
0.979 ⋆
Unified Multimodal Models
Table 2: Comparison with representative models on image generation. More results of OneIG are provided in Table 8 . Bold denotes the best result among UMMs, while underline denotes the second-best result. Hereinafter the same.
Models
Size
ImgEdit
GEdit CN
GEdit EN
Add
Adjust
Extract
Replace
Remove
Background
Style
Hybrid
Action
Overall
G_SC
G_PQ
G_O
G_SC
G_PQ
G_O
Editing-specific Models
LongCat-Image-Edit
6B
4.15
4.24
3.39
4.47
3.95
3.57
4.28
3.39
4.70
4.02
8.13
6.44
7.15
8.05
6.44
7.10
Qwen-Image-Edit-2511
20B
4.22
4.24
3.34
4.37
3.98
3.61
4.39
3.25
4.69
4.01
8.25
6.42
7.19
8.20
6.46
7.17
Qwen-Image-2.1
7B
4.17
4.24
2.16
4.48
4.13
3.75
4.27
3.15
2.89
3.69
7.60
6.45
6.73
7.91
6.40
6.92
Unified Multimodal Models
Table 3: Comparison with representative models on image editing. Both benchmarks are judged by GPT-5.2. More results of GEdit are provided in Table 9 .
Model
Understanding
Generation
Name
Size
MMBench EN
RealWorldQA
MMStar
HallusionBench
AI2D
OCRBench
Avg.
OneIG Avg.
HiUni -Dense-4B
4B+0.5B
72.60
65.49
53.93
48.96
77.95
71.30
65.04
0.314
HiUni -Dense-2B
2B+0.5B
64.94
60.39
46.67
38.70
70.47
70.10
58.55
0.291
HiUni -MoT-2B
4B+0.5B
67.97
63.27
47.00
35.33
71.76
70.02
59.23
0.345
HiUni -Und-4B
4B
72.52
63.79
53.33
46.96
77.85
73.00
64.58
–
HiUni -Gen-4B
4B+0.5B
–
–
–
–
–
–
–
0.390
Table 4: Comparison between dense and task-routed MoT backbones under controlled settings. HiUni -Dense-2B and HiUni -MoT-2B differ only in backbone design while HiUni -Dense-4B examines a different parameter allocation with stronger pretrained initialization. HiUni -Und-4B and HiUni -Gen-4B serve as understanding-only and generation-only baselines, respectively.
Figure 8
Configuration
Understanding
Generation
MMBench EN
RealWorldQA
MMStar
HallusionBench
AI2D
OCRBench
Avg.
OneIG Avg.
Pixel Features
72.60
65.49
53.93
48.96
77.95
71.30
65.04
0.314
ViT Features
74.61
67.45
53.07
49.05
77.82
69.00
65.17
0.284
No Gate
72.91
65.88
54.13
46.70
77.75
73.00
65.06
0.267
Table 5: Analysis of HiGate with different insertion positions. Pixel Features and ViT Features use features from different stages of the visual pathway as the input to HiGate , while No Gate removes the gated residual pathway. Pixel Features corresponds to the default design of HiUni .
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Generation
Pretraining
SFT
Warmup
Phase I
Phase II
Phase III
Phase IV
Phase I
Phase II
Hyperparameters
Peak learning rate
1×10−4
1×10−4
1×10−4
1×10−4
2×10−5
2×10−5
2×10−5
Min learning rate
-
-
-
2×10−5
-
-
2×10−6
LR scheduler
Constant
Constant
Constant
Cosine decay
Constant
Constant
Cosine decay
Optimizer
AdamW (β1=0.9,β2=0.95,ϵ=10−8)
Appendix
Table 6: Training recipe of HiUni . Image resolutions denote the maximum resolution of an image, subject to the image number and sequence length constraint. “-” denotes an inapplicable value.
Configuration
Number of Transformer blocks
10
Hidden dimension
D=2560
Attention
GQA with 32 query heads and 8 key-value heads
Head dimension
80
Feed-forward network
SwiGLU, intermediate dimension 9728
Flattened pixel dimension
3×32×32=3072
Appendix
Table 7: Configuration of the Pixel Diffusion Head and HiGate .
Models
Size
OneIG-ZH
OneIG-EN
Alignment
Text
Reasoning
Style
Diversity
Overall
Alignment
Text
Reasoning
Style
Diversity
Overall
Generation-only Models
GLM-Image ⋆
9B + 7B
0.738
0.976
0.284
0.335
0.221
0.511
0.805
0.969
0.298
0.353
0.213
0.528
Qwen-Image ⋆
20B
0.825
0.963
0.267
0.405
0.279
0.548
0.882
0.891
0.306
0.418
0.197
0.539
ERNIE-Image ⋆
8B
0.842
0.898
0.266
0.421
0.177
0.521
0.891
0.967
0.295
0.447
0.169
0.554
Unified Multimodal Models
Appendix
Table 8: Detailed results on OneIG.
Table 9: Detailed results on GEdit.
Figure 6: Diffusion training trajectories from development runs with and without AdaLN in the Pixel Diffusion Head. The run without AdaLN (right) shows fewer loss spikes. The two curves are collected at different development stages and therefore provide only a qualitative comparison rather than a controlled isolation of the effect of AdaLN.
Model
Overall
Cultural
Time
Space
Biology
Physics
Chemistry
DeepGen 1.0 ⋆
0.57
0.65
0.41
0.72
0.39
0.59
0.45
Bagel w/cot ⋆
0.63
0.78
0.63
0.57
0.38
0.55
0.51
SenseNova-U1-8B ⋆
0.68
0.69
0.60
0.74
0.63
0.72
0.69
SenseNova-U1-8B w/cot ⋆
0.79
0.80
0.74
0.86
0.76
0.78
0.80
Qwen-Image-2.0 ⋆
0.80
0.82
0.65
0.90
0.79
0.80
0.75
GPT-Image-1.5 ⋆
0.83
0.89
0.69
0.88
0.80
0.76
0.78
Appendix
Table 10: Results on the WISE benchmark for chain of thought image generation.
Configuration
Understanding
Generation
MMBench EN
RealWorldQA
MMStar
HallusionBench
AI2D
OCRBench
Avg.
OneIG Avg.
w/o Editing Supervision
69.58
64.58
50.20
48.43
72.54
70.50
62.64
0.538
w/ Editing Supervision
72.29
66.80
55.00
43.83
79.24
69.80
64.49
0.528
Appendix
Table 11: Effect of image-editing supervision on image understanding and generation. All other settings are kept fixed, with image-editing data as the only changed factor.
Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.
Wujian Peng, Lingchen Meng, Yuxuan Cai +7
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · Qwen Team, Alibaba Inc.
We present UniSpark-9B, a unified multimodal model that investigates hybrid attention for visual understanding and generation. Built on a Mixture-of-Transformer-Experts backbone with approximately 9B parameters in each branch, UniSpark combines full attention for explicit token interactions with GatedDeltaNet linear attention for compact context propagation. Autoregressive text prediction and visual flow matching jointly support visual question answering, text-to-image generation, instruction-guided editing, and interleaved text-image prediction. In interleaved generation, noisy visual targets are isolated from subsequent predictions, while clean visual representations provide context for later outputs. A blockwise protocol enforces this distinction through attention masks in full-attention layers and recurrent-state retention rules in linear-attention layers. The selected training sources contain 159.7M generation samples, 213.4M understanding samples, and 624.8K interleaved sequences. A four-stage curriculum jointly trains multimodal understanding and visual generation, refining answer accuracy and instruction following while extending image synthesis to 2K. UniSpark scores 2470.3 on MME-S and 0.83 on GenEval. Qualitative examples further illustrate its capabilities in interleaved image generation, video synthesis, and video editing. Together, these results demonstrate that hybrid attention provides an effective backbone for unified multimodal understanding and generation.
Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities. To address these challenges, we present Argus-Unified, a compact, effective and unified multimodal model built with low demand on computation and data. Instead of aligning modalities from scratch, Argus-Unified effectively leverages pretrained vision-language models (VLMs) that provide strong multimodal priors. Specifically, we introduce hybrid visual tokens that preserve continuous tokens for understanding while learning discrete tokens for generation from a frozen unified vision encoder. Our training pipeline includes two stages: the first stage learns a quantizer and image decoder on top of the frozen vision encoder, the second stage trains the LLM initialized from a pretrained VLM for the unified multimodal modeling. Using by far the least amount of data (15.6M) and the lowest cost (~$2,000), we demonstrate that unified multimodal models can be trained economically while achieving strong performance in both understanding and generation. Notably, our model attains state-of-the-art multimodal understanding on GQA, POPE, and VQAv2, and competitive generation quality compared to models with dedicated vision encoders (e.g., Janus, Janus-Pro), all at ~10x lower cost and with ~5x less data. We envision Argus-Unified as a useful baseline that lowers the development barrier for unified models.