A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research.
Figures & tables
Figure 1: Cheers Capabilities. (a) Performance on general understanding and generation benchmarks compared with unified multimodal models (UMMs) of similar scale. (b) Generated image samples of Cheers .
Figure 2: Architectural comparison between prior UMMs and Cheers . (a) Separated visual spaces for understanding and generation. (b) Single semantic-centric space with limited structural details. (c) Fused feature representation with potential interference. (d) Cheers (Ours): A unified vision tokenizer that integrates structural and semantic features to ensure stable semantic understanding while enhancing generative details.
Figure 3: Overview of Cheers , a unified framework for multimodal understanding and image generation. The Unified Vision Tokenizer converts visual inputs into semantic tokens that are jointly processed with text tokens by the LLM for understanding tasks, and detail tokens that serve as step-adaptive high-frequency injection into the CFM Head during generation. During generation, the CFM Head predicts a continuous-time velocity field in the latent space, enabling iterative sampling from Gaussian noise z0 to the terminal latent z1 , which is finally decoded by the VAE decoder.
Stage
Vision-Language Alignment
General Pre-Training
Refined Pre-Training
Supervised Fine-Tuning
Dataset
Image caption Text-to-image
Pure text corpus Detailed image caption Text-to-image OCR data
Pure text corpus Synthetic generation data General VQA
Pure text corpus Instruction text-to-image High-quality instruction data
Learning rate
1e−4
1e−4
4e−5
2e−5
Training parts
Projection & CFM Head & Gate
All parameters w/o VAE
All parameters w/o VAE
All parameters w/o VAE
Schedules
constant
constant
constant
cosine
Batch size
512
512
512
128
Training steps
30 K
60 K
65 K
30 K
Table 1: Training setup and hyperparameters across different stages for Cheers .
Table 3: Performances on GenEval. Obj.: Object; Attr.: Attribute; #Params.: LLM backbone parameters; #Data: total training samples (visual, image, and text).
Model
#Params.
#Data
Global
Entity
Attribute
Relation
Other
Overall
Generation Only
SDXL [ 77 ]
2.6B
-
83.27
82.43
80.91
86.76
80.41
74.65
DALL-E 3 [ 78 ]
-
-
90.97
89.61
88.39
90.58
89.83
83.50
SD3-Medium [ 79 ]
2B
-
87.90
91.01
88.83
80.70
88.68
84.08
Understanding & Generation
JanusFlow [ 73 ]
1.3B
-
87.03
87.31
87.39
89.79
88.10
80.09
Table 4: Performances on DPG-Bench. #Params.: LLM backbone parameters; #Data: total training samples (visual, image, and text).
Figure 4: Overall training pipeline and the progression of the GenEval score. The curve above illustrates the GenEval score as a function of cumulative training steps.
Figure 5: (a) Heatmap of high frequency injection across different generation steps. (b) High frequency injection intensity at each step. The reported values are aggregated over multiple runs, with per-run normalization applied prior to averaging across samples at the same step, ensuring a faithful representation of the overall trend.
Model
HFI
Fine-tuning Data
SEEDBench
MMBench
ChartQA
POPE
AI2D
GenEval
DPG-Bench
Cheers
✓
Understanding
70.8
65.2
58.5
87.0
67.7
-
-
Cheers
✗
Generation & Understanding
70.0
66.3
58.8
86.2
67.3
0.17
39.11
Cheers
✓
Generation & Understanding
69.8
67.1
59.9
87.5
68.1
0.30
51.63
Table 5: Ablation results. The first row corresponds to a model fine-tuned using understanding data only. The second and third rows report jointly trained models optimized for both understanding and generation, without and with HFI, respectively.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Pixel-Reconstruct
Training Data
ChartQA
OCRBench
TextVQA
MMBench
MMStar
SEEDBench
Cheers
✗
558K & 858K
13.9
2.5
9.6
23.2
27.7
38.4
Cheers
✗
4.6M & 858K
14.2
2.2
9.5
42.8
32.1
56.9
Cheers (Ours)
✓
558K & 858K
42.1
31.5
43.4
66.09
40.2
69.2
Appendix
Table 6: Effect of Pixel-Reconstruct on OCR-centric understanding and Others. We scale the alignment dataset from 558K to 4.6M samples (both fine-tuned on the same 858K data) to ensure sufficient training, and then introduce Pixel-Reconstruct under the same 4.6M alignment setting for comparison.
Change the color of the background to blue.
Turn this apple into a watermelon.
Place the red bell pepper from Figure 1 and the green broccoli from Figure 2 into one image.
Appendix
Table 7: Emergent capabilities of Cheers after Refined Pre-Training.
Discribe this image. This image captures a cozy corner of a room with a stunning view of the ocean. The foreground features a stone wall with a wooden ceiling, creating a warm and rustic atmosphere. A black chair is tucked under a small table, which holds a vase with red flowers and a framed picture. large window dominates the scene, offering an impressive view of the sea and distant land. The ocean stretches out to the horizon, meeting a clear blue sky. In the distance, a city skyline and mountains can be seen, adding depth and interest to the landscape. Inside the room, a straw-like hanging lamp adds a touch of texture and soft lighting. A potted palm tree adds a tropical feel to the space. The overall scene is serene and inviting, with a perfect blend of natural and man-made elements. It’s a peaceful coastal retreat that combines comfort with a breathtaking view.
Please answer the question requiring a floating-point number with one decimal place and provide the final value, e.g. , 1.2, 1.3, 1.4, at the end. Question: What is the difference of largest and smallest bar? To find the difference between the largest and smallest bar, we need to identify the values of the largest and smallest bars from the given data. The largest bar represents Iceland, with a value of 100.0. The smallest bar represents Cambodia, with a value of 52.4. Now, we need to calculate the difference between these two values: 100.0 - 52.4 = 47.6 Therefore, the difference between the largest and smallest bar is 47.6.
Appendix
Table 8: Emergent capabilities of Cheers after Refined Pre-Training.
Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.
Wujian Peng, Lingchen Meng, Yuxuan Cai +7
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · Qwen Team, Alibaba Inc.
Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE.
Yaqi Zhao, Wang Lin, Zijian Zhang +5
Peking University · Tencent Hunyuan · Zhejiang University
Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities. To address these challenges, we present Argus-Unified, a compact, effective and unified multimodal model built with low demand on computation and data. Instead of aligning modalities from scratch, Argus-Unified effectively leverages pretrained vision-language models (VLMs) that provide strong multimodal priors. Specifically, we introduce hybrid visual tokens that preserve continuous tokens for understanding while learning discrete tokens for generation from a frozen unified vision encoder. Our training pipeline includes two stages: the first stage learns a quantizer and image decoder on top of the frozen vision encoder, the second stage trains the LLM initialized from a pretrained VLM for the unified multimodal modeling. Using by far the least amount of data (15.6M) and the lowest cost (~$2,000), we demonstrate that unified multimodal models can be trained economically while achieving strong performance in both understanding and generation. Notably, our model attains state-of-the-art multimodal understanding on GQA, POPE, and VQAv2, and competitive generation quality compared to models with dedicated vision encoders (e.g., Janus, Janus-Pro), all at ~10x lower cost and with ~5x less data. We envision Argus-Unified as a useful baseline that lowers the development barrier for unified models.