Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous representation. We empirically demonstrate that reducing channel dimension is significantly more effective than spatial downsampling for both reconstruction and generation. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. Furthermore, we validate that the transfusion architecture surpasses query-based designs in convergence and consistency. Experiments demonstrate that UniCom achieves state-of-the-art generation performance among unified models. Notably, by preserving rich semantic priors, it delivers exceptional controllability in image editing and maintains image consistency even without relying on VAE.
Figures & tables
Figure 1: High-quality samples generated by UniCom. Built on compressed continuous representations, our unified multimodal model demonstrates exceptional capabilities in text-to-image generation, precise image editing, and fine-grained controllable generation.
Figure 2: Overview of the proposed framework. For a controlled comparison, both pathways are built upon the same compressed representations and jointly optimized with cross-entropy loss ( Lce ) and flow matching loss ( Lfm ).
Tokenizer
Res.
rFID ↓
PSNR ↑
SSIM ↑
Specialized tokenizers
SD-VAE Esser et al. (2024b)
256
1.06
28.62
0.86
GigaTok Xiong et al. (2025)
256
0.51
21.32
0.69
VA-VAE Yao et al. (2025)
256
0.26
28.59
0.80
DC-AE Chen et al. (2024)
512
0.22
26.15
0.71
MAE-Tok Chen et al. (2025a)
512
0.62
-
-
Table 1: Evaluation on image reconstruction ability on the ImageNet validation set . d denotes the feature dimension of compressed representations z~ . Models marked with † are re-evaluated using official checkpoints.
Figure 3: Comparison of the results for image editing, highlighting UniCom performance in tasks such as image manipulation, object swapping, and color adjustment. See more visualization in the Appendix.
Figure 4: Overview of the proposed diffusion decoder and reconstruction analysis. Given an input image, the vision encoder extracts semantic features which are then processed by compressor and decompressor to condition the diffusion model for image reconstruction. During training, the compression modules and the diffusion backbone are optimized, while the encoder remains frozen. We investigate the bottleneck capacity by varying the token number n and feature dimension d . As illustrated in the bottom row, compressing the dimension d yields superior reconstruction fidelity, whereas reducing the token number n leads to noticeable blurring in fine details.
Figure 5: Visual comparison of image reconstruction results . Notably, our method (d64) maintains high fidelity in high-frequency details (e.g., text characters) and preserves facial identity better than these semantic-based baselines, achieving quality comparable to the specialized Flux.1-dev VAE.
GenEval
DPG
WISE
Models
Single
Two
Count
Colors
Pos
Col-Attr
Overall
Overall
Cult.
Time
Space
Bio.
Phy.
Chem.
Overall
Generation-only Models
SD3-Medium ( Esser et al., 2024a )
0.99
0.94
0.72
0.89
0.33
0.60
0.74
-
-
-
-
-
-
-
-
FLUX.1 [Dev] ( Labs, 2024 )
0.98
0.93
0.75
0.93
0.68
0.65
0.82
84.00
0.48
0.58
0.62
0.42
0.51
0.35
0.50
Unified Multimodal Models
MetaQuery-XL † ( Pan et al., 2025b )
-
-
-
-
-
-
0.80
-
0.56
0.55
0.62
0.49
0.63
0.41
0.55
Table 2: Image generation results on GenEval, DPG-Bench, and WISE . † refers to methods using LLM rewriters on GenEval. Abbreviations for WISE: Cult. (Cultural), Bio. (Biology), Phy. (Physics), Chem. (Chemistry).
ImgEdit-Bench
GEdit-Bench
KRIS-Bench
WorldEdit
Models
Add
Adj.
Ext.
Rep.
Rm.
Bg.
Sty.
Hyb.
Act.
Overall
G-SC
G-PQ
G-Overall
Fact.
Conc.
Proc.
Overall
Overall
Generation-only Models
FLUX.1 Kontext [Pro] ( Batifol et al., 2025 )
4.25
4.15
2.35
4.56
3.57
4.26
4.57
3.68
4.63
4.00
7.02
7.60
6.56
57.22
55.06
46.69
54.17
3.21
Qwen-Image ( Wu et al., 2025a )
4.38
4.16
3.43
4.66
4.14
4.38
4.81
3.82
4.69
4.27
8.00
7.86
7.56
-
-
-
-
-
Specialized Editing Models
Instruct-Pix2Pix ( Brooks et al., 2023 )
2.45
1.83
1.44
2.01
1.50
1.44
3.55
1.20
1.46
1.88
3.58
5.49
3.68
23.33
25.59
17.28
22.82
2.44
Table 3: Comparison of image editing capabilities. We evaluate on ImgEdit-Bench, GEdit-Bench, KRIS-Bench and WorldEdit. For ImgEdit-Bench, performance is evaluated across nine distinct operation categories (e.g., ‘Add’, ‘Adjust’, ‘Extract’, ‘Replace’, ‘Remove’, ‘Background’, ‘Style’, ‘Hybrid’, and ‘Action’. ). For GEdit-Bench, metrics include ‘G-Semantic Consistency’ (G-SC) and ‘G-Perceptual Quality’ (G-PQ). For KRIS-Bench, we report Factual (Fact.), Conceptual (Conc.), and Procedural (Proc.) knowledge scores. Bold : best results. Underline : second-best.
Projector
Tokens ( n )
Dim ( d )
Reconstruction
rFID ↓
PSNR ↑
SSIM ↑
–
1024
1152
0.40
23.26
0.69
–
256
1152
0.72
20.29
0.56
MLP
1024
256
0.62
21.73
0.66
MLP
1024
64
0.55
22.17
0.66
MHA
1024
64
0.56
22.61
0.69
Table 4: Reconstruction performance of different compressed feature shapes on ImageNet50k ( Deng et al., 2009 ) .
Figure 10
Figure 8: T-SNE visualization of feature distributions. We randomly select 6 distinct classes from ImageNet, sampling 150 images per class. MHA compression to 64-d preserves the structure, MLP projection yields scattered 64-d features, and FLUX-VAE has no semantic separability due to pixel-reconstruction focus.
Model Config.
General VQA
Knowledge
Text-Centric
GQA
RealWorldQA
SEED
MMMU
ChartQA
OCRBench
Baseline
65.25
64.31
74.63
44.56
69.04
55.40
Compression Strategies
MLP Proj.
62.80
60.39
69.92
43.00
56.80
31.70
MHA Proj.
64.01
63.14
71.75
44.11
62.12
36.00
Concatenation Strategies
Table 5: Visual understanding performance on six benchmarks using compressed representations as visual inputs.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Reducing dimensions maintains higher reconstruction fidelity than sequence reduction. We compare the reconstruction training dynamics (Loss and PSNR) of different compressed representations ( z~ ).
Figure 10: Additional visual comparison of image reconstruction results.
Figure 11: Qualitative reconstruction samples using Decoder (d64). Left: Reconstructed image. Right: Original image. Even with an 18× channel compression ratio, our model accurately recovers complex high-frequency details.
Figure 16
Cause Category
Metric
GPT-4o
Nano-Banana
Seedream4.0
UniCom
WorldEdit
Flux-Kontext
Bagel-Think
Bagel
Omnigen
Omnigen2
Emu2
Anyedit
Ip2p
Magicbrush
Time
VC
4.02
4.22
3.92
3.08
4.18
4.6
2.38
2.34
3.02
2.22
1.48
1.63
3.04
1.40
VQ
5.00
5.00
4.86
4.78
4.55
4.88
4.52
4.02
4.20
4.35
4.56
3.25
4.51
4.28
IF
3.86
3.90
4.25
4.08
3.71
1.46
2.46
1.66
1.38
1.33
1.54
1.79
1.51
1.19
KP
4.20
4.14
4.41
4.38
3.94
1.42
2.86
1.80
1.48
1.25
1.75
1.92
1.65
1.34
Avg
4.27
4.32
4.36
4.08
4.10
3.09
3.06
2.46
2.52
2.29
2.33
2.15
2.68
2.05
Temperature
VC
4.02
4.22
4.06
3.40
3.52
3.86
2.08
1.94
2.54
2.26
1.38
1.45
2.62
1.81
Appendix
Table 7: The performance of both commercial and open-source models on image editing tasks in WorldEdit-Test, evaluated across different causes and metrics.
Figure 13: More results on single image editing. From top to bottom: (1) Add/Remove/Extract : adding objects, removing elements, and extracting subjects; (2) Replace : substituting objects or materials while preserving context; (3) Style Transfer : transforming image appearance into different artistic styles; (4) Background : modifying or replacing image backgrounds; (5) Subject-Driven Generation : generating new images conditioned on reference subjects; (6) Controllable Generation : generating images with spatial or structural control signals.
Figure 14: More results on multi-element composition. Our model can compose multiple reference subjects into a coherent scene while preserving the identity of each element.
Figure 15: More results on intelligent image editing. These examples require complex world knowledge such as biological transformations, geographical understanding, and cultural context to perform accurate editing.
Figure 16: Chain-of-thought enhanced generation and editing. For each example, we show the generated image along with the corresponding thinking process enclosed in <think> tokens. The model first reasons about the required visual transformation, then generates the output image accordingly.
Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text within a single autoregressive framework. This unified design offers architectural simplicity and cross-modal synergy, which facilitates shared parameterization, consistent training objectives, and seamless transfer between modalities. However, the large number of visual tokens required by such models introduces substantial computation and memory overhead, and this inefficiency directly hinders deployment in resource constrained scenarios such as embodied AI systems. In this work, we propose a unified token compression algorithm UniCompress that significantly reduces visual token count while preserving performance on both image understanding and generation tasks. Our method introduces a plug-in compression and decompression mechanism guided with learnable global meta tokens. The framework is lightweight and modular, enabling efficient integration into existing models without full retraining. Experimental results show that our approach reduces image tokens by up to 4 times, achieves substantial gains in inference latency and training cost, and incurs only minimal performance degradation, which demonstrates the promise of token-efficient unified modeling for real world multimodal applications.
A recent cutting-edge topic in multimodal modeling is to unify visual comprehension and generation within a single model. However, the two tasks demand mismatched decoding regimes and visual representations, making it non-trivial to jointly optimize within a shared feature space. In this work, we present Cheers, a unified multimodal model that decouples patch-level details from semantic representations, thereby stabilizing semantics for multimodal understanding and improving fidelity for image generation via gated detail residuals. Cheers includes three key components: (i) a unified vision tokenizer that encodes and compresses image latent states into semantic tokens for efficient LLM conditioning, (ii) an LLM-based Transformer that unifies autoregressive decoding for text generation and diffusion decoding for image generation, and (iii) a cascaded flow matching head that decodes visual semantics first and then injects semantically gated detail residuals from the vision tokenizer to refine high-frequency content. Experiments on popular benchmarks demonstrate that Cheers matches or surpasses advanced UMMs in both visual understanding and generation. Cheers also achieves 4x token compression, enabling more efficient high-resolution image encoding and generation. Notably, Cheers outperforms the Tar-1.5B on the popular benchmarks GenEval and MMBench, while requiring only 20% of the training cost, indicating effective and efficient (i.e., 4x token compression) unified multimodal modeling. We will release all code and data for future research.
Yichen Zhang, Da Peng, Zonghao Guo +19
Tsinghua University · Xi’an Jiaotong University · University of Chinese Academy of Sciences
Unified Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual tokens without additional re-encoding. UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on multimodal understanding benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.
Wujian Peng, Lingchen Meng, Yuxuan Cai +7
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · Qwen Team, Alibaba Inc.