Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device

Abdelrahman Shaker; Ahmed Heakl; Jaseel Muhammad; Ritesh Thawkar; Omkar Thawakar; Senmao Li; Hisham Cholakkal; Ian Reid

cs.CVFeb 23, 2026

Mobile-O: Unified Multimodal Understanding and Generation on Mobile Device

Authors: Abdelrahman Shaker, Ahmed Heakl, Jaseel Muhammad, Ritesh Thawkar, Omkar Thawakar, Senmao Li, Hisham Cholakkal, Ian Reid,

Organizations: Mohamed bin Zayed University of Artificial Intelligence · Carnegie Mellon University · Linköping University

Abstract

Unified multimodal models can both understand and generate visual content within a single architecture. Existing models, however, remain data-hungry and too heavy for deployment on edge devices. We present Mobile-O, a compact vision-language-diffusion model that brings unified multimodal intelligence to a mobile device. Its core module, the Mobile Conditioning Projector (MCP), fuses vision-language features with a diffusion generator using depthwise-separable convolutions and layerwise alignment. This design enables efficient cross-modal conditioning with minimal computational cost. Trained on only a few million samples and post-trained in a novel quadruplet format (generation prompt, image, question, answer), Mobile-O jointly enhances both visual understanding and generation capabilities. Despite its efficiency, Mobile-O attains competitive or superior performance compared to other unified models, achieving 74% on GenEval and outperforming Show-O and JanusFlow by 5% and 11%, while running 6x and 11x faster, respectively. For visual understanding, Mobile-O surpasses them by 15.3% and 5.1% averaged across seven benchmarks. Running in only ~3s per 512x512 image on an iPhone, Mobile-O establishes the first practical framework for real-time unified multimodal understanding and generation on edge devices. We hope Mobile-O will ease future research in real-time unified multimodal intelligence running entirely on-device with no cloud dependency. Our code, models, datasets, and mobile application are publicly available at https://amshaker.github.io/Mobile-O/

Figures & tables

Figure 2: Overview of Mobile-O . Left: The proposed framework consists of an efficient image encoder with a compact autoregressive language model for visual understanding. For image generation, a lightweight linear diffusion transformer (DiT) is employed alongside a simple yet effective VAE-based encoder–decoder. Right: Our novel Mobile Conditioning Projector (MCP) bridges the understanding and generation tasks by directly conditioning the diffusion model on weighted hidden states from the VLM without the need for intermediate query tokens. The projector leverages layer-wise feature fusion, depthwise separable convolutions, and efficient channel attention to produce high-fidelity conditioning signals with minimal cost, enabling seamless deployment on edge devices.

Figure 3: Overview of the proposed unified multimodal post-training pipeline. We jointly optimize multimodal understanding and generation through a multi-task objective using a quadruplet format ( generation prompt, image, question, answer ). Both I2T and T2I losses are computed simultaneously, enabling aligned cross-modal learning where each training sample supports both multimodal understanding and generation.

Type	Model	# Total Params	MMMU $\uparrow$	TextVQA $\uparrow$	MMVet $\uparrow$	SEED $\uparrow$	ChartQA $\uparrow$	POPE $\uparrow$	GQA $\uparrow$	Average $\uparrow$
Und. Only $>$ 1B	LLaVA-Phi [ 50 ]	$3.1$ B	-	$48.6$	$28.9$	-	-	$85.0$	-	-
	LLaVA-v $1.5$ -Phi- $1.5$ [ 50 ]	$1.6$ B	$30.7$	-	-	-	-	$84.1$	$56.5$	-
	MobileVLM [ 7 ]	$1.7$ B	-	$41.5$	-	-	-	$84.5$	$56.1$	-
	MobileVLM-V2 [ 8 ]	$1.7$ B	-	$52.1$	-	-	-	$84.3$	$59.3$	-
	LLaVa-OV [ 16 ]	$1.6$ B	$31.4$	-	$29.1$	$65.5$	$61.4$	-	-	-
Und. Only $\leq$ 1B	Smol-VLM-0.5B [ 23 ]	$0.6$ B	$33.7$	$60.2$	-	-	$62.8$	-	-	-

Table 1: Comparison with recent multimodal understanding models. “Und.” and “Gen.” denote “understanding” and “generation”, respectively. Total Params represent the sum of visual encoder, language model, and diffusion/unet components (when applicable). Compared to unified models with similar size (

\leq

2B), our Mobile-O-0.5B achieves superior overall performance with a score of 61.9 averaged over seven datasets. Further, Mobile-O-0.5B also outperforms its understanding-only counterpart (FastVLM) by 1.6% in average performance.

Type	Method	# Total Params	Single Obj.	Two Obj.	Counting	Colors	Position	Color Attri.	Overall $\uparrow$
Gen. Only $>$ 1B	LlamaGen [ 33 ]	$3.8$ B	$0.71$	$0.34$	$0.21$	$0.58$	$0.07$	$0.04$	$0.32$
	LDM [ 29 ]	$1.5$ B	$0.92$	$0.29$	$0.23$	$0.70$	$0.02$	$0.05$	$0.37$
	PixArt- $\alpha$ [ 3 ]	$4.9$ B	$0.98$	$0.50$	$0.44$	$0.80$	$0.08$	$0.07$	$0.48$
	SDXL [ 27 ]	$3.0$ B	$0.98$	$0.74$	$0.39$	$0.85$	$0.15$	$0.23$	$0.55$
	SDv $2.1$ [ 29 ]	$1.9$ B	$0.98$	$0.51$	$0.44$	$0.85$	$0.07$	$0.17$	$0.50$
	SANA-0.6B [ 42 ]	$2.6$ B	$0.99$	$0.77$	$0.62$	$0.88$	$0.21$	$0.47$	$0.66$

Table 2: Evaluation of text-to-image generation performance on the GenEval benchmark. “Und.” and “Gen.” denote “understanding” and “generation”, respectively. Total Params represent the sum of the visual encoder, language model, and diffusion/unet components (when applicable). Compared to unified models with similar size (

\leq

2B), our Mobile-O-0.5B achieves superior overall score of 0.74 and outperforms Show-o-Clip-ViT [ 43 ] by 5.0%.

Figure 4: Qualitative comparison of text-to-image generation (left) and visual understanding (right) across unified multimodal models . Each column shows Janus, JanusFlow, Show-O, and Mobile-O (ours) for the same prompts/questions. Mobile-O yields more consistent, detailed, and semantically faithful images with high fidelity and style diversity for image generation. For visual understanding, it delivers more accurate and contextually coherent responses. Additional results are presented in suppl. material. Best viewed zoomed in.

Figure 5: Qualitative image editing results of Mobile-O-0.5B. Given a source image and a textual editing instruction, Mobile-O-0.5B produces the edited output. The model is fine-tuned on only 46k editing samples from ShareGPT4V [ 5 ] .

Model	Und. Acc. (%)	Gen. Acc. (%)
FastVLM-1.5B [ 39 ]	64.8%	–
SANA-1.5B [ 42 ]	–	66%
Mobile-O-1.5B (SFT)	64.8%	75%
Mobile-O-1.5B (Post-train)	66.2%	78%

Table 3: Mobile-O-1.5B : Scaling to FastVLM-1.5B and SANA-1.5B components. Understanding performance is averaged over seven benchmarks (MMMU, TextVQA, SEED-Bench, ChartQA, POPE, GQA, MM-Vet). Generation quality is measured by GenEval overall score. The proposed post-training stage consistently improves both capabilities.

Proj.	# Layers	Fusion	Compress	CA	Acc. (%)	Params (M)
MLP	–	–	–	–	68.5	3.3
MCP	1	Uniform	✓	–	68.4	1.7
MCP	4	Uniform	✓	–	69.6	1.7
MCP	4	Learnable	✓	–	70.0	1.7
MCP	4	Learnable	✓	✓	70.4	2.4

Table 4: Ablation on the Mobile Conditioning Projector (MCP). We study the effect of layer fusion, learnable weighting, and the refinement block.

Method	Und. Acc. (%)	Gen. Acc. (%)
SFT	60.5	73.3
SFT + Post-Train (image-text pairs)	60.6	73.4
SFT + Post-Train (quadruplets)	62.1	74.2

Table 5: Effect of Unified Post-Training. Our post-training data improves both understanding and generation alignment when using joint quadruplets.

MacBook M2 Pro
Model	Vision Enc. (ms)	TTFT (ms)	Latency (s)
Janus	$783\pm 244$	$289\pm 19$	$201\pm 15614$
JanusFlow	$1909\pm 466$	$935\pm 152$	$24\pm 0.8$
Show-O	$699\pm 107$	$797\pm 5$	$47\pm 0.2$
Mobile-O-0.5B (Ours)	$\mathbf{56}\pm\mathbf{7}$	$\mathbf{187}\pm\mathbf{31}$	$\mathbf{4}\pm\mathbf{0.5}$
Jetson Orin Nano

Table 6: Image understanding and generation performance comparison on MacBook M2 Pro, Jetson Orin Nano, and iPhone for Mobile-O-0.5B . Vision Enc. and TTFT denote understanding latency, while Latency indicates image generation latency.

Proj.	# Layers	Fusion	Compress	CA	Accuracy (%)
MCP	1	-	✓	✓	68.7
MCP	2	Learnable	✓	✓	69.8
MCP	4	Learnable	✓	✓	70.4
MCP	8	Learnable	✓	✓	70.2

Table 7: Ablation on the number of layers for the Mobile Conditioning Projector (MCP). We systematically vary the number of VLM layers aggregated by MCP to condition the diffusion process. All configurations use the final design of MCP: learnable fusion, compression , and channel attention (CA).

Figure 6: Mobile-O running natively on iPhone 17 Pro. We demonstrate real-world deployment of Mobile-O ’s unified capabilities on consumer hardware. (a) Text-to-image generation: Given a detailed prompt describing a Bengal tiger. (b) Image-to-text generation: Mobile-O provides detailed visual descriptions, analyzing composition and subject positioning

	Stage 1: Cross-Modal Alignment	Stage 2: Supervised Fine-tuning	Stage 3: Unified Post-Training
Data Source	JourneyDB (4M) +	BLIP3o-60K +	BLIP3o-60K +
	BLIP3o-Short (5M)	ShareGPT-4o (45K)	ShareGPT-4o (45K)
Total Samples	9M	105K	105K
Format	Prompt-image pairs	Prompt-image pairs	Quadruplet $\{p,\mathbf{x}_{\text{img}},q,a\}$
Learning Rate	2e-4	2e-4	1e-4
Batch Size	512	384	128

Table 8: Three-stage training setup for Mobile-O. Stage 1 establishes cross-modal alignment using large-scale image-text pairs. Stage 2 performs targeted fine-tuning to address weaknesses in complex gestures, common objects, and landmarks. Stage 3 introduces unified multimodal post-training with quadruplet samples

\{p,\mathbf{x}_{\text{img}},q,a\}

for joint understanding and generation. All experiments were conducted on 8

\times

A100 GPUs.

Figure 7: Qualitative comparison on dense text understanding and information extraction. We evaluate Mobile-O against other models on a challenging OCR and comprehension task requiring the model to read, parse, and summarize the back cover text of a book. Green text indicates correctly extracted information, while red indicates hallucinations or errors. Mobile-O demonstrates superior performance in accurately extracting key bibliographic details, including the correct title, author, editors, and price information from the densely-packed text on the book cover.

Figure 8: Qualitative comparison with SANA-0.6B on text-to-image generation. We compare Mobile-O (1.6B total parameters) against SANA-0.6B (2.6B total parameters), our generation baseline, on challenging prompts requiring photorealistic rendering, complex lighting, and fine-grained details. Mobile-O demonstrates competitive or superior visual quality across diverse scenarios, including wildlife photography, landscape composition, and portrait rendering. Best viewed zoomed in.

Figure 9: Qualitative comparison of image-to-text across unified models below 2B. Mobile-O is compared against Janus [ 41 ] , JanusFlow [ 22 ] , and Show-O [ 43 ] on diverse visual question answering tasks, including scientific reasoning, OCR, object recognition, and cultural knowledge. Green indicates correct answers, red indicates errors. Mobile-O demonstrates competitive visual understanding, despite its mobile-optimized architecture, correctly answering complex questions that require fine-grained visual analysis and domain knowledge.

Figure 10: Qualitative comparison of text-to-image generation across unified models below 2B. Mobile-O is compared against Janus [ 41 ] , JanusFlow [ 22 ] , and Show-O [ 43 ] on challenging prompts spanning fantasy, photorealism, and scientific visualization. Despite its mobile-optimized architecture, Mobile-O maintains competitive visual quality and prompt adherence. Best viewed zoomed in.

Figure 11: Additional Text-to-Image generation examples of Mobile-O . Best viewed zoomed in.

Detailed Prompts for Image Generation in Fig. 11 — $(i,j)$ denotes the image at row $i$ , column $j$
Row 1	Portrait Photography
[2pt/2pt] (1,1)	Young woman with freckles, green eyes with detailed iris and catchlight, natural skin texture, flowing red hair catching golden hour sunlight, warm vibrant lighting, gentle expression, shallow depth of field
(1,2)	Elderly Asian man with silver hair and peaceful expression, kind brown eyes, colorful traditional clothing, bright natural window lighting, detailed skin texture
(1,3)	Young man with curly hair and beard, bright turquoise shirt, outdoor lighting
(1,4)	Child with bright blue eyes and blonde hair, curious expression, rosy cheeks, individual eyelashes visible, natural freckles, bright daylight from side creating dimension, colorful playground in blurred background
Row 2	Flowers & Garden Scenes

Table 9: Text-to-image generation prompts used for visualization.

Unified Multimodal Models

Unified multimodal models combine multiple data types in shared systems for cross-modal understanding, generation, reasoning, and interaction.

See all 290 papers

Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation
Jul 28, 2026Weiming Zhuang, Jiabo Huang, Jingtao Li +4Unified Multimodal Models Visual Tokenization
Sony AI
Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
May 7, 2026Zeyu Liu, Zanlin Ni, Yang Yue +5Unified Multimodal Models Image Generation
1Tsinghua University · 2Kolors Team, Kuaishou Technology
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Unified Multimodal Models Multimodal Generation
1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1

Jul 28, 2026J/K move · Enter open · S save

Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

Weiming Zhuang, Jiabo Huang, Jingtao Li, Zhizhong Li+3

Sony AI

Unified Multimodal Models Visual Tokenization Multimodal Generation Multimodal Understanding

Open paper AskPDF arXiv

Unifying visual understanding and generation in one model holds immense promise, but remains challenging and expensive due to heavy compute and data demands and conflicts between the visual features needed for these two capabilities. To address these challenges, we present Argus-Unified, a compact, effective and unified multimodal model built with low demand on computation and data. Instead of aligning modalities from scratch, Argus-Unified effectively leverages pretrained vision-language models (VLMs) that provide strong multimodal priors. Specifically, we introduce hybrid visual tokens that preserve continuous tokens for understanding while learning discrete tokens for generation from a frozen unified vision encoder. Our training pipeline includes two stages: the first stage learns a quantizer and image decoder on top of the frozen vision encoder, the second stage trains the LLM initialized from a pretrained VLM for the unified multimodal modeling. Using by far the least amount of data (15.6M) and the lowest cost (~$2,000), we demonstrate that unified multimodal models can be trained economically while achieving strong performance in both understanding and generation. Notably, our model attains state-of-the-art multimodal understanding on GQA, POPE, and VQAv2, and competitive generation quality compared to models with dedicated vision encoders (e.g., Janus, Janus-Pro), all at ~10x lower cost and with ~5x less data. We envision Argus-Unified as a useful baseline that lowers the development barrier for unified models.

Abstract

Unified Multimodal Models

Explore similar work

Argus-Unified: Towards A Compact and Economical Unified Model for Image Understanding and Generation

Steering Visual Generation in Unified Multimodal Models with Understanding Supervision

Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards