On-device inference is booming, but the momentum is almost all in language models. Diffusion pipelines are memory hungry, latency-sensitive, and require orchestrating an embedder, a transformer, a decoder, and often further postprocessing that is not as standardized as LLM inference loops are. We navigate the trade-off between performance, quality, and model footprint to reach as many client devices in the wild as possible. We make three contributions: an embedding translator that maps a small text encoder into a large encoder space to cut weight and latency; a reproducible sweep recipe for navigating the speed/quality/memory triangle in diffusion pipelines; and an interactive on-device image generation editor achieving sub-second TTFI on recent GPUs.
Figures & tables
Figure 1. FLUX.2-klein with its 4B text encoder replaced by Qwen3-0.6B plus a 187M-parameter translator (trained post-hoc; transformer and VAE frozen). Prompts: a horned owl at golden hour; an erupting volcano at night; Jupiter; a sumi-e ink-wash raccoon. Four generated images: an owl, an erupting volcano, the planet Jupiter, and an ink-wash anthropomorphic raccoon.
Figure 2. Same prompt and seed across encoders (columns): the 4B reference, and Qwen3-0.6B with a 187M, 29M, and 3.5M translator. The 187M translator tracks the 4B; the 29M drifts to plausible but wrong content; and the 3.5M collapses toward a single dominant style regardless of the prompt. Rows: a red fox in a snowy forest; a bowl of ramen; a lighthouse on a cliff at sunset. All images are 1024×1024 (1K).
Translator
Params
Mem
Encode
LPIPS ↓
CLIP ↑
4B (reference)
4000 M
8.0 GB
435 ms
0.553 †
0.328
d3072 (MLP)
187 M
1.77 GB
107 ms
0.514
0.306
d2048 × 8
158 M
1.72 GB
105 ms
0.550
0.299
d2048 × 7
98 M
1.60 GB
99 ms
0.564
0.295
d2048 wide
91 M
1.58 GB
98 ms
0.580
0.298
d2048
91 M
1.58 GB
98 ms
0.578
0.295
Table 1. Translator ablation on FLUX.2-klein (Qwen3-0.6B + translator vs. the 4B encoder). Mem = encoder-path bf16 weights (shared 0.6B base ≈1.4 GB + translator, params × 2 B); the 4B row is the 4B encoder alone. Encode = text-encode latency (0.6B + translator), PyTorch/MPS, bf16. LPIPS/CLIPScore: end-to-end on 24 PartiPrompts vs. 4B at matched seed. † seed-variance floor (4B at two seeds). Attention rows timed on CPU (MPS SDPA fallback).
Prec.
Step
Weights
Act.
CLIP ↑
LPIPS ↓
BF16
105 ms
7,9 GiB
510 MiB
ref.
FP16
108 ms
7,9 GiB
510 MiB
0.99±0.01
0.02±0.02
FP8
73 ms
4,3 GiB
462 MiB
0.96±0.05
0.10±0.07
NVFP4
57 ms
2,8 GiB
475 MiB
0.92±0.08
0.23±0.09
Table 2. Transformer precision on RTX PRO 6000 Blackwell, 1024×1024 , single step, 4B encoder. Weights/Act. = weight/peak-activation memory. Quality vs. BF16 on 24 PartiPrompts: CLIP image similarity and LPIPS ( Zhang et al., 2018 ) (mean ± std).
Figure 3. Cross-device sweep (4 steps, 10242 ): peak VRAM vs. per-step latency. Label = resident-weight budget (100% = full); line = one (encoder, precision) trajectory; filled = 0.6B+translator, open = 4B. RTX 4070 Ti (12 GB): above the ≈11.2 GB budget the denoiser pages, latency jumps ∼40× (shaded). RTX PRO 6000 (96 GB): all resident—full-resident is fastest and precision drives the front. BF16 exact, FP16 ≥0.99 vs BF16.
Encoder
Prec.
Encode
Step
Decode
TTFI
Total
VRAM
4B
BF16
12 ms
105 ms
41 ms
158 ms
0.47 s
15.1 GB
0.6B+tr
BF16
4 ms
105 ms
40 ms
149 ms
0.47 s
10.6 GB
0.6B+tr
FP8
4 ms
73 ms
41 ms
119 ms
0.34 s
7.2 GB
0.6B+tr
NVFP4
4 ms
57 ms
40 ms
102 ms
0.27 s
5.7 GB
Table 3. On-device timings, NVIDIA RTX PRO 6000 Blackwell Workstation Edition, TensorRT-RTX, 1024×1024 , 4 steps. TTFI = encode + one denoising step + VAE decode. VRAM = peak device memory across the full pipeline.
The remarkable generation quality of modern diffusion models often comes at the cost of massive parameter counts, which necessitate server-side inference with significant computational costs and potential privacy risks. Consequently, there is growing momentum toward developing efficient on-device alternatives. While recent efforts have optimized text-to-image models for mobile hardware, they remain relatively bulky, typically ranging from 0.5B to 1B parameters. We present BlazeEdit, a highly efficient, generalist image-to-image diffusion model tailored for on-device deployment. By identifying that many practical image editing tasks do not require text-based guidance, we eliminate the text-conditioning components and develop a multi-task architecture that consolidates object removal, outpainting, tone correction, relighting, and sticker generation into a single, compact model of only 195M parameters. BlazeEdit achieves a substantial reduction in download size and memory overhead while maintaining competitive generation quality. It completes a full inference pass in just 290ms on a Pixel 10, delivering a seamless, privacy-preserving, and lightning-fast experience for generalist image editing on the edge.
Diffusion models have made significant progress in both text-to-image (T2I) generation and text-guided image editing. However, these models are typically built with billions of parameters, leading to high latency and increased deployment challenges. While on-device diffusion models improve efficiency, they largely focus on T2I generation and lack support for image editing. In this paper, we propose DreamLite, a compact unified on-device diffusion model (0.39B) that supports both T2I generation and text-guided image editing within a single network. DreamLite is built on a pruned mobile U-Net backbone and unifies conditioning through in-context spatial concatenation in the latent space. It concatenates images horizontally as input, using a (target | blank) configuration for generation tasks and (target | source) for editing tasks. To stabilize the training of this compact model, we introduce a task-progressive joint pretraining strategy that sequentially targets T2I, editing, and joint tasks. After high-quality SFT and reinforcement learning, DreamLite achieves GenEval (0.72) for image generation and ImgEdit (4.11) for image editing, outperforming existing on-device models and remaining competitive with several server-side models. By employing step distillation, we further reduce denoising processing to just 4 steps, enabling our DreamLite could generate or edit a 1024 x 1024 image in less than 1s on a Xiaomi 14 smartphone. To the best of our knowledge, DreamLite is the first unified on-device diffusion model that supports both image generation and image editing.
Diffusion Transformers (DiT) have established a new state-of-the-art in high-fidelity image synthesis; however, their massive computational complexity and memory requirements hinder local deployment on resource-constrained edge devices. In this paper, we introduce EdgeDiT, a family of hardware-efficient generative transformers specifically engineered for mobile Neural Processing Units (NPUs), such as the Qualcomm Hexagon and Apple Neural Engine (ANE). By leveraging a hardware-aware optimization framework, we systematically identify and prune structural redundancies within the DiT backbone that are particularly taxing for mobile data-flows. Our approach yields a series of lightweight models that achieve a 20-30% reduction in parameters, a 36-46% decrease in FLOPs, and a 1.65-fold reduction in on-device latency without sacrificing the scaling advantages or the expressive capacity of the original transformer architecture. Extensive benchmarking demonstrates that EdgeDiT offers a superior Pareto-optimal trade-off between Frechet Inception Distance (FID) and inference latency compared to both optimized mobile U-Nets and vanilla DiT variants. By enabling responsive, private, and offline generative AI directly on-device, EdgeDiT provides a scalable blueprint for transitioning large-scale foundation models from high-end GPUs to the palm of the user.