Asuka-FM: A System-Architectural Co-Design Framework for Efficient Text-to-Image Pretraining on Consumer Hardware
Authors: Víctor Rodrigo González, Pablo M. Olmos, Ignacio Peis, Alejandro Lancho
Organizations: Department of Signal Theory and Communications, Universidad Carlos III de Madrid, Spain · Department of Applied Mathematics and Computer Science, Technical University of Denmark (DTU), Denmark · Math Institute, Universidad de Granada, Spain
Frontier text-to-image (T2I) foundation models predominantly rely on multi-billion parameter Diffusion Transformers (DiTs) pretrained across massive compute clusters, creating an accessibility barrier for academic laboratories. When training DiTs from scratch on single consumer GPUs (e.g., an NVIDIA RTX 4090 with 24,GB VRAM), practitioners encounter three primary bottlenecks: quadratic attention sequence costs, Just-In-Time (JIT) compilation graph breaks induced by variable aspect-ratio bucketing, and an autoencoder throughput wall where frozen Variational Autoencoder (VAE) operations dominate step latency. In this work, we introduce Asuka-FM, an end-to-end system and architectural framework demonstrating that competitive dual-stream DiTs (∼206M parameters) can be pretrained from scratch on a single consumer GPU in ∼160 GPU hours for under $55. Asuka-FM integrates four core designs: (i) an AdaLN-free dual-stream DiT backbone reducing parameter count; (ii) structured sequence sparsification via Sparse-Dense Residual Fusion (SPRINT) to cut attention FLOPs by over 50%; (iii) continuous aspect-normalized Rotary Position Embeddings (RoPE) coupled with shifted square crop training, guaranteeing static tensor shapes while unlocking zero-shot virtual camera control; and (iv) a pixel adaptation phase utilizing a lightweight (10M parameter) Patch Detailer Head operating under velocity regression (v-prediction). Evaluated on open-domain benchmarks without benchmark-specific fine-tuning, Asuka-FM achieves GenEval 0.588 / DPG 78.96 scores at 256px and GenEval 0.609 / DPG 78.51 scores at 512px, setting a new Pareto-optimal efficiency standard for democratic foundation modeling.
Figures & tables
Figure 2: High-fidelity zero-shot visual synthesis from scratch on a single consumer GPU. Curated open-domain samples synthesized by Asuka-FM (206M backbone) following two-stage pretraining ( 2562→5122 ) on a single NVIDIA RTX 4090 GPU in 149.0 hours ($49.17).
Figure 3: Comprehensive multi-stage training roadmap of Asuka-FM. Pretraining advances through two latent resolution tiers ( 2562→5122 ) coupling 75% SPRINT sequence sparsification with full-token dense refinement and Warmup-Stable-Decay (WSD). Stage 3 transitions the converged 512px foundation prior to pixel space using a 10M convolutional Patch Detailer Head (PDH) trained under native v -prediction across synthetic and real datasets, permanently removing the VAE decode barrier during few-step inference.
Architectural Hyperparameter
Specification / Value
Latent Channel Dimension ( C )
32 (FLUX.2-dev VAE, f=8 )
Latent Patch Dimension ( p )
2×2 (Effective P=16 )
Hidden Dimension ( D )
768
Number of Attention Heads ( H )
12
Attention Head Dimension ( dh )
64
Feedforward Network (FFN)
SwiGLU ( Dffn=2048 )
Table 1: Asuka-FM architecture specifications. 206M-parameter Dual-Stream SPRINT DiT backbone.
Figure 4: Asuka-FM system-architectural co-design pipeline. Top Row (Stage 1): Raw images Iorig undergo shifted square cropping to enforce static tensor dimensions ( X×X ) and eliminate JIT recompilations, while continuous aspect-normalized RoPE prevents phase collapse. Precomputed FLUX.2 VAE latents zt receive prepended in-context temporal conditioning ( ztime ) alongside text representations from a 2-layer adapter without per-block AdaLN parameters. Bottom Row (Stage 2): The 13-layer dual-stream DiT executes SPRINT sequence sparsification: shallow blocks ( fθ ) supply high-frequency dense skips ft , deep middle blocks ( gθ ) process macro-semantics over a 75%-sparse grid, and dense decoder blocks ( hθ ) fuse representations to predict flow trajectories. Finally, the lightweight 10.09M Patch Detailer Head (PDH) transfers representations directly to RGB pixels under native v -prediction.
Figure 5: Continuous RoPE coordinate geometry and frequency calibration analysis. Top Row: (A) Adjacent patch phase step Δθk across channels k∈[0,7] : uncalibrated normalized coordinates collapse phase shifts to <7.16∘ ( K→1.0 ), while calibrated frequencies ( ω∈[π/4,2π] ) restore angular separation ( 22.5∘–45∘ ). (B) Center-row coordinate trajectories comparing discrete integer indices [0,15] against continuous coordinates [−rW,rW] . Bottom Row: Spatial attention kernel heatmaps K(pi,pj) centered at probe patch (8,8) (cyan star). (C) Discrete integer baseline exhibits natural radial decay. (D) Uncalibrated continuous RoPE undergoes phase collapse ( K≈1.0 ). (E) Calibrated continuous RoPE restores sharp isotropic radial decay across arbitrary canvas dimensions without learned parameters.
Table 2: Pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). All stages use continuous aspect-normalized RoPE, bfloat16, and torch.compile . Pretraining uses an effective batch size of 256. Unabridged configurations and token processing rates are detailed in Table 8 (Appendix A ). Evaluated at $0.33/GPU-hour.
Model
Params
Hardware Budget
Training Data
GenEval Total ↑
DPG-Bench Total ↑
Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute)
SDXL Base [ 35 ]
2.6B
Hundreds of A100s
Millions (Web)
0.406 ∗
72.40 ∗
SD3 Medium [ 16 ]
2.0B
Massive Cluster
Millions (Web)
0.620 ∗
84.08 ∗
FLUX.1 [Dev] [ 3 ]
12.0B
Massive Cluster
Web-Scale
0.660 ∗
83.84 ∗
Janus-Pro-7B [ 9 ]
7.0B
Large Cluster
Multimodal Web
0.800 ∗
84.19 ∗
i1 (Open Foundation) [ 49 ]
3.0B
TPU v5p-128 Pod
Curated Open
0.840
86.73
Table 3: Quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench). Results marked with ( ∗ ) denote numbers reported in prior literature [ 40 , 49 , 12 , 43 ] . Models marked with † indicate fine-tuning on the BLIP3o-60K dataset, which specifically targets GenEval metrics. Asuka-FM is trained from scratch on open data without benchmark-specific fine-tuning. The full GenEval breakdown is provided in Table 9 in Appendix A .
Model
Params
Hardware
Wall Time
GPU-h
Cost ($)
FLUX.1 [Dev]
12.0B
H100 SXM5 Pod
Weeks
>100k
>\175\text{k}$
SD3 Medium
2.0B
H100 SXM5 Pod
Weeks
>50k
>\70\text{k}$
i1 .
3.0B
TPU v5p-128
29.5 Days
90.6k
\approx\380\text{k}$
PixelGen-XXL
1.1B
8 × H800
6.0 Days
1,152
\approx\2,880$
MiniT2I-B/32
0.26B
8 × H100
3.0 Days
576
\approx\2,016$
Looped-DiT
0.26B
8 × H100
3.0 Days
576
\approx\2,016$
Table 4: Pretraining hardware and financial cost comparison across foundation T2I models. Cloud rates: 0.33/hr(RTX4090),1.50/hr (H800), 2.89/hr(H100),537.60/hr (TPU v5p-128 pod).
Table 5: Micro-architectural profiling across hardware architectures at 2562 and 5122 . Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference, micro-batch 64 with 16,384 tokens/step at 2562 and 65,536 tokens/step at 5122 ). Canvas dimensions and token lengths are reported in Table 10 (Appendix A ).
Stage / Phase
Res. ( X2 )
Drop ( r )
Samples
Throughput (img/s)
MFU (%) Theo / Act †
Compute (EFLOPs) ∗
Wall Time
Cost ($)
Phase 0: Offline Synth Generation (A40) ‡
5122
–
0.10M
1.85 img/s
–
–
15.0 h
$6.00
Stage 1: Decoupled PDH Warmup (Synth)
5122
0.75
0.50M
61.8 img/s
62.4% / 72.8%
0.52
2.25 h
$0.74
Stage 2A: Joint SPRINT Sparse (Synth)
5122
0.75
2.50M
57.7 img/s
58.1% / 67.8%
2.60
12.0 h
$3.98
Stage 2B: Full-Token Dense Refine (Synth)
5122
0.00
1.50M
44.4 img/s
59.8% / 69.8%
2.71
9.4 h
$3.10
Stage 3: Real-Data SFT Refinement (Real)
5122
0.00
5.16M
42.1 img/s
56.7% / 66.2%
9.32
34.0 h
$11.22
Cumulative Pixel DiT Adaptation Total (Stages 1–3)
9.66M
46.5 img/s
57.8% / 67.5%
15.15
57.65 h
$19.04
Table 6: Latent-to-Pixel adaptation progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Adapts the converged 512px latent foundation model to direct RGB pixel space using a 10.09M convolutional Patch Detailer Head (PDH) under native v -prediction at effective batch size 128. Hardware costs evaluated at 0.33/GPU−hourforRTX4090and0.40/GPU-hour for A40 (Phase 0). Unabridged telemetry and hyperparameter matrices are detailed in Table 13 in Appendix C .
Configuration / Stage
Patch Size ( P )
Target Objective
Training Schedule
GenEval Total ↑
DPG-Bench Total ↑
Sampling Latency / Img (20-step / 30-step)
Baseline Latent Foundation Model (Pretraining Anchor)
Asuka-512 (Base Latent)
P=16
v -pred
139 Epochs (81.0h)
0.609
78.51
0.250 s / 0.374 s
Latent-to-Pixel Adaptation Progression (DiP Architecture, Zero VAE Decode)
Stage 2A: Joint SPRINT Sparse ( r=0.75 )
P=16
v -pred
25 Epochs (12.0h)
0.465
69.63
0.329 s / 0.493 s
Stage 2B: Full-Token Refine (Synth, v -pred)
P=16
v -pred
15 Epochs (9.4h)
0.505
72.35
0.329 s / 0.493 s
Stage 2B: Full-Token Refine (Synth, x0 -pred)
P=16
x0 -pred
15 Epochs (9.4h)
0.477
71.49
0.329 s / 0.493 s
Table 7: Quantitative benchmark summary of Latent-to-Pixel adaptation on GenEval and DPG-Bench. Evaluated at 5122 canvas resolution using a 100-step Euler ODE solver with CFG scale wcfg=6.0 . Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Full task-by-task GenEval breakdowns are provided in Table 14 in Appendix C .
Figure 6: Sampling latency trade-off at 1MP: Latent DiT vs. DiP Pixel Engine on NVIDIA A40. Pixel sampling is faster in few-step regimes ( N≤8 , 3.34× speedup at 1 step), while latent sampling is faster for multi-step ancestral sampling ( N≥13 ).
Table 8: Comprehensive pretraining progression and compute metrics of Asuka-FM on a single consumer NVIDIA RTX 4090 GPU (24GB). Unabridged reference for Table 2 . Sequence lengths account for visual image tokens plus context ( 227 text+1 time=228 tokens ). MFUact evaluates actual tensor operations on silicon under selective SwiGLU activation checkpointing ( 7PD work factor, where P denotes active model parameters and D denotes tokens per sample); MFUtheo denotes standard theoretical algorithmic utilization ( 6PD factor). Hardware costs evaluated at $0.33/GPU-hour.
Model
Params
Hardware Budget
Training Data
GenEval Task Breakdown
GenEval Overall ↑
DPG-Bench Total Score ↑
Single
Two-Obj
Count
Colors
Position
Color- Attr
Industrial Foundation Models (Multi-Billion Parameters, Datacenter Compute)
SDXL Base [ 35 ]
2.6B
Hundreds of A100s
Millions (Web)
–
–
–
–
–
–
0.406 ∗
72.40 ∗
SD3 Medium [ 16 ]
2.0B
Massive Cluster
Millions (Web)
0.98
0.84
0.66
0.74
0.40
0.43
0.620 ∗
84.08 ∗
FLUX.1 [Dev] [ 3 ]
12.0B
Massive Cluster
Web-Scale
0.99
0.81
0.79
0.74
0.20
0.47
0.660 ∗
83.84 ∗
Janus-Pro-7B [ 9 ]
7.0B
Large Cluster
Multimodal Web
–
–
–
–
–
–
0.800 ∗
84.19 ∗
Appendix
Table 9: Comprehensive quantitative comparison on representative text-to-image benchmarks (GenEval and DPG-Bench) with full task breakdown. Unabridged reference for Table 3 . Results marked with ( ∗ ) denote numbers reported in prior literature. Models marked with † indicate fine-tuning on the BLIP3o-60K instruction dataset. Asuka-FM is trained from scratch on open-source data without benchmark-specific fine-tuning.
Figure 7: Per-step execution latency breakdown across resolutions on consumer hardware. While In-RAM latent caching eliminates online VAE overhead at 2562 (devoting >80% of step latency to the DiT backbone), on-the-fly VAE encoding at 10242 (1MP) consumes 72.6% of wall-clock time ( 2,548.1ms out of 3,507.6ms ). Decoupled latent-to-pixel adaptation permanently bypasses this throughput wall.
Table 10: Comprehensive micro-architectural profiling across hardware architectures at 2562 and 5122 . Unabridged reference for Table 5 . Profiled across 30 active iterations via PyTorch FlopCounterMode (165.2 TFLOPS peak reference). Reports explicit spatial dimensions, per-step token volumes, and isolated backbone (‘Iso.‘) vs. end-to-end system (‘Sys‘) MFUact .
Resolution
Mode
Steps
Total Time
Latency / Image
Step Cost ( tstep )
Fixed Overhead ( tfixed )
Peak VRAM
Rate
1024×1024
Latent + VAE
30
46.43 s
2.902 s / img
1.387 s / step
4.82 s (VAE Decode)
6.75 GB
0.34 img/s
1024×1024
Latent + VAE
20
32.56 s
2.035 s / img
1.387 s / step
4.82 s (VAE Decode)
6.75 GB
0.49 img/s
1024×1024
Pixel (DiP)
30
53.28 s
3.330 s / img
1.773 s / step
0.09 s (Zero VAE)
13.22 GB
0.30 img/s
1024×1024
Pixel (DiP)
20
35.55 s
2.222 s / img
1.773 s / step
0.09 s (Zero VAE)
13.22 GB
0.45 img/s
512×512
Latent + VAE
30
5.99 s
0.374 s / img
0.181 s / step
0.56 s (VAE Decode)
2.78 GB
2.67 img/s
512×512
Pixel (DiP)
30
7.89 s
0.493 s / img
0.260 s / step
0.09 s (Zero VAE)
4.40 GB
2.03 img/s
Appendix
Table 11: Sampling execution profile across latent and pixel-space adaptation on NVIDIA A40 ( B=16 samples, TorchInductor compilation, FlashAttention-2, bfloat16). Latent mode evaluates 206M DiT followed by a terminal VAE decode. Pixel mode evaluates 216M DiT + PDH without an autoencoder.
Figure 8: Quantitative convergence on the toy-diffusion benchmark. Evaluation of SPRINT token drop schedules ( r=0.75→0.00 ) on FID.
Backbone / Architecture Variant
Params
BS
Epochs
FID ↓
Part I: Latent Architecture & Conditioning Ablations
Single-Stream DiT ( D=768,L=14 )
218M
256
600
3.89
Efficient U-Net Baseline
185M
256
400
3.45
Dual-Stream DiT + MLP Text Adapter
200M
256
600
3.42
Dual-Stream DiT + LLM (Frozen LFM2.5)
206M
512
1000
3.12
Dual-Stream Baseline (Constant LR)
206M
512
600
3.06
Appendix
Table 12: Architectural and conditioning ablations on the toy-diffusion framework (evaluated on 87,000 facial portraits at 256×256 , patch size P=16 ).
Stage / Phase
Modules Trained
Patch Size ( P )
Drop ( r )
Micro-BS (Per GPU)
Peak LR ( η )
Warmup / Schedule
Epochs
Samples
Throughput (img/s)
MFU (%) Theo / Act
Wall Time
Cost ($)
Phase 0: Synth Gen (A40)
– (Sampling)
–
–
32
–
–
–
0.10M
1.85
–
15.0 h
$6.00
Stage 1: PDH Warmup
Embed + PDH
16
0.75
16
1.0×10−4
25% Const
5
0.50M
61.8
62.4% / 72.8%
2.25 h
$0.74
Stage 2A: Joint SPRINT
Full Model
16
0.75
16
6.0×10−5
10% Const
25
2.50M
57.7
58.1% / 67.8%
12.0 h
$3.98
Stage 2B: Dense Refine
Full Model
16
0.00
8
6.0×10−5
10% WSD
15
1.50M
44.4
59.8% / 69.8%
9.4 h
$3.10
Stage 3: Real Data SFT
Full Model
16
0.00
8
6.0×10−5
10% WSD
13
5.16M
42.1
56.7% / 66.2%
34.0 h
$11.22
Cumulative Pixel DiT Adaptation Total (Stages 1–3)
58
9.66M
46.5
57.8% / 67.5%
57.65 h
$19.04
Appendix
Table 13: Complete pretraining and adaptation progression for Latent-to-Pixel transfer at 512×512 on NVIDIA RTX 4090 (24GB). Hardware evaluated at 0.33/GPU−hour(0.40/h for A40 in Phase 0). Effective batch size is maintained at 128 across all stages using gradient accumulation.
Configuration / Stage
Patch Size ( P )
Target Obj.
Steps / Epochs
GenEval Task Breakdown
GenEval Overall ↑
DPG-Bench Total ↑
Latency / Img (20-step / 30-step)
Single
Two-Obj
Count
Colors
Position
Color- Attr
Baseline Latent Foundation Model (Pretraining Anchor)
Asuka-512 (Base Latent)
P=16
v -pred
139 Ep
0.950
0.811
0.347
0.817
0.278
0.450
0.609
78.51
0.250 s / 0.374 s
Stage 2: Synthetic Data SPRINT Adaptation Progression (100k Synthetic Images)
Stage 2A: SPRINT Sparse ( r=0.75 )
P=16
v -pred
25 Ep (12.0h)
0.928
0.543
0.259
0.662
0.183
0.215
0.465
69.63
0.329 s / 0.493 s
Stage 2B: Dense Refine ( r=0.00 )
P=16
x0 -pred
15 Ep (9.4h)
0.928
0.586
0.291
0.628
0.195
0.233
0.477
71.49
0.329 s / 0.493 s
Appendix
Table 14: Comprehensive benchmark evaluation of latent-to-pixel adaptation configurations on GenEval and DPG-Bench across training stages. All models evaluate at 5122 canvas resolution using a 100-step Euler ODE solver with CFG scale wcfg=6.0 . Sampling throughput and per-image latency profiled on a single NVIDIA A40 GPU at batch size 32. Intermediate stages demonstrate that v -prediction achieves faster initial convergence and prevents color desaturation.
Figure 9: Qualitative and quantitative comparison of x0 -prediction vs. native v -prediction in 5122 Latent-to-Pixel adaptation under identical random seeds. Native v -prediction eliminates dynamic clipping artifacts and color blowout.
Figure 10: Horizontal ( X -axis) viewport panning under identical noise seeds. Modulating the horizontal offset xshift of the continuous coordinate mesh translates the virtual viewport across the manifold, shifting the dachshund horizontally relative to the centered baseline ( xshift=0.00 ) while preserving knit cap details and snow texture geometry.
Figure 11: Vertical ( Y -axis) viewport framing under identical noise seeds. Adjusting the vertical offset yshift alters the virtual camera elevation: yshift=−0.25 elevates the framing to resolve upper floral tiers and the ballroom chandelier, whereas yshift=+0.25 lowers the viewpoint to ground the ornate metallic pedestal stand.
Figure 12: Continuous focal zoom scaling via RoPE coordinate grid compression. Rescaling the continuous evaluation mesh by optical magnification factor z∈{0.75×,1.00×,1.25×} simulates continuous wide-angle expansion and focal zoom under identical latent noise initializations. At z=1.25× , transformer attention capacity concentrates on local topological features, resolving fine rose petal contours and porcelain teacup glazes while strictly maintaining global structural alignment.
Figure 13: Zero-shot continuous virtual camera manipulation on anime illustrations under identical random seeds. Manipulating the continuous coordinate mesh allows continuous horizontal panning (Row 1), vertical elevation adjustment (Row 2), and optical zoom (Row 3) purely via position map manipulation, without retraining, camera adapters, or prompt modifications.
Figure 14: Failure mode comparison under continuous RoPE zoom on a complex organic composition. (a) At canonical scale ( z=1.0× ), the 206M dual-stream backbone synthesizes a coherent scene with sharp lighting, structured perspective, and clean floral silhouettes. (b) Applying continuous RoPE focal zoom ( z>1.0× ) compresses the evaluation grid, inducing: (i) painterly high-frequency texture smearing on the stone pathway, (ii) semantic attribute bleeding (pink pigment leaking into ground cobblestones), and (iii) spatial non-rigidity and warping along linear wooden bench slats.
Figure 15: Ablation of RoPE positional formulations across non-square aspect ratios under identical random seeds. Switching to continuous aspect-normalized coordinates with frequency spectrum ω∈[π/4,2π] completely resolves anatomical tiling without requiring pretraining restarts.
Phase
Epochs
Drop
RoPE Mode
Samples
Patches
PFLOPs exec
Time
1A: Base Sparse
0–100
0.75
Discrete
33.70M
8.63B
7.2
28.0 h
1B: RoPE Adaptation
100–120
0.75
Continuous
6.74M
1.73B
1.4
5.6 h
1C: Dense FT
120–145
0.00
Continuous
8.43M
2.16B
4.2
10.0 h
1D: Cosine Decay
145–170
0.00
Continuous
8.43M
2.16B
4.2
10.0 h
Total Stage 1 (Swapped Run)
57.30M
14.68B
17.0
53.6 h
Appendix
Table 15: Stage 1 RoPE swap progression on the 337K Danbooru corpus: Successful coordinate adaptation from discrete integer grid to continuous aspect-normalized RoPE within 20 epochs ( ≈ 6.74M samples) on a single RTX 4090.
Figure 16: Strict semantic color disentanglement under identical latent initializations. Modifying exclusively the hair color prompt token updates the pigment channels without perturbing facial geometry, eye catchlights, clothing ruffles, hand gestures, or background sparkle lighting.
Figure 17: Visual impact of SPRINT sequence sparsification during training under identical noise seeds. Sparse pretraining establishes robust macro-semantic layout at >50% compute savings, after which dense refinement recovers razor-sharp textural boundaries.
Figure 18: Resolution scaling dynamics at native 1MP ( 10242 , N=4096 ) across sparse and dense pretraining phases.
Table 16: Pretraining progression and compute metrics of Asuka-FM on the curated anime dataset ( Ntotal=337,000 ) on a single consumer NVIDIA RTX 4090 GPU (24GB). Formatted symmetrically with Table 2 . Pretraining executed using an effective batch size of 256 for Stages 1 and 2 (micro-batch 64 with 4 gradient accumulation steps), and effective batch size 128 for Stage 3 (micro-batch 16 with 8 accumulation steps). Throughput, compute, and MFU are derived directly from per-stage epochs and wall-clock hours using the medium context tier ( 152 text+1 time=153 tokens). Hardware costs evaluated at $0.33/GPU-hour.
Pipeline Component
Micro-BS 16 (Accum 8)
Micro-BS 32 (Accum 4)
Latency (ms)
Share (%)
Latency (ms)
Share (%)
Data Loading / IO (CPU)
0.16 ms
0.0%
0.18 ms
0.0%
Online Frozen VAE Encoding
2,548.10 ms
72.6%
4,906.91 ms
72.5%
Text Encoder Forward Pass
31.84 ms
0.9%
63.23 ms
0.9%
Backbone Forward Pass (Sparse)
229.31 ms
6.5%
457.20 ms
6.8%
Backbone Backward Pass (Sparse)
659.65 ms
18.8%
1,299.48 ms
19.2%
Appendix
Table 17: Execution profiling breakdown of Asuka-FM at native 1024×1024 resolution on a single NVIDIA A40 GPU across variable micro-batch sizes (FLUX.2 VAE, N=4096 tokens, TorchInductor compilation, bfloat16 precision).
Figure 19: High-quality zero-shot samples from the anime-specialized Asuka-FM model. Our 206M dual-stream backbone demonstrates sharp lineart, intricate anatomical rendering, and character-concept binding pretrained from scratch on a single consumer RTX 4090 despite a compact open dataset.
Diffusion Transformers (DiTs) are a dominant backbone for high-fidelity text-to-image generation due to strong scalability and alignment at high resolutions. However, quadratic self-attention over dense spatial tokens leads to high inference latency and limits deployment. We observe that denoising is spatially non-uniform with respect to aesthetic descriptors in the prompt. Regions associated with aesthetic tokens receive concentrated cross-attention and show larger temporal variation, while low-affinity regions evolve smoothly with redundant computation. Based on this insight, we propose AccelAes, a training-free framework that accelerates DiTs through aesthetics-aware spatio-temporal reduction while improving perceptual aesthetics. AccelAes builds AesMask, a one-shot aesthetic focus mask derived from prompt semantics and cross-attention signals. When localized computation is feasible, SkipSparse reallocates computation and guidance to masked regions. We further reduce temporal redundancy using a lightweight step-level prediction cache that periodically replaces full Transformer evaluations. Experiments on representative DiT families show consistent acceleration and improved aesthetics-oriented quality. On Lumina-Next, AccelAes achieves a 2.11× speedup and improves ImageReward by +11.9% over the dense baseline. Code is available at https://github.com/xuanhuayin/AccelAes.
Xuanhua Yin, Chuanzhi Xu, Haoxian Zhou +2
School of Computer Science, The University of Sydney, NSW 2006, Australia
Text-to-image (T2I) diffusion models typically require substantial computational resources and cloud infrastructure, posing significant challenges for edge deployment in terms of latency, cost, and user privacy. We present JuZhou 1.0, an ultra-lightweight T2I foundation model designed for fully offline, on-device execution. JuZhou 1.0 achieves its efficiency through four key designs: (1) a compact image-generation backbone consisting of a 0.385B-parameter denoising U-Net and a 1.90M-parameter distilled decoder, totaling approximately 0.387B parameters; (2) Rectified Flow training combined with DMD2 distillation, reducing inference to 4 sampling steps; (3) Chinese semantic alignment trained on 9M curated image-text pairs, enabling direct Chinese prompting without external translation at inference time; and (4) a training and distillation pipeline completed on domestically developed Sugon K100 AI accelerators without relying on NVIDIA GPUs for training or distillation. Despite its compact scale, the 28-step base model of JuZhou 1.0 achieves an overall GenEval score of 0.69, outperforming published baselines including SDXL (2.6B, 0.55), SD3-Medium (2B, 0.62), and IF-XL (4.3B, 0.61). We further validate the full poetry-to-image pipeline on Android and the core CLIP-U-Net-VAE generation branch on iOS. On a smartphone powered by the Snapdragon 8 Elite Gen 5 Mobile Platform, the 4-step U-Net denoising branch runs in approximately 1.6 seconds, while the full Android poetry-to-image pipeline takes 4.5 seconds with on-device prompt refinement on Xiaomi 17 Pro Max. These results position JuZhou 1.0 as a practical approach to mobile text-to-image generation and provide a concrete reference for Chinese-native generation, domestic-compute training, and fully offline on-device deployment after one-time installation.
We introduce Lens, a 3.8B-parameter T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters across various benchmarks, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The training efficiency of Lens stems from two key strategies beyond its compact model size. First, we maximize data information density per training batch by (i) training on Lens-800M, a dataset of 800M densely captioned image-text pairs whose captions are generated by GPT-4.1 and contain approximately 109 words on average, providing richer semantic supervision than conventional short captions, and (ii) constructing each batch from images with multiple resolutions and diverse aspect ratios, thereby enlarging the effective visual coverage of each optimization step. Second, we improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. After pre-training, we apply RL with taxonomy-driven prompts (Lens-RL-8K) and structured reward rubrics to suppress artifacts and improve visual quality, a reasoner module with training-free system prompt search to better align user requests with the model, and distillation-based acceleration for 4-step inference. Through efficient training and systematic optimization, Lens generalizes to arbitrary aspect ratios from 1:2 to 2:1 and resolutions up to 1440^2, and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a 1024^2 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds.