We present Nucleus-Image, a text-to-image generation model that establishes a new Pareto frontier in quality-versus-efficiency by matching or exceeding leading models on GenEval, DPG-Bench, and OneIG-Bench while activating only approximately 2B parameters per forward pass. Nucleus-Image employs a sparse mixture-of-experts (MoE) diffusion transformer architecture with Expert-Choice Routing that scales total model capacity to 17B parameters across 64 routed experts per layer. We adopt a streamlined architecture optimized for inference efficiency by excluding text tokens from the transformer backbone entirely and using joint attention that enables text KV sharing across timesteps. To improve routing stability when using timestep modulation, we introduce a decoupled routing design that separates timestep-aware expert assignment from timestep-conditioned expert computation. We construct a large-scale training corpus of 1.5B high-quality training pairs spanning 700M unique images through multi-stage filtering, deduplication, aesthetic tiering, and caption curation. Training follows a progressive resolution curriculum (256 to 512 to 1024) with multi-aspect-ratio bucketing at every stage, coupled with progressive sparsification of the expert capacity factor. We adopt the Muon optimizer and share our parameter grouping recipe tailored for diffusion models with timestep modulation. Nucleus-Image demonstrates that sparse MoE scaling is a highly effective path to high-quality image generation, reaching the performance of models with significantly larger active parameter budgets at a fraction of the inference cost. These results are achieved without post-training optimization of any kind: no reinforcement learning, no direct preference optimization, and no human preference tuning. We release the training recipe, making Nucleus-Image the first fully open-source MoE diffusion model at this quality.
Figures & tables
Figure 1: Nucleus-Image generations of human subjects and portraits, spanning diverse cultures, ages, and artistic styles. From expressive character studies to fine-grained close-ups with intricate skin texture and detail.
Figure 2: Nucleus-Image generations spanning fantasy, surrealism, animation, and the natural world.
Figure 3: Nucleus-Image generations across product photography, architecture, typography, food, and world culture demonstrating versatility in commercial, conceptual, and everyday imagery.
Figure 4: Overall performance computed as the average of GenEval, DPG-Bench, and OneIG-Bench benchmark scores
Figure 5: Dataset retention across the data pipeline. Block height indicates retained corpus size, and ring markers indicate retained fraction relative to the raw corpus.
Field
Type
Purpose
id
string
Stable record identifier for tracing and debugging.
media_path
string
Content-addressed locator in object storage.
width , height
int
Original geometry; used for aspect-ratio bucketing and validation.
captions
list[string]
Candidate captions at multiple granularities.
caption_sources
list[string]
Provenance tag per caption, aligned with captions .
caption_lengths
list[int]
Token counts per caption.
Table 1: Core training-facing metadata schema.
Figure 6: Representative quality tiers for real images. Real-image samples are ranked using aesthetic scoring together with downstream quality signals, then grouped into the shared A1-A5 quality taxonomy used by the sampler.
Figure 7: Representative quality tiers for synthetic images. Synthetic samples bypass aesthetic scoring and are assigned to the same A1-A5 taxonomy using non-aesthetic quality heuristics
Figure 8: Average caption length across quality tiers and episodic buckets. Higher quality tiers generally carry longer captions, and later buckets trend toward richer textual supervision.
Figure 9: Joint view of static quality tiers and episodic buckets. Columns denote quality tiers A1-A5 and rows denote curriculum buckets B1-B8, illustrating that episodic bucketing acts as a second control axis layered on top of quality tiering.
Figure 10: Overview of the Nucleus-Image architecture. Left : A single transformer block consisting of cross-attention (GQA), modulation, and MoE layers, repeated 32 times. The first 3 blocks use dense FFN with hidden size 2,048 instead of MoE. Right : The expert-choice MoE module with 64 routed experts and a shared expert, where the router selects tokens based on the unmodulated input and timestep.
Dense gradients: only 256 image tokens across 64 experts
512
4.0 (uniform)
Moderate sparsity as representations mature
1024
4.0 / 2.0 (per-layer)
Full progressive schedule (Table 3 )
Table 4: Capacity factor across training resolution stages.
Loss component
Coefficient
Stage
Rectified flow (MSE)
1.0
All phases
Z-loss
5×10−7
All phases
Orthogonal loss
1×10−7
All phases (direct weight update)
Wavelet loss
0.1
High-resolution fine-tuning
Table 5: Loss components and coefficients.
Group
Parameters
Optimizer
η
β1/β2 or μ
λ
Muon
Attention & expert FFN weights
Muon
1×10−4
μ=0.95
0.01
AdamW
Patch embed, modulation, output head
AdamW
1×10−4
0.9 / 0.95
0.01
No-decay
Biases, norms, router gates
AdamW
1×10−4
0.9 / 0.95
0
Table 6: Optimizer groups and hyperparameters.
Stage
Base Resolution
Global Batch Size
Capacity Factor
Notes
1
256
4,096
8.0 (uniform)
Dense gradients; multiple aspect ratios
2
512
1,024
4.0 (uniform)
Moderate sparsity; larger token sequences
3
1024
256
4.0 / 2.0 (per-layer)
Full progressive schedule; wavelet loss enabled
Table 7: Multi-stage training curriculum. Each stage trains with multiple aspect ratios derived from its base resolution. The capacity factor is progressively reduced as expert specialization matures.
Parameter
Value
Hidden dimension
2048
Number of layers
32
Attention heads (Q / KV)
16 / 4
Number of experts
64 + shared
Experts per token (effective)
~2
Expert hidden dimension
1344
Table 8: Model and training hyperparameters.
Model
Single
Two
Count
Colors
Position
Attr.
Overall
Show-o [ 44 ]
0.95
0.52
0.49
0.82
0.11
0.28
0.53
PixArt- α [ 45 ]
0.98
0.50
0.44
0.80
0.08
0.07
0.48
Emu3-Gen [ 46 ]
0.98
0.71
0.34
0.81
0.17
0.21
0.54
SD3 Medium [ 27 ]
0.98
0.74
0.63
0.67
0.34
0.36
0.62
FLUX.1 Dev [ 40 ]
0.98
0.81
0.74
0.79
0.22
0.45
0.66
SD3.5 Large [ 27 ]
0.98
0.89
0.73
0.83
0.34
0.47
0.71
Table 9: GenEval results.
Figure 11: GenEval overall scores for top-performing models. Nucleus-Image matches Qwen-Image at 0.87 and leads all models on spatial position understanding.
Figure 12: DPG-Bench overall scores for top-performing models. Nucleus-Image achieves the highest overall score of 88.79, leading in entity, attribute, and other subcategories.
Model
Global
Entity
Attribute
Relation
Other
Overall
SD v1.5 [ 36 ]
74.63
74.23
75.39
73.49
67.81
63.18
PixArt- α [ 45 ]
74.97
79.32
78.60
82.57
76.96
71.11
Lumina-Next [ 52 ]
82.82
88.65
86.44
80.53
81.82
74.63
SDXL [ 53 ]
83.27
82.43
80.91
86.76
80.41
74.65
Playground v2.5 [ 54 ]
83.06
82.59
81.20
84.08
83.50
75.47
HunyuanDiT [ 55 ]
84.59
80.59
88.01
74.36
86.41
78.87
Table 10: DPG-Bench results.
Figure 13: OneIG-Bench overall scores for top-performing models. Nucleus-Image scores 0.522, surpassing Imagen4 and Recraft V3, with particularly strong style and text rendering performance.
Model
Alignment
Text
Reasoning
Style
Diversity
Overall
Janus-Pro [ 48 ]
0.553
0.001
0.139
0.276
0.365
0.267
BLIP3-o [ 59 ]
0.711
0.013
0.223
0.361
0.229
0.307
Show-o2-1.5B [ 60 ]
0.798
0.002
0.219
0.317
0.186
0.304
Show-o2-7B [ 60 ]
0.817
0.002
0.226
0.317
0.177
0.308
SD 1.5 [ 36 ]
0.565
0.010
0.207
0.383
0.429
0.319
SDXL [ 53 ]
0.688
0.009
0.237
0.332
0.296
0.316
Table 11: OneIG-Bench (EN) results.
Benchmark
Score (0–100)
Configuration
GenEval
87.00
1024×1024
DPG-Bench
88.79
1024×1024
OneIG-Bench
52.20
1024×1024
Overall performance (avg. of normalized 3)
76.00
—
Table 12: Nucleus-Image benchmark summary.
Figure 14: Expert allocation and diversity across three domains. Each column shows a different generation: stylized text rendering (left), photorealistic scene composition (center), and portrait photography (right). Top: generated images. Middle: normalized expert allocation aggregated over all 29 MoE layers and 50 denoising steps—bright regions attract more expert capacity. Bottom: expert diversity, measuring the number of unique dominant experts per spatial position across all timesteps. Across all three domains the router concentrates computation on semantically salient structures (letterforms, helicopter fuselage, facial features) while assigning fewer experts to uniform backgrounds. Foreground regions exhibit lower diversity (stable expert specialization), whereas backgrounds show higher diversity (fluctuating, uncommitted assignments).
Figure 15: Timestep evolution of expert allocation at layer 17. Each row shows the number-of-experts-per-token heatmap overlaid on the generated image, sampled at 10 evenly spaced denoising steps (left to right, top to bottom: steps 0, 5, 11, 17, 22, 27, 33, 38, 44, 49). Early steps exhibit diffuse, spatially unstructured allocation; mid-steps develop coarse semantic structure; late steps produce the sharpest, most content-aligned allocation maps.