In this paper, we uncover the hidden potential of Diffusion Transformers (DiTs) to significantly enhance generative tasks. Through an in-depth analysis of the denoising process, we demonstrate that introducing a single learned scaling parameter can significantly improve the performance of DiT blocks. Building on this insight, we propose Calibri, a parameter-efficient approach that optimally calibrates DiT components to elevate generative quality. Calibri frames DiT calibration as a black-box reward optimization problem, which is efficiently solved using an evolutionary algorithm and modifies just ~100 parameters. Experimental results reveal that despite its lightweight design, Calibri consistently improves performance across various text-to-image models. Notably, Calibri also reduces the inference steps required for image generation, all while maintaining high-quality outputs.
Figures & tables
Figure 2: Illustration of DiT architectural components.
Figure 3: Motivational Experiment: Our findings reveal that the contribution of DiT blocks is not fully optimized. We demonstrate that their performance can be enhanced through a straightforward output scaling using a scalar multiplier.
Figure 4: Illustration of calibration parameter search procedure.
Scaling
N params
iters
HPSv3
IR
Q-Align
–
–
–
11.41
1.15
4.85
Block
57
200
13.29
1.17
4.91
Layer
76
410
13.41
1.24
4.90
Gate
114
960
13.48
1.18
4.88
Table 1: Comparison of various granularity levels for internal-layer calibration applied to the Flux model. All experiments are conducted using the Flux model, with evaluation performed on the HPSv3 test set.
Figure 5: Quantitative comparison of various granularity levels for internal-layer calibration applied to the Flux model.
Figure 6: Comparison between Calibri Ensemble and original model across several inference steps.
Model
Calibri
HPSv3
IR
Q-Align
NFE
FLUX
✗
11.41
1.15
4.85
30
✓
13.48
1.18
4.88
15
SD-3.5M
✗
11.15
1.10
4.74
80
✓
14.10
1.17
4.91
30
Qwen Image
✗
11.26
1.16
4.55
100
✓
12.95
1.18
4.73
30
Table 2: Quantitative evaluation of generation quality improvements across various baseline models. Notably, Calibri achieves superior metric scores while requiring fewer inference steps.
Figure 7: Qualitative evaluation of generation quality improvements across various baseline models. Models have same NFE as in Table 2 .
Methods
Overall Preference
Text Alignment
Calibri
Equal
Original
Calibri
Equal
Original
Flux
51.87
7.33
40.80
38.71
37.68
23.61
Qwen-Image
54.62
7.91
37.47
40.29
37.65
22.06
Table 3: Human evaluation: Calibri vs. Baselines win rates %.
Flow-GRPO
Calibri
HPSv3
PickScore
Q-Align
NFE
✗
✗
11.15
22.40
4.74
80
PickScore
12.47
23.13
4.91
30
PickScore
✗
12.67
23.78
4.92
80
PickScore
12.96
23.93
4.85
30
GenEval
✗
10.16
22.22
4.69
80
HPSv3
14.18
22.22
4.88
30
Table 4: Comparison of Calibri and Flow-GRPO [ 24 ] on SD-3.5M. Calibri achieves comparable performance with 105 fewer parameters and can be combined with alignment methods to boost either the same or different target metrics.
Figure 8: Qualitative comparison of Calibri and Flow-GRPO [ 24 ] on SD-3.5M. Calibri achieves comparable performance with 105 fewer parameters and can be combined with alignment methods to boost either the same or different target metrics.
Model
Flux
SD-3.5M
Qwen-Image
Scaling type
Block
Layer
Gate
Gate
Gate
CFG type
CFG Distill
Standard CFG
N params
57
76
114
216
482
N iters
200
410
960
2280
630
GPU hours
32
64
150
356
286
Table 5: Calibration cost.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Limitations of modern reward models.
Flow-GRPO
Calibri
HPSv3
PickScore
Q-Align
Dino Diversity
NFE
✗
✗
11.15
22.40
4.74
0.20±0.06
80
✗
9.5
22.04
4.51
0.25±0.08
30
PickScore
12.47
23.13
4.91
0.20±0.06
30
PickScore
✗
12.67
23.78
4.92
0.15±0.06
80
✗
12.514
23.76
4.91
0.15±0.05
30
PickScore
12.96
23.93
4.85
0.15±0.05
30
Appendix
Table 6: Comparison of Generation Diversity for SD-3.5M [ 8 ] Optimized by Calibri and Flow-GRPO [ 24 ] .
Figure 10: Illustration of Calibri with different rewards as objective.
Calibri
HPSv3
IR
Q-Align
PickScore
NFE
✗
11.41
1.15
4.85
22.88
30
HPSv3
13.41
1.24
4.90
23.07
15
ImageReward
11.06
1.17
4.70
22.47
15
Q-Align
11.65
1.0
4.89
22.36
15
PickScore
13.34
1.2
4.89
23.24
15
Appendix
Table 7: Quantitative comparison of Calibri layer scale on Flux [ 22 ] across different reward models.
Figure 11: Comparison of CMA-ES and Flow-GRPO performance in optimizing Calibri coefficients.
Diffusion Transformers use attention-based networks to generate and transform images, video, audio, and other structured data through iterative denoising.