Generating text-bearing images requires coherent scenes and accurate characters. Large text-to-image models improve both capabilities through repeated denoising, at substantial sampling cost. Few-step distillation reduces this cost but requires the student to organize scenes and refine glyphs within a short trajectory. We present a four-step distillation framework combining local response regularization with teacher-guided text training. Built on distribution matching, the framework distills the GLM-Image DiT while retaining its autoregressive visual prior, glyph encoder, and VAE. A path-length penalty promotes consistent local responses of the conditional denoising field. Progressive training combines synthetic strings with teacher-generated imagery to preserve character accuracy and scene-integrated typography. The student achieves GenEval Overall 0.84 and CVTG-2K word accuracy 0.5045, compared with 0.81 and 0.1690 for four-step TDM. A coefficient study favors 10−5, and Qwen-Image experiments show additional gains from response regularization and text supervision. Single-image inference takes 1.742 s, compared with 5.818 s for the teacher, yielding a 3.34× speedup at the same 40 GB GPU memory requirement.
Figures & tables
Method
Main supervision
Relation to the proposed response prior
DMD2 [ 39 ]
Teacher/fake-score distribution matching; inference-state simulation
Supplies the base distributional supervision complemented by our conditional response term.
TDM [ 24 ]
Step-aware trajectory distribution matching
Supervises trajectory distributions; our added term shapes local input responses.
Phased DMD [ 7 ]
Progressive subinterval distribution and score matching
Allocates supervision and capacity by phase; our added term acts on a shared conditional denoiser.
rCM [ 45 ]
Continuous-time consistency plus score distillation
Uses differential consistency and teacher scores; our VJP measures response magnitude at a fixed condition and time.
GAD [ 15 ]
Teacher–student directional derivative alignment
Matches response vectors to a teacher target; our target is an adaptive scalar response scale.
StyleGAN2 [ 17 ]
Style-to-image path-length regularization
Supplies the regularization principle; we apply it to the latent-to-denoising field before guidance.
Table 1: Training objectives of the methods most relevant to our work. Our framework integrates a conditional response-magnitude prior with teacher-guided text supervision while retaining the distribution-matching backbone and the teacher’s conditioning interface.
Figure 2: Four-step distillation with complementary response and data supervision. Frozen autoregressive and glyph encoders retain semantic and character conditions. Distribution matching transfers teacher synthesis behavior, while the path-length term regularizes the student’s local response. Teacher-guided progressive training combines explicit text examples with visually integrated typography; the VAE remains fixed.
Stage
Training examples
Primary purpose
1
Generic prompts and teacher images
Preserve scene fidelity
2
Synthetic short text and teacher images
Learn glyph-level alignment
3
Dense layouts and teacher images
Refine multi-region text
Table 2: Three-stage text-training curriculum for GLM-Image. Teacher images preserve scene appearance throughout, while synthetic text supervision progresses from short strings to dense layouts.
Component
Setting
Teachers
GLM-Image and Qwen-Image
Sampling budget
Teacher: 50 steps; student: four steps
Resolution
1024×1024
Student time labels
1000, 750, 500, 250
Update ratio
Four fake-score updates per generator update
Response penalty
Enabled or disabled
Table 3: Shared distillation and evaluation settings. Each student uses four denoising transitions while retaining the teacher’s conditioning and decoding interfaces.
Figure 3: Literal text preservation under a shared garden prompt and random seed. The full student renders “Welcome” correctly in four denoising steps, matching the target spelling of the 50-step teacher. Intermediate variants show generation without text training and with synthetic text alone.
Setting
Value
Optimizer
AdamW
Learning rate
10−6
Betas; weight decay
(0.9,0.999) ; 0.01
Batch size
16/GPU, 8 GPUs, total 128
PL target decay
0.99
Default PL coefficient
10−5
Table 4: Optimization settings. Response regularization adds a training term while preserving the student’s inference architecture.
Model
Steps
Branch evals.
Reduction
Teacher
50
100
1×
TDM
4
8
12.5×
Ours
4
8
12.5×
Table 5: Scheduled denoising workload under classifier-free guidance. Reductions apply to branch-evaluation counts; total latency also includes conditioning and VAE decoding.
Model
Time (s/image)
Memory (GB)
Speedup
Teacher (50 steps)
5.818
40
1.00×
Ours (4 steps)
1.742
40
3.34×
Table 6: Measured single-image inference cost for GLM-Image on an NVIDIA A100 GPU. Our four-step student reduces latency by 70.1% while retaining the teacher’s 40 GB GPU memory requirement.
Method
WA-2 ↑
WA-3 ↑
WA-4 ↑
WA-5 ↑
WA ↑
NED ↑
CLIPScore ↑
Teacher
0.9103
0.9209
0.9169
0.8975
0.9116
0.9557
0.7877
TDM [ 24 ]
0.2179
0.2088
0.1300
0.1194
0.1690
0.4265
0.6902
Student w/o text training
0.2032
0.1758
0.1087
0.0951
0.1457
0.3717
0.6895
Student with synthetic text
0.2448
0.2298
0.1910
0.1691
0.2086
0.5050
0.7283
Student with teacher-image mixing
0.5548
0.5744
0.4868
0.4022
0.5045
0.7130
0.7508
Table 7: Text-rendering fidelity on CVTG-2K with GLM-Image. Teacher-image mixing achieves the strongest four-step result on every metric, outperforming TDM and both text-training variants. The teacher uses 50 steps. Higher is better; boldface denotes the best four-step score.
Method
WA-2 ↑
WA-3 ↑
WA-4 ↑
WA-5 ↑
WA ↑
NED ↑
CLIPScore ↑
Teacher
0.8370
0.8364
0.8313
0.8158
0.8288
0.9116
0.8017
Student w/o text training
0.4684
0.3576
0.3203
0.2751
0.3554
0.4520
0.7785
Full student
0.6878
0.6625
0.6217
0.5860
0.6395
0.8542
0.7873
Table 8: Text-rendering fidelity on CVTG-2K with Qwen-Image. Dedicated text training improves all region-count groups, WA, NED, and CLIPScore. The teacher uses 50 steps and students use four. Higher is better; boldface denotes the best student score.
Method
Single
Two
Count
Color
Position
Attr.
Overall
Teacher
0.99
0.93
0.89
0.90
0.78
0.77
0.88
TDM [ 24 ]
0.97
0.88
0.80
0.80
0.72
0.69
0.81
Student w/o PL
0.97
0.88
0.81
0.82
0.71
0.68
0.81
Full student
0.98
0.90
0.84
0.85
0.73
0.72
0.84
Table 9: Compositional fidelity on GLM-Image. Our four-step student improves all six GenEval categories over four-step TDM, raising Overall from 0.81 to 0.84. The teacher uses 50 steps. Higher is better; boldface marks the best distilled-model score.
λPL
WA
NED
CLIPScore
GenEval
10−6
0.4475
0.6821
0.7404
0.82
10−5
0.5045
0.7130
0.7508
0.84
10−4
0.4648
0.6997
0.7460
0.83
Table 10: PL-weight ablation on GLM-Image after text-oriented training. The intermediate coefficient 10−5 gives the best result on all four metrics. Every setting uses four-step inference. Higher is better; boldface marks the best score.
Method
Single
Two
Count
Color
Position
Attr.
Overall
Teacher
0.99
0.92
0.89
0.88
0.76
0.77
0.87
Student w/o PL
0.98
0.88
0.83
0.84
0.72
0.70
0.82
Full student
0.98
0.89
0.84
0.84
0.73
0.71
0.83
Table 11: Effect of response regularization on Qwen-Image. PL improves four GenEval categories and raises Overall from 0.82 to 0.83. The teacher uses 50 steps and students use four. Higher is better; boldface marks the best student score, including ties.
Figure 4: CVTG-2K word accuracy across text-region counts. Teacher-image mixing gives the strongest GLM student results and exceeds four-step TDM in every group; dedicated text training also improves Qwen rendering throughout. Curves connect the measurements in Tables 7 and 8 .
VAE
FID
PSNR
Scale
Resp. ratio
Without regularization
1.68
25.480
0.248
42.159
With regularization
1.65
25.495
0.323
30.629
SD3 VAE reference
1.69
25.784
0.209
38.167
EDM2 variant
FID
FD-DINOv2
Baseline
5.268
311.906
Regularized
4.953
276.889
Table 12: Auxiliary VAE reconstruction and downstream EDM2 generation. Response regularization improves reconstruction FID and both generation distances. Lower FID and FD-DINOv2 and higher PSNR indicate better quality. Scale denotes the latent scale factor; response ratios describe empirical sensitivity variation.
VAE
FID
PSNR
Scale
Resp. ratio
Baseline
1.68
25.480
0.248
42.159
1 × PL
1.64
25.078
0.280
32.833
8 × PL
1.65
25.495
0.323
30.629
40 × PL
1.86
25.088
0.396
20.020
Table 13: VAE regularization-strength sweep. The 1× setting minimizes reconstruction FID, while 8× supports the best downstream generation in Table 12 . Scale is the latent scale factor. Lower FID and higher PSNR indicate better reconstruction.
Figure 5: ImageNet samples and text-scene illustrations arranged in a continuous grid. Rows correspond to the baseline and 1× , 8× , and 40× PL settings. Quantitative comparisons are reported in Tables 12 and 13 .
Figure 6: Qualitative comparison of four-step GLM-Image students. Within each three-column group, outputs are ordered as the student without text training, TDM, and our complete framework. Cases cover (a) an English poster, (b) a Chinese poster, (c) a bilingual sign, (d) product packaging, (e) multiline text, (f) a bilingual display, (g) object counting, and (h) spatial relations. The examples illustrate differences in literal text preservation, object counts, and spatial arrangement while retaining coherent scene appearance.
VAE
Low
High
Ratio
Baseline
0.069
2.909
42.159
1× PL
0.102
3.349
32.833
8× PL
0.143
4.380
30.629
40× PL
0.247
4.945
20.020
Table 14: Directional response statistics for the auxiliary VAE sweep. Low and High report empirical finite-difference response magnitudes; Ratio is High/Low. PL narrows this relative spread, complementing the reconstruction and generation comparisons in Tables 12 and 13 .
Efficient text-to-image generation requires both reinforcement-learning (RL)-based reward alignment and few-step distillation, yet these procedures are typically performed sequentially, increasing training cost and risking the loss of reward gains during compression. We take an RL-native perspective: diffusion RL already generates reward-scored finite-step trajectories, whose intermediate states provide distillation supervision. Based on this insight, we propose REST (Reward-Enhanced Scored-Trajectory Distillation), a single-stage co-training framework in which a decoupled student learns from the evolving RL teacher's trajectories without changing teacher optimization. Advantage-Modulated Distillation (AMD) transforms rollout advantages into signed weights, strengthening imitation of preferred trajectories and aligning distillation priorities with task value. The resulting framework is general and lightweight, requires no extra image rollouts, no separate distillation dataset, and no adversarial training. Experiments on compositional generation, visual text rendering, and human-preference alignment demonstrate competitive few-step, CFG-free generation with RAM or DiffusionNFT teachers. With only four sampling steps, REST-RAM achieves a DrawBench PickScore of 23.97, outperforming both the 40-step RAM teacher (23.95) and RTDMD (23.71).
Yuhan Li, Fangao Zeng, Sicong Kang +7
Shanghai Jiao Tong University · Taobao & Tmall group of Alibaba
Few-step distillation has emerged as a critical component in the development of advanced visual generative foundation models, substantially reducing inference overhead while enabling real-time generation and cost-efficient deployment across a broad range of practical scenarios. However, prior work has predominantly focused on advancing training objectives, while comparatively overlooking the training recipe, which has become increasingly critical in the era of large-scale foundation models. In this work, we systematically revisit the training recipe under the well-established distribution matching distillation (DMD) framework for both text-to-image generation and image editing, focusing on three key dimensions: training data composition, teacher guidance within DMD, and task mixture. Our empirical analysis reveals several non-obvious and counterintuitive phenomena, ultimately motivating the development of Qwen-Image-Flash. These findings highlight that effective few-step distillation depends not only on carefully designed objectives, but also on a principled training recipe.
Distilled text-to-image models generate images quickly, but classifiers trained on those images can perform substantially worse than classifiers trained on images from their slower base models. We adapt released distilled generators to produce better training data while retaining fast generation. We fit a low-rank update to the student's first prediction so that it reaches an intermediate state produced by its unguided base, then disable the update and let the unchanged student finish generation. The base supplies training targets and is absent at inference. Guided targets and shorter jumps give classifiers lower accuracy in our ablations, showing why the state used for supervision matters. Across three generator families, the adapted images train better classifiers than images from the released students. With class-name templates at full ImageNet-1k scale, ResNet-50 top-1 rises from 8.4% to 34.2% for FLUX.2-klein and from 12.7% to 32.1% for SD-Turbo, using two and three denoiser evaluations per image, respectively. Fixed adapters also improve results on additional classes, fine-grained datasets, and other learner architectures and training recipes. Template-based adaptation remains useful against richer-prompt alternatives, while the benefit of combining adaptation with LLM generation prompts depends on the generator. Adapting a single generation step makes fast generators substantially more useful for training visual models.
Patryk Bartkowiak, Dominik Michels, Sören Pirk +1
Adam Mickiewicz University · GreenMatterAI · KAUST +1