Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved image example consumes several hundred context tokens, making large-K settings prohibitively expensive at inference scale. We propose Instruction Distillation: an offline procedure in which the MLLM itself generates, for each individual training image, a structured identification instruction encoding general appearance cues, features that differentiate the class from visually similar ones, and a common confusion point. Unlike prior work that produces a single description per class, our instructions are generated per training image, preserving the intra-class visual diversity that per-class descriptions collapse. At inference time, we study five configurations sharing a single CLIP retrieval index: zero-shot, image ICL, instruction-only ICL, and two hybrid variants in which retrieved neighbors are split between images and instructions. Across seven fine-grained benchmarks and two MLLM backbones, instruction based pipelines match, or exceeds image ICL at K=1 and reduces per-query tokens by 2.9× and inference latency by 3.3× at K=5. Hybrid configurations further show that visual and textual ICL signals are complementary, images give visual patterns to learn and see, while instructions give explicit rules and logic. When both of these are provided, the quality of context improves, which is noticeable in the performance.
Figures & tables
Figure 1: Instruction Distillation. Images are replaced by text-based instructions and passed as context during inference. It results in an improved accuracy score with reduced token consumption.
Figure 2: Inference pipeline overview. The four context-construction strategies. P1 (blue) places K retrieved images in context. P2 (orange) replaces them with K text instructions, the most token-efficient configuration. P3 (green) uses K images as primary context and K′ instructions as supplementary. P4 (purple) reverses the role assignment: K instructions are primary and K′ images are supplementary.
Configuration
Tokens
P0: Zero-Shot
≈ 574
P1(a): Image ICL ( K=5 )
≈ 3,400
P1(b): Image-compressed ICL ( K=5 )
≈ 1940
P2: Instruction ICL ( K=5 )
≈ 1,165
P3: Hybrid ( K=3 , K′=2 )
≈ 2,106
P4: Flip Hybrid ( K=3 , K′=2 )
≈ 1,950
Table 1: Token Cost. Approximate input token cost for each inference configuration at K=5 .
Figure 3: Accuracy versus token budget. Mean per-class accuracy (%) plotted against input tokens for P1(a) (solid blue), P1(b) (dashed blue), P2 (orange), P3 with K′=3 (green), and P4 with K′=3 (purple). Top row: Gemma-3-4B-IT. Bottom row: Qwen2.5-VL-7B-Instruct. Each marker corresponds to one (K) or (K,K′) configuration; markers are connected in order of increasing token cost.
K
Per-class
Caption
Single rule
Paragraph
2-rule
3-rule
1
0.6609
0.6178
0.6140
0.6361
0.7286
0.7224
5
0.6972
0.7893
0.7856
0.7830
0.8304
0.8276
10
0.6565
0.7860
0.7920
0.8010
0.8215
0.8545
25
0.6672
0.7544
0.7678
0.7711
0.8124
0.8390
Table 2: Ablation study with PETS and FGVC. Comparison of three basic instruction formats: caption, paragraph, and a single rule with our 3-rule format, which outperforms all alternatives across K , indicating that both structured decomposition and the inclusion of general and error-avoidance cues contribute to performance.
Pipeline
Tokens
PETS
FOOD
DTD
ESAT
FGVC
FLWR
S397
Avg.
Gemma-3-4B-IT
Low budget ( ∼ 1.5k tokens)
P1(a) ( K=1 )
1,152
67.3
76.3
71.6
27.8
25.7
69.0
56.2
56.3
P1(b) ( K=3 )
1,402
83.7
85.4
78.2
86.9
32.8
84.5
66.9
74.0
P2 ( K=10 )
1,700
85.5
88.1
72.4
89.2
37.4
85.1
73.4
75.9
P3 ( K=1,K′=3 )
1,493
83.9
85.4
79.7
88.5
38.9
83.4
63.9
74.8
Table 3: Pipeline comparison across token budgets and seven fine-grained benchmarks. Mean per-class accuracy (%) for P1(a), P1(b), P2, P3 and P4 on Gemma-3-4B-IT (top) and Qwen2.5-VL-7B-Instruct (bottom), evaluated at three token budgets: low ( ∼ 1.5k), mid ( ∼ 3.3k), and high ( ∼ 6.3k). The Tokens column reports the actual input cost of the chosen (K,K′) configuration. Best per dataset within each budget block is shown in bold .
Figure 4: Average accuracy by pipeline and token budget. Mean per-class accuracy averaged across seven fine-grained benchmarks under three token budgets, for P1(a), P1(b), P2, P3, and P4. Instruction-based pipelines (P1(b), P2, P3, P4) substantially outperform full-resolution image ICL (P1(a)) at the Low budget.
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
Abbr.
Dataset
Images
Classes
FOOD
Food-101
25,250
101
DTD
DTD
1,692
47
FGVC
FGVC Aircraft
3,333
100
PETS
Oxford-IIIT Pets
3,699
37
ESAT
EuroSAT
8,100
10
FLWR
Flowers102
2,463
102
Appendix
Table 4: Dataset details. Summary details of the datasets used in our experiments.
Figure 5: Pipeline accuracy across token budgets and datasets. Mean per-class accuracy for the five inference pipelines (P1(a): full-resolution image ICL, P1(b): compressed-image ICL, P2: instruction ICL, P3: hybrid, P4: flip hybrid) on seven fine-grained benchmarks, evaluated under three per-query token budgets: Low ( ∼ 1.5k), Mid ( ∼ 3.3k), and High ( ∼ 6.3k). P1(b) is reported at the Low and Mid budgets only. Top: Gemma-3-4B-IT. Bottom: Qwen2.5-VL-7B-Instruct. At matched token budgets, the hybrid P3 is competitive with, or stronger than, P1(a) on most datasets, while P2 at low budgets approaches mid-budget P3 on PETS, FOOD, ESAT, and FLWR. This is a per-dataset bar-chart view of the same results plotted as curves in Figure 3 in the main text.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.5286
–
–
–
–
–
–
–
–
1
0.6728
0.663
0.7224
0.8016
0.8356
0.8388
0.8259
0.8508
0.8714
2
0.7827
0.7725
0.8077
0.8555
0.8626
0.8713
0.8441
0.8619
0.8762
3
0.8375
0.8366
0.8175
0.87
0.8697
0.8779
0.8541
0.8688
0.8772
5
0.8742
0.8624
0.8276
0.8754
0.8858
0.8903
0.8521
0.8647
0.8778
Appendix
Table 5: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for PETS dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1103
–
–
–
–
–
–
–
–
1
0.7627
0.752
0.7850
0.8154
0.8410
0.8537
0.8238
0.8464
0.8628
2
0.8332
0.8292
0.8236
0.8520
0.8618
0.8693
0.8500
0.8626
0.8689
3
0.8565
0.8536
0.8413
0.8650
0.8727
0.8788
0.8615
0.8687
0.8730
5
0.8726
0.8694
0.8650
0.8671
0.8718
0.874
0.8712
0.8722
0.8760
Appendix
Table 6: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for FOOD dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1335
–
–
–
–
–
–
–
–
1
0.7159
0.7028
0.7664
0.7452
0.7893
0.7968
0.7765
0.8015
0.7994
2
0.7755
0.7601
0.7686
0.8175
0.8271
0.8313
0.7803
0.7920
0.7893
3
0.7978
0.7815
0.7510
0.8180
0.8340
0.8335
0.7702
0.7787
0.7770
5
0.8132
0.8027
0.7579
0.8218
0.8335
0.8404
0.7537
0.7824
0.7813
Appendix
Table 7: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for DTD dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.0437
–
–
–
–
–
–
–
–
1
0.2781
0.2781
0.8571
0.8529
0.8759
0.8847
0.8811
0.8924
0.9000
2
0.7900
0.7900
0.8715
0.8892
0.9066
0.9054
0.8906
0.9006
0.9049
3
0.8690
0.8690
0.8892
0.8846
0.9074
0.9053
0.8995
0.9043
0.9094
5
0.8996
0.8996
0.8894
0.8859
0.9035
0.9109
0.8901
0.8971
0.9078
Appendix
Table 8: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for ESAT dataset. ESAT images are already low-resolution, so Acc. P1(a) and P1(b) coincide.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.0792
–
–
–
–
–
–
–
–
1
0.2565
0.2371
0.30153
0.34177
0.37143
0.3894
0.3288
0.3774
0.405
2
0.31023
0.2885
0.33783
0.38793
0.4041
0.4119
0.3306
0.3717
0.3903
3
0.36513
0.3277
0.34713
0.42184
0.43444
0.44344
0.3399
0.3706
0.3819
5
0.40144
0.366
0.35823
0.447
0.4578
0.4647
0.3309
0.3675
0.3675
Appendix
Table 9: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for FGVC dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1068
–
–
–
–
–
–
–
–
1
0.6901
0.688
0.7921
0.7803
0.8186
0.8343
0.8882
0.9176
0.9264
2
0.8284
0.813
0.8843
0.9
0.9058
0.9137
0.8931
0.9088
0.9225
3
0.8715
0.845
0.8892
0.9225
0.9382
0.9421
0.8941
0.9078
0.9215
5
0.9049
0.901
0.8715
0.9421
0.933
0.9382
0.8843
0.9186
0.9205
Appendix
Table 10: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for FLWR dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1925
–
–
–
–
–
–
–
–
1
0.5618
0.5549
0.5817
0.5601
0.6016
0.639
0.6453
0.6837
0.6984
2
0.6319
0.6406
0.6512
0.6446
0.6671
0.6925
0.6849
0.6926
0.7029
3
0.67
0.6686
0.6756
0.6848
0.699
0.7087
0.6967
0.7053
0.7096
5
0.7065
0.7092
0.7111
0.7136
0.7214
0.7266
0.7051
0.7127
0.7215
Appendix
Table 11: Accuracy metrics for Gemma-3-4b-it across various shot counts ( K ) for S397 dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.3571
–
–
–
–
–
–
–
–
1
0.7915
0.7856
0.7969
0.8128
0.8336
0.8435
0.8253
0.8461
0.8601
2
0.8329
0.8291
0.8368
0.8387
0.8422
0.8501
0.8383
0.8596
0.8671
3
0.8466
0.8389
0.8472
0.8551
0.8563
0.8586
0.8467
0.8659
0.872
5
0.8615
0.8454
0.8599
0.8687
0.8702
0.8703
0.8563
0.8719
0.875
Appendix
Table 12: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for FOOD dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1672
–
–
–
–
–
–
–
–
1
0.759
0.7218
0.7452
0.7648
0.776
0.7898
0.7712
0.7787
0.7905
2
0.8
0.773
0.7735
0.7973
0.8042
0.8132
0.7877
0.7973
0.8026
3
0.8085
0.7936
0.7835
0.8106
0.8212
0.8281
0.7856
0.8015
0.8048
5
0.8266
0.7989
0.7957
0.8138
0.8239
0.8329
0.7745
0.7835
0.7888
Appendix
Table 13: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for DTD dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.6903
–
–
–
–
–
–
–
–
1
0.8285
0.8077
0.8111
0.8574
0.8705
0.8743
0.8078
0.8323
0.846
2
0.8552
0.8331
0.843
0.8757
0.8841
0.8852
0.8192
0.8601
0.8746
3
0.8721
0.8504
0.8577
0.8887
0.892
0.8912
0.819
0.8708
0.886
5
0.8972
0.8864
0.8778
0.9013
0.9117
0.9084
0.8252
0.8781
0.8917
Appendix
Table 14: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for PETS dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.2413
–
–
–
–
–
–
–
–
1
0.8529
0.84
0.8441
0.9147
0.9098
0.9127
0.9245
0.9421
0.946
2
0.9107
0.899
0.8588
0.949
0.9509
0.949
0.9303
0.9441
0.951
3
0.9264
0.9086
0.8725
0.9578
0.9617
0.9568
0.9334
0.948
0.9509
5
0.947
0.9194
0.8784
0.9627
0.9637
0.9637
0.9313
0.945
0.9529
Appendix
Table 15: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for FLWR dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.2826
–
–
–
–
–
–
–
–
1
0.4794
0.4449
0.4392
0.4967
0.5265
0.5391
0.4689
0.5163
0.5433
2
0.5345
0.4875
0.4569
0.5313
0.5616
0.5535
0.4752
0.5304
0.5409
3
0.5598
0.5139
0.4784
0.5709
0.5778
0.5766
0.4836
0.5286
0.543
5
0.5916
0.5584
0.5148
0.5892
0.5979
0.5955
0.4926
0.5397
0.5442
Appendix
Table 16: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for FGVC dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.1164
–
–
–
–
–
–
–
–
1
0.6797
0.6797
0.8521
0.752
0.7875
0.7816
0.8759
0.8841
0.883
2
0.8217
0.8217
0.8486
0.8218
0.8156
0.8141
0.8789
0.8878
0.8885
3
0.8452
0.8452
0.8529
0.8402
0.828
0.8277
0.87
0.8941
0.8915
5
0.8406
0.8406
0.8519
0.8362
0.844
0.8351
0.8563
0.8778
0.8785
Appendix
Table 17: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for ESAT dataset. ESAT images are already low-resolution, so Acc. P1(a) and P1(b) coincide.
K
Acc. P1
Acc.
Accuracy P3
Accuracy P4
P1(a)
P1(b)
P2
k′=1
k′=2
k′=3
k′=1
k′=2
k′=3
0
0.2137
–
–
–
–
–
–
–
–
1
0.6255
0.6163
0.6233
0.6803
0.6996
0.7157
0.6685
0.6868
0.6977
2
0.6755
0.6544
0.6813
0.7022
0.7114
0.7228
0.7009
0.7151
0.7238
3
0.6986
0.6865
0.7048
0.7179
0.7269
0.7324
0.7186
0.73
0.7361
5
0.7202
0.7089
0.7247
0.7256
0.7348
0.7475
0.7376
0.7465
0.7516
Appendix
Table 18: Accuracy metrics for Qwen2.5-VL-7B-Instruct across various shot counts ( K ) for S397 dataset. Acc. P1(a) is with full-resolution images in context, P1(b) uses compressed images.
Dataset
Groundedness
Genericness
Hallucination
PETS
4.67
3.03
3.92
FOOD
4.82
3.01
3.47
DTD
4.84
3.22
4.66
FGVC
4.78
3.54
4.42
FLWR
4.80
3.26
4.53
S397
4.81
3.00
3.48
Appendix
Table 19: Instruction Quality. Mean scores given by LLM as a judge for various datasets acrross different parameters.
K
Label-only
Per-image
Δ (pp)
1
0.7172
0.7224
+0.5
2
0.7922
0.8077
+1.6
3
0.7991
0.8175
+1.8
5
0.8072
0.8276
+2.0
10
0.8272
0.8545
+2.7
25
0.8023
0.8390
+3.7
Appendix
Table 20: Label-prior control. Top-1 accuracy for label-only vs. per-image instructions on Oxford-IIIT Pets (top) and FGVC (bottom) across K∈{1,2,3,5,10,25} . Per-image instructions outperform the label-only control at every K on both datasets, with no exceptions.
K
Pipeline
Latency (s)
QPS
Speedup
Latency
QPS
1
P1
0.2444
17.68
—
—
1
P2
0.1405
26.35
1.74x
1.49x
5
P1
0.5486
2.57
—
—
5
P2
0.1678
24.40
3.27x
9.49x
10
P1
1.1492
1.18
—
—
Appendix
Table 21: Latency and QPS comparison between P1 and P2 pipelines across different values of K.
Figure 6: Per-query latency vs. shot count K for P1 and P2 (Table 21 ). P1’s latency grows steeply with K since each additional shot adds a full image ( ≈ 550 tokens); P2’s grows far more slowly, so the speedup widens from 1.74 × at K=1 to 5.44 × at K=10 .
Figure 7: Amortization of the one-time instruction-generation cost. P1 has no upfront cost; P2 includes a worst-case ≈ 1-hour offline generation cost, amortized against per-query savings at K=5 .
Budget
Pipeline
Tokens
Input TPS
Output TPS
Total TPS
Low (~1.5k)
P1 (K=1)
1,152
2,436.65
17.42
2,454.07
P2 (K=10)
1,700
6,480.92
21.25
6,502.18
P3 (K=1, K’=3)
1,493
4,165.58
18.71
4,184.28
P4 (K=3, K’=1)
1,600
3,237.63
19.83
4,547.96
Mid (~3.3k)
P1 (K=5)
3,400
3,045.66
8.20
3,053.86
P2 (K=25)
3,305
8,548.72
13.28
8,561.99
Appendix
Table 22: Token-level comparison across low, mid, and high token budgets.
Figure 8: Projected P1 vs. P2 completion time for 100K queries. Reductions: 69.4% (Serial, K=5 ), 89.5% (Throughput, K=5 ), 39.4% (Serial, token-matched Mid budget). At 1M queries the same ratios scale linearly, extending the savings to ≈ 105.8h (Serial) and ≈ 96.7h (Throughput).
Component
Tokens (avg)
System prompt
15–40
Per image example (patches)
≈ 550
Per compressed image example (patches)
≈ 258
Per image example (label text)
≈ 12
Per instruction (3 rules)
≈ 95
Per instruction (label text)
≈ 12
Appendix
Table 23: Per-component token counts . Instruction tokens are measured as the mean token count of generated instructions across all training images and all models.
Vision-language models (VLMs) struggle with in context learning compared to their textual counterparts. Our preliminary study shows that VLMs rely on textual shortcuts and underutilize the visual demonstrations. We therefore hypothesize that VLMs are insufficiently trained to perform visual in-context learning (vICL). To this end, we construct MIME, a task-agnostic vICL training set without textual shortcuts. We ensure the instruction does not state the rule linking the demonstration images to their answers, so the model must infer it from the visual demonstrations. Fine-tuning on MIME improves vICL accuracy by 8.1 to 18.0 points across three VLMs, preserves single-image performance, and improves multi-image and text only in-context learning. In contrast, stating the rule in the training data sharply reduces this gain. We further propose DTAD (Demonstration Text Attention Dropout), which randomly withholds demonstration labels during training so that the VLM must infer the rule from the demonstration images. While achieving comparable vICL accuracy to fine-tuning on MIME, DTAD yields more consistent attention to the demonstration images. On TrueMICL, Qwen3-VL-8B trained on MIME matches frontier models such as Claude Opus 5, and surpasses them with DTAD. Finally, adding MIME to the instruction-tuning mix of LLaVA 1.5 13B more than doubles its vICL accuracy without hurting single-image abilities, whereas a same-size multi-image set improves it only slightly. These results suggest that vICL data should be a standard component of VLM training.
Multimodal large language models (MLLMs) perform well on many vision-language tasks but often struggle with vision-centric problems that require fine-grained visual perception. Recent evidence suggests that this limitation arises not from weak visual representations, but from under-utilization of visual information during instruction tuning, where many tasks can be partially solved using language priors alone. We propose a simple and lightweight approach that augments visual instruction tuning with a small number of visually grounded self-supervised tasks expressed as natural language instructions. By reformulating classical self-supervised pretext tasks, such as rotation prediction, color matching, and cross-view correspondence, as image-instruction-response triplets, we introduce supervision that cannot be solved without relying on visual evidence. Our approach requires no human annotations, no architectural modifications, and no additional training stages. Across multiple models, training regimes, and benchmarks, injecting only a small fraction (3-10%) of such visually grounded instructions consistently improves performance on vision-centric evaluations. Our findings highlight instruction tuning with visually grounded SSL tasks as a powerful lever for improving visual perception in MLLMs through simple adjustments to the training data distribution. Code available at: https://github.com/sirkosophia/V-GIFT
Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc +2
Sorbonne Universite, CNRS, ISIR, F-75005 Paris, France · Institut universitaire de France (IUF)
Few-step distillation has emerged as a critical component in the development of advanced visual generative foundation models, substantially reducing inference overhead while enabling real-time generation and cost-efficient deployment across a broad range of practical scenarios. However, prior work has predominantly focused on advancing training objectives, while comparatively overlooking the training recipe, which has become increasingly critical in the era of large-scale foundation models. In this work, we systematically revisit the training recipe under the well-established distribution matching distillation (DMD) framework for both text-to-image generation and image editing, focusing on three key dimensions: training data composition, teacher guidance within DMD, and task mixture. Our empirical analysis reveals several non-obvious and counterintuitive phenomena, ultimately motivating the development of Qwen-Image-Flash. These findings highlight that effective few-step distillation depends not only on carefully designed objectives, but also on a principled training recipe.