Timing Matters Twice: When to Query and When to Skip in Flow-Matching On-Policy Distillation

Renye Yan; Junyi Zhan; Jikang Cheng; Wei Peng

cs.CVOct 25, 2026

Timing Matters Twice: When to Query and When to Skip in Flow-Matching On-Policy Distillation

Authors: Renye Yan, Junyi Zhan, Jikang Cheng, Wei Peng

Organizations: Peking University · Stanford University

Abstract

On-policy distillation (OPD) enables flow-matching models to acquire complementary capabilities from specialized teachers through supervision on student-generated trajectories. However, different evaluation dimensions, such as aesthetics and OCR accuracy, and different tasks, such as image generation and editing, emphasize different aspects of the denoising process, suggesting that their effective supervision may occur at different stages. Existing methods typically select supervision states from predefined timestep distributions, which may overlook these phase-dependent differences and incur unnecessary computational overhead. This motivates two critical timing decisions: when to adaptively query teachers for effective supervision based on the evolving generative process, and when to skip the remaining rollout to avoid redundant computation. In this paper, we introduce PhaseOPD, a phase-aware query-and-skip framework that identifies generative phases online from revisions of predicted-clean latents, using existing student velocity predictions without additional model evaluations. For each task-assigned teacher, PhaseOPD selects a supervision state based on its phase preference and skips the unused rollout suffix once all query decisions are finalized, preserving the single-query velocity-distillation objective. On SD3.5-M, PhaseOPD executes only 5.56 of 16 rollout positions per local batch, reducing training time by 41.3% relative to DanceOPD while improving Aesthetic, GenEval, and OCR scores.

Figures & tables

Figure 1: Teacher supervision is not equally useful throughout denoising: semantic composition, text rendering, and aesthetics benefit most from different query intervals. This motivates adapting query timing to each teacher-task pair and terminating the rollout after the required queries, rather than using a shared preset schedule and completing every denoising step.

Figure 2: Overall framework of our method

Method	Aesthetic $\uparrow$	GenEval $\uparrow$	OCR $\uparrow$	Time (min) $\downarrow$	Update (s) $\downarrow$	GPU-h $\downarrow$	ImageReward $\uparrow$	VQAScore $\uparrow$
Base model and single-teacher references
SD3.5-M Base	5.3973	0.6620	0.5611	–	–	–	0.9565	0.7416
Aesthetic Teacher	6.2133	0.7246	0.5852	–	–	–	1.5708	0.7840
GenEval Teacher	5.2709	0.9336	0.6473	–	–	–	0.8062	0.7225
OCR Teacher	5.2665	0.6631	0.9428	–	–	–	0.9243	0.7275
Multi-teacher distillation methods

Table 1: Final generation quality, independent diagnostic metrics, and training cost. Aesthetic, GenEval, and OCR are training objectives. ImageReward and VQAScore, shaded in light blue, are excluded from training and assess cross-metric generalization and potential reward hacking. The base model and individual teachers serve as capability references. Among multi-teacher methods, the best and second-best results are shown in bold and underlined, respectively.

Figure 3: Quality over the first 160 training minutes. Panels show Aesthetic, GenEval, and OCR for DanceOPD, DiffusionOPD, and Ours. Time uses each method’s matched mean update time measured with exclusive GPU access.

Figure 4: Plug-and-play comparison across three OPD objectives. Each panel compares SD3.5-M Base, the corresponding OPD method, and the method augmented with Ours. Quality metrics share scales across panels; Overall Quality is the geometric mean of the three normalized quality scores. Training efficiency uses inverse training time normalized within each panel and is omitted for the base model.

Figure 5: Qualitative comparison with the primary teacher set. Rows correspond to Aesthetic, GenEval, and OCR; columns show SD3.5-M Base, DanceOPD, DiffusionOPD, and Ours. All methods use archived, paired outputs generated with the same seed.

Method	HPSv3 $\uparrow$	SigLIP $\uparrow$	OCR $\uparrow$	Time (min) $\downarrow$	Update (s) $\downarrow$	GPU-h $\downarrow$	ImageReward $\uparrow$	VQAScore $\uparrow$
Base model and single-teacher references
SD3.5-M Base	2.613	0.1472	0.5611	–	–	–	0.9565	0.7416
HPSv3 Teacher	12.44	0.1361	0.5648	–	–	–	1.309	0.7392
SigLIP Teacher	7.091	0.1973	0.6083	–	–	–	1.274	0.7904
OCR Teacher	2.673	0.1507	0.9428	–	–	–	0.9243	0.7275
Multi-teacher distillation methods

Table 2: Generation quality, cross-metric generalization, and training cost with alternative teachers. HPSv3, SigLIP, and OCR are training objectives; ImageReward and VQAScore are held-out metrics for assessing generalization and potential reward hacking. Bold and underlined values indicate the best and second-best multi-teacher results, respectively. Base and single-teacher models serve as references. Ours offers a favorable quality–efficiency balance across teacher combinations.

Figure 6: Qualitative comparison with the alternative teacher set. Rows correspond to HPSv3, SigLIP, and OCR; columns show SD3.5-M Base, DanceOPD, DiffusionOPD, and Ours.

Figure 7: Cross-backbone and cross-task generalization on BAGEL. This setting changes both the student backbone and teacher tasks, covering text-to-image generation, global editing, and local editing.

Figure 8: Teacher-specific query distributions across three OPD hosts. Panels correspond to Ours, DiffusionOPD + Ours, and FlowOPD + Ours, each showing GenEval, OCR, and Aesthetic teachers. The horizontal axis indicates query position; the vertical axis shows the percentage of trajectories normalized separately for each teacher.

Figure 9: Mean dynamics of online detection signals. The blue curve shows the change in the spatial-scale revision ratio,

\Delta F_{k}

, and the orange curve shows its first-order difference

g_{k}

. Background colors and vertical dashed lines indicate three generation stages identified by near-zero slopes after sustained increases and decreases in

\Delta F_{k}

Figure 12

Variant	AT	ES	Aesthetic $\uparrow$	GenEval $\uparrow$	OCR $\uparrow$	Time (min) $\downarrow$	Update (s) $\downarrow$	GPU-h $\downarrow$
Ours (w/o AT and ES)	$\times$	$\times$	5.9044	0.9496	0.8626	275.333	5.5067	18.356
Ours (w/o AT)	$\times$	$\checkmark$	5.8073	0.8892	0.9305	241.78	4.836	16.119
Ours (w/o ES)	$\checkmark$	$\times$	6.1642	0.9341	0.9275	277.667	5.5533	18.511
Full Ours	$\checkmark$	$\checkmark$	6.1705	0.9461	0.9413	161.893	3.2379	10.793

Table 4: Effects of individual components on final quality and training efficiency. AT denotes adaptive triggering, and ES denotes early skipping. The best and second-best results for each metric are shown in bold and underlined, respectively.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Training time to reach quality targets: Aesthetic

\geq 5.70

, GenEval

\geq 0.89

, and OCR

\geq 0.90

Figure 12: Stage sensitivity of teacher supervision. The three panels report Aesthetic, GenEval, and OCR gains from controlled single-teacher experiments. Each bar shows the change relative to the Beta query strategy when queries are restricted to the corresponding generation interval.

Query interval	Aesthetic	GenEval	OCR
R1	6.3258	0.8042	0.8926
R2	6.4515	0.7833	0.9171
R3	6.4623	0.7458	0.9114
R4	6.2956	0.6542	0.8569
Beta	6.4341	0.6833	0.9034

Appendix

Table 5: Single-teacher scores for fixed query intervals.

OCR stage policy
Configuration	Aesthetic $\uparrow$	GenEval $\uparrow$	OCR $\uparrow$
Stage-2 only	5.9199	0.9379	0.9139
Stage-1/2 mixture (Ours)	6.1705	0.9461	0.9413
Stage-2/3 mixture	5.9154	0.9339	0.9139
Teacher routing policy
Balanced (Ours)	6.1705	0.9461	0.9413

Appendix

Table 6: OCR stage-policy and teacher-routing results at 3,000 updates. Both Ours rows reproduce Table 1 .

Figure 13: Time–quality trajectories with the alternative teacher set under a 90-minute budget. The panels report HPSv3, SigLIP cosine, and OCR. All methods start from the same SD3.5-M Base scores.

DanceOPD + Ours
CFG	Step	Time (min)	GPU-h	Aesthetic	GenEval	OCR
4.5	3000	161.89	10.7929	6.1705	0.9461	0.9413
1.0	3000	81.39	5.640	5.6135	0.9396	0.9129
DiffusionOPD + Ours
4.5	1500	395.57	26.4101	6.1003	0.8915	0.9281
1.0	1500	205.94	13.7814	5.8450	0.9301	0.9239

Appendix

Table 7: Quality and training cost under different training CFG settings. Evaluation CFG is fixed at 4.5.

Figure 14: Text accuracy. PhaseOPD improves target-text reproduction; final punctuation in (d) remains imperfect.

Figure 15: Text visual quality. PhaseOPD unifies readable text, coherent typography, materials, and scene integration.

Figure 16: Aesthetic quality. PhaseOPD preserves lighting and materials while clarifying subjects in selected scenes.

Figure 17: Semantic alignment. PhaseOPD improves counts over DanceOPD in the shown examples while preserving attributes and spatial relations.

Figure 18: Joint text and scene quality. PhaseOPD combines readable text with clear subjects and coherent surroundings.

Figure 19: Alternative teachers. PhaseOPD retains scene detail and improves count alignment in the macaron example under HPSv3, SigLIP, and OCR supervision.

Figure 20: Cross-host transfer. PhaseOPD retains readable text and enriches material appearance with both OPD hosts.

On-Policy Distillation

On-policy distillation trains a model on its own outputs using teacher guidance, aligning supervision with the states it encounters at deployment.

See all 517 papers

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
Aug 4, 2026Siming Fu, Zheming Fu, Ruizhe He +7Flow Matching Diffusion Model Distillation
1Joy Future Academy · 2Zhejiang University
Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Aug 27, 2026Shiyi Zhang, Mushui Liu, Yunze Tong +8Flow Matching On-Policy Self-Distillation
1Tsinghua University · 2Zhejiang University · 3Alibaba Group
DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models
Aug 10, 2026Mingfeng Lin, Chengfei Cai, Lin Xu +2Flow Matching RL for Image Generation
Harbin Institute of Technology (Shenzhen) · Zhejiang University · Harbin Institute of Technology

Aug 4, 2026J/K move · Enter open · S save

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging

Siming Fu, Zheming Fu, Ruizhe He, Hualiang Wang+6

1Joy Future Academy · 2Zhejiang University

Flow Matching Diffusion Model Distillation T2I Generation Cross-Architecture Knowledge Distillation On-Policy Distillation

Open paper AskPDF arXiv

On-policy distillation, in which a teacher corrects samples that the student itself generates, presupposes that the two models speak the same language: identical VAE latents, matching architectures, and a common timestep grid. We ask what happens when none of this holds, as when the strongest teacher available and the student one wishes to deploy come from different model families, and find that the standard recipes have no answer: teacher latents cannot serve as targets in a foreign coordinate system, per-pixel losses against a teacher that stochastically re-draws local detail degenerate into blur or divergence, and timestep indices lose their meaning across mismatched schedules. We present Any-OPD, to our knowledge the first framework for on-policy distillation between arbitrary pairs of latent flow-matching generators. Any-OPD treats the teacher purely as a black-box sampler and connects the two models at exactly one point: a frozen, model-agnostic vision representation in which their independently decoded outputs are compared, sidestepping every assumption about latents, features, or architecture. Trajectory correspondence is recovered by matching continuous noise levels instead of step indices, and a brief anchoring phase, in which teacher samples are re-encoded through the student's own VAE, ensures the on-policy gradient measures sample quality rather than domain mismatch. Distilling the 12B FLUX.1-dev into the 2.5B SD3.5-Medium, Any-OPD lifts the student's PickScore from 0.846 to 0.884 and HPSv3 from 9.12 to 10.97, rivaling the teacher at a fifth of its size, where direct latent regression fails to train at all.

Abstract

On-Policy Distillation

Explore similar work

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher

DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models