FlowMap-OPD: Rollout--Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators

Zhiqi Li; Bo Zhu

cs.CVSep 29, 2026

FlowMap-OPD: Rollout--Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators

Authors: Zhiqi Li, Bo Zhu

Organizations: Georgia Institute of Technology

Abstract

Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distillation framework that separates student-state acquisition from teacher--student distribution comparison. A formulation based on state marginals establishes this separation, while flow--velocity consistency connects local supervision to the deployed long-range map. Within this framework, we develop flow-map, induced-velocity, and instantaneous-velocity distribution supervision, each paired with a separately specified native flow-map rollout. Cross-capacity ImageNet experiments across three teacher rewards identify instantaneous-velocity distribution supervision with independently tunable student consistency as the most effective choice. In text-to-image experiments, FlowMap-OPD demonstrates strong multi-specialist consolidation capabilities and surpasses multi-reward Flow-Map GRPO in task performance and convergence speed.

Figures & tables

Figure 1: Three specialist teachers, one native student. By separating native rollout from the teacher-supervision kernel, FlowMap-OPD rapidly distills the complementary capabilities of three specialist teachers into a single few-step student: visual preferences, text rendering, and object relations. Large images show outputs from the same student across tasks; the lower-right thumbnails show the corresponding specialist outputs.

Panel of Figure 2 — Figure 2: From local-transition OPD to few-step flow-map OPD with separate rollout and teacher comparison. (1) Flow/Diffusion OPD [ 4 , 13 ] uses the same local transition distributions from $dx_{t}=[v_{t}^{\theta}-\tfrac{g_{t}^{2}}{2}\nabla_{x}\log p_{t}]dt+g_{t}\,d\overline{W}_{t}$ for rollout and teacher–student comparison. (2) FlowMap-OPD collects states through long-range map rollouts, $x_{t_{i+1}}=\psi^{\theta}_{t_{i}\to t_{i+1}}(x_{t_{i}})$ . (3) Independently of the rollout transition, optimization compares teacher–student distributions through $\mathrm{KL}(\mathcal{N}(\mu^{\theta},\Sigma)\|\mathcal{N}(\mu^{T},\Sigma))$ , with means constructed from direct velocities $v^{\theta}(x,t)$ (Section 4.5 ), flow maps $\psi^{\theta}_{t\to r}(x)$ with their stochastic corrections (Section 4.3 ), or induced velocities $V^{\theta,\mathrm{src}}(x,r,t)$ and $V^{\theta,\mathrm{dst}}(x,r,t)$ (Section 4.4 ).

Figure 3: Specialist capabilities consolidated in the same few-step student. Each row compares the base, all three specialists, and FlowMap-OPD on the same prompt and initial noise with the same few-step sampler. Gold borders mark the specialist for each task; the green column shows the same student across all three tasks: a seed packet labeled “Magic Beans Inside,” a medieval wolf adventurer, and a truck to the left of a refrigerator. Full task scores are reported in Table 2 .

Panel of Figure 4 — Figure 4: Source and destination consistency. Gray S-curves show the continuous trajectory; dashed arcs show separate long- and short-range flow-map jumps, and black straight arrows show local velocity steps. (a) Source: the direct map $\psi_{t\to r}$ agrees to first order with a step $x_{t-\varepsilon}=x_{t}-\varepsilon v_{t}$ followed by $\psi_{t-\varepsilon\to r}$ . (b) Destination: $\psi_{t\to r-\varepsilon}$ agrees to first order with $\psi_{t\to r}$ followed by $x_{r-\varepsilon}=x_{r}-\varepsilon v_{r}$ . Here $v_{t}=v(x_{t},t)$ and $v_{r}=v(x_{r},r)$ . These two routes give the identities in Eq. 11 .

Figure 5: Cross-capacity transfer under three teacher rewards. Separate XL/2-to-B/2 runs compare representative supervision recipes. Panels show MMD-setting FID, classifier target log-probability, and DINO cosine similarity under native four-step generation. Every marker is a recorded evaluation; curves stop at the common epoch 300. Dashed lines show the frozen teacher actually used in each sweep. Complete coefficient sweeps and five-step curves appear in Appendix E .

Flow-map supervision
Supervision	MMD	Classifier	DINO
	FID $\downarrow$	$\log p_{y}\uparrow$	Cosine $\uparrow$
B/2 base	22.68	-0.5448	0.7207
XL/2 teacher	14.68	-0.4526	0.8084
Stochastic map, $\lambda=0.1$	29.15	-0.9623	0.7004
Stochastic map, $\lambda=0.2$	29.15	-0.9621	0.7003

Table 1: Three separate XL/2-to-B/2 reward-transfer settings. All students use epoch 300, with native four-step sampling. Columns report MMD-teacher transfer by FID, classifier transfer by target log-probability, and DINO transfer by cosine similarity. These are different reward settings, not three scores of one student. Best and second-best student results are bold and underlined, respectively.

Task-specialized teachers
	Task scores			DrawBench scores
Model / supervision	GenEval	OCR	PickScore	PickScore	Aesthetic	DeQA	ImgRwd	UniRwd
Base	0.5041	0.3491	20.9758	21.6298	5.5368	4.1712	0.3918	2.7187
GenEval teacher	0.8454	—	—	21.8184	5.3905	3.2086	0.6426	2.8719
OCR teacher	—	0.8504	—	21.9869	5.5111	4.1396	0.5386	2.8049
PickScore teacher	—	—	23.0772	23.3711	6.1510	4.0829	1.1402	3.2005

Table 2: Three specialist teachers distilled into one five-step student. The student uses

\lambda_{c}=0.001

at update 300. Task scores and DrawBench scores are shown side by side; higher is better.

Figure 6: Three-teacher transfer during training. Full-test-set OCR and PickScore scores during training and at update 300; markers are actual evaluations and lines connect measurements without smoothing. Dashed lines denote the respective specialists. Both panels report the same student with

\lambda_{c}=0.001

Figure 7: Two specialist teachers versus direct mixed-reward optimization. EMA OCR and PickScore scores for two-teacher OPD and Flow-Map GRPO with equal OCR/PickScore reward weights. Dashed lines denote the task specialists. The shared starting marker is a visual reference.

Figure 8: Consistency (solid, left) versus teacher-map error.(dashed, right) across

\lambda_{c}

$\lambda_{c}$	GenEval $\uparrow$	OCR $\uparrow$	PickScore $\uparrow$
0	0.827	0.830	22.8533
0.001	0.838	0.845	22.9794
0.01	0.725	0.806	22.6529
0.1	0.568	0.646	22.0283
1	0.450	0.340	21.5939

Table 3: MeanFlow consistency sweep at update 200.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Native family	Source interface	Destination interface	Flow-map interface
MeanFlow, two-time map	Self-induced or teacher-directed source predictor; instantaneous-velocity supervision with consistency is also available	Learned destination derivative at the student proposal	Direct flow-map regression or local-anchor Gaussian comparison
CM, endpoint plus re-noising	Endpoint source tangent with declared teacher-velocity closure	Unavailable natively: no learned destination-time derivative	Endpoint-anchor Gaussian comparison or endpoint regression

Appendix

Table 4: Supervision choices supported by each native parameterization.

Figure 9: All twelve supervision configurations for each of the three teacher rewards, under four-step (top) and five-step (bottom) generation. Markers are recorded evaluations, dashed horizontal lines are the frozen teachers, and the dotted vertical line marks the common epoch-300 comparison. MMD FID uses a logarithmic scale; classifier log-probability uses a symmetric logarithmic scale with a linear region between

-1

and

1

, retaining unstable configurations. Curves end at the last available evaluation.

	MMD: FID $\downarrow$		Classifier: $\log p_{y}\uparrow$		DINO: Cosine $\uparrow$
Supervision	4 steps	5 steps	4 steps	5 steps	4 steps	5 steps
B/2 base	22.68	22.72	-0.5448	-0.5451	0.7207	0.7215
XL/2 teacher	14.68	14.73	-0.4526	-0.4500	0.8084	0.8096
Stochastic map, $\lambda=0.1$	29.15	28.70	-0.9623	-0.9474	0.7004	0.7040
Stochastic map, $\lambda=0.2$	29.15	28.70	-0.9621	-0.9472	0.7003	0.7040
Stochastic map, $\lambda=0.4$	29.15	28.70	-0.9619	-0.9469	0.7001	0.7037

Appendix

Table 5: Four- and five-step reward-transfer results at epoch 300. Each reward setting reports its corresponding metric. Best student results in each column are bold.

Supervision	Epoch	4 steps	5 steps	8 steps	16 steps
B/2 base	0	22.48	22.72	22.88	22.87
Velocity, $\lambda_{c}=0$	300	23.58	21.44	19.29	19.17
Velocity, $\lambda_{c}=0.01$	300	18.14	18.33	18.33	18.56
Velocity, $\lambda_{c}=0.1$	300	21.59	21.01	20.42	20.41
Velocity, $\lambda_{c}=1$	300	41.40	40.33	37.19	34.74
Velocity, $\lambda_{c}=10$	300	395.48	392.19	388.29	386.52

Appendix

Table 6: Fixed epoch-300 checkpoints under the fixed-tail deployment sweep. Lower test FID is better; best student results are bold.

Supervision	Four-step FID $\downarrow$	Five-step FID $\downarrow$
Stochastic endpoint map, $\lambda=0.1$	41.01	39.72
Stochastic endpoint map, $\lambda=0.2$	40.91	39.67
Stochastic endpoint map, $\lambda=0.4$	40.66	39.28
Stochastic endpoint map, $\lambda=0.8$	40.34	38.78
Stochastic endpoint map, $\lambda=1.0$	40.12	38.54
Deterministic endpoint map	40.96	39.68

Appendix

Table 7: Supplementary CM distillation at epoch 150. We report FID under four- and five-step generation; lower is better.

$\lambda_{c}$	Source residual	Velocity mismatch	Average-velocity mismatch	Map composition error
0	0.800	0.017	0.031	0.003391
0.001	0.649	0.020	0.032	0.003148
0.01	0.763	0.053	0.043	0.003504
0.1	0.401	0.088	0.060	0.002558
1	0.179	0.171	0.090	0.001996

Appendix

Table 8: Consistency and teacher-matching errors after 200 training steps, averaged over the preceding 20 steps at the last supervised interval. Velocity errors are mean squared differences from the teacher; map-composition error measures agreement between direct and composed student maps.

Figure 10: Complementary specialist capabilities in one student. Rows compare text rendering, visual preferences and object relations. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher and the same student.

Figure 11: Cross-task specialist–student comparison. Rows cover OCR, visual preferences and compositional generation. Each row compares the base, three specialist teachers and the same student under a shared prompt, initial noise and few-step sampler.

Figure 12: Cross-task specialist–student comparison. Rows cover OCR, visual preferences and compositional generation. Each row compares the base, three specialist teachers and the same student under a shared prompt, initial noise and few-step sampler.

Figure 13: Paired OCR examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Figure 14: Paired OCR examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Figure 15: Paired PickScore examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Figure 16: Paired PickScore examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Figure 17: Paired GenEval examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Figure 18: Paired GenEval examples. Columns show the base, OCR teacher, PickScore teacher, GenEval teacher, and the same student. Each row shares the prompt, initial noise and few-step sampler.

Explore similar work

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
Aug 27, 2026Shiyi Zhang, Mushui Liu, Yunze Tong +8Uni-Opd Teacher
1Tsinghua University · 2Zhejiang University · 3Alibaba Group
Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging
Aug 4, 2026Siming Fu, Zheming Fu, Ruizhe He +7Rp-Opsd Conditional Variational Autoencoder
1Joy Future Academy · 2Zhejiang University
Distribution Matching Distillation without Fake Score Network
May 19, 2026Youngjoong Kim, Deokyeong Lee, Jaesik ParkDistribution Matching Distillation Dataset Distillation
Department of Computer Science and Engineering, Seoul National University · Department of Computer Science and Engineering, Sogang University Seoul, Republic of Korea

Aug 27, 2026, cs.CVJ/K move · Enter open · S save

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher

Shiyi Zhang, Mushui Liu, Yunze Tong, Wanggui He+7

1Tsinghua University · 2Zhejiang University · 3Alibaba Group

Uni-Opd Teacher

Open paper AskPDF arXiv

On-policy distillation (OPD), which leverages a pre-trained, specialized teacher model to provide dense supervisory signals, has achieved significant success in Large Language Models (LLMs) and has recently been adapted to flow matching models. However, this paradigm suffers from two major issues: First, training a separate, task-specific teacher for every new objective incurs high computational costs. Second, the discrepancy between teacher and student distributions often leads to compounding errors along the generation trajectory. In this paper, we introduce \textbf{Self-OPD}, a teacher-free OPD framework for flow matching models that turns the student's own self-exploration into step-wise supervision. At each timestep, Self-OPD branches the deterministic next-state prediction into

K

stochastic SDE candidates, rolls them out with the ODE sampler, and compares their rewards against a deterministic self-reference baseline to obtain normalized advantages. The velocity field is optimized with an all-branch pull-push objective, where high-advantage branches attract the student and low-advantage branches repel it under direction-aware attenuation and SDE-variance normalization. For multi-objective alignment, Self-OPD fuses normalized scores at the reward level, avoiding direct gradient conflict. Experiments on single and mixed reward benchmarks show that Self-OPD outperforms prior RL and OPD methods without task-specific teachers.

Abstract

Explore similar work

Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher

Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging

Distribution Matching Distillation without Fake Score Network