Few-step generation with pretrained diffusion and flow models can be accelerated by lightweight training that optimizes the sampling trajectory rather than the network. A recent approach parameterizes the stochastic interpolant (SI) scheduler as a smooth curve whose control points enforce the three properties an SI scheduler must satisfy: fixed boundary conditions, a monotone signal-to-noise ratio (SNR), and differentiability. Existing parameterizations use globally supported polynomial bases, where every control point moves the whole curve and higher expressiveness needs a higher degree, which couples distant regions of the schedule during optimization. We introduce I-SplineFlow, which parameterizes the scheduler with integrated monotone splines (I-splines). I-splines decouple the polynomial degree from the number of mixture weights, so support width and smoothness can be chosen per model at a fixed weight count, and the compactly supported derivative basis makes the scheduler Jacobian orders of magnitude better conditioned than a Bézier basis. Boundary conditions and a strictly monotone SNR hold by construction, with no ordering constraint on the parameters and closed-form velocity derivatives. Across diffusion (EDM) and flow (ReFlow, Simple ReFlow) models, I-SplineFlow improves few-step FID over Bézier scheduling in most settings, most clearly at the lowest NFEs, and trains in minutes. Ablations show that both the degree freedom and the monotonicity constraint are needed. The code will be released upon acceptance.
Figures & tables
Figure 2: FID against I-spline degree for iPNDM (left) and UniPC (right) at NFE 4 – 10 , extending Table 2 to p=31 . Each curve dips at an interior degree and rises toward the global limit.
Figure 3: Training cost is set by the sampling budget, not the degree. Across p∈{4,…,31} the per-configuration wall-clock time varies by under 5% ( ≤8 minutes), while it grows about 2.2× from NFE 4 to 10 .
Figure 4: Local support yields sparse, banded parameter coupling. Top: derivative sensitivity ∣∂αˉ˙(s)/∂θi∣ over parameter index i and time s , per row normalized; the I-spline basis is nonzero on only a few knot spans, while the Bernstein derivative has a broader footprint. Bottom: parameter coupling ∣⟨Ji,Jj⟩∣/(∥Ji∥∥Jj∥) , banded for I-SplineFlow (bandwidth 1.34 ) and broad for BézierFlow ( 3.52 ). Degree-16 operating point.
Figure 5: Local support yields a better-conditioned scheduler. The left panel plots the effective condition number κeff=σ1/σK−1 of ∂αˉ/∂θ against I-spline degree, at initialization and at the trained checkpoint, with BézierFlow (degree 31 ) as the horizontal reference; κeff grows with degree toward the global Bernstein limit and is about five orders of magnitude smaller than BézierFlow at the operating degree, and it is nearly identical at initialization and after training. The shaded band marks the FID-optimal degree range of Table 2 . The right panel plots the normalized singular-value spectra of the same Jacobian: a low-degree spline keeps a full spectrum, while high degree and the Bernstein basis collapse toward rank deficiency.
Figure 6: Better conditioning gives faster, lower convergence. Validation LPIPS (running best) against training epoch at a matched budget, same K=32 , same initialization, and same teacher data. I-SplineFlow reaches BézierFlow’s fully trained value about three epochs earlier and settles 6 – 15% lower in both featured few-step settings.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Architecture Diagram of I-SplineFlow
Figure 7
Figure 9: FID against I-spline degree on CIFAR-10 with ReFlow, K=32 , for RK1 (top) and RK2 (bottom) at NFE 4 – 10 . Degree 3 is best at the higher budgets (RK1 NFE =10 , RK2 NFE =8 and 10 ), where the near-straight ReFlow trajectory leaves little curvature to fit, while NFE =4 prefers p=10 on both solvers and quality falls off toward the global limit p=31 . The best degree therefore depends on the model and the budget, and only the I-spline can choose it: BézierFlow is fixed at p=K−1=31 , the worst setting in six of eight columns (Contribution 1) .
Figure 10: The x axis is the fraction of the grid where the B-spline’s SNR is decreasing, ν=∣{s:ρˉ′(s)<0}∣/G ; the I-spline has ν=0 always (Theorem 1 ) . (a) ν against absolute FID: the more monotonicity the B-spline loses, the worse its FID. (b) ν against the FID gap to the I-spline at the same solver and NFE: where the B-spline stays monotone it matches the I-spline, and where it loses monotonicity it degrades, by up to 11 FID. This is why the I-spline’s built-in monotonicity gives better FID.
Flow Matching (FM) constructs linear conditional probability paths, but the learned marginal velocity field inevitably exhibits strong curvature due to trajectory superposition. This curvature severely inflates numerical truncation errors, bottlenecking few-step sampling. To overcome this, we introduce Isokinetic Flow Matching (Iso-FM), a lightweight, Jacobian-free dynamical regularizer that directly penalizes pathwise acceleration. By using a self-guided finite-difference approximation of the material derivative Dv/Dt, Iso-FM enforces local velocity consistency without requiring auxiliary encoders or expensive second-order autodifferentiation. Operating as a pure plug-and-play addition to single-stage FM training, Iso-FM dramatically improves few-step generation. On CIFAR-10 (DiT-S/2), Iso-FM slashes conditional non-OT FID at 2 steps from 78.82 to 27.13 - a 2.9x relative efficiency gain - and reaches a best-observed FID at 4 steps of 10.23. These results firmly establish acceleration regularization as a principled, compute-efficient mechanism for fast generative sampling.
We present FlowLM, a flow matching language model transformed from pre-trained diffusion language models via efficient fine-tuning. By re-aligning the curved sampling trajectories of diffusion models into straight-line flows, FlowLM enables high quality few-step generation that rivals or even outperforms the quality of 2,000-step diffusion sampling with very few training epochs. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution. Empirical results demonstrate that our approach is highly effective for high-quality, few-step text generation.
Modern diffusion and flow-based models are increasingly moving toward few-step, latent-free generation to bypass the computational overhead of multi-step sampling and the reconstruction bottlenecks of external autoencoders. We propose the Interval Denoiser, a theoretically rigorous framework for latent-free generation. Derived directly from the flow matching ODE, it establishes an exact analytical mapping for intermediate trajectory states. Unlike prior formulations, our prediction is shown to reside on a low-dimensional manifold across any time interval, making the regression tractable for a network operating directly on pixels. Furthermore, by avoiding empirical algebraic substitutions, our formulation correctly isolates the pure time derivative to prevent biased gradient evaluations and ensure exact first-order optimization. By analyzing this objective, we equip our framework with residual clipping and a time-sampling curriculum, enabling effective long-interval training and improving few-step performance. Trained from scratch on ImageNet 256x256, our model achieves an FID of 4.55 in one step (1-NFE) and 3.98 in two steps (2-NFE) without perceptual losses.
Alexander Zaytsev, Dmitry Baranchuk, Alexander Korotin +1
HSE University · Yandex Research · Applied AI Institute +2