Conditional generative modeling with limited target data can benefit from representations learned in a related source population. We develop a transfer learning framework for conditional flow matching that accommodates distinct source and target response distributions and dimensions. Two procedures exploit a representation learned from source data: TFM-Freeze estimates a target velocity head with the representation fixed, whereas TFM-Adapt jointly learns the head and a representation adaptation under a functional anchoring penalty. For explicit neural network classes, we establish bounds on conditional squared Wasserstein risk that account for source estimation, target fitting, representation mismatch, and numerical integration. When the source representation admits a smooth correction, adaptation replaces the frozen mismatch contribution by the cost of learning the correction and a controllable regularization bias. An information lower bound identifies regimes in which adaptation attains a strictly faster conditional squared Wasserstein risk rate than any generator restricted to the frozen context information. We also develop independent validation procedure for selecting among fitted conditional generators. We show that the selection procedure satisfies an oracle inequality, and the switching rule guarantees, with high probability, no increase in conditional energy risk relative to a target-only generator. Simulations and real data applications illustrate the gains from our proposed methods and the effectiveness of target validation in guarding against negative transfer.
Figures & tables
Figure 1: Mean energy discrepancy against source sample size, with nt=1024 . Columns correspond to related and unrelated representations; rows correspond to (p,d)=(50,1),(200,5),(500,10) . Error bars are pointwise 95% Monte Carlo intervals from 100 repetitions. +TV denotes target validation.
Figure 2: Mean joint empirical Wasserstein-1 distance against source sample size, with nt=1024 . Panel arrangement, methods, and intervals are as in Figure 1 .
Figure 3: Mean joint empirical Wasserstein-2 distance against source sample size, with nt=1024 . Panel arrangement, methods, and intervals are as in Figure 1 .
Materials
Diamonds
Method
CRPS
Harm
CRPS
Harm
Target-only diffusion
0.31245
—
0.07064
—
Target-only FM
0.31826
—
0.06914
—
Tian–Shen
0.41195
40/40
0.06959
11/40
TFM-Freeze
0.41191
40/40
0.06598
2/40
TFM-Adapt
0.35067
35/40
0.06357
0/40
Table 1: Mean test CRPS and harmful-repetition counts for Materials and Diamonds, using 40 paired repetitions. Harm compares TFM procedures with target-only FM and Tian–Shen with target-only diffusion; exact fallback ties are excluded. Bold type marks the lowest mean CRPS in each application.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Retained
Unrelated
d
Procedure
Related
Unrelated
Harm
1
TFM-Adapt
49
0
100 / 0
1
TFM-Adapt ( λ=0 )
51
0
100 / 0
5
TFM-Adapt
100
0
100 / 0
5
TFM-Adapt ( λ=0 )
100
0
100 / 0
10
TFM-Adapt
98
0
94 / 0
Appendix
Table 2: Target validation at ns=8000 in the simulations. Retained is the number of repetitions selecting transfer. Harm gives the count before / after validation in the unrelated regime, based on energy discrepancy. All counts are out of 100.
Comparator
Difference
Paired 95% interval
TFM-Adapt
Target-only diffusion
−0.00706
[−0.00785,−0.00628]
Target-only FM
−0.00557
[−0.00638,−0.00475]
Tian–Shen
−0.00602
[−0.00672,−0.00532]
TFM-Freeze
−0.00241
[−0.00299,−0.00183]
TFM-Freeze + TV
−0.00248
[−0.00308,−0.00188]
Appendix
Table 3: Paired CRPS comparisons on Diamonds. Differences are adaptive procedure minus comparator. Intervals are pointwise 95% paired Student- t intervals across 40 training repetitions.
We introduce a transfer learning framework for regression that leverages heterogeneous source domains to improve predictive performance in a data-scarce target domain. Our approach fits a conditional generative model to each source domain and calibrates a shared linear combination of the generated responses via conditional quantile matching at the observed target covariates. The calibration aligns the conditional response distributions while accommodating simultaneous shifts in both the covariate and conditional response distributions. The calibrated responses then serve as pseudo-labels to augment the target data for a broad class of downstream learning tasks. Theoretically, we derive an excess risk bound for an empirical risk minimizer (ERM) trained on the augmented data, separating the benefit of an enlarged training sample from errors due to density ratio estimation, source generator estimation, conditional calibration, and irreducible transfer bias. Empirically, simulations and real-data applications show that the proposed method can improve target-domain prediction when the transfer bias is small.
Yikun Zhang, Steven Wilkins-Reeves, Wesley Lee +1
Department of Statistics, University of Washington, Seattle, USA · Central Applied Science, Meta, Menlo Park, USA
We propose Coreset-Induced Conditional Velocity Flow Matching (CCVFM), a generative model that augments hierarchical rectified flow with a data-informed source distribution. Standard flow matching regresses only the conditional mean of the velocity; hierarchical flow matching instead models the full conditional velocity law with a second, inner flow in velocity space, but this inner flow is asked to transport isotropic Gaussian noise to a multimodal target velocity distribution from scratch. Our key observation is that this inner source can be replaced by a closed-form surrogate built from a coreset of the target. CCVFM first compresses the target into weighted atoms using an entropic Sinkhorn coreset and lifts them to a Gaussian mixture. The induced conditional velocity law is then a closed-form Gaussian mixture that can be sampled without a learned neural sampler. A lightweight correction flow, trained from this surrogate source, then refines the remaining surrogate-to-target residual rather than learning an entire noise-to-data map. We prove that the surrogate transport cost equals the target-surrogate Wasserstein gap under an explicit compression assumption, whereas the noise-source analogue has a dimension-scale lower bound. We further characterize the conditional second moment of the training target when the source velocity is drawn from the surrogate law independently of the target velocity, and show that the part of it that depends on the source is small when the surrogate conditional law is close to the true conditional velocity law in mean and covariance. Empirically, on MNIST, CIFAR-10, ImageNet-32, and CelebA-HQ, the proposed method reaches competitive few-step generation under matched architectures. Code is available at https://github.com/ziwaa-se/CCVFM.
Flow matching has emerged as a promising alternative to diffusion-based generative modeling, offering a continuous-time ordinary differential equation formulation with fast sampling and a simple regression-based training objective. Despite its empirical success, some theoretical analyses idealize the learning procedure by assuming that the velocity field is obtained by exact empirical risk minimization. This assumption does not reflect the practical finite-dataset setting, where the empirical flow-matching loss is only approximately minimized by a finite number of stochastic-gradient updates. We develop an empirical-risk-minimizer-free analysis for flow matching that explicitly accounts for both finite-sample statistical error and finite-time empirical optimization error. Our analysis considers a finite dataset of N flow-matching samples and a learned parameter θT obtained after T stochastic gradient updates on the empirical objective. Under regularity assumptions on the interpolation path, sub-exponential source and target distributions, approximation by a bounded-parameter smooth neural velocity class, empirical-loss optimization, and uniform flow stability, we prove a guarantee that holds with high probability over the finite training dataset and in expectation over the stochastic-gradient randomness. The resulting bound separates approximation error, statistical error from the finite dataset, tail error from unbounded source and target distributions, and empirical optimization error from approximate minimization of the finite-sample objective.
Mudit Gaur, Prashant Trivedi, Shuchin Aeron +3
Department of Electrical and Computer Engineering, Ohio State Univeristy · Department of Computer Science and Information Systems, BITS Pilani · Department of Electrical and Computer Engineering, Tufts University +2