cs.LGSep 26, 2026

Beyond the Manifold Hypothesis: Hybrid Spectral Parameterizations for Flow Matching

Authors: Ségolène Martin, Anne Gagneux, Quentin Bertrand, Rémi Emonet, Mathurin Massias

Organizations: Inria, ENS de Lyon, CNRS, Université Claude Bernard Lyon 1, LIP UMR 5668, 69342 Lyon Cedex 07, France · ENS de Lyon, CNRS, Université Claude Bernard Lyon 1, Inria, LIP UMR 5668, 69342 Lyon Cedex 07, France · Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Inria, Laboratoire Hubert Curien UMR 5516, F-42023 Saint-Étienne, France · Institut Universitaire de France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Flow matching and diffusion can be trained to predict different quantities, most commonly the data x1x_1, the source noise x0x_0, or the velocity~vv. Although theoretically equivalent, these can lead to substantially different performances. We identify two main drivers for these differences: the source--data signal-to-noise ratio, and the information bottleneck induced by the neural architecture. We show that, beyond intrinsic data dimension, the factor affecting the optimal parametrization the most is a certain signal-to-noise ratio in each data covariance direction. From this analysis, we introduce new spectral hybrid parameterizations that adapt across time and data covariance directions; we show that these are optimal for Gaussian data. We also show that architecture-induced compression changes which parameterization is easier to learn, with vv-prediction being more sensitive to discarded directions than x1x_1-prediction. Experiments across architectures and source scales show that our spectral parameterizations are robust across regimes, can substantially accelerate optimization, while incurring essentially no additional training cost compared with standard parameterizations.

Figures & tables

Figure 1: Disentangling ambient dimensionality and source–data SNR. True data are shown in orange and generated samples in blue . For all columns, x0∼N(0,σ02ID)x_{0}\sim{\mathcal{N}}(0,\sigma_{0}^{2}I_{D}) . Left: fixed σ0=1\sigma_{0}=1 ; as the dimension DD increases, the source–data SNR ρ\rho decreases. Center: the source-data SNR ρ=1\rho=1 is kept fixed (by adapting σ0\sigma_{0} ) as DD increases. Right: at a fixed D=256D=256 , the source–data SNR varies as ρ∈{0.01,1,100}\rho\in\{0.01,1,100\} , by using σ0∈{0.6,0.06,0.006}\sigma_{0}\in\{0.6,0.06,0.006\} . The conclusions of Li and He, (2025) (left panel) no longer hold when the SNR is kept fixed (middle panel). The right panel identifies the SNR as a critical factor .
Figure 2: Validation of the theoretical switching time on the anisotropic Gaussian model, for ρ=1\rho=1 ( σ0=1\sigma_{0}=1 ). Left: directional MSEs of x0x_{0} - and x1x_{1} -prediction for three selected directions uju_{j} (for x0x_{0} -pred, the curves are superposed). The crossing (black dot) occurs earlier in high-variance directions and later in low-variance directions. Right: For each spectral direction jj , we compare the theoretical switching time tjswitcht_{j}^{\mathrm{switch}} with the empirical transition time (black dot in left figure). The theory agrees strongly with empirical observation.
Figure 3: Joint effect of architectural compression and global SNR on CIFAR-10 generation. Increasing compression favors x1x_{1} -prediction, whereas increasing ρ\rho favors velocity prediction. Generated samples are shown in Figure 10 .
Figure 4
Figure 6: Generative performance (FID) for a ViT with patch size 44 and compression factor 22 . Left: On CelebA- 6464 , we compare x1x_{1} -pred, vv -pred, a fixed x1/x0x_{1}/x_{0} switch at t=1/2t=1/2 (uniform switch), and the binary and continuous spectral parameterizations. The two hybrid parameterizations YbinY^{\text{bin}} and YcontY^{\text{cont}} are robust to SNR variations and architectural compression, and in the worst case nearly match the best fixed prediction target. Right : FID throughout training on Bedroom- 6464 . Our spectral hybrid parameterizations converge substantially faster.
Figure 7: AFHQ- 256256 samples generated from the same fixed source sample throughout training, using a ViT- 1616 without bottleneck. The best fixed prediction target here is x1x_{1} -pred.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

 x1x_{1} -lossvv -lossx0x_{0} -loss
x1x_{1} -pred111(1−t)2\tfrac{1}{(1-t)^{2}}t2(1−t)2\tfrac{t^{2}}{(1-t)^{2}}
vv -pred(1−t)2(1-t)^{2}11t2t^{2}
x0x_{0} -pred(1−t)2t2\tfrac{(1-t)^{2}}{t^{2}}1t2\tfrac{1}{t^{2}}11
Appendix
Table 1: Combinations of prediction and loss. The network NθpN_{\theta}^{p} outputs p∈{x0,v,x1}p\in\{x_{0},v,x_{1}\} (row); the loss regresses against another (column), possibly after algebraic conversion. Every cell reduces to a weighted regression onto the network’s own output, Ex0,x1,t[w(t) ∥Nθp(xt,t)−p∥2]\mathbb{E}_{x_{0},x_{1},t}\big[w(t)\,\|N^{p}_{\theta}(x_{t},t)-p\|^{2}\big]
Figure 8: Validation of the theoretical switching time on the anisotropic Gaussian model, for ρ=100\rho=100 (i.e., σ02=0.01\sigma_{0}^{2}=0.01 ). Left: directional MSEs of x0x_{0} - and x1x_{1} -prediction for three selected directions uju_{j} . The crossing (black dot) occurs earlier in high-variance directions and later in low-variance directions. Right: For each spectral direction jj , we compare the theoretical switching time tjswitcht_{j}^{\mathrm{switch}} with the empirical transition time at which, in this direction, the denoising error of the x0x_{0} -prediction model becomes smaller than that of the x1x_{1} -prediction model (black dot in left figure). The theory agrees strongly with empirical observation.
Figure 9: Binary versus continuous spectral parameterizations for selected directions. Left: analytical coefficient aj(t)a_{j}(t) . Right: normalized coefficient αj(t)=taj(t)\alpha_{j}(t)=ta_{j}(t) , highlighting the hard binary switch and its smooth continuous relaxation.
Figure 10: Representative CIFAR-10 samples generated by a ViT with a patch bottleneck, for different source-to-data ratios ρ\rho and compression factors β\beta . Each panel shows the same generated sample index across the sweep, comparing x1x_{1} -prediction and vv -prediction.
Figure 11: Target-dependent spectral allocation under a rank- 4848 bottleneck. Top: directional residual Rt,jp\mathcal{R}_{t,j}^{p} across the data spectrum. Bottom: theoretical and learned retention of the corresponding PCA directions, measured by rjp(t)r^{p}_{j}(t) ( 48 ). The learned bottlenecks closely follow the allocation predicted by minimizing the residual ( 47 ) : x1x_{1} consistently favors the leading PCA directions, whereas velocity and the binary hybrid exhibit more time-dependent allocations.
Figure 12: Spectral allocation of a rank- 4848 bottleneck shared across time. The time-averaged theoretical allocation is compared with the retention measured from the learned bottleneck.
Figure 13: Patch-PCA retention of learned rank- 2424 bottlenecks in CIFAR-10 ViTs. The two prediction parameterizations learn different retained subspaces.
Figure 14: Generation with a fixed rank- 4848 time-averaged velocity-optimal bottleneck (i.e., the one in Figure 12 ) on non-Gaussian CIFAR-10 patches. The spectral hybrids again track the target covariance and reduce marginal error across discarded directions, showing that the effect is not specific to a top-PCA bottleneck.
Figure 15: FID on Bedroom- 6464 throughout training, for a U-Net (left) and a ViT with patch size 44 and compression factor 22 (right). The two spectral hybrid parameterizations, YbinY^{\mathrm{bin}} and YcontY^{\mathrm{cont}} , converge substantially faster toward their final FID than the fixed prediction targets.
Figure 16: AFHQ- 256256 samples generated from two fixed source samples throughout training using a ViT with patch size 1616 , token dimension 768768 , and no bottleneck. The same source samples are used at every checkpoint, allowing the evolution of individual generated images to be compared directly.
Figure 17: True data are shown in orange and generated samples in blue . Top: the SNR is fixed to ρ=1\rho=1 while the ambient dimension varies. Bottom: the ambient dimension is fixed to D=256D=256 while the source–data SNR varies; ρ∈{0.01,1,100}\rho\in\{0.01,1,100\} corresponds respectively to σ0∈{0.6,0.06,0.006}\sigma_{0}\in\{0.6,0.06,0.006\} . We observe that the ybiny^{\mathrm{bin}} parametrization achieves good performance in every scenario, whereas the yconty^{\mathrm{cont}} parametrization fails most of the time.
More in this topic

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 663 papers

Explore similar work

CardsList

  1. Training Flow Matching: The Role of Weighting and Parameterization

    Mar 6, 2026Anne Gagneux, Ségolène Martin, Rémi Gribonval +1Flow MatchingAdaptive Loss Weighting

    ENS de Lyon, CNRS, Universit´e Claude Bernard Lyon 1, Inria, LIP, UMR 5668, France · Technische Universität Berlin, Germany · Inria, ENS de Lyon, CNRS, Universit´e Claude Bernard Lyon 1, LIP, UMR 5668, France

  2. Preconditioned Flow Matching

    Mar 2, 2026Shadab Ahamed, Eshed Gal, Md Shahriar Rahim Siddiqui +3Flow MatchingFlow-Based Generative Modeling

    University of British Columbia, Vancouver, Canada · University of Cambridge, Cambridge, United Kingdom

  3. Spectral Edge Dynamics of Training Trajectories: Signal--Noise Geometry Across Scales

    Mar 14, 2026Yongzhong XuTransformer Training DynamicsNeural Network Training Dynamics