Discriminative Flow Matching: Beyond Time-Conditioning in Generative Restoration via Flow-State Representations

Shrishti Saha Shetu; Emanuël A. P. Habets; Andreas Brendel

eess.ASSep 3, 2026

Discriminative Flow Matching: Beyond Time-Conditioning in Generative Restoration via Flow-State Representations

Authors: Shrishti Saha Shetu, Emanuël A. P. Habets, Andreas Brendel

Organizations: International Audio Laboratories Erlangen∗ Am Wolfsmantel 33, 91058 Erlangen, Germany · Fraunhofer IIS, Am Wolfsmantel 33, 91058 Erlangen, Germany

Abstract

Existing Conditional Flow Matching (CFM) formulations describe transport progress using an explicit interpolation coordinate, commonly interpreted as time, assuming that a single global variable adequately represents a sample's position along the generative trajectory. In restoration tasks, however, transport progress is sample-dependent because the initial distribution may exhibit varying statistical dependencies with the target distribution. Thus, samples at the same interpolation coordinate can differ substantially in degradation level, distance to the target distribution, and restoration difficulty. We investigate whether signal representations learned by discriminatively trained models provide a meaningful description of generative transport state in CFM-based restoration. Through systematic latent-space analysis, we show that discriminative representations organize according to degradation severity and follow a consistent trajectory toward the clean-data manifold during generation. Motivated by these observations, we introduce the Discriminative Flow-State Hypothesis, which posits that discriminative representations encode a transport state governing generative restoration. Based on this hypothesis, we propose Discriminative Flow Matching, which conditions the Flow-Matching velocity field on Discriminative Flow-State Representations rather than explicit time coordinates. Experiments on speech enhancement and image denoising show that these representations characterize restoration progress, enable adaptive inference, and consistently outperform CFM and diffusion-related baselines. Our findings suggest that discriminative representations provide an effective state-aware alternative to explicit time conditioning and offer a novel perspective on the relationship between discriminative and CFM-based generative modeling.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of conditioning strategies for Flow-based restoration tasks. (a) CFM conditions the velocity field on $t$ . (b) Autonomous Rectified Flow (ARF) removes $t$ . (c) CFM+DL conditions CFM with the additional discriminative latent (DL) representation $z_{t}=\mathcal{E}_{\phi}(x_{t})$ . (d) DFM replaces $t$ with the Discriminative Flow-State Representation $z=\mathcal{E}_{\phi}(x_{\zeta}).$

Figure 2: t-SNE visualization of DCCRN latent representations under (a) rain and (b) machinery noise.

Panel of Figure 3 — Figure 3: Prediction of observable Flow-State quantities from either the DFSR or the anchor-based signal pair.

Target	Method	MAE $\downarrow$	RMSE $\downarrow$	Pearson $\uparrow$
iSNR	Latent	0.032	0.047	0.968
	Anchor	0.035	0.048	0.971
gSNR	Latent	0.117	0.154	0.491
	Anchor	0.142	0.181	0.301
$\zeta$	Latent	0.154	0.190	0.741
	Anchor	0.155	0.211	0.751

Table 1: Prediction performance for quantities related to the Flow-State. Best results are shown in bold .

Figure 4: Evolution of signal-space and latent-space Flow-States over NFE.

Cat.	Method	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )	SCOREQ Ref. ( $\downarrow$ )	SCOREQ Non-Ref. ( $\uparrow$ )
Unproc.	Noisy	9.06	1.58	0.93	2.78
	Clean	–	–	0.00	4.60
Disc.	DCCRN	17.36	2.91	0.31	4.15
	GCRN	16.71	2.63	0.42	3.87
Diff.	SGMSE+	16.86	2.81	0.29	4.15
	BBED	19.10	2.81	0.26	4.38

Table 2: SE results on the DNS Challenge synthetic non-reverberant test set.

Method	P808 MOS ( $\uparrow$ )	SIG ( $\uparrow$ )	BAK ( $\uparrow$ )	OVRL ( $\uparrow$ )
Noisy	3.05	3.05	2.51	2.26
RVAE	3.29	3.16	2.91	2.44
CDiffuse	3.14	3.15	3.19	2.55
MetricGAN+	3.26	2.88	3.39	2.45
DisCoGAN	3.65	3.32	3.91	2.98
SGMSE	3.38	3.22	3.02	2.52

Table 3: Generalization results on the DNS Real Recordings test set ( Reddy et al. 2020 ) . Baseline results are taken from their respective original publications.

Method	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )	SCOREQ Ref. ( $\downarrow$ )	SCOREQ Non-Ref. ( $\uparrow$ )
Noisy	9.06	1.58	0.93	2.78
Clean	–	–	0	4.60
CFM	18.74	2.78	0.27	4.32
ARF	18.83	2.79	0.26	4.38
CFM+DL	18.96	2.75	0.26	4.39
DFM (Ours)	19.17	2.88	0.24	4.44

Table 4: Results on the DNS Challenge non-reverberant test set under the data-prediction training objective.

Figure 5: Performance for DFM with adaptive inference.

Figure 6: Complexity-performance trade-off (baselines shown in red, proposed methods in green)

Figure 7: Image denoising performance relative to CFM .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A8: Relative kNN and MMD distances between noisy latent representations and the clean latent set across different SNR levels. Both measures decrease monotonically with increasing SNR, indicating that intermediate samples progressively approach the clean latent representations.

Metric	Value
Pearson ( $r$ )	$-0.9645$
Spearman ( $\rho$ )	$-0.9832$
Monotonicity violations	$0/8$

Appendix

Table A5: Monotonic relationship between SNR and latent proximity.

Category	Method	Conditioning	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )
Unprocessed	Noisy	–	9.06	1.58
Discriminative	DCCRN	NA	17.36	2.91
	GCRN	NA	16.71	2.63
	TaylorSENet	NA	16.59	2.65
	ULCNet	NA	16.70	2.56
CFM	FlowSE	$t$	18.99	2.86

Appendix

Table A6: DFM results on the DNS Challenge synthetic non-reverberant test set for utilizing discriminative features from different discriminatively trained SE methods.

Input	STFT Match	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )
$\mu_{\zeta}$	No	19.48	2.91
$\mu_{\zeta}$	Yes	19.87	2.94
$x_{\zeta}$	No	19.74	2.89
$x_{\zeta}$	Yes	19.86	2.96
$x_{0}$	No	19.55	2.92
$x_{0}$	Yes	19.63	2.96

Appendix

Table A7: Ablation study of DFM on the DNS Challenge synthetic non-reverberant test set where the discriminative model used for conditioning is trained with different inputs and STFT configurations.

Method	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )	SCOREQ Ref. ( $\downarrow$ )	SCOREQ Non-Ref. ( $\uparrow$ )
Noisy	9.06	1.58	0.93	2.78
Clean	–	–	0	4.60
CFM	6.81	1.31	0.76	3.2
ARF	6.86	1.31	0.7	3.35
DFM (Ours)	7.20	1.34	0.67	3.46

Appendix

Table A8: Results on the DNS Challenge non-reverberant test set without using the noisy signal

x_{0}

for conditioning the velocity network.

Method	SI-SDR ( $\uparrow$ )	PESQ ( $\uparrow$ )	SCOREQ Ref. ( $\downarrow$ )	SCOREQ Non-Ref. ( $\uparrow$ )
Noisy	-31.55	1.32	0.84	2.99
Clean	-		0	4.60
DCCRN	-29.06	1.91	0.87	2.73
CFM	-27.64	2.35	0.59	3.45
DFM (Ours)	-27.19	2.44	0.58	3.47

Appendix

Table A9: Results on the DNS Challenge reverberant test set for a speech dereverberation task.

Panel of Figure 18 — Figure A9: Architecture of the MNIST and Fashion-MNIST Image Classifier.

Figure A10: Semantic latent structure and latent restoration geometry.

Panel of Figure 20 — Figure A11: Architecture of the velocity network for image denoising task.

Figure A12: Adaptive inference with uniform Euler solver for image denoising on the MNIST dataset.

Figure A13: PESQ and SI-SDR improvements along with their respective $95\%$ confidence intervals for the compared methods presented in Table 2 .

Baseline Model	PESQ		SI-SDR (dB)
	Baseline Mean	$p$ -value	Baseline Mean	$p$ -value
DCCRN	$2.91$	$0.300$	$17.36$	$<0.001$
GCRN	$2.63$	$<0.001$	$16.71$	$<0.001$
BBED	$2.81$	$<0.001$	$19.10$	$<0.001$
SGMSE+	$2.81$	$<0.001$	$16.86$	$<0.001$
SToRM	$2.70$	$<0.001$	$17.56$	$<0.001$

Appendix

Table A10: Statistical significance test results (paired Wilcoxon signed-rank test) comparing our proposed DFM method (PESQ:

2.96

, SI-SDR:

19.63\text{ dB}

) against various baseline models.

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 656 papers

Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement
Jun 23, 2026Wangyi Pu, Michele ScarpinitiFlow Matching Speech Enhancement
Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Italy
Representation Learning via Composite Subspace Flow Matching
Oct 23, 2026José Afonso, Mário FigueiredoSelf-Supervised Visual Representation Learning Flow Matching
Instituto Superior Técnico, Lisboa, Portugal · Instituto de Telecomunicações, Instituto Superior Técnico, Lisboa, Portugal
Dino Forcing Flow Models: Do not denoise what you can predict
Oct 8, 2026Arijit Ghosh, Lucas Degeorge, Paul Couairon +3Self-Conditioning Flow-Based Generative Modeling
ENPC, IP Paris · Ecole Polytechnique, IP Paris · AMIAD +1

Jun 23, 2026J/K move · Enter open · S save

Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement

Wangyi Pu, Michele Scarpiniti

Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome, Italy

Flow Matching Speech Enhancement Flow-Based Generative Modeling Latent Space Alignment

Open paper AskPDF arXiv

Generative models, particularly diffusion and score-based approaches, have recently achieved strong performance in speech enhancement, but their iterative sampling process limits real-time deployment. Flow Matching offers an efficient alternative by transporting noisy speech toward clean speech through an ordinary differential equation with few function evaluations. In this work, we propose a skip-free encoder-decoder backbone for flow-matching speech enhancement, guided by Latent Representation Alignment (LRA). Instead of relying on U-Net skip connections, which may transfer noise-correlated low-level features to the decoder, the proposed model aligns its bottleneck and decoder representations with clean latent features extracted from a frozen Descript Audio Codec encoder-decoder without quantization. This codec-aligned supervision promotes compact clean-speech representations while preserving efficient few-step inference. Experiments on WSJ0-CHiME3 and VoiceBank-DEMAND show improved PESQ and perceptual quality, especially on VoiceBank-DEMAND, using only five function evaluations.

Abstract

Flow Matching

Explore similar work

Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement

Representation Learning via Composite Subspace Flow Matching

Dino Forcing Flow Models: Do not denoise what you can predict