The Flow Matching objective supervises the velocity field at intermediate states, whereas waveform metrics are evaluated after decoding and may be non-differentiable. To align Flow Matching models with waveform-level and perceptual quality objectives during post-training, FlowGRPO optimizes their trajectories using endpoint rewards. In this work, we focus on music separation and use multi-resolution short-time Fourier transform (MRSTFT), scale-invariant signal-to-distortion ratio (SI-SDR), and Virtual Speech Quality Objective Listener (ViSQOL) as reward components. Before policy optimization, we analyze the relationship between data quality factors and the within-group gap between the maximum and mean rollout rewards for each component. This analysis motivates a revised ViSQOL reward that matches channel-wise root-mean-square (RMS) levels and evaluates the interval from the first to the last active frame, retaining inactive frames within that interval. We also apply a sample-quality filter based on predefined waveform checks. Together, the revised reward and quality filter enable FlowGRPO to improve the pretrained vocal separator in both separation performance and perceptual quality.
Figures & tables
Factor
RMRSTFT
RSI-SDR
RViSQOL
RMS level
−0.005
−0.062
−0.077
Activity ratio
0.030
0.006
0.142
Edge-silence ratio
−0.018
−0.015
−0.063
Stereo RMS gap
−0.048
−0.012
−0.107
Channel onset lag
−0.016
−0.029
−0.008
Table 1: Pre-training analysis of the within-group gap on 1500 training mixtures. Entries are standardized regression coefficients αq,m . Bold denotes ∣αq,m∣>0.05 .
Row
Stage
Reward components
Filter
ViSQOL ↑
SI-SDR ↑
MRSTFT ↓
SDR ↑
PESQ ↑
1
Pretrain
–
No
3.525
5.927
1.148
6.946
2.076
2
FlowGRPO
RMRSTFT
No
3.594
6.858
1.067
7.991
2.103
3
FlowGRPO
RMRSTFT+RSI-SDR+RViSQOLact
No
3.591
6.687
1.108
7.887
2.079
4
FlowGRPO
RMRSTFT+RSI-SDR+RViSQOL
Yes
3.641
6.900
1.098
8.244
2.066
5
FlowGRPO
RMRSTFT+RSI-SDR+RViSQOLact
Yes
3.675
7.045
1.084
8.293
2.112
Table 2: Vocal-separation quality. The reward components are combined using Eqs. 11 and 7 . The ViSQOL column always reports the original metric.
Figure 1: Training rewards for Rows 3–5, plotted as the raw reward components before batch and group normalization. Panel (c) shows RViSQOL for Row 4 and RViSQOLact for Rows 3 and 5. Shading denotes ±1 standard error.
Generative modeling provides a flexible way to model mixture-conditioned source distributions, but iterative diffusion and flow matching models are costly for long music signals. This paper studies joint vocal-accompaniment separation through latent flow matching, where a pretrained variational autoencoder (VAE) maps mixtures and sources into a compact latent space and a flow matching model generates vocal and accompaniment latents jointly. The proposed framework decouples semantic separation from acoustic velocity prediction through a Separation Encoder and a Velocity Decoder. To reduce sampling cost, we further apply latent adversarial post-training inspired by Flow2GAN for few-step generation. Experiments show that latent adversarial refinement can improve perceptual and separation metrics under a reduced sampling budget.
Lishi Zuo, Youzhi Tu, Lu Yi +4
Dept. of Electrical and Electronic Engineering, Hong Kong Polytechnic University, Hong Kong SAR, China
Music source separation (MSS) has been conventionally addressed by discriminative methods. Recently, flow matching has emerged as a strong generative paradigm for modeling complex data distributions, with successful applications across diverse domains. These advances motivate us to systematically investigate flow matching for MSS in a four-source (Vocals, Bass, Drums, Other) setting in full-band stereo. Specifically, we investigate prediction targets, flow paths, single-source versus multi-source modeling, and mixture consistency constraints. Beyond the signal-level metrics commonly used for MSS, such as signal-to-distortion ratio, we further evaluate MSS methods using embedding-based metrics, as well as a listening test, providing a comprehensive benchmark. On MUSDB18-HQ, our proposed model exhibits competitive performance in both objective and subjective tests, highlighting the potential of flow matching for generative MSS. Demo: https://ik4sumii.github.io/FLOMSS/
Single-channel speech separation remains challenging for real-world deployment due to source permutation ambiguity, sampling variability of generative models, and the difficulty of processing long recordings with chunk-wise inference. We address these issues with a conditional flow-matching-based method that produces an ordered two-source output conditioned on the mixture. A frozen speaker encoder defines the source order during training and is reused at inference for biometric best-of-N candidate selection and chunk-level channel alignment. We evaluate separation quality on Libri2Mix benchmark using SI-SDR, PESQ, and ESTOI, and measure downstream impact using cpWER for automatic speech recognition and EER for speaker verification. The results show that the proposed Transformer U-Net variant is competitive with strong baselines in objective separation metrics and achieves the lowest downstream automatic speech recognition and speaker verification error rates in all evaluated settings.
Anastasia Zorkina, Alexandr Anikin, Nikita Khmelev +5
ITMO University, Speech Processing Group, Russia · Speech Technology Center Ltd., R&D department, Russia.