FlowGRPO for Vocal Separation: From Reward Analysis to Reward Design

Lishi Zuo; Chong-Xin Gan; Zezhong Jin; Youzhi Tu; Man-Wai Mak; KongAik Lee

cs.SDOct 11, 2026

FlowGRPO for Vocal Separation: From Reward Analysis to Reward Design

Authors: Lishi Zuo, Chong-Xin Gan, Zezhong Jin, Youzhi Tu, Man-Wai Mak, KongAik Lee

Organizations: Dept. of Electrical and Electronic Engineering, Hong Kong Polytechnic University, Hong Kong SAR, China

Abstract

The Flow Matching objective supervises the velocity field at intermediate states, whereas waveform metrics are evaluated after decoding and may be non-differentiable. To align Flow Matching models with waveform-level and perceptual quality objectives during post-training, FlowGRPO optimizes their trajectories using endpoint rewards. In this work, we focus on music separation and use multi-resolution short-time Fourier transform (MRSTFT), scale-invariant signal-to-distortion ratio (SI-SDR), and Virtual Speech Quality Objective Listener (ViSQOL) as reward components. Before policy optimization, we analyze the relationship between data quality factors and the within-group gap between the maximum and mean rollout rewards for each component. This analysis motivates a revised ViSQOL reward that matches channel-wise root-mean-square (RMS) levels and evaluates the interval from the first to the last active frame, retaining inactive frames within that interval. We also apply a sample-quality filter based on predefined waveform checks. Together, the revised reward and quality filter enable FlowGRPO to improve the pretrained vocal separator in both separation performance and perceptual quality.

Figures & tables

Factor	$R_{\mathrm{MRSTFT}}$	$R_{\mathrm{SI\text{-}SDR}}$	$R_{\mathrm{ViSQOL}}$
RMS level	$-0.005$	$\mathbf{-0.062}$	$\mathbf{-0.077}$
Activity ratio	$0.030$	$0.006$	$\mathbf{0.142}$
Edge-silence ratio	$-0.018$	$-0.015$	$\mathbf{-0.063}$
Stereo RMS gap	$-0.048$	$-0.012$	$\mathbf{-0.107}$
Channel onset lag	$-0.016$	$-0.029$	$-0.008$

Table 1: Pre-training analysis of the within-group gap on

1500

training mixtures. Entries are standardized regression coefficients

\alpha_{q,m}

. Bold denotes

|\alpha_{q,m}|>0.05

Row	Stage	Reward components	Filter	ViSQOL $\uparrow$	SI-SDR $\uparrow$	MRSTFT $\downarrow$	SDR $\uparrow$	PESQ $\uparrow$
1	Pretrain	–	No	3.525	5.927	1.148	6.946	2.076
2	FlowGRPO	$R_{\mathrm{MRSTFT}}$	No	3.594	6.858	1.067	7.991	2.103
3	FlowGRPO	$R_{\mathrm{MRSTFT}}+R_{\mathrm{SI\text{-}SDR}}+R_{\mathrm{ViSQOL}_{\mathrm{act}}}$	No	3.591	6.687	1.108	7.887	2.079
4	FlowGRPO	$R_{\mathrm{MRSTFT}}+R_{\mathrm{SI\text{-}SDR}}+R_{\mathrm{ViSQOL}}$	Yes	3.641	6.900	1.098	8.244	2.066
5	FlowGRPO	$R_{\mathrm{MRSTFT}}+R_{\mathrm{SI\text{-}SDR}}+R_{\mathrm{ViSQOL}_{\mathrm{act}}}$	Yes	3.675	7.045	1.084	8.293	2.112

Table 2: Vocal-separation quality. The reward components are combined using Eqs. 11 and 7 . The ViSQOL column always reports the original metric.

Figure 1: Training rewards for Rows 3–5, plotted as the raw reward components before batch and group normalization. Panel (c) shows $R_{\mathrm{ViSQOL}}$ for Row 4 and $R_{\mathrm{ViSQOL}_{\mathrm{act}}}$ for Rows 3 and 5. Shading denotes $\pm 1$ standard error.

Music Source Separation

See all 20 papers

Decoupled Latent Flow Matching for Few-Step Joint Vocal-Accompaniment Separation
Aug 31, 2026Lishi Zuo, Youzhi Tu, Lu Yi +4Flow Matching Flow-Based Generative Modeling
Dept. of Electrical and Electronic Engineering, Hong Kong Polytechnic University, Hong Kong SAR, China
FLOMSS: Generative Music Source Separation by Flow Matching
Oct 13, 2026Shuyang Cui, Zhi Zhong, Keisuke Toyama +10Music Source Separation Audio Source Separation
Sony Group Corporation
Flow Matching-Based Speech Source Separation with Best-of-N Biometric Sampling
Jul 7, 2026Anastasia Zorkina, Alexandr Anikin, Nikita Khmelev +5Flow Matching Speech Processing
ITMO University, Speech Processing Group, Russia · Speech Technology Center Ltd., R&D department, Russia.

Aug 31, 2026J/K move · Enter open · S save

Decoupled Latent Flow Matching for Few-Step Joint Vocal-Accompaniment Separation

Lishi Zuo, Youzhi Tu, Lu Yi, Zezhong Jin+3

Dept. of Electrical and Electronic Engineering, Hong Kong Polytechnic University, Hong Kong SAR, China

Flow Matching Flow-Based Generative Modeling Audio Source Separation Conditional Generative Modeling Music Source Separation

Open paper AskPDF arXiv

Generative modeling provides a flexible way to model mixture-conditioned source distributions, but iterative diffusion and flow matching models are costly for long music signals. This paper studies joint vocal-accompaniment separation through latent flow matching, where a pretrained variational autoencoder (VAE) maps mixtures and sources into a compact latent space and a flow matching model generates vocal and accompaniment latents jointly. The proposed framework decouples semantic separation from acoustic velocity prediction through a Separation Encoder and a Velocity Decoder. To reduce sampling cost, we further apply latent adversarial post-training inspired by Flow2GAN for few-step generation. Experiments show that latent adversarial refinement can improve perceptual and separation metrics under a reduced sampling budget.

Abstract

Music Source Separation

Explore similar work

Decoupled Latent Flow Matching for Few-Step Joint Vocal-Accompaniment Separation

FLOMSS: Generative Music Source Separation by Flow Matching

Flow Matching-Based Speech Source Separation with Best-of-N Biometric Sampling