Prime-EVC: One-Step Expressive Emotional Voice Conversion via Emotion-Primed Flow Matching

Yu Pan; Xiaojuan Cai; Liang Zhang; Yang Hou; Tianjiao Zhang; Yuguang Yang; Jixun Yao; Hiroshi Koide

cs.SDOct 12, 2026

Prime-EVC: One-Step Expressive Emotional Voice Conversion via Emotion-Primed Flow Matching

Authors: Yu Pan, Xiaojuan Cai, Liang Zhang, Yang Hou, Tianjiao Zhang, Yuguang Yang, Jixun Yao, Hiroshi Koide,

Organizations: Kyushu University, Japan · Donghua University, China · National Institute of Informatics, Japan · Independent Researcher ·

Abstract

Recent diffusion- and conditional flow matching (CFM)-based emotional voice conversion (EVC) systems achieve strong conversion quality. However, they typically rely on multiple sampling steps, because the conditioning specifies which emotion to impose while its acoustic realization is left to the sampler. In this paper, we propose Prime-EVC, a one-step expressive EVC architecture based on emotion-primed flow matching. Our key idea is to reallocate the emotional conversion between the conditioning and the sampler: a FiLM-fused speaker-emotion style, an emotional variance adaptor, and direct emotion and spectro-temporal supervision assemble an emotion-rich conditioning upstream, rendering the residual conversion left to the CFM decoder attainable within one Euler step. Experiments on the ESD corpus demonstrate that single-step Prime-EVC surpasses five representative EVC systems on every quality metric and exceeds 4- and 100-step DurFlex-EVC in subjective naturalness and emotion while running 3.4--50 times faster than both.

Figures & tables

Method	nMOS $\uparrow$	eMOS $\uparrow$	UTMOS $\uparrow$	CER $\downarrow$	WER $\downarrow$	EECS $\uparrow$	SECS $\uparrow$	RTF $\downarrow$
GT	4.02 $\pm 0.03$	4.06 $\pm 0.02$	3.65	2.15	5.69	0.96	0.84	-
GT (vocoded)	3.96 $\pm 0.05$	4.01 $\pm 0.03$	3.42	2.39	5.85	0.95	0.84	-
StarGAN-EVC	2.05 $\pm 0.06$	2.11 $\pm 0.04$	1.39	49.77	72.38	0.37	0.57	0.005
Seq2seq-EVC	2.72 $\pm 0.03$	2.69 $\pm 0.05$	1.48	22.19	37.62	0.39	0.61	0.233
Emovox	3.06 $\pm 0.05$	3.17 $\pm 0.03$	2.11	18.69	34.18	0.49	0.69	0.211
Textless-EVC	3.43 $\pm 0.06$	3.39 $\pm 0.04$	2.84	15.17	26.73	0.65	0.66	0.434

Table 1: Overall comparison on the ESD corpus. Best among the conversion systems in bold .

Model	WER $\downarrow$	UTMOS $\uparrow$	EECS $\uparrow$	SECS $\uparrow$
Prime-EVC	9.61	3.37	0.87	0.80
w/o FiLM	9.86	3.34	0.83	0.77
w/o intensity	9.55	3.35	0.82	0.80
w/o classifier	9.52	3.41	0.81	0.81
w/o STFT	9.17	3.23	0.85	0.80
w/o EVA	8.25	2.89	0.46	0.78

Table 2: Ablation of Prime-EVC. Above the rule, each row removes one conditioning component from the full model. Below the rule are the conditioning alone and alternative decoder objectives.

Prime-EVC
NFE	UTMOS $\uparrow$	WER(%) $\downarrow$	EECS $\uparrow$	SECS $\uparrow$	RTF $\downarrow$
1	3.37	9.61	0.87	0.80	0.038
4	3.35	9.70	0.88	0.80	0.045
16	3.37	9.64	0.87	0.80	0.078
Global-cond. CFM
1	2.80	8.07	0.37	0.77	0.036

Table 3: Effect of NFE for Prime-EVC and the Global-cond. CFM , in which the global style is the only emotion pathway.

Voice Conversion

Voice conversion changes a speaker's vocal identity or delivery while preserving the spoken content.

See all 35 papers

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion
Sep 30, 2026Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka +1Zero-Shot Voice Conversion Flow Matching
NTT, Inc., Japan
TargetSEC: Plug-and-Play In-the-Wild Speech Emotion Conversion via Arousal-Conditioned Latent Style Diffusion
Jun 5, 2026Constantin Alexander AugaLatent Diffusion Models Audio Diffusion Models
Hasso Plattner Institute / University of Potsdam, Potsdam, Germany
MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows
Feb 20, 2026Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka +1Voice Conversion Mean Flow
NTT, Inc., Japan

Sep 30, 2026J/K move · Enter open · S save

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion

Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka, Yuto Kondo

NTT, Inc., Japan

Zero-Shot Voice Conversion Flow Matching Speech Processing Flow-Based Generative Modeling Voice Conversion

Open paper AskPDF arXiv

Flow-matching approaches to voice conversion (VC) have gained attention owing to their high speech quality and strong speaker similarity. Among them, one-step models such as MeanVoiceFlow are particularly attractive because they enable efficient inference; however, their reliance on a computationally intensive content encoder remains a bottleneck. We therefore propose MeanVoiceFlow2, a framework that jointly optimizes a flow-based conversion module and a computationally efficient content encoder. The model is trained through conversion distillation using MeanVoiceFlow and the reconstruction of real data. We further incorporate diffusion-GAN training with sample mixing and teacher-guided conditioning augmentation to enhance realism and disentanglement. Experiments on zero-shot VC showed that MeanVoiceFlow2 achieved higher perceptual quality and approximately

9\times

faster inference than MeanVoiceFlow while maintaining comparable speaker similarity. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow2/.

Abstract

Voice Conversion

Explore similar work

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion

TargetSEC: Plug-and-Play In-the-Wild Speech Emotion Conversion via Arousal-Conditioned Latent Style Diffusion

MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows