StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion

Yisi Liu; Nicholas Lee; Gopala K. Anumanchipalli

cs.SDOct 20, 2026

StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion

Authors: Yisi Liu, Nicholas Lee, Gopala K. Anumanchipalli

Organizations: University of California, Berkeley

Abstract

Voice style conversion (VSC) aims to transform a source utterance to match the timbre, accent, and emotion of a target voice while preserving linguistic content. StyleStream 1.0 introduced the first streamable zero-shot VSC system with state-of-the-art conversion quality, but remained limited in two ways: approximately 1s end-to-end latency on consumer hardware, and the lack of a text-based interface for designing or editing target voices. We present StyleStream 2.0, a fast and controllable streaming VSC framework that addresses these limitations. To reduce latency, we train a few-step pixel MeanFlow model and further fine-tune it with autoregressive feedback inspired by Self Forcing, improving robustness under streaming inference. This reduces end-to-end latency to 520~ms on a consumer GPU, a 2x speedup over StyleStream 1.0, while maintaining conversion quality. To enable controllable style manipulation, we remove the mel-spectrogram context and instead route target style information through a compact style embedding. In this embedding space, we train a unified text-conditioned flow matching model that supports both text-based voice design, which maps natural language prompts to voice styles, and instruction-based voice editing, which modifies a specific attribute of a source utterance while preserving the rest. Experiments show that StyleStream 2.0 achieves the strongest target style fidelity on VSC, voice design, and voice editing, while remaining competitive on intelligibility and non-edited attribute preservation.

Figures & tables

Figure 1: Left : Stylizer architecture. $[r,t]$ is the MeanFlow interval, $\omega$ is the CFG strength. Right : Voice Controller architecture. When used for voice design, the auxiliary input is null $\varnothing$ ; for voice editing, source style embedding is also part of the input.

Model	NFE $\downarrow$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$	NMOS $\uparrow$	S-SMOS $\uparrow$	A-SMOS $\uparrow$	E-SMOS $\uparrow$
Ground Truth	—	4.42	—	—	—	3.72 $\pm$ .12	—	—	—
Qwen3-TTS	—	4.76	0.927	0.833	0.941	—	—	—	—
CosyVoice 2.0	10	8.07	0.814	0.484	0.674	3.54 $\pm$ .12	3.64 $\pm$ .13	3.25 $\pm$ .15	2.61 $\pm$ .15
CosyVoice 3.0	10	6.56	0.832	0.508	0.668	3.48 $\pm$ .13	3.78 $\pm$ .12	3.21 $\pm$ .14	2.80 $\pm$ .15
Vevo	32	13.33	0.853	0.659	0.745	3.42 $\pm$ .13	4.00 $\pm$ .14	3.94 $\pm$ .13	3.63 $\pm$ .13
Vevo 2.0	32	5.79	0.843	0.676	0.723	3.44 $\pm$ .13	4.06 $\pm$ .13	3.98 $\pm$ .13	3.11 $\pm$ .15

Table 1: Voice style conversion results (SS denotes StyleStream). The best results are shown in bold , and the second-best results are underlined . Ground-truth NMOS is reported as an upper bound and excluded from ranking.

Chunk (ms)	Model	Proc (ms) $\downarrow$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
160	SS 1.0	500.1	16.59	0.821	0.624	0.724
160	SS 2.0	130.5	14.02	0.860	0.669	0.763
200	SS 1.0	528.8	15.25	0.825	0.632	0.732
200	SS 2.0	132.0	13.22	0.863	0.675	0.764
300	SS 1.0	559.3	13.34	0.833	0.644	0.748
300	SS 2.0	140.7	12.03	0.866	0.684	0.765

Table 2: Streaming chunk-size analysis on RTX 4060. We compare StyleStream 1.0 (SS 1.0, 12 NFE) and StyleStream 2.0 (SS 2.0, 4 NFE) across chunk sizes from 160 ms to 600 ms, reporting per-chunk processing time and quality. Best per chunk-size group is in bold .

		Objective				Subjective
Model	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	NMOS $\uparrow$	SC-MOS $\uparrow$
CapTTS	7.36	41.11	94.33	52.00	29.47	3.27 $\pm$ .13	3.54 $\pm$ .13
Qwen3-TTS	4.86	44.33	92.39	14.73	36.33	3.35 $\pm$ .14	2.97 $\pm$ .15
SS 2.0 (streaming)	10.42	67.94	98.94	46.73	30.13	3.23 $\pm$ .13	3.78 $\pm$ .12
SS 2.0 (offline)	6.89	70.67	99.44	52.73	32.20	3.45 $\pm$ .12	3.96 $\pm$ .10

Table 3: Voice design results. WER and VoxProfile accuracies are percentages. We use Age-acc, G-acc, A-acc, and E-acc to denote age, gender, accent and emotion classification accuracy. NMOS measures naturalness and SC-MOS measures style consistency with the prompt.

		Style Consistency				MOS			Style Preservation
Model	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	SC-MOS $\uparrow$	SP-MOS $\uparrow$	NMOS $\uparrow$	S-pres. $\uparrow$	A-pres. $\uparrow$	E-pres. $\uparrow$
Vevo	12.35	28.33	94.83	16.89	6.56	2.64 $\pm$ .16	3.64 $\pm$ .12	3.44 $\pm$ .13	0.9092	0.6303	0.8902
Vevo2	7.82	24.67	65.17	6.78	7.11	2.26 $\pm$ .15	3.28 $\pm$ .14	3.32 $\pm$ .14	0.8853	0.5948	0.8832
SS 2.0 (streaming)	9.62	67.17	99.67	46.56	13.67	3.64 $\pm$ .13	3.50 $\pm$ .13	3.21 $\pm$ .15	0.7644	0.4864	0.8794
SS 2.0 (offline)	6.43	71.67	99.17	54.33	14.67	3.76 $\pm$ .13	3.58 $\pm$ .12	3.48 $\pm$ .13	0.7899	0.5102	0.8851

Table 4: Voice editing results. WER and VoxProfile accuracies are percentages. We use Age-acc, G-acc, A-acc, and E-acc to denote age, gender, accent and emotion classification accuracy. SC-MOS measures instruction consistency, SP-MOS measures preservation of non-edited attributes, and NMOS measures naturalness.

Setting	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
w/ AR-ft	12.03	0.866	0.684	0.765
w/o AR-ft	19.54	0.866	0.676	0.767

Table 5: Ablation studies. Left : Effect of with or without autoregressive fine-tuning (AR-ft) under streaming inference. Right : Different style encoder effect under offline inference.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Samples	Audio source
Voice design	2,000,005	Synthesized
Accent editing	3,000,028	Synthesized
Emotion editing	2,999,992	Synthesized
Age editing	1,499,961	Natural
Gender editing	1,481,314	Natural
Joint age+gender editing	1,000,001	Natural

Appendix

Table 6: Training tiers for the text-based StyleStream 2.0 corpus.

Design prompt type	Target axes	Count
Timbre	age, gender	600
Accent	accent	600
Emotion	emotion	600
Timbre+accent	age, gender, accent	300
Timbre+emotion	age, gender, emotion	300
All attributes	age, gender, accent, emotion	600

Appendix

Table 7: Voice design prompt mix for evaluation.

Panel of Figure 2 — Figure 2: Naturalness MOS (NMOS) and Emotion SMOS (E-SMOS) evaluation interfaces. The S-SMOS and A-SMOS surveys are analogous to E-SMOS, with the emotion target replaced by speaker identity or accent and the listener instructed to ignore the other style axes.

Panel of Figure 3 — Figure 3: Voice-design (VD) and voice-editing (VE) evaluation interfaces. Each item pairs a source and a test audio sample with a natural-language style prompt (VD) or edit instruction (VE), and asks for naturalness plus task-specific match (and preservation, for VE) ratings.

$\omega$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
1	9.53	0.8806	0.6852	0.7462
2	7.19	0.8835	0.7071	0.7711
3	7.03	0.8791	0.7055	0.7751
4	6.98	0.8741	0.7000	0.7766
5	6.97	0.8694	0.6929	0.7747
6	7.00	0.8653	0.6870	0.7722

Appendix

Table 8: CFG guidance-scale ablation on the offline StyleStream 2.0 model at NFE

{}=4

NFE	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
1	7.06	0.8755	0.6976	0.7689
2	7.05	0.8791	0.7052	0.7740
4	7.02	0.8787	0.7058	0.7744
8	7.06	0.8785	0.7047	0.7768
16	7.09	0.8791	0.7056	0.7756

Appendix

Table 9: NFE ablation on the offline StyleStream 2.0 model at

\omega=3

Text conditioning	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$
Trainable embeddings	6.80	59.67	81.17	45.60	29.47
Flan-T5-small	7.02	61.06	97.94	47.87	33.60
Flan-T5-base	7.05	64.56	98.33	49.20	39.93
Flan-T5-XL	6.91	58.94	97.61	49.13	38.40

Appendix

Table 10: Voice-design text-encoder ablation after 150k controller training steps. WER and attribute accuracies are percentages. Flan-T5 encoders are frozen; trainable embeddings are randomly initialized. Bold / underline mark the best/second-best values within this ablation.

Text conditioning	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	S-pres. $\uparrow$	A-pres. $\uparrow$	E-pres. $\uparrow$
Trainable embeddings	6.77	60.00	60.00	43.56	12.56	0.5873	0.4480	0.8723
Flan-T5-small	6.86	54.17	97.00	47.44	16.11	0.5795	0.4350	0.8689
Flan-T5-base	6.86	61.50	92.17	49.00	22.89	0.5960	0.4535	0.8721
Flan-T5-XL	6.95	55.00	97.33	49.44	24.33	0.5992	0.4433	0.8718

Appendix

Table 11: Voice-editing text-encoder ablation after 150k controller training steps, under the same conditions as Table 10 . WER and accuracies are percentages; preservation scores are similarities to the source. Best/second-best values include ties.

Asset	License / terms	Source
Emilia / Emilia-EN	CC BY-NC 4.0; audio copyright remains with original owners	https://huggingface.co/datasets/amphion/Emilia
CapSpeech	CC BY-NC 4.0	https://github.com/WangHelin1997/CapSpeech
ESD	Free for research use only	https://hltsingapore.github.io/ESD/download.html
GLOBE	CC0 1.0	https://huggingface.co/datasets/MushanW/GLOBE
LibriTTS	CC BY 4.0	https://www.openslr.org/60/
RAVDESS	CC BY-NC-SA 4.0	https://zenodo.org/records/1188976

Appendix

Table 12: Licenses and terms for existing assets used in this work.

Voice Conversion

Voice conversion changes a speaker's vocal identity or delivery while preserving the spoken content.

See all 35 papers

Feb 23, 2026

StyleStream: Real-Time Zero-Shot Voice Style Conversion

Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.

Yisi Liu, Nicholas Lee, Gopala Anumanchipalli

UC Berkeley, USA

Zero-Shot Voice Conversion Speech Processing Voice Conversion

Ask PDF

Jun 8, 2026

MeanVC 2: Robust Low-Latency Streaming Zero-Shot Voice Conversion

Streaming zero-shot voice conversion (VC) has become increasingly popular due to its potential for real-time applications. The recently proposed MeanVC achieves lightweight streaming zero-shot VC, but it has several limitations: its chunk-wise autoregressive denoising doubles the effective training sequence length, conversion quality degrades under small-chunk settings, and its timbre encoder directly relies on reference mel-spectrograms, making it sensitive to reference audio quality. To address these limitations we propose MeanVC 2. We introduce future-receptive chunking (FRC), which explicitly schedules past and future receptive fields across diffusion transformer decoder layers and removes clean-chunk teacher forcing. By incorporating bounded future context, FRC enables stable conversion with a 40 ms chunk size. We further introduce a universal timbre token encoder, which constructs a timbre representation from a global speaker embedding and retrieves fine-grained timbre cues via cross-attention, improving robustness to low-quality references and enhancing zero-shot speaker similarity. Experimental results show that MeanVC 2 significantly outperforms MeanVC, while reducing latency from 211 ms to 110 ms. Audio samples are publicly available. The source code will be publicly released.

Guobin Ma, Yuxuan Xia, Yuepeng Jiang +6

The University of New South Wales, Australia · WeNet Open Source Community, China

Zero-Shot Voice Conversion Diffusion Transformer Audio Diffusion Models Speech Processing Voice Conversion

Ask PDF

Sep 30, 2026

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion

Flow-matching approaches to voice conversion (VC) have gained attention owing to their high speech quality and strong speaker similarity. Among them, one-step models such as MeanVoiceFlow are particularly attractive because they enable efficient inference; however, their reliance on a computationally intensive content encoder remains a bottleneck. We therefore propose MeanVoiceFlow2, a framework that jointly optimizes a flow-based conversion module and a computationally efficient content encoder. The model is trained through conversion distillation using MeanVoiceFlow and the reconstruction of real data. We further incorporate diffusion-GAN training with sample mixing and teacher-guided conditioning augmentation to enhance realism and disentanglement. Experiments on zero-shot VC showed that MeanVoiceFlow2 achieved higher perceptual quality and approximately

9\times

faster inference than MeanVoiceFlow while maintaining comparable speaker similarity. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow2/.

Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka +1

NTT, Inc., Japan

Zero-Shot Voice Conversion Flow Matching Speech Processing Flow-Based Generative Modeling Voice Conversion

Ask PDF

Abstract

Voice Conversion

Explore similar work

StyleStream: Real-Time Zero-Shot Voice Style Conversion

MeanVC 2: Robust Low-Latency Streaming Zero-Shot Voice Conversion

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion