StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion

Yisi Liu; Nicholas Lee; Gopala K. Anumanchipalli

cs.SDOct 20, 2026

StyleStream 2.0: Fast and Controllable Streaming Voice Style Conversion

Authors: Yisi Liu, Nicholas Lee, Gopala K. Anumanchipalli

Organizations: University of California, Berkeley

Abstract

Voice style conversion (VSC) aims to transform a source utterance to match the timbre, accent, and emotion of a target voice while preserving linguistic content. StyleStream 1.0 introduced the first streamable zero-shot VSC system with state-of-the-art conversion quality, but remained limited in two ways: approximately 1s end-to-end latency on consumer hardware, and the lack of a text-based interface for designing or editing target voices. We present StyleStream 2.0, a fast and controllable streaming VSC framework that addresses these limitations. To reduce latency, we train a few-step pixel MeanFlow model and further fine-tune it with autoregressive feedback inspired by Self Forcing, improving robustness under streaming inference. This reduces end-to-end latency to 520~ms on a consumer GPU, a 2x speedup over StyleStream 1.0, while maintaining conversion quality. To enable controllable style manipulation, we remove the mel-spectrogram context and instead route target style information through a compact style embedding. In this embedding space, we train a unified text-conditioned flow matching model that supports both text-based voice design, which maps natural language prompts to voice styles, and instruction-based voice editing, which modifies a specific attribute of a source utterance while preserving the rest. Experiments show that StyleStream 2.0 achieves the strongest target style fidelity on VSC, voice design, and voice editing, while remaining competitive on intelligibility and non-edited attribute preservation.

Figures & tables

Figure 1: Left : Stylizer architecture. $[r,t]$ is the MeanFlow interval, $\omega$ is the CFG strength. Right : Voice Controller architecture. When used for voice design, the auxiliary input is null $\varnothing$ ; for voice editing, source style embedding is also part of the input.

Model	NFE $\downarrow$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$	NMOS $\uparrow$	S-SMOS $\uparrow$	A-SMOS $\uparrow$	E-SMOS $\uparrow$
Ground Truth	—	4.42	—	—	—	3.72 $\pm$ .12	—	—	—
Qwen3-TTS	—	4.76	0.927	0.833	0.941	—	—	—	—
CosyVoice 2.0	10	8.07	0.814	0.484	0.674	3.54 $\pm$ .12	3.64 $\pm$ .13	3.25 $\pm$ .15	2.61 $\pm$ .15
CosyVoice 3.0	10	6.56	0.832	0.508	0.668	3.48 $\pm$ .13	3.78 $\pm$ .12	3.21 $\pm$ .14	2.80 $\pm$ .15
Vevo	32	13.33	0.853	0.659	0.745	3.42 $\pm$ .13	4.00 $\pm$ .14	3.94 $\pm$ .13	3.63 $\pm$ .13
Vevo 2.0	32	5.79	0.843	0.676	0.723	3.44 $\pm$ .13	4.06 $\pm$ .13	3.98 $\pm$ .13	3.11 $\pm$ .15

Table 1: Voice style conversion results (SS denotes StyleStream). The best results are shown in bold , and the second-best results are underlined . Ground-truth NMOS is reported as an upper bound and excluded from ranking.

Chunk (ms)	Model	Proc (ms) $\downarrow$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
160	SS 1.0	500.1	16.59	0.821	0.624	0.724
160	SS 2.0	130.5	14.02	0.860	0.669	0.763
200	SS 1.0	528.8	15.25	0.825	0.632	0.732
200	SS 2.0	132.0	13.22	0.863	0.675	0.764
300	SS 1.0	559.3	13.34	0.833	0.644	0.748
300	SS 2.0	140.7	12.03	0.866	0.684	0.765

Table 2: Streaming chunk-size analysis on RTX 4060. We compare StyleStream 1.0 (SS 1.0, 12 NFE) and StyleStream 2.0 (SS 2.0, 4 NFE) across chunk sizes from 160 ms to 600 ms, reporting per-chunk processing time and quality. Best per chunk-size group is in bold .

		Objective				Subjective
Model	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	NMOS $\uparrow$	SC-MOS $\uparrow$
CapTTS	7.36	41.11	94.33	52.00	29.47	3.27 $\pm$ .13	3.54 $\pm$ .13
Qwen3-TTS	4.86	44.33	92.39	14.73	36.33	3.35 $\pm$ .14	2.97 $\pm$ .15
SS 2.0 (streaming)	10.42	67.94	98.94	46.73	30.13	3.23 $\pm$ .13	3.78 $\pm$ .12
SS 2.0 (offline)	6.89	70.67	99.44	52.73	32.20	3.45 $\pm$ .12	3.96 $\pm$ .10

Table 3: Voice design results. WER and VoxProfile accuracies are percentages. We use Age-acc, G-acc, A-acc, and E-acc to denote age, gender, accent and emotion classification accuracy. NMOS measures naturalness and SC-MOS measures style consistency with the prompt.

		Style Consistency				MOS			Style Preservation
Model	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	SC-MOS $\uparrow$	SP-MOS $\uparrow$	NMOS $\uparrow$	S-pres. $\uparrow$	A-pres. $\uparrow$	E-pres. $\uparrow$
Vevo	12.35	28.33	94.83	16.89	6.56	2.64 $\pm$ .16	3.64 $\pm$ .12	3.44 $\pm$ .13	0.9092	0.6303	0.8902
Vevo2	7.82	24.67	65.17	6.78	7.11	2.26 $\pm$ .15	3.28 $\pm$ .14	3.32 $\pm$ .14	0.8853	0.5948	0.8832
SS 2.0 (streaming)	9.62	67.17	99.67	46.56	13.67	3.64 $\pm$ .13	3.50 $\pm$ .13	3.21 $\pm$ .15	0.7644	0.4864	0.8794
SS 2.0 (offline)	6.43	71.67	99.17	54.33	14.67	3.76 $\pm$ .13	3.58 $\pm$ .12	3.48 $\pm$ .13	0.7899	0.5102	0.8851

Table 4: Voice editing results. WER and VoxProfile accuracies are percentages. We use Age-acc, G-acc, A-acc, and E-acc to denote age, gender, accent and emotion classification accuracy. SC-MOS measures instruction consistency, SP-MOS measures preservation of non-edited attributes, and NMOS measures naturalness.

Setting	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
w/ AR-ft	12.03	0.866	0.684	0.765
w/o AR-ft	19.54	0.866	0.676	0.767

Table 5: Ablation studies. Left : Effect of with or without autoregressive fine-tuning (AR-ft) under streaming inference. Right : Different style encoder effect under offline inference.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Tier	Samples	Audio source
Voice design	2,000,005	Synthesized
Accent editing	3,000,028	Synthesized
Emotion editing	2,999,992	Synthesized
Age editing	1,499,961	Natural
Gender editing	1,481,314	Natural
Joint age+gender editing	1,000,001	Natural

Appendix

Table 6: Training tiers for the text-based StyleStream 2.0 corpus.

Design prompt type	Target axes	Count
Timbre	age, gender	600
Accent	accent	600
Emotion	emotion	600
Timbre+accent	age, gender, accent	300
Timbre+emotion	age, gender, emotion	300
All attributes	age, gender, accent, emotion	600

Appendix

Table 7: Voice design prompt mix for evaluation.

Panel of Figure 2 — Figure 2: Naturalness MOS (NMOS) and Emotion SMOS (E-SMOS) evaluation interfaces. The S-SMOS and A-SMOS surveys are analogous to E-SMOS, with the emotion target replaced by speaker identity or accent and the listener instructed to ignore the other style axes.

Panel of Figure 3 — Figure 3: Voice-design (VD) and voice-editing (VE) evaluation interfaces. Each item pairs a source and a test audio sample with a natural-language style prompt (VD) or edit instruction (VE), and asks for naturalness plus task-specific match (and preservation, for VE) ratings.

$\omega$	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
1	9.53	0.8806	0.6852	0.7462
2	7.19	0.8835	0.7071	0.7711
3	7.03	0.8791	0.7055	0.7751
4	6.98	0.8741	0.7000	0.7766
5	6.97	0.8694	0.6929	0.7747
6	7.00	0.8653	0.6870	0.7722

Appendix

Table 8: CFG guidance-scale ablation on the offline StyleStream 2.0 model at NFE

{}=4

NFE	WER ( $\%$ ) $\downarrow$	S-SIM $\uparrow$	A-SIM $\uparrow$	E-SIM $\uparrow$
1	7.06	0.8755	0.6976	0.7689
2	7.05	0.8791	0.7052	0.7740
4	7.02	0.8787	0.7058	0.7744
8	7.06	0.8785	0.7047	0.7768
16	7.09	0.8791	0.7056	0.7756

Appendix

Table 9: NFE ablation on the offline StyleStream 2.0 model at

\omega=3

Text conditioning	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$
Trainable embeddings	6.80	59.67	81.17	45.60	29.47
Flan-T5-small	7.02	61.06	97.94	47.87	33.60
Flan-T5-base	7.05	64.56	98.33	49.20	39.93
Flan-T5-XL	6.91	58.94	97.61	49.13	38.40

Appendix

Table 10: Voice-design text-encoder ablation after 150k controller training steps. WER and attribute accuracies are percentages. Flan-T5 encoders are frozen; trainable embeddings are randomly initialized. Bold / underline mark the best/second-best values within this ablation.

Text conditioning	WER $\downarrow$	Age-acc $\uparrow$	G-acc $\uparrow$	A-acc $\uparrow$	E-acc $\uparrow$	S-pres. $\uparrow$	A-pres. $\uparrow$	E-pres. $\uparrow$
Trainable embeddings	6.77	60.00	60.00	43.56	12.56	0.5873	0.4480	0.8723
Flan-T5-small	6.86	54.17	97.00	47.44	16.11	0.5795	0.4350	0.8689
Flan-T5-base	6.86	61.50	92.17	49.00	22.89	0.5960	0.4535	0.8721
Flan-T5-XL	6.95	55.00	97.33	49.44	24.33	0.5992	0.4433	0.8718

Appendix

Table 11: Voice-editing text-encoder ablation after 150k controller training steps, under the same conditions as Table 10 . WER and accuracies are percentages; preservation scores are similarities to the source. Best/second-best values include ties.

Asset	License / terms	Source
Emilia / Emilia-EN	CC BY-NC 4.0; audio copyright remains with original owners	https://huggingface.co/datasets/amphion/Emilia
CapSpeech	CC BY-NC 4.0	https://github.com/WangHelin1997/CapSpeech
ESD	Free for research use only	https://hltsingapore.github.io/ESD/download.html
GLOBE	CC0 1.0	https://huggingface.co/datasets/MushanW/GLOBE
LibriTTS	CC BY 4.0	https://www.openslr.org/60/
RAVDESS	CC BY-NC-SA 4.0	https://zenodo.org/records/1188976

Appendix

Table 12: Licenses and terms for existing assets used in this work.

Voice Conversion

Voice conversion changes a speaker's vocal identity or delivery while preserving the spoken content.

See all 35 papers

StyleStream: Real-Time Zero-Shot Voice Style Conversion
Feb 23, 2026Yisi Liu, Nicholas Lee, Gopala AnumanchipalliZero-Shot Voice Conversion Speech Processing
UC Berkeley, USA
MeanVC 2: Robust Low-Latency Streaming Zero-Shot Voice Conversion
Jun 8, 2026Guobin Ma, Yuxuan Xia, Yuepeng Jiang +6Zero-Shot Voice Conversion Diffusion Transformer
The University of New South Wales, Australia · WeNet Open Source Community, China
MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion
Sep 30, 2026Takuhiro Kaneko, Hirokazu Kameoka, Kou Tanaka +1Zero-Shot Voice Conversion Flow Matching
NTT, Inc., Japan

Feb 23, 2026J/K move · Enter open · S save

StyleStream: Real-Time Zero-Shot Voice Style Conversion

Yisi Liu, Nicholas Lee, Gopala Anumanchipalli

UC Berkeley, USA

Zero-Shot Voice Conversion Speech Processing Voice Conversion

Open paper AskPDF arXiv

Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.

Abstract

Voice Conversion

Explore similar work

StyleStream: Real-Time Zero-Shot Voice Style Conversion

MeanVC 2: Robust Low-Latency Streaming Zero-Shot Voice Conversion

MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversion