Organizations: School of Software Engineering, Xi’an Jiaotong University, Xi’an, China · SYKI-SPEECH Team, Xi’an, China · Fudan University, Shanghai, China · Division of Music and Audio Union Wheatland Culture and Media Ltd., China
This paper presents the submission of the S4 team to the Singing Voice Conversion Challenge 2025 (SVCC2025)-a novel singing style conversion system that advances fine-grained style conversion and control within in-domain settings. To address the critical challenges of style leakage, dynamic rendering, and high-fidelity generation with limited data, we introduce three key innovations: a boundary-aware Whisper bottleneck that pools phoneme-span representations to suppress residual source style while preserving linguistic content; an explicit frame-level technique matrix, enhanced by targeted F0 processing during inference, for stable and distinct dynamic style rendering; and a perceptually motivated high-frequency band completion strategy that leverages an auxiliary standard 48kHz SVC model to augment the high-frequency spectrum, thereby overcoming data scarcity without overfitting. In the official SVCC2025 subjective evaluation, our system achieves the best naturalness performance among all submissions while maintaining competitive results in speaker similarity and technique control, despite using significantly less extra singing data than other top-performing systems. Audio samples are available online.
Figures & tables
Fig. 1: The overall architecture of the proposed System S4. (a) Training Stage: The system employs a boundary-aware semantic bottleneck to suppress style leakage in Whisper features, while the VITS decoder is conditioned on the ground-truth technique matrix T to learn explicit style rendering. (b) Inference Stage: The pipeline is driven by a target technique matrix, which guides both the explicit pitch dynamics module (for F0 refinement) and the decoder. Finally, a High-Frequency Band Completion module extends the output to 48 kHz fidelity.
Fig. 2: Illustration of the core disentanglement and control mechanisms. The Boundary-Aware Semantic Bottleneck pools frame-level Whisper features within phoneme boundaries to wash out residual source style, while the technique matrix T explicitly drives style generation.
Fig. 3: A scatter plot comparing naturalness and style similarity metrics. Our system S4B is positioned on the far right, indicating superior naturalness performance.
Fig. 4: Official evaluation results. (a) Naturalness MOS boxplots. The red dot indicates the mean score, and the black line indicates the median. (b) Singing style similarity. (c) Singer identity similarity. The black diamonds ( ⧫ ) represent the binary accuracy.
Setting
Spk Sim ↑
MOS ↑
Converted ↑
Leakage ↓
ContentVec (encoder)
0.724
4.18
72.2%
13.9%
WeNet (encoder)
0.694
4.04
63.9%
22.2%
Whisper + pooling, λ=1.0
0.692
4.32
66.7%
27.8%
Whisper + pooling, λ=0
0.428
4.19
66.7%
11.1%
Whisper w/o pooling, λ=0.1
0.665
4.21
58.3%
25.0%
Whisper + pooling, λ=0.1 (ours)
0.732
4.22
83.3%
8.3%
TABLE I: Ablation results on semantic features and the proposed semantic bottleneck (phoneme-aware pooling + scaling).
Singing style is a crucial aspect of a natural and expressive singing voice. Singers utilize singing styles to convey the feeling or emotion of the songs. Several works have been proposed to control singing style for making the more expressive singing voice. Recently, VibE-SVC successfully controls vibrato by predicting high-frequency F0 contour. In this paper, we introduce a singing voice conversion framework, called VibE-SVC2, to improve singing style conversion performance and controllability. The model offers control over two types of singing styles: a pitch style and a timbre style. For the pitch style, to resolve the pitch-energy entanglement issue that is unresolved in our previous work, we introduce a novel Energy Style Converter to address remaining style information in the energy contour. In addition, we propose a Zero-shot Pitch Style Converter, which mimics the pitch style of reference audio. To expand the controllability of the model, we propose vibrato rate scaling that is an independent control of vibrato extent, which is unavailable in VibE-SVC. For the timbre style, we extend the model to handle a variety of phonation styles. However, addressing specific styles such as vocal fry poses a challenge, as conventional F0 extraction often fails due to their inherent subharmonic characteristics, which degrades the conversion quality. To address this, we propose a novel Subharmonic Correction algorithm to refine the F0 contour for more natural timbre conversion. Through comprehensive objective and subjective evaluations, we demonstrate that VibE-SVC2 provides fine-grained, independent control over two types of singing styles, outperforming existing methods.
Joon-Seung Choi, Dong-Min Byun, Seong-Whan Lee
Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea
Singing voice beautifying (SVB) aims to correct pitch and rhythm of amateur singing while enhancing vocal quality, preserving lyrics and the singer's timbre. Existing methods, however, suffer from limited generation quality and efficiency, and tend to neglect the preservation of the singer's style. We propose SRF-SVB, a style-consistent model for SVB via rectified flow, which achieves high-fidelity and efficient beautification covering pitch and rhythm correction. Furthermore, we design a context-guided masked mel-spectrogram inpainting mechanism that effectively preserves the amateur singer's style, including unique timbre and expressive patterns. Experiments on both English and Chinese test sets show that SRF-SVB outperforms baseline models in most objective and subjective metrics.
Voice style conversion aims to transform an input utterance to match a target speaker's timbre, accent, and emotion, with a central challenge being the disentanglement of linguistic content from style. While prior work has explored this problem, conversion quality remains limited, and real-time voice style conversion has not been addressed. We propose StyleStream, the first streamable zero-shot voice style conversion system that achieves state-of-the-art performance. StyleStream consists of two components: a Destylizer, which removes style attributes while preserving linguistic content, and a Stylizer, a diffusion transformer (DiT) that reintroduces target style conditioned on reference speech. Robust content-style disentanglement is enforced through text supervision and a highly constrained information bottleneck. This design enables a fully non-autoregressive architecture, achieving real-time voice style conversion with an end-to-end latency of 1 second. Samples and real-time demo: https://berkeley-speech-group.github.io/StyleStream/.