Scalable Neural Vocoder from Range-Null Space Decomposition

Andong Li; Tong Lei; Zhihang Sun; Rilin Chen; Xiaodong Li; Dong Yu; Chengshi Zheng

cs.SDMar 9, 2026

Scalable Neural Vocoder from Range-Null Space Decomposition

Authors: Andong Li, Tong Lei, Zhihang Sun, Rilin Chen, Xiaodong Li, Dong Yu, Chengshi Zheng

Organizations: the Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, 100190, China, and also with University of Chinese Academy of Sciences, Beijing, 100049, China. · Tencent AI Lab. · School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China.

Abstract

Although deep neural networks have facilitated significant progress of neural vocoders in recent years, they usually suffer from intrinsic challenges like opaque modeling, inflexible retraining under different input configurations, and parameter-performance trade-off. These inherent hurdles can heavily impede the development of this field. To resolve these problems, in this paper, we propose a novel neural vocoder in the time-frequency (T-F) domain. Specifically, we bridge the connection between the classical range-null decomposition (RND) theory and the vocoder task, where the reconstruction of the target spectrogram is formulated into the superimposition between range-space and null-space. The former aims to project the representation in the original mel-domain into the target linear-scale domain, and the latter can be instantiated via neural networks to further infill the spectral details. To fully leverage the spectrum prior, an elaborate dual-path framework is devised, where the spectrum is hierarchically encoded and decoded, and the cross- and narrow-band modules are leveraged for effectively modeling along sub-band and time dimensions. To enable inference under various configurations, we propose a simple yet effective strategy, which transforms the multi-condition adaption in the inference stage into the data augmentation in the training stage. Comprehensive experiments are conducted on various benchmarks. Quantitative and qualitative results show that while enjoying lightweight network structure and scalable inference paradigm, the proposed framework achieves state-ofthe-art performance among existing advanced methods. Code is available at https://github.com/Andong-Li-speech/RNDVoC.

Figures & tables

Fig. 1: Illustrations of the previous and the proposed neural vocoder generation pipelines with range-null decomposition (RND) theory. (a) In previous T-F domain neural vocoders, the mapping relation from mel-spectrogram to the target spectrogram/waveform was designed in a black-box manner. (b) We exploit the linear degradation prior to develop a more transparent generation pipeline, where the range-space module is to transform the acoustic feature in the original mel-scale domain into the target linear-scale domain, and the null-space module is responsible for generating fine-grained spectral details.

Fig. 2: Comparison of model parameters and PESQ scores between the proposed RNDVoC and other mainstream vocoders on the LibriTTS benchmark. Larger bubbles indicate higher computational complexity. All generators are updated for 1M steps.

Fig. 3: Illustrations of the proposed RNDVoC.

Category	Signal-Level	Spectral-Level (Vocoder)
Compression Matrix	$\mathbf{A}$	Mel-Filter $\mathcal{A}$
Observation	$\mathbf{y}$	Input Mel-Spectrogram $\mathbf{Y}$ , Eq. $\left(\ref{eqn:5}\right)$
Target Signal	$\mathbf{x}$	Linear-Scale Spectrum $\mathbf{S}$
Range-Space	$\mathbf{A}^{\dagger}\mathbf{y}$	Range-Space Spectrum $\left\|\tilde{\mathbf{S}}\right\|_{range}$ , Eq. $\left(\ref{eqn:9}\right)$
Null-Space	$\left(\mathbf{I}-\mathbf{A}^{\dagger}\mathbf{A}\right)\hat{\mathbf{x}}$	Null-Space Spectrum $\left(\mathbf{I}-\mathcal{A}^{\dagger}\mathcal{A}\right)\left\|\tilde{\mathbf{S}}\right\|_{null}$ , Eq. $\left(\ref{eqn:10}\right)$
Estimation	$\mathbf{\tilde{x}}$	Superimposed Spectrum $\mathbf{\tilde{S}},$ , Eq. $\left(\ref{eqn:11}\right)$

TABLE I: Notation tables between classical signal model and mel degradation formulation.

Fig. 4: Framework diagram of the proposed RNDVoC, where the range-space module only involves the pseudo-inverse matrix operation and its output will serve as the input of the null-space module. (a) Main framework diagram of the null-space module. (b) Detailed structure of the band-aware spectral encoding module (BAEM). (c) Detailed structure of the band-aware spectral magnitude/phase module (BAMM/BAPM), which shares a similar network structure. (d) Detailed structure of the dual-path block (DPB). Note that the nonshared scheme is adopted in (b)-(c).

Fig. 5: Illustrations of the spectral encoding and decoding module with the proposed parameter-shared strategy, and here the number of regions $I$ is set to 3 as an example. (a) Spectral encoding process. (b) Spectral decoding process, we only take one branch (magnitude or phase) as an example.

Methods	Type	Publish	Domain	Code Link
HiFiGAN-V1 [ 33 ]	GAN	NeurIPS 2020	T	https://github.com/jik876/hifi-gan
iSTFTNet-V1 [ 34 ]	GAN	ICASSP 2022	Hybrid	https://github.com/deepvk/istftnet
PriorGrad [ 30 ]	DDPM	ICLR 2022	T	https://github.com/microsoft/NeuralSpeech/tree/master/PriorGrad-vocoder
Avocodo [ 74 ]	GAN	AAAI 2023	T	https://github.com/ncsoft/avocodo
HiFTNet [ 75 ]	GAN	Arxiv 2023	Hybrid	https://github.com/yl4579/HiFTNet
BigVGAN [ 39 ]	GAN	ICLR 2023	T	https://github.com/NVIDIA/BigVGAN

TABLE II: Descriptions of the baseline methods used in the LJSpeech or LibriTTS benchmark.

Fig. 6: Illustration of the proposed MCDA strategy.

Settings		P	#Param.	#MACs	Inference Speed		PESQ $\uparrow$	MCD $\downarrow$	Periodicity $\downarrow$	V/UV $\uparrow$	Pitch $\downarrow$
Settings		P	(M)	(Giga/5s)	CPU	GPU	PESQ $\uparrow$	MCD $\downarrow$	RMSE	F1	RMSE
(1)	Remove Narrow-Band module	-	7.88	8.36	0.0419(23.88 $\times$ )	0.0048(210 $\times$ )	1.697	4.890	0.291	0.866	89.166
(2)	Remove Cross-Band module	2	8.98	19.81	0.0527(18.98 $\times$ )	0.0050(199 $\times$ )	3.157	3.428	0.130	0.953	32.555
(3)	Replace the proposed module with RNN	2	21.59	150.81	0.2098(4.77 $\times$ )	0.0211(47 $\times$ )	3.781	2.402	0.098	0.966	22.451
(4)	Increase the number of ConvNext v2 blocks	1	8.68	16.67	0.0581(17.22 $\times$ )	0.0061(164 $\times$ )	3.651	2.462	0.104	0.963	24.652
(5)		2	9.48	24.98	0.0770(12.98 $\times$ )	0.0063(158 $\times$ )	3.763	2.394	0.099	0.966	23.419

TABLE III: The ablation studies conducted on the LJSpeech dataset. For the inference speed, “

a\times

” denotes the inference ratio over the real-time. “P” denotes the number of ConvNext v2 blocks within each narrow-band module. The inference speed on a CPU is based on a CPU Intel(R) Core(TM) i7-14700F, and for GPU it is based on NVIDIA GeForce RTX 4060 Ti.

\uparrow

denotes the higher the better, and

\downarrow

denotes the opposite situation. The best results are labeled in bold.

Fig. 7: Illustration of the proposed omnidirectional phase loss.

Fig. 8: A typical spectral visualization where either narrow-band or cross-band module is removed. Within each figure, the x- and y-coordinates represent time and frequency, respectively.

Models	#Param.	#MACs	Inference Speed		M-STFT	PESQ	MCD	Periodicity	V/UV	Pitch	UTMOS	VISQOL
Models	(M)	(Giga/5s)	CPU	GPU	M-STFT	PESQ	MCD	RMSE	F1	RMSE	UTMOS	VISQOL
HiFiGAN-V1	13.94	152.90	0.1669(5.99 $\times$ )	0.0069(145 $\times$ )	1.167	3.574	3.671	0.134	0.947	33.687	4.218	4.771
iSTFTNet-V1	13.26	107.76	0.0949(10.54 $\times$ )	0.0038(266 $\times$ )	1.188	3.535	3.625	0.135	0.947	35.106	4.236	4.756
Avocodo	13.94	152.95	0.1611(6.21 $\times$ )	0.0063(158 $\times$ )	1.165	3.604	3.657	0.139	0.946	32.989	4.152	4.771
HiFTNet	22.43	163.05	0.1559(6.41 $\times$ )	0.0085(117 $\times$ )	0.923	3.992	2.039	0.092	0.966	20.780	4.343	4.849
BigVGAN-base	13.94	152.90	0.3856(2.59 $\times$ )	0.0368(27 $\times$ )	0.978	3.603	2.331	0.120	0.956	30.277	4.208	4.822

TABLE IV: Objective comparisons among different GAN-based baselines on the LJSpeech benchmark. The best and second-best performances are respectively highlighted in bold and underlined.

Models	#Param.	#MACs	M-STFT	PESQ	MCD	Periodicity	V/UV	Pitch	UTMOS	VISQOL
Models	(M)	(Giga/5s)	M-STFT	PESQ	MCD	RMSE	F1	RMSE	UTMOS	VISQOL
HiFiGAN-V1	14.01	166.41	1.104	3.056	4.284	0.167	0.921	52.529	3.511	4.721
iSTFTNet-V1	13.33	117.30	1.157	2.880	4.480	0.167	0.918	53.072	3.464	4.655
Avocodo	14.01	166.47	1.113	3.217	4.314	0.161	0.913	51.600	3.358	4.762
BigVGAN-base (1M steps) $\dagger$	14.01	166.41	0.879	3.519	-	0.129	0.946	-	-	-
BigVGAN (1M steps) $\dagger$	112.39	454.08	0.800	4.027	-	0.102	0.960	-	-	-

TABLE V: Objective comparisons among different GAN-based baselines on the LibriTTS benchmark. “-” denotes the results are not reported, and

\dagger

denotes the results are calculated using the open-sourced model checkpoints.

Fig. 9: Loss curves of the null-constraint loss and idempotent loss.

Fig. 10: Spectral visualization of the range-space, null-space and the reconstruction error between the estimated and target mel-spectrograms for Id5, Id11-Id13.

Fig. 11: Average preference scores (in %) of AB tests between RNDVoC-shared and three other GAN-based baselines. (a)-(d) Mel-spectrograms are obtained from speech clips in the LibriTTS test set. (e)-(h) Mel-spectrograms are obtained from vocal clips in the MUSDB18 test set.

Models	GT	HiFiGAN-V1	Avocodo	BigVGAN-base	BigVGAN	APNet2	Vocos	RNDVoC-nonshared	RNDVoC-shared
MUSHRA	89.45 $\pm$ 0.45	69.99 $\pm$ 1.15	68.16 $\pm$ 1.26	74.27 $\pm$ 1.12	79.33 $\pm$ 0.92	54.95 $\pm$ 1.11	73.18 $\pm$ 1.15	78.76 $\pm$ 0.94	**80.74 $\pm$ 0.99
Note: ${}^{*}p<0.05$ , ${}^{}p<0.1$

TABLE VI: MUSHRA scores among different methods on the LibriTTS benchmark. The confidence level is 95%, and we performed a t-test comparing RNDVoc-shared with BigVGAN.

Models	EARS (English Corpus with Diverse Emotions)							VCTK (English Corpus with Unseen Speakers)
Models	PESQ	Periodicity	V/UV F1	Pitch	UTMOS	VISQOL	MUSHRA	PESQ	Periodicity	V/UV F1	Pitch	UTMOS	VISQOL	MUSHRA
GT	-	-	-	-	3.300	-	89.06 $\pm$ 0.48	-	-	-	-	4.117	-	89.52 $\pm$ 0.45
HiFiGAN-V1	2.907	0.149	0.876	48.417	2.835	4.570	70.82 $\pm$ 1.12	3.090	0.115	0.943	33.289	3.952	4.723	78.49 $\pm$ 1.10
Avocodo	3.065	0.149	0.866	48.494	2.809	4.641	69.15 $\pm$ 1.29	3.337	0.114	0.943	33.041	3.905	4.758	73.14 $\pm$ 1.13
BigVGAN-base	3.797	0.087	0.946	26.050	3.040	4.871	78.19 $\pm$ 0.95	3.859	0.079	0.965	28.847	3.970	4.893	79.73 $\pm$ 0.86
BigVGAN	4.236	0.064	0.963	17.655	3.190	4.946	80.80 $\pm$ 0.84	4.282	0.062	0.972	20.316	4.034	4.959	80.59 $\pm$ 0.84

TABLE VII: Metric comparisons on two out-of-domain datasets among different GAN-based methods. All the models are pretrained on the LibriTTS dataset, and BigVGAN-base and BigVGAN are pretrained for 5M steps. For the MUSHRA test, the confidence level is 95%, and we conducted one-tailed two-paired KS-test comparing RNDVoC-shared with BigVGAN.

Models	Bass	Drums	Mixture	Other	Vocals	Mean
HiFiGAN-V1	4.068	4.452	4.349	4.346	4.631	4.3691
Avocodo	4.491	4.568	4.479	4.480	4.685	4.5406
BigVGAN-base	4.674	4.792	4.742	4.727	4.851	4.757
BigVGAN	4.825	4.885	4.874	4.853	4.924	4.872
APNet2	4.274	4.230	4.122	4.246	4.539	4.282
Vocos	4.7035	4.785	4.684	4.677	4.835	4.737

TABLE VIII: VISQOL scores of multiple methods on the test set of MUSDB18 test set. Note that BigVGAN-base and BigVGAN are trained for 5M steps while other models are trained for 1M steps.

Fig. 12: Average preference scores (in %) of AB tests between RNDVoC-shared and three other diffusion-based baselines. Clips are from the vocal type of the MUSDB18 test set.

Fig. 13: Spectral visualization of different vocoder methods. The audio clip is a singing voice from the MUSDB18 test set.

Fig. 14: Metric comparisons in terms of whether the proposed MCDA strategy is adopted. (a)-(c): Metric performance under varying number of mel filters

F_{m}

, and

f_{\text{max}}

is fixed to 12 kHz. (d)-(f): Metric performance under varying

f_{\text{max}}

, and

F_{m}

is fixed to 100.

Models	NFE	#Param.	#MACs	LJSpeech			LibriTTS			EARS			VCTK
Models	NFE	(M)	(Giga/5s)	PESQ	UTMOS	VISQOL	PESQ	UTMOS	VISQOL	PESQ	UTMOS	VISQOL	PESQ	UTMOS	VISQOL
FreGrad	50	1.82	1589	3.775	3.934	4.574	3.793	3.217	4.699	3.495	2.753	4.526	3.803	3.670	4.579
PriorGrad	50	2.70	8364	3.955	4.006	4.659	4.017	3.329	4.737	3.847	2.863	4.636	4.030	3.740	4.685
WaveFM	6	19.50	1188	-	-	-	3.954	3.191	4.943	3.856	2.709	4.926	3.960	3.631	4.928
PeriodWave	16	29.81	4986.56	4.235	4.374	4.722	4.240	3.652	4.749	4.238	3.218	4.686	4.248	4.082	4.671
RNDVoC-nonshared	1	9.48	28.29	3.892	4.138	4.812	4.085	3.607	4.873	3.856	3.078	4.821	4.022	3.937	4.863

TABLE IX: Metric comparisons on multiple datasets between our method and advanced diffusion-based methods. For EARS and VCTK evaluations, models are pretrained on the LibriTTS benchmark. “NFE” denotes the number of function evaluations, and we choose the default NFE setup in the original literature.

Subband	w/i Even	#Pram.	#MACs (Giga/5s)			M-STFT	PESQ	MCD	V/UV	Periodicity	UTMOS	VISQOL
Number $N$	w/i Even	(M)	BAEM	LKCAM	BAMM+BAPM	M-STFT	PESQ	MCD	F1	RMSE	UTMOS	VISQOL
1	-	200.89	4.30	73.74	15.97	0.912	3.490	3.220	0.947	0.117	3.423	4.810
6	$\times$	3.88	0.37	5.91	41.23	0.891	3.680	3.025	0.949	0.116	3.398	4.860
12	$\times$	3.38	0.37	11.82	21.73	0.836	3.939	2.500	0.958	0.097	3.552	4.904
24	$\times$	3.14	0.37	23.61	13.09	0.746	4.226	1.913	0.970	0.074	3.657	4.915
24	✓	3.09	0.37	23.64	10.58	0.830	4.018	2.319	0.959	0.090	3.623	4.904

TABLE X: Objective performance under different number of subbands.

N=1

denotes the full-band model is adopted, and all subband-related operations are removed. Results are based on the LibriTTS benchmark.

Sets	$F_{m}$ range	$\Delta F_{m}$	$f_{\text{max}}$ range (Hz)	$\Delta f_{\text{max}}$ (Hz)
MCDA-1	$\{88,96,100\}$	-	$[9\text{k},10\text{k}]$	100
MCDA-2	$[64,128]$	16	$[8\text{k},12\text{k}]$	100
MCDA-3	$[64,128]$	1	$[8\text{k},12\text{k}]$	50

TABLE XI: Detailed setups of three MCDA strategies with varying configuration pools.

\left\{\Delta F_{m},\Delta f_{\text{max}}\right\}

denote the sampling interval of

\left\{F_{m},f_{\text{max}}\right\}

, respectively.

Models	#Parm.	Sets	Inference Speed		PESQ	VISQOL
Models	(M)	Sets	CPU	GPU	PESQ	VISQOL
HiFiGAN-V2	0.92	LJSpeech	0.0257(38.71 $\times$ )	0.0014(725 $\times$ )	2.848	4.542
HiFiGAN-V2	0.92	LibriTTS	0.0288(34.72 $\times$ )	0.0014(704 $\times$ )	2.195	4.370
RNDVoC-Lite	0.71	LJSpeech	0.0267(37.43 $\times$ )	0.0019(531 $\times$ )	3.769	4.782
RNDVoC-Lite	0.71	LibriTTS	0.0261(38.33 $\times$ )	0.0020(502 $\times$ )	3.834	4.874
RNDVoC-UltraLite	0.08	LJSpeech	0.0251(39.04 $\times$ )	0.0018(557 $\times$ )	3.264	4.701

TABLE XII: Objective comparisons among lightweight neural vocoders.

Fig. 15: Spectral visualizations between lightweight neural vocodes. Audio clips are from the LibriTTS test set.

Fig. 16: Effect of the proposed RND strategy for different neural vocoders.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 17: Implementation process of the region-oriented and subband-oriented spectral encoding and decoding, respectively.

Fig. 18: Complexity comparisons between the non-shared and shared schemes for encoding/decoding modules.

Scheme	Non-shared Scheme			Shared Scheme
Channels $C$	#Param.(M)	#MACs(G)	Time(s)	#Param.(M)	#MACs(G)	Time(s)
64	0.67	0.06	24.61	0.11	0.52	12.23
128	2.12	0.19	39.30	0.32	1.11	13.30
256	7.38	0.64	59.06	0.93	2.50	18.04
384	15.79	1.38	62.53	2.15	4.16	21.53

Appendix

TABLE XIII: A case comparison in terms of trainable parameters (in million), computational complexity (in Giga), and processing time (in second) under different channel settings.

Fig. 19: Framework diagrams of the RNDVoC with a cascaded generation style.

Fig. 20: Framework diagrams of the RNDVoC defined in the magnitude and complex-valued domains. (a) The proposed RNDVoC defined in the magnitude domain. (b) The proposed RNDVoC defined in the complex domain.

Model Type	M-STFT $\downarrow$	PESQ $\uparrow$	MCD $\downarrow$	UTMOS $\uparrow$
RNDVoC-Cascaded	0.827	3.949	2.623	3.437
Ref. (Main Text)	0.746	4.226	1.913	3.657

Appendix

TABLE XIV: Performance comparisons between RNDVoC-Cascaded and the adopted format in the main text. Results are based on the LibriTTS benchmark.

Setting		P	#Param.	#MACs	Inference Speed		PESQ $\uparrow$	MCD $\downarrow$	Periodicity $\downarrow$	V/UV $\uparrow$	Pitch $\downarrow$
Setting		P	(M)	(Giga/5s)	CPU	GPU	PESQ $\uparrow$	MCD $\downarrow$	RMSE	F1	RMSE
(1)	RNDVoC-Mag	2	9.48	24.98	0.0770(12.98 $\times$ )	0.0063(158 $\times$ )	3.763	2.394	0.099	0.966	23.419
(2)	RNDVoC-Com	2	11.07	26.39	0.0988(10.12 $\times$ )	0.0076(131 $\times$ )	3.335	2.814	0.120	0.958	27.867

Appendix

TABLE XV: The comparisons between RNDVoC-Mag and RNDVoC-Com on the LJSpeech dataset.

Type	$F_{m}$	$f_{min}$ (Hz)	$f_{max}$ (kHz)	M-STFT	PESQ $\uparrow$	MCD $\downarrow$	UTMOS $\uparrow$
w/o MCDA	72	0	12	1.505	2.393	4.118	2.477
	80	0	12	1.315	2.951	3.420	3.109
	88	0	12	1.226	3.266	3.062	3.204
	96	0	12	1.087	3.350	3.093	3.246
	100	0	12	0.780	4.085	2.082	3.552
	104	0	12	1.023	3.570	2.430	3.434

Appendix

TABLE XVI: Objective performance between “w/o MCDA” and three MCDA variants in terms of different number of mel-filters.

f_{max}

is fixed to 12 kHz.

Type	$F_{m}$	$f_{min}$ (Hz)	$f_{max}$ (kHz)	M-STFT	PESQ $\uparrow$	MCD $\downarrow$	UTMOS $\uparrow$
w/o MCDA	100	0	8	2.549	3.557	4.676	3.403
	100	0	9	2.220	3.613	3.687	3.433
	100	0	10	1.897	3.654	2.631	3.440
	100	0	11	1.502	3.523	2.464	3.422
	100	0	11.5	1.343	3.479	2.496	3.424
	100	0	12	0.798	4.085	2.250	3.552

Appendix

TABLE XVII: Objective performance between “w/o MCDA” and three MCDA variants in terms of different upper-bound frequency values.

F_{m}

is fixed to 100.

Fig. 21: Spectral visualizations given different deviations of mel-filter. (a): Target. (b): Default configuration (Slaney strategy with region normalizaiton). (c): Slaney strategy without normalization. (d): Slaney strategy with $L_{1}$ normalization. (e): Slaney strategy with $L_{2}$ normalization. (f): HTK strategy with region normalization. (g): HTK without normalization. (h): HTK with $L_{1}$ normalization. (i): HTK with $L_{2}$ normalization. (j)-(o): Based on (b), add Gaussian noise with varying strength $\sigma$ , ranging from $10^{-7}$ to $10^{-2}$ .

Fig. 22: Sensitivity analysis of mel-filter in terms of PESQ and UTMOS. Results are calculated based on the LibriTTS benchmark.

	layer name	input size	hyper-parameters	output size
	RSM	$100\times T$	-	$513\times T$
	BAEM	$513\times T$	$(3,12),(3,24),(3,44)$	$24\times 256\times T$
Cross	reshape_1	$24\times 256\times T$	-	$T\times 256\times 24$
	gconv1d_1	$T\times 256\times 24$	$(256,3,1,8)$	$T\times 256\times 24$
	pconv1d_1	$T\times 256\times 24$	$(64,1,1,1)$	$T\times 64\times 24$
	band_mixer	$T\times 64\times 24$	$(24,64)$	$T\times 64\times 24$

Appendix

TABLE XVIII: Detailed parameter illustrations of the RNDVoC-Shared. For BAEM, hyper-parameters denote kernel sizes for three spectral regions along the time and frequency axes, respectively. For conv1d layers, hyper-parameters denote (output_channel, kernel_size, stride_size, group_num). For band_mixer layer, hyper-parameters denotes (output_size, group_num).

Test sets	Sound Effect from [ 101 ]			AISHELL3 (Mandarin)
Models	M-STFT $\downarrow$	MCD $\downarrow$	VISQOL $\uparrow$	M-STFT $\downarrow$	PESQ $\uparrow$	MCD $\downarrow$	V/UV $\uparrow$	Period. $\downarrow$	UTMOS $\uparrow$	VISQOL $\uparrow$
Models	M-STFT $\downarrow$	MCD $\downarrow$	VISQOL $\uparrow$	M-STFT $\downarrow$	PESQ $\uparrow$	MCD $\downarrow$	F1	RMSE	UTMOS $\uparrow$	VISQOL $\uparrow$
GT	-	-	-	-	-	-	-	-	2.700	-
HiFiGAN-V1	1.462	6.971	4.291	1.059	2.823	2.827	0.934	0.163	2.432	4.651
iSTFTNet-V1	1.540	7.352	4.150	1.132	2.595	3.061	0.932	0.164	2.351	4.486
Avocodo	1.488	6.795	4.426	1.040	3.003	2.864	0.937	0.154	2.126	4.731

Appendix

TABLE XIX: Objective comparisons among different baselines on the out-of-distribution sound effect evaluation set from [8] and AISHELL3 datasets. All models are pretrained on the LibriTTS dataset.

Split-Type	Total Subbands	Range (Hz)	Width (Hz)	Sub-band number	Kernel Size	Stride Size
Uneven	6	0-3375	1125	3	(48, 3)	(48, 1)
		3375-7875	2250	2	(96, 3)	(96, 1)
		7875-12000	4125	1	(176, 3)	(176, 1)
	12	0-3375	562.5	6	(24, 3)	(24, 3)
		3375-7825	1125	4	(48, 3)	(48, 1)
		7825-12000	2062.5	2	(88, 3)	(88, 1)

Appendix

TABLE XX: Different subband division schemes, including total number of sub-bands, frequency range, bandwidths, kernel size, and stride size.

SE Models	Discriminative				Generative
Vocoders	PESQ ( $\uparrow$ )	eSTOI (in %) $\uparrow$	P808-DNSMOS $\uparrow$	VISQOL $\uparrow$	PESQ $\uparrow$	eSTOI (in %) $\uparrow$	P808-DNSMOS $\uparrow$	VISQOL $\uparrow$
Mixture	1.163	57.48	2.580	2.512	1.163	27.48	2.580	2.512
HiFiGAN-V1	1.903	75.89	3.268	3.601	1.739	71.78	3.440	3.318
iSTFTNet-V1	1.818	74.76	3.275	3.551	1.677	70.87	3.416	3.268
Avocodo	1.892	75.47	3.212	3.613	1.720	71.48	3.389	3.311
BigVGAN-base	2.051	76.62	3.252	3.698	1.839	72.77	3.422	3.375

Appendix

TABLE XXI: Comparisons among different neural vocoders on the SE task. Note that BigVGAN-base and BigVGAN are pretrained for 5M steps while other models are trained for 1M steps.

Fig. 23: Detailed structure of the adopted discriminative SE networks.

Fig. 24: Framework diagram of the adopted discriminative and generative networks for SE task. For both methods, a pre-trained neural vocoder is followed for target waveform generation. (a) Discriminative method, where a discriminative SE network is utilized for target mel-spectrogram mapping. (b) Generative method, where a Schrödinger bridge is established between noisy and target mel-spectrograms.

Fig. 25: Spectral Visualizations generated by different neural vocoders. The file is a vocal sound from the song “Al James - Schoolboy Facination” in the MUSDB18 corpus.

Fig. 26: Spectral Visualizations generated by different neural vocoders. The file is a vocal sound from the song “Ben Carrigan - We’ll Talk About It All Tonight” in the MUSDB18 corpus.

Fig. 27: Spectral Visualizations generated by different neural vocoders. The file is a vocal sound from the song “BKS - Bulldozer - We’ll Talk About It All Tonight” in the MUSDB18 corpus.

Fig. 28: Spectral Visualizations generated by different neural vocoders. The file is a vocal sound from the song “Mu - Too Bright” in the MUSDB18 corpus.

Fig. 29: Screenshot of speech similarity for MUSRA testing.

Neural Vocoders

See all 9 papers

Sep 11, 2026

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction

A vocoder is a pivotal component of modern text-to-speech (TTS) systems. Despite the significant progress of neural network-based vocoders, accurate phase reconstruction remains the main challenge limiting both audio quality and modeling efficiency. We introduce PhaseGAN, a lightweight vocoder that addresses this limitation through a "mel

\rightarrow

Amplitude

\rightarrow

Phase" reconstruction pipeline. By reconstructing amplitude and phase spectra via distinct methodologies, the proposed PhaseGAN outperforms state-of-the-art baselines while utilizing fewer model parameters and reduced computational requirements. The compact version generates high-fidelity audio with approximately 500K parameters and 1 GMAC computational load, making it highly suitable for real-time applications on edge devices. In addition, our approach exhibits exceptional musical audio synthesis capabilities despite no training on musical data, illustrating unprecedented cross-domain generalization. See https://github.com/phasegan/phasegan-audio-demo for demos of our work.

Wenzheng Zhang, Xueliang Zhang, Shulin He +7

College of Computer Science, Inner Mongolia University, China

Neural Vocoders Speech Processing Audio Generation Generative Adversarial Networks

Ask PDF

Mar 12, 2026

Toward Complex-Valued Neural Networks for Waveform Generation

Neural vocoders have recently advanced waveform generation, yielding natural and expressive audio. Among these approaches, iSTFT-based vocoders have recently gained attention. They predict a complex-valued spectrogram and then synthesize the waveform via iSTFT, thereby avoiding learned upsampling stages that can increase computational cost. However, current approaches use real-valued networks that process the real and imaginary parts independently. This separation limits their ability to capture the inherent structure of complex spectrograms. We present ComVo, a Complex-valued neural Vocoder whose generator and discriminator use native complex arithmetic. This enables an adversarial training framework that provides structured feedback in complex-valued representations. To guide phase transformations in a structured manner, we introduce phase quantization, which discretizes phase values and regularizes the training process. Finally, we propose a block-matrix computation scheme to improve training efficiency by reducing redundant operations. Experiments demonstrate that ComVo achieves higher synthesis quality than comparable real-valued baselines, and that its block-matrix scheme reduces training time by 25%. Audio samples and code are available at https://hs-oh-prml.github.io/ComVo/.

Hyung-Seok Oh, Deok-Hyeon Cho, Seung-Bin Kim +1

Department of Artificial Intelligence Korea University Seoul, Republic of Korea

Neural Vocoders Complex-Valued Neural Networks Speech Processing Generative Adversarial Networks

Ask PDF

Dec 23, 2025

Aliasing-Free Neural Audio Synthesis

In neural audio synthesis, neural vocoders and codecs are models that reconstruct waveforms from acoustic and latent representations, which are essential to the resulting audio quality. While current models are capable of generating perceptually natural speech, they still struggle with high-fidelity music and singing voice synthesis, as severe aliasing artifacts are introduced by non-linear activation functions and upsampling layers in existing architectures. Although various anti-aliasing techniques have been proposed in digital signal processing, their integration into neural vocoders and codecs remains under-explored. This paper incorporates differentiable anti-aliasing techniques into the activation and upsampling modules to bridge this gap, and thus presents Pupu-Vocoder and Pupu-Codec. We build a test signal benchmark to evaluate the anti-aliased modules, and validate our proposed models on speech, singing voice, music, and audio. Experimental results show that Pupu-Vocoder and Pupu-Codec outperform existing systems on singing voice, music, and audio, while achieving comparable performance on speech. Demos, codes, and checkpoints are available at: VocodexElysium.github.io/AliasingFreeNeuralAudioSynthesis/.

Yicheng Gu, Junan Zhang, Chaoren Wang +3

Spellbrush, Akihabara, Tokyo 101-0021, Japan · Acoustic Laboratory, Department of Information Communications Engineering, Aalto University, Espoo 02150, Finland · School of Data Science, The Chinese University of Hong Kong, Shenzhen, Guangdong 518172, China

Singing Voice Synthesis Neural Audio Codecs Neural Vocoders Speech Processing Audio Generation

Ask PDF

Abstract

Neural Vocoders

Explore similar work

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction

Toward Complex-Valued Neural Networks for Waveform Generation

Aliasing-Free Neural Audio Synthesis