Unsupervised Instantaneous Phase and Frequency Tracking by Inverse Voice Synthesis

Chin-Yun Yu; György Fazekas

eess.ASOct 2, 2026

Unsupervised Instantaneous Phase and Frequency Tracking by Inverse Voice Synthesis

Authors: Chin-Yun Yu, György Fazekas

Organizations: Centre for Digital Music, Queen Mary University of London, Mile End Road, London E1 4NS, UK

Abstract

Knowledge-driven neural vocoders struggle to learn reliable fundamental frequency end-to-end, because spectral objectives provide weak supervision of periodic structure and lack phase information. We address this with a source-filter model whose alias-free additive source makes the instantaneous phase of the glottal cycle explicit; differentiating it yields the instantaneous frequency, and thus $F_0$ , without an external tracker. Waveform error supervises only the deterministic harmonic path, while a spectral loss covers the full signal. On M4Singer and LM-SSD, the reconstruction is phase-aligned, reaching a signal-to-reconstruction-error ratio of 8.1 dB, while neural baselines remain negative. However, GOLF, given an external $F_0$ , still reaches lower spectral distortion. On LM-SSD, the recovered $F_0$ attains the highest overall accuracy of any method tested, including supervised neural pitch trackers applied off the shelf, and the glottal closure instants come within 0.53 points of REAPER's identification rate, without any $F_0$ label.

Figures & tables

Panel of Figure 1 — Figure 1: Loss routing in GOLF-inv. MSE compares only $\hat{S}_{h}(t)$ with $S(z)$ , whereas MSS compares the spectrum of the full reconstruction $\hat{S}(z)$ with the target $S(z)$ .

Figure 2: Synchronised waveforms and GCIs on two test LM-SSD excerpts. The reconstruction

\hat{s}(t)

follows the target

s(t)

cycle by cycle.

Model	MSS( $\downarrow$ )	MCD( $\downarrow$ )	PESQ( $\uparrow$ )	SRER( $\uparrow$ )
GOLF-inv	0.981	4.469 dB	2.613	8.103 dB
GOLF	0.868	3.917 dB	2.838	-2.801 dB
DDSP+ $F_{0}$	1.072	5.175 dB	2.485	-2.665 dB
QHM	0.871	4.668 dB	2.967	13.745 dB

Table 1: Synthesis evaluation on the test split of M4Singer and LM-SSD datasets.

Model	Train		Test
Model	RPA ( $\uparrow$ )	OA ( $\uparrow$ )	RPA ( $\uparrow$ )	OA ( $\uparrow$ )
GOLF-inv	92.45	95.19	92.91	95.79
$\hookrightarrow$ $\hat{f}(t_{l})$ only	91.79	94.85	92.41	95.54
Fixed window	91.82	94.83	92.28	95.44
$\hookrightarrow$ $\hat{f}(t_{l})$ only	86.43	92.07	86.89	92.75
DDSP+ $F_{0}$	91.19	93.29	91.52	93.96

Table 2:

F_{0}

estimation results (%) on the LM-SSD dataset.

Model	IDR ( $\uparrow$ )	FAR ( $\downarrow$ )	MR ( $\downarrow$ )	$\sigma$ ( $\downarrow$ )
GOLF-inv	96.93%	1.10%	1.97%	0.77 ms
REAPER	97.46 %	0.82 %	1.72 %	0.76 ms

Table 3: GCI detection performance on LM-SSD test set.

Neural Vocoders

See all 9 papers

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction
Sep 11, 2026Wenzheng Zhang, Xueliang Zhang, Shulin He +7Neural Vocoders Speech Processing
College of Computer Science, Inner Mongolia University, China
Aliasing-Free Neural Audio Synthesis
Dec 23, 2025Yicheng Gu, Junan Zhang, Chaoren Wang +3Singing Voice Synthesis Neural Audio Codecs
Spellbrush, Akihabara, Tokyo 101-0021, Japan · Acoustic Laboratory, Department of Information Communications Engineering, Aalto University, Espoo 02150, Finland · School of Data Science, The Chinese University of Hong Kong, Shenzhen, Guangdong 518172, China
PitchFlower: A flow-based neural audio codec with pitch controllability
Oct 29, 2025Diego Torres, Axel Roebel, Nicolas ObinNeural Audio Codecs Controllable Speech Generation
STMS Lab IRCAM - CNRS - Sorbonne Universit´e Paris, France

Sep 11, 2026J/K move · Enter open · S save

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction

Wenzheng Zhang, Xueliang Zhang, Shulin He, Fei Zhao+6

College of Computer Science, Inner Mongolia University, China

Neural Vocoders Speech Processing Audio Generation Generative Adversarial Networks

Open paper AskPDF arXiv

A vocoder is a pivotal component of modern text-to-speech (TTS) systems. Despite the significant progress of neural network-based vocoders, accurate phase reconstruction remains the main challenge limiting both audio quality and modeling efficiency. We introduce PhaseGAN, a lightweight vocoder that addresses this limitation through a "mel

\rightarrow

Amplitude

\rightarrow

Phase" reconstruction pipeline. By reconstructing amplitude and phase spectra via distinct methodologies, the proposed PhaseGAN outperforms state-of-the-art baselines while utilizing fewer model parameters and reduced computational requirements. The compact version generates high-fidelity audio with approximately 500K parameters and 1 GMAC computational load, making it highly suitable for real-time applications on edge devices. In addition, our approach exhibits exceptional musical audio synthesis capabilities despite no training on musical data, illustrating unprecedented cross-domain generalization. See https://github.com/phasegan/phasegan-audio-demo for demos of our work.

Abstract

Neural Vocoders

Explore similar work

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction

Aliasing-Free Neural Audio Synthesis

PitchFlower: A flow-based neural audio codec with pitch controllability