We present AudioNoisePrints, a training-free watermarking pipeline for flow matching and diffusion TTS models, which requires minimal extra computation during inference and does not require retraining the TTS model or reducing the generation quality. We exploited the fact that there are strong correlations between the initial Gaussian noises and the generated outputs in diffusion and flow matching models, such that a simple cosine correlation between the initial noise and the generated output can be used to perform watermaking. Moreover, we train a lightweight detector on top for more aggressive augmentations. Our method outperforms AudioSeal, a strong baseline for audio watermarking under strong augmentations. We experimented on F5TTS and other TTS and vocoder models, and concluded that they all exhibit similar spatial correlation properties, suggesting our watermarking scheme can be used for more flow-matching TTS models and even vocoders in the future.
Figures & tables
Figure 1: General overview of our approach ( AudioNoisePrints ) comparing to the commonly used post-hoc watermarking schemes. As shown above, the post-hoc watermarking model required altering the audio after it is generated, which creates computation overhead and alters the quality of the audio. Our method simply changes the initial noise x^init during generation (without altering the quality of the generated audio), and uses either the model-free option (calculating the cosine similarity of the audio-Mel-spectrogram and the initial noise) or the more robust detector option to determine if the audio is in fact watermarked.
Figure 2: Accuracy comparing the AudioSeal and our approaches. While AudioSeal has a higher top-end score (1.0 accuracy), under strong augmentation, our method (using either the model-free Cosine similarity approach or the Detector approach) significantly outperforms the post-hoc watermark baseline. Especially in the speed augmentation, where AudioSeal fails even witha 1 percent change in stretching.
τp
0.80
0.85
0.90
0.95
1.00
CoSim
0.905
0.930
0.949
0.975
0.988
L1
0.392
0.422
0.450
0.476
0.497
L2
0.405
0.424
0.448
0.474
0.497
Dot
0.885
0.922
0.955
0.972
0.986
Table 1: Performance comparison across similarity thresholds and distance functions (F5-TTS).
F5-TTS
Augmentation
CosSim
Detecor
AudioSeal
AAC compression
0.987
0.9947
1.0
Echo
0.504
0.499
1.0
Table 2: All three methodologies have near 1.0 accuracy AAC (mp4) compression. Yet the current methodology did not perform as well with the “echo” augmentation.
Figure 3: (Top) Distributions of random noise similarity versus originated noise similarity (red line) for the DiffWave vocoder [Left] and MatchTTS [Middle] and F5TTS models [Right]. The generated audio is significantly more similar to the original noise than random noises.
Department of Electrical and Computer Engineering, Queen’s University, Kingston, ON, Canada · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, Canada