GAMW: Generation-Aware Mel-Domain Watermarking for TTS Traceability
Organizations: Zhejiang University
Abstract
As text-to-speech (TTS) systems continue to advance, provenance tracking and security verification of synthesized speech have become increasingly important. Audio watermarking provides active provenance by embedding imperceptible, machine-recoverable information into speech. However, existing post-hoc methods typically modify the waveform after synthesis, leaving the watermark vulnerable to compound perturbations. To address this limitation, we propose GAMW, a watermark injection framework integrated into the TTS generation process. Before waveform synthesis, GAMW embeds a content-adaptive multi-bit watermark residual into the generated representation, and its decoding objective combines focal modulation with dynamic per-bit weighting. We train and evaluate GAMW separately on two TTS backbones. With a 16-bit payload, GAMW improves decoding accuracy over the strongest baseline by 9.2 and 23.5 percentage points under individual and compound perturbations, respectively, while maintaining competitive speech quality. Reliable clean-speech decoding is also retained at higher payloads. Code is available at https://github.com/ZConJugaTe/GAMW.
Figures & tables
| Method | PESQ | ViSQOL | MOS | ACC |
|---|---|---|---|---|
| CosyVoice2 | ||||
| WavMark | 4.3008 | 4.8340 | 4.0628 | 1.0000 |
| AudioSeal | 4.3672 | 4.8884 | 4.0592 | 1.0000 |
| SilentCipher | 4.5701 | 4.9352 | 4.0745 | 0.8654 |
| TimbreWM | 3.8464 | 4.6662 | 3.9354 | 1.0000 |
| GAMW (ours) | 4.3001 | 4.9726 | 4.0775 | 1.0000 |
| (bits) | PESQ | ViSQOL | MOS | ACC |
|---|---|---|---|---|
| CosyVoice2 | ||||
| 16 | 4.3001 | 4.9726 | 4.0775 | 1.0000 |
| 32 | 4.2518 | 4.9614 | 4.0766 | 0.9976 |
| 50 | 4.2332 | 4.9562 | 4.0763 | 0.9992 |
| 100 | 4.0607 | 4.9205 | 4.0266 | 0.9978 |
| GLM-TTS | ||||
| Perturbation | WavMark | AudioSeal | SilentCipher | TimbreWM | GAMW |
|---|---|---|---|---|---|
| CosyVoice2 | |||||
| Gaussian noise (SNR = 10 dB) | 0.5008 | 0.6711 | 0.1669 | 0.6777 | 0.9837 |
| Gaussian noise (SNR = 20 dB) | 0.5946 | 0.8911 | 0.1751 | 0.9054 | 0.9977 |
| Gaussian noise (SNR = 30 dB) | 0.8572 | 0.9876 | 0.2370 | 0.9922 | 0.9996 |
| Background noise (SNR = 10 dB) | 0.6176 | 0.7256 | 0.2343 | 0.7224 | 0.9663 |
| Background noise (SNR = 20 dB) | 0.8580 | 0.8991 | 0.2635 | 0.8918 | 0.9963 |
| Position | PESQ | ViSQOL | MOS | ACC | ACC |
|---|---|---|---|---|---|
| Waveform | 2.1911 | 3.8863 | 3.2943 | 0.9991 | 0.9048 |
| Mel (ours) | 4.2914 | 4.9721 | 4.0790 | 0.9999 | 0.9795 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Perturbation | Training parameter |
|---|---|
| Gaussian noise | SNR dB |
| Background noise | SNR dB |
| Low-pass filtering | Normalized cutoff |
| High-pass filtering | Normalized cutoff |
| Smoothing | Window samples |
| Echo | Delay s; gain |
| Perturbation | Definition | Reported parameter(s) |
|---|---|---|
| Gaussian noise | Add zero-mean Gaussian noise scaled according to the signal power and target SNR. | SNR dB |
| Background noise | Repeat or truncate a real background-noise recording to the utterance length and scale it to the target SNR before mixing. | SNR dB |
| Low-pass filtering | Apply the Julius low-pass filter to the waveform. | Normalized cutoff |
| High-pass filtering | Apply the Julius high-pass filter to the waveform. | Normalized cutoff |
| MP3 compression | Encode and decode the waveform using MP3 compression with a fixed bitrate. | Bitrate kbps |
| Opus compression | Encode and decode the waveform using Opus with a fixed bitrate and encoder complexity. | Bitrate kbps; complexity |
| Pipeline | Ordered operations and parameters |
|---|---|
| Gaussian noise + background noise | Gaussian noise (20 dB) background noise (20 dB) |
| EnCodec + Opus | EnCodec (12 kbps) Opus (16 kbps, complexity 1) |
| Echo + Gaussian noise | Echo (0.3 s, amplitude 0.4) Gaussian noise (20 dB) |
| EnCodec + MP3 | EnCodec (12 kbps) MP3 (16 kbps) |
| Time stretching + smoothing + Opus | Time stretching (1.1) smoothing (22 samples) Opus (16 kbps, complexity 1) |
| Low-pass filtering + Gaussian noise + EnCodec | Low-pass filtering (0.2) Gaussian noise (20 dB) EnCodec (12 kbps) |
| Perturbation | WavMark | AudioSeal | SilentCipher | TimbreWM | GAMW |
|---|---|---|---|---|---|
| CosyVoice2 | |||||
| Gaussian noise (SNR = 10 dB) | 0.5008 | 0.6711 | 0.1669 | 0.6777 | 0.9837 |
| Gaussian noise (SNR = 20 dB) | 0.5946 | 0.8911 | 0.1751 | 0.9054 | 0.9977 |
| Gaussian noise (SNR = 30 dB) | 0.8572 | 0.9876 | 0.2370 | 0.9922 | 0.9996 |
| Background noise (SNR = 10 dB) | 0.6176 | 0.7256 | 0.2343 | 0.7224 | 0.9663 |
| Background noise (SNR = 20 dB) | 0.8580 | 0.8991 | 0.2635 | 0.8918 | 0.9963 |
| Perturbation pipeline | WavMark | AudioSeal | SilentCipher | TimbreWM | GAMW |
|---|---|---|---|---|---|
| CosyVoice2 | |||||
| Gaussian noise + background noise | 0.5925 | 0.8136 | 0.1748 | 0.8192 | 0.9949 |
| EnCodec + Opus | 0.4694 | 0.3681 | 0.1756 | 0.5910 | 0.8981 |
| Echo + Gaussian noise | 0.5610 | 0.8170 | 0.1824 | 0.8693 | 0.9890 |
| EnCodec + MP3 | 0.4530 | 0.3666 | 0.1718 | 0.5656 | 0.8490 |
| Time stretching + smoothing + Opus | 0.4968 | 0.5647 | 0.1787 | 0.7751 | 0.8865 |
| Perturbation | CosyVoice2 | GLM-TTS | ||
|---|---|---|---|---|
| 16 | 32 | 16 | 32 | |
| Gaussian noise (10 dB) | 0.9837 | 0.9467 | 0.9744 | 0.9401 |
| Gaussian noise (20 dB) | 0.9977 | 0.9896 | 0.9944 | 0.9844 |
| Gaussian noise (30 dB) | 0.9996 | 0.9974 | 0.9984 | 0.9954 |
| Background noise (10 dB) | 0.9663 | 0.8655 | 0.9603 | 0.8713 |
| Background noise (20 dB) | 0.9963 | 0.9642 | 0.9916 | 0.9594 |