VocalTrax recovers interpretable articulatory controls from a recording by gradient descent through a differentiable Pink Trombone synthesizer. Its vocal tract is a Kelly-Lochbaum waveguide updated twice per audio sample, so every gradient step backpropagates through a long recurrence, which is slow on GPUs. We propose Convolutional VocalTrax, which computes the exact frequency response of the same waveguide and applies it as a causal convolution. We assume that the tract is stationary within 5.8 ms blocks and that its lossy impulse response can be truncated after 93 ms. Gradients then pass through FFTs and a short recursion along the tract instead of through time. On 600 AudioMNIST recordings, the tract forward and backward pass runs 47 to 63 times faster on an A100 GPU, and a classifier trained only on real speech recognizes our reconstructions more often than those of VocalTrax (14.5% vs. 10.0%), although the log-mel distance to the target is 11% higher. Parameters fitted with our model keep this gain when rendered by the original VocalTrax (13.8%).
Figures & tables
Block H
Fixed
5% motion
20% motion
Time (s)
64
8.1×10−6
0.79%
2.95%
0.36
128
8.1×10−6
0.86%
3.33%
0.19
256
8.1×10−6
1.04%
4.00%
0.17
512
8.1×10−6
1.46%
5.55%
0.11
Table 1: Validation of the two assumptions. Top: relative waveform error of our model against VocalTrax for fixed geometry and for diameter motion of 5% and 20%, and CPU time of one forward pass over 8192 samples. Bottom: sensitivity of one output sample of VocalTrax to past source samples, as the lag containing 99% and 99.9% of the energy and the energy fraction beyond 100 ms.
Figure 1: Spectrograms of the digit “zero” (speaker 55) and its 1000-step reconstructions. (d) renders the parameters fitted by our model with the original recurrent VocalTrax, without further optimization. All panels share one dB scale, and no signal is normalized.
Audio
Resampled to 44.1 kHz, right-padded to a multiple of 1024 samples, one extra control frame at the end
F0
CREPE full model, 23 ms hop; confidence ≤0.5 set to 1 Hz; interpolated to the control frames; fixed during fitting
Controls
5 per 1024-sample frame, normalized to [0,1] : tongue diameter [2.05,3.5] , tongue position [12,29] , throat and lip constriction [−0.99,0.99] , tenseness [0.1,1]
Projection to [0,1] after every update and every smoothing
Smoothing
Savitzky-Golay, window 10, order 7, every 200 steps, averaged 1:1 with the current values; skipped below 10 frames
Table 2: Fitting protocol shared by both models.
Acc. (%) ↑
Log-mel ↓
F0 (ct) ↓
V-F1 ↑
Real
86.7 [81.6, 91.1]
VocalTrax
10.0 [7.7, 12.3]
0.250
46.0
0.843
Ours
14.5 [11.8, 17.0]
0.278
40.1
0.846
Transfer
13.8 [10.9, 16.5]
0.310
41.1
0.849
Table 3: Recognition by a classifier trained on real speech, and distances to the reference, on 600 AudioMNIST recordings. Brackets give 95% intervals. Transfer renders our parameters with VocalTrax. F0 : RMS error in cents; V-F1: voicing F1 score.
Tract, fwd + bwd
Full optimizer step
Audio
VocalTrax
Ours
Ratio
VocalTrax
Ours
Ratio
1 s
1.111
0.023
47.5
1.45
0.50
2.9
8 s
9.30
0.148
62.6
11.77
3.62
3.3
16 s
17.63
0.297
59.3
22.98
7.13
3.2
30 s
34.46
0.546
63.1
42.46
13.35
3.2
60 s
68.06
1.092
62.3
85.19
28.98
2.9
Table 4: Time per call in seconds on one A100 at batch size one. The tract columns add the forward and backward passes of the tract. The full step includes the shared LF source, the loss and the optimizer.
Descriptor
VocalTrax
Ours
Transfer
Log-STFT L1 ↓
1.566
1.579
1.714
MFCC L2 ↓
7.19
7.80
7.74
MFCC DTW cost ↓
6.35
6.66
6.46
Centroid error (Hz) ↓
2587
2727
2813
Contrast error (dB) ↓
5.27
5.83
5.35
Δ MFCC L1 ↓
0.589
0.572
0.576
Table 5: Further descriptors averaged over the 600 recordings. Pitch measures use the 581 recordings voiced in all conditions.
Figure 2: Time per call of both tracts and the shared LF source, at batch size one on one A100.
Figure 3: Per digit (60 recordings each). Top: log-mel MSE. Bottom: frame-to-frame change of the four tract controls, ours divided by VocalTrax.