We introduce AudioCodex, a fully causal autoencoder for live generative audio applications that compresses stereo audio into compact continuous latent sequences at ~10 Hz. It combines a Mean Flow magnitude autoencoder with latent hypersphere projection, bounded-noise augmentation, representation alignment, and adversarial phase recovery to learn representations that support both efficient downstream generation and high-quality reconstruction. Through extensive ablations, we show that AudioCodex outperforms the state-of-the-art in terms of generation-reconstruction trade-off, whilst remaining strictly causal in both encoding and decoding. This work paves the way towards truly real-time interactive and streaming generative audio applications.
Figures & tables
Figure 1: AudioCodex’s autoencoder main architecture. Blue denotes learnable components.
a) Autoencoder
Formulation
rFAD
gFAD
CM cSTFT
2.17
4.81
v -MF mSTFT∅CFG
0.83
3.63
x -MF mSTFT∅CFG
0.82
3.61
Bottleneck
tanh
0.82
5.14
Table 1: Architecture ablations for: a) the autoencoder, evaluated using r/gFAD; b) the phase estimator using rFAD.
Model
rFAD
rFAD C
gFAD
STFT
MEL
Codec Reference
DAC [ 9 ]
0.601
0.044
N/A
0.094
0.057
Baselines
M2L [ 2 ]
0.943
0.233
5.241
0.132
0.090
CDC [ 17 ]
0.304
0.088
5.065
0.114
0.079
SAO [ 10 ]
1.151
0.155
9.518
0.106
0.068
Table 2: Comparison to baselines. Metrics are defined in 5 . For all metrics, lower is better.
Component
Time (ms)
RTF
Single latent streaming
Encoding
6.80
12.56x
Combined decoding
41.38
2.06x
10s sample
Encoding
18.91
528.8x
Combined decoding
87.64
114.1x
Table 3: Inference latency and real-time factor (RTF) measured on a NVIDIA A100 GPU with 48 kHz stereo audio.