cs.SDAug 22, 2024

Modeling Time-Dependent Responses of Optical Compressors with Selective State Space Models

Authors: Riccardo Simionato, Stefano Fasciani

Organizations: Department of Musicology University of Oslo Oslo, Norway

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

This paper presents a method for modeling optical dynamic range compressors using deep neural networks with Selective State Space models. The proposed approach surpasses previous methods based on recurrent layers by employing a Selective State Space block to encode the input audio. It features a refined technique integrating Feature-wise Linear Modulation and Gated Linear Units to adjust the network dynamically, conditioning the compression's attack and release phases according to external parameters. The proposed architecture is well-suited for low-latency and real-time applications, crucial in live audio processing. The method has been validated on the analog optical compressors TubeTech CL 1B and Teletronix LA-2A, which possess distinct characteristics. Evaluation is performed using quantitative metrics and subjective listening tests, comparing the proposed method with other state-of-the-art models. Results show that our black-box modeling methods outperform all others, achieving accurate emulation of the compression process for both seen and unseen settings during training. We further show a correlation between this accuracy and the sampling density of the control parameters in the dataset and identify settings with fast attack and slow release as the most challenging to emulate.

Figures & tables

Figure 1: Proposed architecture: the input x\boldsymbol{x} , which contains the current and past samples, is fed to a linear FC layer and subsequently to a S6 block (detailed in Figure 2 ). The resulting vector passes through the conditioning layer and another identical S6 block before reaching the output layer, which is a linear FC layer with one unit. The number of units (u) is indicated next to each layer. The output is a coefficient gg which multiplied by the current input sample xnx_{n} yields the current output sample yny_{n}
Figure 2: Internal architecture of the S6 block featured in the in architecture show in Figure 1 . The first FC layer has a number of units equal to twice the length of the input dimension and, in turn, is used to create a double projection of its input. The projection is split into two equal-sized vectors, one passing the convolutional layer and, afterward, the swish function and, finally, the S6 layer; the other is element-wise multiplied by the output of the S6 layer after the swish function. The result from the residual connection feeds a FC layer with a number of units equal to the length of the block input vector. Next to the layers are the reported number of units (u), activation functions, and kernel (k) when applicable.
Figure 3: Internal architecture of the conditioning block featured in the architecture shown in Figure 1 . The device control parameters, conditioning the model, are split into two vectors: those influencing the amount of compression pco\boldsymbol{p_{co}} and those determining the timing behavior pti\boldsymbol{p_{ti}} . The FiLM method, followed by a GLU featuring the softsign function, is used for pco\boldsymbol{p_{co}} . Similarly, the temporal FiLM method, followed by a GLU featuring the softsign function, is used for pti\boldsymbol{p_{ti}} . In both cases, the vector f\boldsymbol{f} computed by a convolutional layer, which takes the magnitude spectrum of the input x\boldsymbol{x} is concatenated to pco\boldsymbol{p_{co}} and pti\boldsymbol{p_{ti}} . In the case of the LA-2A, pco\boldsymbol{p_{co}} corresponds to the peak reduction and the switch mode, while pti\boldsymbol{p_{ti}} pertains to the vector f\boldsymbol{f} . For the CL 1B, pco\boldsymbol{p_{co}} includes both the threshold and ratio, whereas pti\boldsymbol{p_{ti}} encompasses the attack and release times. Next to the layers are the reported number of units (u) and kernel (k) when applicable.
Figure 4: Recurrent blocks used as alternative to the S6 block in the Figure 1 architecture for comparing performances. From left to right: S4D, ED, and LSTM blocks. In the ED block, the convolutional layer is included only before the conditioning block. Next to the layers are the reported number of units (u), activation function, and kernel (k) when applicable.
 FLOPs/sampleParameters
S61242, 1290984, 1000
S4D1220, 12681027, 1043
ED1252, 13001009, 1025
LSTM1668, 1716989, 1005
ED-b1762, 17941005, 1021
LSTM-b2453, 2869994, 994
Table 1: FLOPs for the inference of a single audio sample and the number of trainable parameters for all architectures used in the comparative experiments. Two values are reported for each architecture: the first corresponds to the LA-2A and the second to the CL 1B. The slight differences between the two sets of values are attributable to the varying number of control parameters in the two devices.
Figure 5: TubeTech CL 1B (top) and Teletronix LA-2A (bottom) optical range dynamic compressors used to collect the dataset for the experiments.
DatasetCombinationsDurationParametersRangeValues
Teletronix LA-2A203,900 sPeak Reduction[0, 100]10 EQS
   Switch[0, 1][0, 1]
TubeTech CL 1B8115,795 sRatio[2:1, 10:1]3 EQS
   Threshold[0, -40] dBu3 EQS
   Attack[0.5, 500] ms3 EQS
   Release[0.005, 10] s3 EQS
Table 2: Overview of datasets collected from the analog compressors and their software emulations, indicating dataset name, parameter combinations, total duration, and parameter sampling details.
SetCombinationsDurationParametersValues
Universal Audio LA 2A test61,170 sPeak Reduction[0.25, 0.55, 0.75]
   Switch[0, 1]
Universal Audio LA 2A train-0.0119437,830Peak Reduction100 EQS
   Switch[0, 1]
Universal Audio LA 2A train-0.05346,630 sPeak Reduction20 EQS
   Switch[0, 1]
Table 3: Overview of the training and test set splits for the Universal Audio LA-2A and Softube CL 1B datasets, indicating the parameter combinations, total duration, and details of the variable parameters with values normalized to the unit range. Fixed parameter values are provided in Table 2 . The numerical value at the end of the name of the training set indicates the step at which the normalized values are sampled.
 Average Training Time per Epoch (min)
SetS6S4DEDLSTMED-bLSTM-bTCN-b
Teletronix LA-2A7777771212444411
TubeTech CL 1B37373535343435352323222233
Universal Audio LA-2A-0.0125252323222236362424242411
Universal Audio LA-2A-0.0555555555665511
Universal Audio LA-2A-0.1022554444222211
Table 4: Average training time per epoch, including validation, across different datasets. Times are expressed in minutes.
DatasetModelMSEMAEESRRMSESFEM-STFTEPSM
LA2AS62.39⋅10−52.39\cdot 10^{-5}2.32⋅10−32.32\cdot 10^{-3}2.23⋅10−22.23\cdot 10^{-2}2.23⋅10−32.23\cdot 10^{-3}4.59⋅10−34.59\cdot 10^{-3}0.460.460.980.98
 S4D2.57⋅10−52.57\cdot 10^{-5}2.42⋅10−32.42\cdot 10^{-3}2.40⋅10−22.40\cdot 10^{-2}2.33⋅10−32.33\cdot 10^{-3}5.16⋅10−35.16\cdot 10^{-3}0.540.540.980.98
 ED3.23⋅10−53.23\cdot 10^{-5}2.87⋅10−32.87\cdot 10^{-3}3.01⋅10−23.01\cdot 10^{-2}2.78⋅10−32.78\cdot 10^{-3}5.60⋅10−35.60\cdot 10^{-3}0.760.760.990.99
 LSTM4.82⋅10−54.82\cdot 10^{-5}3.42⋅10−33.42\cdot 10^{-3}4.50⋅10−24.50\cdot 10^{-2}3.34⋅10−33.34\cdot 10^{-3}5.38⋅10−35.38\cdot 10^{-3}0.730.730.990.99
 ED-b8.15⋅10−58.15\cdot 10^{-5}5.46⋅10−35.46\cdot 10^{-3}7.61⋅10−27.61\cdot 10^{-2}5.12⋅10−35.12\cdot 10^{-3}7.22⋅10−37.22\cdot 10^{-3}0.780.780.980.98
 LSTM-b8.06⋅10−48.06\cdot 10^{-4}1.36⋅10−21.36\cdot 10^{-2}7.52⋅10−17.52\cdot 10^{-1}1.36⋅10−21.36\cdot 10^{-2}1.02⋅10−21.02\cdot 10^{-2}2.932.930.970.97
Table 5: Loss and performance metrics for models trained on the TubeTech CL 1B and Teletronix LA-2A datasets, measured on the test set.
Figure 6: Box plot displaying the MUSHRA rating results for the Teletronix LA-2A (top) and TubeTech CL 1B (bottom).
Figure 7: Spectrograms of the output of the Teletronix LA-2A and of the output of models, trained using the proposed S6 and alternative architectures. Each column presents spectrograms for different compression settings labeled above the columns. The model that produced each spectrogram is specified at the top of the respective spectrogram. ‘Target’ refers to the analog hardware device.
Figure 8: Spectrograms of the output of the TubeTech CL 1B, and of the output of models trained using the proposed S6 and alternative architectures. Each column presents spectrograms for different compression settings labeled above the columns. The model that produced each spectrogram is specified at the top of the respective spectrogram. ‘Target’ refers to the analog hardware device.
Figure 9: RMS levels measured at the output of the Teletronix LA-2A (top) and TubeTech CL 1B (bottom) compared with those at the output of models trained using the proposed S6 and alternative architectures. The applied compression settings are indicated above the plots. The legend associates the different color curves with their corresponding models, with ‘Target’ denoting the analog hardware device.
Figure 10: Spectral flux measured at the output of the Teletronix LA-2A (top) and TubeTech CL 1B (bottom) compared with those at the output of models trained using the proposed S6 and alternative architectures. The applied compression settings are indicated above the plots. The legend associates the different color curves with their corresponding models, with ‘Target’ denoting the analog hardware device.
SetModelMSEMAEESRRMSESFEM-STFTEPSM
0.10S66.10⋅10−46.10\cdot 10^{-4}1.19⋅10−2\mathbf{1.19\cdot 10^{-2}}1.17⋅10−11.17\cdot 10^{-1}1.10⋅10−21.10\cdot 10^{-2}1.02⋅10−2\mathbf{1.02\cdot 10^{-2}}1.861.860.950.95
 S4D8.04⋅10−48.04\cdot 10^{-4}1.34⋅10−21.34\cdot 10^{-2}1.54⋅10−11.54\cdot 10^{-1}1.24⋅10−21.24\cdot 10^{-2}1.33⋅10−21.33\cdot 10^{-2}2.212.210.950.95
 ED5.91⋅10−4\mathbf{5.91\cdot 10^{-4}}1.19⋅10−2\mathbf{1.19\cdot 10^{-2}}1.14⋅10−1\mathbf{1.14\cdot 10^{-1}}1.09⋅10−2\mathbf{1.09\cdot 10^{-2}}1.37⋅10−21.37\cdot 10^{-2}1.26\mathbf{1.26}0.950.95
 LSTM6.41⋅10−46.41\cdot 10^{-4}1.20⋅10−21.20\cdot 10^{-2}1.23⋅10−11.23\cdot 10^{-1}1.11⋅10−21.11\cdot 10^{-2}1.38⋅10−21.38\cdot 10^{-2}1.671.670.950.95
0.05S65.90⋅10−4\mathbf{5.90\cdot 10^{-4}}1.18⋅10−2\mathbf{1.18\cdot 10^{-2}}1.13⋅10−1\mathbf{1.13\cdot 10^{-1}}1.09⋅10−2\mathbf{1.09\cdot 10^{-2}}1.50⋅10−21.50\cdot 10^{-2}1.37\mathbf{1.37}0.950.95
 S4D1.09⋅10−31.09\cdot 10^{-3}1.54⋅10−21.54\cdot 10^{-2}2.10⋅10−12.10\cdot 10^{-1}1.44⋅10−21.44\cdot 10^{-2}1.10⋅10−2\mathbf{1.10\cdot 10^{-2}}2.232.230.950.95
Table 6: Loss and performance metrics for models trained on Universal Audio LA-2A datasets with different parameter sampling densities. The ‘Set’ columns indicate the training sets used, corresponding to the suffix of the training set names detailed in Table 3 . Models with the lowest loss and the minimum value for each metric are highlighted in bold.
SetModelMSEMAEESRRMSESFEM-STFTEPSM
0.20S61.05⋅10−41.05\cdot 10^{-4}6.80⋅10−36.80\cdot 10^{-3}2.12⋅10−12.12\cdot 10^{-1}6.50⋅10−36.50\cdot 10^{-3}1.02⋅10−2\mathbf{1.02\cdot 10^{-2}}1.861.860.940.94
 S4D1.60⋅10−41.60\cdot 10^{-4}8.41⋅10−38.41\cdot 10^{-3}3.25⋅10−13.25\cdot 10^{-1}8.07⋅10−38.07\cdot 10^{-3}1.33⋅10−21.33\cdot 10^{-2}2.212.210.930.93
 ED6.99⋅10−5\mathbf{6.99\cdot 10^{-5}}5.55⋅10−3\mathbf{5.55\cdot 10^{-3}}1.41⋅10−1\mathbf{1.41\cdot 10^{-1}}5.22⋅10−3\mathbf{5.22\cdot 10^{-3}}1.37⋅10−21.37\cdot 10^{-2}1.26\mathbf{1.26}0.920.92
 LSTM9.29⋅10−59.29\cdot 10^{-5}6.27⋅10−36.27\cdot 10^{-3}1.87⋅10−11.87\cdot 10^{-1}5.97⋅10−35.97\cdot 10^{-3}1.38⋅10−21.38\cdot 10^{-2}1.671.670.910.91
0.10S65.41⋅10−5\mathbf{5.41\cdot 10^{-5}}4.81⋅10−3\mathbf{4.81\cdot 10^{-3}}1.09⋅10−1\mathbf{1.09\cdot 10^{-1}}4.56⋅10−3\mathbf{4.56\cdot 10^{-3}}1.30⋅10−21.30\cdot 10^{-2}1.14\mathbf{1.14}0.920.92
 S4D8.74⋅10−58.74\cdot 10^{-5}6.27⋅10−36.27\cdot 10^{-3}1.76⋅10−11.76\cdot 10^{-1}5.91⋅10−35.91\cdot 10^{-3}9.69⋅10−3\mathbf{9.69\cdot 10^{-3}}1.511.510.960.96
Table 7: Loss and performance metrics for models trained on Softube CL 1B dynamic datasets with different parameter sampling densities. The ‘Set’ columns indicate the training sets used, corresponding to the suffix of the training set names detailed in Table 3 . Models with the lowest loss and the minimum value for each metric are highlighted in bold.
SetModelMSEMAEESRRMSESFEM-STFTEPSM
0.20S63.32⋅10−4\mathbf{3.32\cdot 10^{-4}}8.83⋅10−3\mathbf{8.83\cdot 10^{-3}}2.43⋅10−1\mathbf{2.43\cdot 10^{-1}}8.48⋅10−38.48\cdot 10^{-3}7.18⋅10−3\mathbf{7.18\cdot 10^{-3}}2.002.000.940.94
 S4D3.49⋅10−43.49\cdot 10^{-4}8.85⋅10−38.85\cdot 10^{-3}2.55⋅10−12.55\cdot 10^{-1}8.20⋅10−38.20\cdot 10^{-3}7.73⋅10−37.73\cdot 10^{-3}1.93\mathbf{1.93}0.960.96
 ED3.52⋅10−43.52\cdot 10^{-4}8.36⋅10−38.36\cdot 10^{-3}2.57⋅10−12.57\cdot 10^{-1}7.94⋅10−3\mathbf{7.94\cdot 10^{-3}}8.63⋅10−38.63\cdot 10^{-3}1.951.950.950.95
 LSTM6.83⋅10−46.83\cdot 10^{-4}1.27⋅10−21.27\cdot 10^{-2}5.31⋅10−15.31\cdot 10^{-1}1.23⋅10−21.23\cdot 10^{-2}8.53⋅10−38.53\cdot 10^{-3}3.313.310.950.95
0.10S64.16⋅10−4\mathbf{4.16\cdot 10^{-4}}9.27⋅10−3\mathbf{9.27\cdot 10^{-3}}2.65⋅10−1\mathbf{2.65\cdot 10^{-1}}8.90⋅10−38.90\cdot 10^{-3}9.38⋅10−39.38\cdot 10^{-3}2.112.110.940.94
 S4D5.78⋅10−45.78\cdot 10^{-4}1.07⋅10−21.07\cdot 10^{-2}3.68⋅10−13.68\cdot 10^{-1}1.01⋅10−21.01\cdot 10^{-2}8.73⋅10−3\mathbf{8.73\cdot 10^{-3}}2.392.390.960.96
Table 8: Loss and performance metrics for models trained on Softube CL 1B time datasets with different parameter sampling densities. The ‘Set’ columns indicate the training sets used, corresponding to the suffix of the training set names detailed in Table 3 . Models with the lowest loss and the minimum value for each metric are highlighted in bold.
Figure 11: Waveforms of input and output of the TubeTech CL 1B for the 3.603.60 -second signal, including bass and drum sounds, utilized as the test set in the experiments investigating the emulation of four different compression temporal profiles. The threshold and ratio are set to −10-10 dBu and 6:16:1 , respectively, while associated attack and release times are indicated above each plot.
ModelValuesMSEESRRMSEMSEESRRMSE
S6    4.03⋅10−54.03\cdot 10^{-5}3.28⋅10−23.28\cdot 10^{-2}2.49⋅10−32.49\cdot 10^{-3}
 [ afa_{f} , rfr_{f} ]3.31⋅10−53.31\cdot 10^{-5}3.11⋅10−23.11\cdot 10^{-2}2.79⋅10−32.79\cdot 10^{-3}↓7.33%\downarrow 7.33\%↓7.33%\downarrow 7.33\%↓11.46%\downarrow 11.46\%
 [ afa_{f} , rsr_{s} ]3.11⋅10−53.11\cdot 10^{-5}4.69⋅10−24.69\cdot 10^{-2}2.89⋅10−32.89\cdot 10^{-3}↑8.08%\uparrow 8.08\%↑8.08%\uparrow 8.08\%↓2.95%\downarrow 2.95\%
 [ asa_{s} , rsr_{s} ]5.80⋅10−55.80\cdot 10^{-5}3.35⋅10−23.35\cdot 10^{-2}3.08⋅10−33.08\cdot 10^{-3}↓2.91%\downarrow 2.91\%↓2.91%\downarrow 2.91\%↓10.89%\downarrow 10.89\%
 [ asa_{s} , rfr_{f} ]2.40⋅10−52.40\cdot 10^{-5}1.15⋅10−21.15\cdot 10^{-2}1.48⋅10−31.48\cdot 10^{-3}↑68.44%\uparrow 68.44\%↑68.44%\uparrow 68.44\%↑32.84%\uparrow 32.84\%
S4D    4.59⋅10−54.59\cdot 10^{-5}3.95⋅10−23.95\cdot 10^{-2}2.69⋅10−32.69\cdot 10^{-3}
Table 9: Average loss and performance metrics obtained over five training iterations using a subset of the TubeTech CL 1B time dataset. The compression parameters of the selected recording are threshold and ratio fixed to −10-10 dBu and 6:16:1 , attack times of 0.50.5 ms ( afa_{f} ) and 500500 ms ( asa_{s} ), and release times of 0.0050.005 s ( rfr_{f} ) and 55 s ( rsr_{s} ). The left half of the table corresponds to models trained on a single compression setting, whereas the right half pertains to models trained across all four timing variations of the compression settings. The top row for each model in the right section summarizes the loss and metrics across all parameter combinations. The symbols ↑\uparrow and ↓\downarrow on the right side signify an increase (worse performance) or decrease (improved performance) in the loss and metrics, respectively, compared to the single-setting models.

Explore similar work

CardsList

  1. Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests

    Jul 5, 2026Anis Hamadouche, Amir HussainNeural NetworkTask-Aware Compression

    The School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK · SDAIA-KFUPM Joint Research Centre for Artificial Intelligence King Fahd University of Petroleum and Minerals Dhahran, Saudi Arabia

  2. DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

    Mar 15, 2026Bingzhou Li, Tao HuangToken CompressionAudio-Visual Reasoning

    Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China

  3. Exposing and Mitigating Neural Codec Vulnerabilities in Audio Deepfake Detection

    Oct 5, 2026Abdullah, Awais Khan, Khalid Mahmood MalikNeural Audio Codecs

    College of Innovation and Technology University of Michigan-Flint Michigan, USA