The Lipschitz continuity of deep neural networks (DNNs) is essential for establishing theoretical guarantees regarding their behavior. From both theoretical and practical perspectives, various methods have been proposed to construct Lipschitz-continuous architectures and control their Lipschitz constants. However, several DNN architectures common in audio signal processing fall outside the scope of existing theoretical frameworks, hindering the development of Lipschitz-continuous models in acoustic applications. In particular, despite their widespread adoption, DNNs that separately process the magnitude and phase of complex-valued signals cannot be Lipschitz continuous under existing frameworks. In this paper, to address this limitation, we establish a theoretical foundation for constructing amplitude modifiers (AMs), a class of DNN architectures that operate solely on the magnitude of a complex-valued input, with provable Lipschitz continuity. Specifically, we derive a necessary and sufficient condition for an AM to be Lipschitz continuous and propose LipsAMs (Lipschitz-continuous AMs) corresponding to common architectures for audio signals, including time-frequency masking. Furthermore, we develop an efficient framework for evaluating their Lipschitz constants and analytically derive these constants for some of the proposed architectures. As an application, we propose CoReM-LipsAM (Controlled Residual Maps via LipsAM) for plug-and-play (PnP) audio signal recovery, integrating a DNN as a data-driven prior within a model-based signal processing algorithm. The convergence of the obtained PnP algorithm is structurally guaranteed by the CoReM-LipsAM architecture and empirically validated through speech dereverberation experiments.
Figures & tables
Fig. 1: Overview of this paper. White boxes denote main theoretical results or algorithms, whereas colored boxes represent the proposed DNN architectures, where “-E” and “-M” indicate direct estimation of the amplitude component and time-frequency (TF) masking, respectively. Ingredient A establishes LipsAM (Lipschitz-continuous Amplitude Modifiers), and Ingredient B provides a framework for evaluating Lipschitz constants within a reduced search space. These components are integrated to construct CoReM-LipsAM (Controlled-Residual Maps via LipsAM), which satisfies the convergence conditions for a PnP (Plug-and-Play) audio signal recovery algorithm.
TABLE I: DNN Architectures of AMs (Amplitude Modifiers) and LipsAMs (Lipschitz-Continuous AMs) *
Fig. 2: Visualization of Example 6 (non-Lipschitz continuous AM-M). The mask value g(M(⋅)) of this AM-M shown in (I) continues to oscillate even for large inputs, causing the amplitude-modifying part A(M,g)(x)=g(M(x))⋅x in (II) to be non-Lipschitz continuous. The proposed LipsAM-M (red) resolves this issue; the derivative of A(M,g) is bounded by the dotted horizontal line in (III) , thus A(M,g) is Lipschitz continuous. Here, we set g(t)=sigmoid(4t) , a=1/5π , and b=−1 .
Fig. 3: Confirmation of the Lipschitz bounds of the proposed LipsAM architectures. The markers show the naive estimates of the maximum Lipschitz constant obtained by Eq. ( 41 ) and the Adam optimizer, where N corresponds to the dimension of the inputs, small jittered dots show the results for the 10 initial values, and large markers show the maximum. The solid lines show the upper bounds obtained in Sect. IV-C . The bounds for LipsAM-E and ReM-LipsAM-E are analytical (see Proposition 16 ), whereas that for LipsAM-M ( g=sigmoid , a=1 , and b=0 ) was numerically estimated using the proposed framework in Sect. IV ; We used the particleswarm function [ 14 ] in MATLAB’s Global Optimization Toolbox to solve Eq. ( 38 ). For each setting, we ran particleswarm 10 times and took the maximum.
Fig. 4: The architecture of learnable mapping E and M for CoReM-LipsAMs. The value L∈(0,1) is parametrized as L=sigmoid(sL) , where sL∈R is a learnable parameter. This structure ensures Lip(E),Lip(M)≤L .
Fig. 5: Performance of the PnP algorithm on the speech dereverberation task. Dotted lines indicate conventional DNN architectures that have no theoretical guarantee. Solid colored lines indicate the proposed CoReM-LipsAMs. The gray line is the result of soft-thresholding as a baseline. (Left) SI-SNR after 1200 iterations with respect to the SNR during training with denoising tasks. The best results for each architecture are marked. (Right) Transition of Δx[k] for the marked results in the left panel.
The robustness of deep neural networks (DNNs) can be certified through their Lipschitz continuity, which has made the construction of Lipschitz-continuous DNNs an active research field. However, DNNs for audio processing have not been a major focus due to their poor compatibility with existing results. In this paper, we consider the amplitude modifier (AM), a popular architecture for handling audio signals, and propose its Lipschitz-continuous variants, which we refer to as LipsAM. We prove a sufficient condition for an AM to be Lipschitz continuous and propose two architectures as examples of LipsAM. The proposed architectures were applied to a Plug-and-Play algorithm for speech dereverberation, and their improved stability is demonstrated through numerical experiments.
Kazuki Matsumoto, Ren Uchida, Kohei Yatabe
Tokyo University of Agriculture and Technology, Tokyo, Japan
Lipschitz continuity is a fundamental principle in the design of certifiably robust deep neural networks (DNNs), wherein adjusting the Lipschitz constant, which quantifies network robustness, is of central theoretical importance. A standard approach to enforcing Lipschitz continuity requires each layer of a DNN to be Lipschitz continuous, thereby guaranteeing overall Lipschitz continuity. However, this layer-wise approach typically imposes overly conservative restrictions by producing a loose estimate of the overall Lipschitz constant, which limits the expressive capacity of the DNN and degrades empirical performance at a prescribed level of robustness. To overcome this loose estimation, the recently proposed LipKernel transfers information across layers to yield a much tighter overall Lipschitz bound than conventional layer-wise construction. In this paper, we extend this concept to cascaded state-space models (SSMs) to construct Lipschitz-continuous DNNs capable of modeling longer-term dependencies. The proposed architecture, named LipSSM, is theoretically justified and empirically evaluated.
Natsuki Yoshino, Ren Uchida, Kazuki Matsumoto +1
Tokyo University of Agriculture and Technology (TUAT), Tokyo, Japan
Trainable denoisers with Lipschitz control have become central to convergent image reconstruction. However, training neural networks that simultaneously offer strong denoising performance and global Lipschitz guarantees is challenging. Existing approaches enforce Lipschitz control only empirically, providing no guarantees beyond the training data. In this work, we show that by exploiting the action of permutations on the image lattice, we can constrain a neural architecture that is globally nonexpansive (Lipschitz bound ⩽1). We integrate the proposed denoiser with forward imaging operators to develop a reconstruction mechanism that is provably contractive and therefore globally convergent. Experiments on standard inverse problems, such as superresolution and deblurring, demonstrate that our reconstruction performance is competitive with softly constrained baselines while providing Lipschitz guarantees.