Learning to reconstruct from saturated data: audio declipping and high-dynamic range imaging

Victor Sechaud; Laurent Jacques; Patrice Abry; Julián Tachella

eess.IVFeb 25, 2026

Learning to reconstruct from saturated data: audio declipping and high-dynamic range imaging

Authors: Victor Sechaud, Laurent Jacques, Patrice Abry, Julián Tachella

Organizations: CNRS and ENS de Lyon, Laboratoire de physique, Lyon, France · UCLouvain, ICTEAM, Louvain-la-Neuve, Belgium · CIFAR Fellow

Abstract

Learning based methods are now ubiquitous for solving inverse problems, but their deployment in real-world applications is often hindered by the lack of ground truth references for training. Recent self-supervised learning strategies offer a promising alternative, avoiding the need for ground truth. However, most existing methods are limited to linear inverse problems. This work extends self-supervised learning to the non-linear problem of recovering audio and images from clipped measurements, by assuming that the signal distribution is approximately invariant to changes in amplitude. We provide sufficient conditions for learning to reconstruct from saturated signals alone and a self-supervised loss that can be used to train reconstruction networks. Experiments on both audio and image data show that the proposed approach is almost as effective as fully supervised approaches, despite relying solely on clipped measurements for training.

Figures & tables

Figure 1: Example in 3 dimensions illustrating Theorem 1 . The black set presents a particularly challenging scenario in which all saturated signals are projected to the same point. In contrast, with high probability on

\bm{A}

, the colored set enables recovery of more signals with moderate norms, as points in

\mathcal{X}

beyond a certain radius— which is connected to the signal norm when

\bm{A}

is Gaussian—are all projected onto a single corner, showing the non-injectivity beyond this radius.

Figure 2:

\mathcal{L}_{\textrm{NMC}}

compared to

\mathcal{L}_{\textrm{MC}}

. On the left: Curve of the

\mathcal{L}_{\textrm{NMC}}

in one dimension for two examples in both cases saturated and not. On the right: Curve of the new

\mathcal{L}_{\textrm{MC}}

for the same two examples.

Figure 3: An example where the reconstruction is learned through the bias (third column) which prevents the reconstruction for low amplitude measurements. We thus observe that removing the bias makes the network naturally homogeneous and improves the reconstruction. On the first row, we consider a signal

\bm{x}

, on the second row

\frac{\bm{x}}{10}

Figure 4: An image with its associated blending mask. The mask is white where a channel is saturated and black when none is saturated.

Figure 5: Example where two different signals $\bm{x}_{1},\bm{x}_{2}$ have the same measurement $\eta(\bm{x}_{1}),\eta(\bm{x}_{2})$ and so the network $f_{\bm{\theta}}$ fails to recover them both (the second and fourth columns). Adding randomness implies that the two measurements are not equal anymore, and so a network $f^{{}^{\prime}}_{\bm{\theta}}$ can reconstruct the original signal (the third and fifth columns).

Figure 6: Plot of the dynamic range of $f_{\bm{\theta}}(\bm{y})$ depending on dynamic range of $\bm{x}$ , where $\bm{y}=\eta(\bm{A}\bm{x})$ . Left: The training is done with the losses $\mathcal{L}_{\textrm{NMC}}+\mathcal{L}_{\textrm{EI}}$ . Right: The training is done with loss $\mathcal{L}_{\textrm{MC}}+\mathcal{L}_{\textrm{EI}}$ . A perfect plot should be the identity line.

Panel of Figure 7 — Figure 7: Average reconstruction performance as a function of saturated part $v$ and signal set dimension $k$ for both supervised (left) and self-supervised (right) methods. The value indicated corresponds to the mean SDR over the test dataset.

Methods	SDR $\uparrow$	PESQ $\downarrow$
Identity	$4.84\pm 2.12$	$2.7\pm 0.65$
Social Sparsity	$9.92\pm 4.46$	$2.09\pm 0.95$
Supervised	$11.69\pm 2.25$	$1.94\pm 0.75$
Supervised 5%	$9.79\pm 1.59$	$2.28\pm 0.78$
Supervised + $\mathcal{L}_{\textrm{EI}}$	$11.72\pm 2.23$	$1.92\pm 0.74$
Proposed self-supervised	$10.48\pm 2.20$	$2.20\pm 0.80$

Table 1: Reconstruction performance, SDR and PESQ are averaged over all music test dataset.

Methods	SDR (dB)
Identity	$6.54\pm 2.34$
Supervised (trained on music)	$10.94\pm 2.00$
Proposed self-supervised (trained on music and voice)	$11.92\pm 2.46$

Table 2: Average SDR performance on the test dataset, which includes both music and voice recordings.

Methods	SDR $\uparrow$	PESQ $\downarrow$
Identity	$5.00\pm 0$	$2.9\pm 0.73$
Supervised	$6.70\pm 1.69$	$2.22\pm 0.52$
Proposed self-supervised	$7.78\pm 8.64$	$2.47\pm 1.12$

Table 3: Average SDR and PESQ performance on the test MAESTRO dataset.

Panel of Figure 8 — Figure 8: Image reconstruction on the test dataset. An exposition correction ( Wikipedia, ) is applied to see details better.

Methods	PSNR $\uparrow$	NIQE $\downarrow$
Identity	$29.54\pm 6.95$	$4.21\pm 1.96$
PnP DPIR	$32.11\pm 5.80$	$4.06\pm 1.93$
Supervised	$36.67\pm 6.19$	$4.04\pm 1.74$
Proposed self-supervised	$35.04\pm 6.33$	$4.10\pm 1.85$

Table 4: Reconstruction performance, PSNR is averaged over all test dataset images.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Section	$\lambda$	$p_{g}$	Architecture	Learning rate	Epochs	Batch size
Section 5.2.1	$1$	$\mathcal{U}(0.1,2)$	Bias-free U-Net, 4 down + 4 up blocks	$5e^{-4}$	$300$	$50$
Section 5.2.2	$1$	$\mathcal{U}(0.5,1.5)$	MLP, input size 100, hidden size 100, depth 5	$1e^{-4}$	$300$	$100$
Section 5.3	$0.1$	$\mathcal{U}(0.1,2.0)$	Bias-free U-Net, 5 down + 5 up blocks	$5e^{-4}$	$100$	$35$
Section 5.4	$0.1$	$\mathcal{U}(0.2,1.5)$	Bias-free U-Net, 4 down + 4 up blocks	$5e^{-5}$	$400$	$12$

Appendix

Table 5: Hyperparameters for different experiments.

Figure 9: SDR performance on the evaluation dataset during training for supervised and self-supervised methods.

Panel of Figure 10 — Figure 10: Image reconstruction on the test dataset. Artifacts can appear in saturation zone.

Inverse Problems

Inverse problems infer hidden causes, parameters, or signals from indirect, noisy measurements using physical models, priors, and statistical methods.

See all 244 papers

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners
Aug 20, 2026Umberto Cappellazzo, Xubo Liu, Stavros Petridis +1Self-Supervised Speech Representation Learning Autoregressive Modeling
Imperial College London · University of Surrey
Frozen CLIP Priors for Robust Self-Supervised Poisson Inverse Problems
Aug 20, 2026Laura C. Diaz-Delgado, Emmanuel Martinez, Henry ArguelloInverse Problems Image Inverse Problems
Department of Computer Science, Universidad Industrial de Santander, Colombia
Learning to Recorrupt: Noise Distribution Agnostic Self-Supervised Image Denoising
Mar 26, 2026Brayan Monroy, Jorge Bacca, Julián TachellaSelf-Supervised Image Denoising Self-Supervised Learning
Universidad Industrial de Santander, Colombia · CNRS, ENS de Lyon, France

Aug 20, 2026J/K move · Enter open · S save

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

Umberto Cappellazzo, Xubo Liu, Stavros Petridis, Maja Pantic

Imperial College London · University of Surrey

Self-Supervised Speech Representation Learning Autoregressive Modeling

Open paper AskPDF arXiv

Self-supervised learning (SSL) has driven substantial progress in audio representation learning, though existing methods have increasingly relied on elaborate pre-training recipes to reach competitive performance. A markedly different pre-training philosophy underpins the most influential progress in language modeling and, more recently, in visual representation learning: rather than train encoders as static feature extractors, models are trained to predict the next element, a discrete token or a continuous embedding, from the preceding context. Autoregressive prediction thereby provides a unified pre-training interface that transfers across modalities, compelling the model to learn the underlying data distribution. We ask whether such a simple causal paradigm can yield strong audio learners, given that audio's temporal structure makes autoregressive prediction of patch embeddings a natural fit. We introduce NAPE (Next-Audio-Patch-Embedding prediction), a self-supervised framework in which a causal Transformer predicts each next patch embedding of a log-mel spectrogram from the previous ones, using causal masking and stop-gradient as its sole training signal. The design is intentionally minimalist, avoiding reconstruction decoders, acoustic tokenizers, student-teacher setups, and auxiliary regularization losses. Across six audio and speech benchmarks, NAPE achieves state-of-the-art fine-tuning performance on several tasks, scales consistently across encoder sizes, and yields strong linear-probing results. NAPE also produces structured attention patterns without explicit supervision.

Abstract

Inverse Problems

Explore similar work

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

Frozen CLIP Priors for Robust Self-Supervised Poisson Inverse Problems

Learning to Recorrupt: Noise Distribution Agnostic Self-Supervised Image Denoising