Room Impulse Responses estimation is a fundamental problem in spatial audio processing and speech enhancement. In this paper, we build upon our previously introduced diffusion-based inpainting framework for Room Impulse Response interpolation and demonstrate its applicability to enhancing the performance of practical multi-microphone array processing tasks. Furthermore, we validate the robustness of this method in interpolating real-world Room Impulse Responses.
Figures & tables
Fig. 1 : System block diagram: Diffusion-based RIR reconstruction followed by beamforming for speech enhancement.
Fig. 2 : Measured (blue) and missing (orange) microphones.
Directional noise
Diffuse noise
Method
SIR [dB] ↑
SI-SDR [dB] ↑
STOI ↑
SIR [dB] ↑
SI-SDR [dB] ↑
STOI ↑
Mics
0
-4
0.59
0
-4
0.58
Full
12.2
9
0.88
6.2
10.2
0.79
Mask 0 - Missing
11
6
0.80
4.8
7
0.70
Mask 0 - Inpainted
12.3
8.8
0.88
6.2
10.1
0.79
Mask 1 - Missing
9.5
4.3
0.78
3.4
6
0.68
TABLE I : MVDR results averaged over all SNRs under directional and diffuse noise.
Dist( h , h^ ) ↓
Mask 0
Mask 1
Mask 2
Mask 3
Inpainted (Ours)
0.02
0.05
0.19
0.52
Interpolated (SCI)
0.1
0.17
0.4
0.63
TABLE II : Alignment between interpolated and gound-truth RIR for proposed and baseline methods (best results in bold).
Fig. 3 : Geometric setup of the room with a source and the microphone array, taken from the MeshRIR dataset.
Fig. 4 : NMSE and CD vs. mask ratio for Three Rows configuration.
Fig. 5 : NMSE and CD vs. mask ratio for Frame configuration.
Fig. 6 : Three Rows array configuration: Measured (blue) and missing (orange) microphones.
Fig. 7 : Comparison between reconstructed and ground-truth impulse responses and EDC for microphone #7.
We address the problem of encoding room impulse responses (RIRs) into high-order Ambisonics (HOA) representations from arbitrary and potentially insufficient or incomplete microphone array measurements. This task is fundamentally ill-posed for microphone arrays with limited spatial capture capabilities, such as irregular or sparse arrays, as classical linear methods fail to reconstruct high-order spatial detail. We introduce a diffusion-based generative framework that models the statistical properties of HOA RIRs. This enables device-agnostic encoding from arbitrary microphone arrays, potentially unseen during data measurement. Our approach incorporates a posterior sampling procedure that enforces consistency between the estimated signals and the measurements while plausibly reconstructing spatial information that is unobservable from the limited measurements alone. Experiments on simulated data demonstrate that our method outperforms linear and neural baselines, achieving accurate HOA RIR estimation up to 12th order. A listening test with binaural renderings, including both simulated and measured RIRs, further confirms that the proposed method yields higher perceptual similarity to reference Ambisonics RIRs than all baselines. The flexibility and accuracy of the proposed framework opens new possibilities for scalable acoustics simulations.
Eloi Moliner, Christoph Hold, Juan Azcarreta Ortiz +5
Reality Labs Research, Meta · Acoustics Lab, DICE, Aalto University, Espoo, Finland
We propose a self-supervised approach for learning room impulse response (RIR) representations from single-channel noisy-reverberant speech. It consists of first training on reverberant data, then on noisy-reverberant data, and finally with a teacher-student approach, where the student learns to replicate the teacher's embeddings when given a noisy version of the reverberant input. We assess their representational capabilities by estimating acoustic room parameters from them. Conditioning a discriminative speech enhancement model on the derived embeddings yields consistent gains across all evaluated metrics, including downstream word error rate, for both reverberant and noisy-reverberant speech.
Spatial room impulse responses (SRIRs) capture directional distributions of acoustic sound-sources and their reflections. However, collecting SRIRs of moving sound-sources remains a challenge, requiring complex interpolations across measurements that account for multi-path spatial-temporal dynamics. This paper investigates the Wasserstein metric and displacement for evaluating interpolated SRIR echo densities in the spherical harmonic domain. We present novel sum-of-magnitude square expansions for efficiently fitting probability density functions, maximizing likelihood, inverse sampling, and computing spherical sliced Wasserstein interpolations. Experiments compare the Wasserstein displacements and metric to linear and geometric interpolations of SRIR image-source densities on a line-path, and demonstrate model-order reduction.