Reverberant Speech Conditioning for Material-Aware Acoustic Parameter Mapping
Organizations: Dolby Laboratories, Inc. · New York University
Abstract
Dense maps of room acoustic parameters (T30, EDT, C50, DRR) describe how a room sounds at every position. Scene-wide estimation predicts such maps from a floor plan but requires a reference impulse response, which is hard to obtain. Our system instead receives a floor plan and easily recorded reverberant speech from one talker-microphone pair. We compare four speech-derived conditioning inputs: MFCCs, an embedding from the blind estimator BERP, BERP's scalar reverberation-time estimate, and an impulse response reconstructed by Rec-RIR. Our main benchmark is the MRAS dataset, where material assignments vary at fixed geometry. The reconstructed impulse response performs best among the four inputs, reducing T30 error from 31.5% with the floor plan alone to 19.9%, 83% of the reduction achieved by the reference response. We find that adding speech mainly sets the overall level of the map, with little improvement in predicting how the acoustics vary across the room.
Figures & tables
| Error (matched observation) | Gap closure | Error increase under mismatch | Pattern (T30) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Input | T30 (%) | EDT (%) | C50 (dB) | DRR (dB) | T30 (%) | EDT (%) | T30 (pp) | EDT (pp) | (%) | |
| Geometry-only | 31.5 | 30.2 | 2.68 | 1.65 | 0 | 0 | – | – | 12.8 | 0.61 |
| scalar (BERP estimate) | 25.2 | 25.2 | 2.29 | 1.49 | 45 [38,51] | 55 [48,61] | 10.3 [8.5,12.2] | 10.0 [8.5,11.6] | 12.5 | 0.63 |
| embedding (BERP) | 24.1 | 24.4 | 2.21 | 1.48 | 52 [46,58] | 63 [57,68] | 13.0 [10.9,15.2] | 12.0 [10.3,13.7] | 13.2 | 0.59 |
| spectral (MFCC) | 23.6 | 24.5 | 2.29 | 1.51 | 56 [50,60] | 63 [57,67] | 13.8 [11.7,16.1] | 10.7 [9.2,12.3] | 13.3 | 0.56 |
| reconstruction (Rec-RIR) | 19.9 | 21.8 | 2.08 | 1.43 | 83 [78,86] | 93 [89,96] | 18.6 [15.8,21.5] | 15.2 [13.1,17.3] | 12.8 | 0.60 |
| Gap closure | ||
| Input | T30 (%) | EDT (%) |
| U-Net map predictor trained per input (three seeds): | ||
| BERP’s estimate | 45 [38,51] | 55 [48,61] |
| Rec-RIR’s broadband T30 | 48 [37,56] | 63 [53,72] |
| Rec-RIR reconstruction | 83 [78,86] | 93 [89,96] |
| True broadband T30 | 63 [54,71] | 66 [56,75] |
Room Acoustics
Room acoustics studies how spaces shape sound through reflection, absorption, reverberation, and spatial propagation.
See all 21 papers