Individualized head-related impulse responses (HRIRs) enable binaural rendering, but dense per-listener measurements are costly. We address HRIR spatial up-sampling from sparse per-listener measurements: given a few measured HRIRs for a listener, predict HRIRs at unmeasured target directions. Prior learning methods often work in the frequency domain, rely on minimum-phase assumptions or separate timing models, and use a fixed direction grid, which can degrade temporal fidelity and spatial continuity. We propose HRIR-Former, a time-domain, grid-free binaural Transformer for reconstructing HRIRs at arbitrary directions from sparse inputs. It uses sinusoidal spatial features, a Conv1D refinement module, and auxiliary interaural time difference (ITD) and interaural level difference (ILD) heads. On SONICOM, it improves normalized mean squared error (NMSE), cosine distance, and ITD/ILD errors over prior methods; ablations validate modules and show minimum-phase preprocessing is unnecessary.
Figures & tables
Figure 1: Illustration of HRIR spatial up-sampling: estimating HRIRs at unmeasured target directions (white) from a sparse set of measured directions (red).
Figure 2: Overview of the proposed HRIR-Former pipeline. Known HRIRs are projected into a latent space using an MLP-based signal encoder, while geometric embeddings derived from direction coordinates are independently projected and added to the signal features. The resulting tokens are processed jointly by a Transformer encoder to capture global spatial dependencies across measured and target directions. A shared MLP decoder maps contextual features to full-length binaural HRIRs for all directions, followed by masked fusion to preserve measured responses and a lightweight Conv1D for temporal refinement.
Method
M=3
M=5
M=19
M=100
ITD-E ( μ s) ↓
ILD-E (dB) ↓
ITD-E ( μ s) ↓
ILD-E (dB) ↓
ITD-E ( μ s) ↓
ILD-E (dB) ↓
ITD-E ( μ s) ↓
ILD-E (dB) ↓
HRIR-Former (Proposed)
18.5
1.14
16.4
1.10
15.4
0.95
10.6
0.70
Nbr [ 35 ]
274.2
7.6
154.2
4.8
108.3
3.0
44.0
1.4
HRTF-Sel-ITD [ 35 ]
26.3
1.4
24.5
1.5
23.2
1.6
20.0
1.4
HRTF-Sel-LSD [ 35 ]
37.5
1.5
35.7
1.4
37.4
1.5
31.4
1.3
NF-CbC [ 45 ]
22.1
1.5
20.7
2.0
14.8
1.8
11.8
1.7
Table 1: ITD-E ( μ s) and ILD-E (dB) under different measurement sparsity levels M on SONICOM. Bold indicates the best result per column. Baseline results are taken from [ 35 ] .
Metric
M=3
M=5
M=19
M=100
NMSE (dB) ↓
-6.90
-7.35
-8.24
-10.20
CD ↓
0.233
0.210
0.163
0.102
Table 2: NMSE and CD of HRIR-Former under different measurement sparsity levels M .
Figure 3: Left-ear HRIR reconstruction for subject P0187 at (ϕ,θ,r)=(90∘,20∘,1.5m) under M=19 : estimated HRIR (dashed) versus ground-truth HRIR (solid). Angular distance to the nearest measured direction: 35.2∘ .
Figure 4: Stacked binaural HRIRs for subject P0187 under M=19 over 793 directions (each column: one direction; left/right ears concatenated along time). Ground truth (left) and estimation (right).
Variant
NMSE (dB) ↓
CD ↓
ITD-E ( μ s) ↓
ILD-E (dB) ↓
HRIR-Former (full)
-7.35
0.210
16.4
1.10
w/o sinusoidal encoding
-4.99
0.298
24.2
1.66
w/o ITD/ILD-prediction heads
-7.52
0.207
19.4
1.11
w/o Conv1D refinement
-6.04
0.257
20.8
1.28
w/o LHRTF
-7.10
0.217
17.3
1.14
w/ MP pre-processing
-6.80
0.225
19.1
1.21
Table 3: Ablation study of HRIR-Former under M=5 .
Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.
Meenakshi Krishnan, You Zhang, Ramani Duraiswami +1
University of Maryland, College Park · Dolby Laboratories, Inc.
We address the problem of encoding room impulse responses (RIRs) into high-order Ambisonics (HOA) representations from arbitrary and potentially insufficient or incomplete microphone array measurements. This task is fundamentally ill-posed for microphone arrays with limited spatial capture capabilities, such as irregular or sparse arrays, as classical linear methods fail to reconstruct high-order spatial detail. We introduce a diffusion-based generative framework that models the statistical properties of HOA RIRs. This enables device-agnostic encoding from arbitrary microphone arrays, potentially unseen during data measurement. Our approach incorporates a posterior sampling procedure that enforces consistency between the estimated signals and the measurements while plausibly reconstructing spatial information that is unobservable from the limited measurements alone. Experiments on simulated data demonstrate that our method outperforms linear and neural baselines, achieving accurate HOA RIR estimation up to 12th order. A listening test with binaural renderings, including both simulated and measured RIRs, further confirms that the proposed method yields higher perceptual similarity to reference Ambisonics RIRs than all baselines. The flexibility and accuracy of the proposed framework opens new possibilities for scalable acoustics simulations.
Eloi Moliner, Christoph Hold, Juan Azcarreta Ortiz +5
Reality Labs Research, Meta · Acoustics Lab, DICE, Aalto University, Espoo, Finland
Binaural audio delivers spatial cues essential for immersion, yet most consumer videos are monaural due to capture constraints. We introduce SIREN, a visually guided mono to binaural framework that explicitly predicts left and right channels. A ViT-based encoder learns dual-head self-attention to produce a shared scene map and end-to-end L/R attention, replacing hand-crafted masks. A soft, annealed spatial prior gently biases early L/R grounding, and a two-stage, confidence-weighted waveform-domain fusion (guided by mono reconstruction and interaural phase consistency) suppresses crosstalk when aggregating multi-crop and overlapping windows. Evaluated on FAIR-Play and MUSIC-Stereo, SIREN yields consistent gains on time-frequency and phase-sensitive metrics with competitive SNR. The design is modular and generic, requires no task-specific annotations, and integrates with standard audio-visual pipelines.