Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.
Figures & tables
Figure 1: HRTF upsampling as operator learning. SPARO learns the upsampling operator F that generalizes across listeners and can be queried at arbitrary directions to produce the HRTF field.
Figure 2: Architecture for HRTF upsampling. Sparse HRTF observations are embedded by the branch projector and aggregated into a fixed set of learned spherical modes according to their angular similarity to mode centers on S2 . Transformer blocks model interactions entirely in the latent mode space. At arbitrary query directions, the transformed modes are interpolated using the same spherical mode geometry, and a final MLP produces the output HRTF.
LSD (dB) ↓
ILD error (dB) ↓
ITD error ( μ s) ↓
Method
3
5
19
100
3
5
19
100
3
5
19
100
Nearest Neighbor
8.7
8.3
5.4
3.4
7.6
4.8
3.0
1.4
274.2
154.2
108.3
44.0
HRTF Selection (ITD) [ 13 ]
5.8
5.6
5.5
5.4
1.4
1.5
1.6
1.4
26.3
24.5
23.2
20.0
NF + LoRA [ 12 ]
4.7
4.6
4.1
3.8
1.3
1.4
1.1
1.1
28.6
24.7
14.7
9.1
RANF [ 13 ]
4.4
4.6
3.6
3.0
1.2
1.2
1.0
0.8
20.5
18.7
14.2
10.0
IOA3D [ 24 ]
5.0
5.3
3.6
2.8
1.3
1.4
0.8
0.7
31.2
30.0
30.4
25.9
Table 1: HRTF upsampling performance on the SONICOM dataset under the LAP24 protocol. Results are reported for different numbers of sparse HRTF measurements. Lower is better for all metrics. Best results are shown in bold.
Figure 3: HRTF magnitude predicted by SPARO and RANF at different sparsity levels and directions. fig:spec
Q
M
SUpDEq
RANF
SPARO
SPARO (all M )
1
6
5.32 ± 0.26
4.04 ± 0.25
3.39 ± 0.31
3.32 ± 0.30
2
14
4.38 ± 0.26
3.49 ± 0.34
2.91 ± 0.25
2.83 ± 0.24
3
26
3.70 ± 0.21
3.26 ± 0.40
2.62 ± 0.23
2.57 ± 0.21
4
38
3.26 ± 0.21
3.07 ± 0.37
2.38 ± 0.18
2.37 ± 0.19
5
50
2.89 ± 0.21
2.97 ± 0.40
2.18 ± 0.17
2.23 ± 0.17
6
74
2.63 ± 0.17
2.82 ± 0.35
2.10 ± 0.18
2.15 ± 0.17
Table 2: HRTF upsampling on simulated HUTUBS dataset. LSD (dB, mean ± std over the 10 test subjects) over all 900 target directions with Lebedev degree Q and M observations. Results are reported for SPARO trained separately for each M and across all M .
Variant
M=6
M=26
M=50
M=110
M=170
SPARO
3.39
2.62
2.18
1.79
1.56
w/o LCA
3.37
2.71
2.47
2.26
2.22
w/o TTO
3.51
2.84
2.41
2.04
1.83
Table 3: Ablation on simulated HUTUBS. LSD (dB) with local cross-attention (LCA) and test-time optimization (TTO) removed. LCA gains increase with M; TTO helps at every level.
M
SPARO w/o TTO
SPARO
RANF
Speedup
3
0.012 s
0.59 s
8.7 s
15 ×
19
0.012 s
0.56 s
29.3 s
53 ×
100
0.012 s
0.56 s
207.2 s
373 ×
Table 4: Per-subject inference cost, including magnitude and ITD for both methods. Speedup is relative to SPARO with TTO. Parameters: 1.73M (SPARO, two networks) vs. 0.82M shared + 2,048 per listener (1.23M total, RANF).
Individualized head-related impulse responses (HRIRs) enable binaural rendering, but dense per-listener measurements are costly. We address HRIR spatial up-sampling from sparse per-listener measurements: given a few measured HRIRs for a listener, predict HRIRs at unmeasured target directions. Prior learning methods often work in the frequency domain, rely on minimum-phase assumptions or separate timing models, and use a fixed direction grid, which can degrade temporal fidelity and spatial continuity. We propose HRIR-Former, a time-domain, grid-free binaural Transformer for reconstructing HRIRs at arbitrary directions from sparse inputs. It uses sinusoidal spatial features, a Conv1D refinement module, and auxiliary interaural time difference (ITD) and interaural level difference (ILD) heads. On SONICOM, it improves normalized mean squared error (NMSE), cosine distance, and ITD/ILD errors over prior methods; ablations validate modules and show minimum-phase preprocessing is unnecessary.
Shaoheng Xu, Chunyi Sun, Jihui Zhang +4
The Australian National University, Australia · The University of Queensland, Australia
In this paper, we propose a reconstruction framework that leverages the Wavelet Scattering Transform (WST) as a multi-scale feature extractor to impose statistical priors under sparse observation conditions. The reconstruction problem is formulated as an optimization task and solved using a neural field, with the WST incorporated into the training loss function. As a proof of concept, we validate the proposed method on HRTF upsampling. A masking strategy is applied to the WST coefficients, resulting in a two-phase procedure. The first phase learns a binary mask from a small multi-subject dataset, while the second phase applies the learned mask to the WST coefficients of an individual HRTF to preserve informative statistical structures during reconstruction. Validation against baseline methods, which also serve as an ablation study of the different components of the framework, demonstrates the effectiveness of the proposed approach.
Xinmeng Luan, Samuel A. Verburg, Efren Fernandez-Grande +1
Schulich School of Music, McGill University, CIRMMT† Montreal, Canada · Dept. of Electrical and Photonics Engineering, Technical University of Denmark Kgs. Lyngby, Denmark · Dept. of Audiovisual Engineering and Communications, Universidad Politécnica de Madrid Madrid, Spain
This paper presents a Head-Related Transfer Function (HRTF)-guided framework for binaural Target Speaker Extraction (TSE) from mixtures of concurrent sources. Unlike conventional TSE methods based on Direction of Arrival (DOA) estimation or enrollment signals, which often distort perceived spatial location, the proposed approach leverages the listener's HRTF as an explicit spatial prior. The proposed framework is built upon a multi-channel deep blind source separation backbone, adapted to the binaural TSE setting. It is trained on measured HRTFs from a diverse population, enabling cross-listener generalization rather than subject-specific tuning. By conditioning the extraction on HRTF-derived spatial information, the method preserves binaural cues while enhancing speech quality and intelligibility. The performance of the proposed framework is validated through simulations and real recordings obtained from a head and torso simulator (HATS).
Yoav Ellinson, Sharon Gannot
Faculty of Engineering, Bar Ilan University, Israel