HRTF Upsampling and Personalization via Neural Operators

Meenakshi Krishnan; You Zhang; Ramani Duraiswami; C. Phillip Brown

cs.SDOct 14, 2026

HRTF Upsampling and Personalization via Neural Operators

Authors: Meenakshi Krishnan, You Zhang, Ramani Duraiswami, C. Phillip Brown

Organizations: University of Maryland, College Park · Dolby Laboratories, Inc.

Abstract

Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.

Figures & tables

Figure 1: HRTF upsampling as operator learning. SPARO learns the upsampling operator $\mathcal{F}$ that generalizes across listeners and can be queried at arbitrary directions to produce the HRTF field.

Figure 2: Architecture for HRTF upsampling. Sparse HRTF observations are embedded by the branch projector and aggregated into a fixed set of learned spherical modes according to their angular similarity to mode centers on $S^{2}$ . Transformer blocks model interactions entirely in the latent mode space. At arbitrary query directions, the transformed modes are interpolated using the same spherical mode geometry, and a final MLP produces the output HRTF.

	LSD (dB) $\downarrow$				ILD error (dB) $\downarrow$				ITD error ( $\mu$ s) $\downarrow$
Method	3	5	19	100	3	5	19	100	3	5	19	100
Nearest Neighbor	8.7	8.3	5.4	3.4	7.6	4.8	3.0	1.4	274.2	154.2	108.3	44.0
HRTF Selection (ITD) [ 13 ]	5.8	5.6	5.5	5.4	1.4	1.5	1.6	1.4	26.3	24.5	23.2	20.0
NF + LoRA [ 12 ]	4.7	4.6	4.1	3.8	1.3	1.4	1.1	1.1	28.6	24.7	14.7	9.1
RANF [ 13 ]	4.4	4.6	3.6	3.0	1.2	1.2	1.0	0.8	20.5	18.7	14.2	10.0
IOA3D [ 24 ]	5.0	5.3	3.6	2.8	1.3	1.4	0.8	0.7	31.2	30.0	30.4	25.9

Table 1: HRTF upsampling performance on the SONICOM dataset under the LAP24 protocol. Results are reported for different numbers of sparse HRTF measurements. Lower is better for all metrics. Best results are shown in bold.

Figure 3: HRTF magnitude predicted by SPARO and RANF at different sparsity levels and directions. fig:spec

$Q$	$M$	SUpDEq	RANF	SPARO	SPARO (all $M$ )
1	6	5.32 $\pm$ 0.26	4.04 $\pm$ 0.25	3.39 $\pm$ 0.31	3.32 $\pm$ 0.30
2	14	4.38 $\pm$ 0.26	3.49 $\pm$ 0.34	2.91 $\pm$ 0.25	2.83 $\pm$ 0.24
3	26	3.70 $\pm$ 0.21	3.26 $\pm$ 0.40	2.62 $\pm$ 0.23	2.57 $\pm$ 0.21
4	38	3.26 $\pm$ 0.21	3.07 $\pm$ 0.37	2.38 $\pm$ 0.18	2.37 $\pm$ 0.19
5	50	2.89 $\pm$ 0.21	2.97 $\pm$ 0.40	2.18 $\pm$ 0.17	2.23 $\pm$ 0.17
6	74	2.63 $\pm$ 0.17	2.82 $\pm$ 0.35	2.10 $\pm$ 0.18	2.15 $\pm$ 0.17

Table 2: HRTF upsampling on simulated HUTUBS dataset. LSD (dB, mean

\pm

std over the 10 test subjects) over all 900 target directions with Lebedev degree

Q

and

M

observations. Results are reported for SPARO trained separately for each

M

and across all

M

Variant	$M{=}6$	$M{=}26$	$M{=}50$	$M{=}110$	$M{=}170$
SPARO	3.39	2.62	2.18	1.79	1.56
w/o LCA	3.37	2.71	2.47	2.26	2.22
w/o TTO	3.51	2.84	2.41	2.04	1.83

Table 3: Ablation on simulated HUTUBS. LSD (dB) with local cross-attention (LCA) and test-time optimization (TTO) removed. LCA gains increase with M; TTO helps at every level.

$M$	SPARO w/o TTO	SPARO	RANF	Speedup
3	0.012 s	0.59 s	8.7 s	15 $\times$
19	0.012 s	0.56 s	29.3 s	53 $\times$
100	0.012 s	0.56 s	207.2 s	373 $\times$

Table 4: Per-subject inference cost, including magnitude and ITD for both methods. Speedup is relative to SPARO with TTO. Parameters: 1.73M (SPARO, two networks) vs. 0.82M shared + 2,048 per listener (1.23M total, RANF).

Spatial Audio

Spatial audio studies how sound is captured, represented, generated, and perceived with direction, distance, motion, and room acoustics.

See all 100 papers

HRIR-Former: Grid-Free Time-Domain Reconstruction of Head-Related Impulse Responses with a Spatially Encoded Transformer
Mar 30, 2026Shaoheng Xu, Chunyi Sun, Jihui Zhang +4Spatial Audio Transformer
The Australian National University, Australia · The University of Queensland, Australia
Masked Wavelet Scattering Transform Neural Field for Sound Field Reconstruction
Jun 3, 2026Xinmeng Luan, Samuel A. Verburg, Efren Fernandez-Grande +1Wavelet Scattering Transforms
Schulich School of Music, McGill University, CIRMMT† Montreal, Canada · Dept. of Electrical and Photonics Engineering, Technical University of Denmark Kgs. Lyngby, Denmark · Dept. of Audiovisual Engineering and Communications, Universidad Politécnica de Madrid Madrid, Spain
HRTF-guided Binaural Target Speaker Extraction with Real-World Validation
Mar 17, 2026Yoav Ellinson, Sharon GannotTarget Speaker Extraction Speech Separation
Faculty of Engineering, Bar Ilan University, Israel

Mar 30, 2026J/K move · Enter open · S save

HRIR-Former: Grid-Free Time-Domain Reconstruction of Head-Related Impulse Responses with a Spatially Encoded Transformer

Shaoheng Xu, Chunyi Sun, Jihui Zhang, Amy Bastine+3

The Australian National University, Australia · The University of Queensland, Australia

Spatial Audio Transformer

Open paper AskPDF arXiv

Individualized head-related impulse responses (HRIRs) enable binaural rendering, but dense per-listener measurements are costly. We address HRIR spatial up-sampling from sparse per-listener measurements: given a few measured HRIRs for a listener, predict HRIRs at unmeasured target directions. Prior learning methods often work in the frequency domain, rely on minimum-phase assumptions or separate timing models, and use a fixed direction grid, which can degrade temporal fidelity and spatial continuity. We propose HRIR-Former, a time-domain, grid-free binaural Transformer for reconstructing HRIRs at arbitrary directions from sparse inputs. It uses sinusoidal spatial features, a Conv1D refinement module, and auxiliary interaural time difference (ITD) and interaural level difference (ILD) heads. On SONICOM, it improves normalized mean squared error (NMSE), cosine distance, and ITD/ILD errors over prior methods; ablations validate modules and show minimum-phase preprocessing is unnecessary.

Abstract

Spatial Audio

Explore similar work

HRIR-Former: Grid-Free Time-Domain Reconstruction of Head-Related Impulse Responses with a Spatially Encoded Transformer

Masked Wavelet Scattering Transform Neural Field for Sound Field Reconstruction

HRTF-guided Binaural Target Speaker Extraction with Real-World Validation