Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonics

Goksenin Yuksel; Marcel van Gerven; Kiki van der Heijden

cs.SDSep 19, 2026

Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonics

Authors: Goksenin Yuksel, Marcel van Gerven, Kiki van der Heijden

Organizations: Dept. of Machine Learning and Neural Computing, Donders Institute, Radboud University, Nijmegen, the Netherlands · Mortimer B. Zuckerman Institute, Columbia University, New York, USA

Abstract

Recently proposed self-supervised audio encoders learn powerful general-purpose representations of sound scenes, yet they are spatially blind. To supply the missing spatial representation of sound scenes, we introduce Bearings. Bearings is a self-supervised framework that learns soundfield embeddings from unlabeled first-order Ambisonics. We pre-train a masked auto-encoder paired with a decoder conditioned on frozen acoustic embeddings from an off-the-shelf single-channel audio encoder. Our results show that the resulting soundfield embeddings form a reusable stream that can be attached to frozen acoustic encoders with a lightweight trainable fusion head. On sound event localization and detection, concatenating our soundfield embeddings with acoustic representations provides the missing spatial information and enables joint detection and localization, raising the location-dependent F-score from below 4 to 50 on TAU-NIGENS 2021 and 39 on STARSS23. To our knowledge, Bearings is the first self-supervised soundfield encoder whose embeddings plug into frozen acoustic encoders without retraining either model.

Figures & tables

Figure 1: Bearings overview. (A) Self-supervised pre-training: a masked ViT encodes seven-channel FOA features (four log-mel spectrograms, three active-intensity vectors), and a decoder that cross-attends to embeddings from a frozen mono encoder reconstructs direction distributions and two-scale diffuseness at the masked patches. (B) Downstream fusion: the frozen Bearings encoder supplies soundfield embeddings that are concatenated with acoustic embeddings from any frozen single-channel acoustic encoder.

		TAU-NIGENS 2021				STARSS23
Backbone	Method	$\mathrm{ER}_{20^{\circ}}\downarrow$	$\mathrm{F}_{20^{\circ}}\uparrow$	$\mathrm{LE}_{\mathrm{CD}}\downarrow$	$\mathrm{LR}_{\mathrm{CD}}\uparrow$	$\mathrm{ER}_{20^{\circ}}\downarrow$	$\mathrm{F}_{20^{\circ}}\uparrow$	$\mathrm{LE}_{\mathrm{CD}}\downarrow$	$\mathrm{LR}_{\mathrm{CD}}\uparrow$
	SELDNet (7-ch FOA + IVs) [ 12 , 15 ]	0.69	33.9	24.1	43.9	0.57	29.9	22.0	47.7
	GRAM-Ambisonics (7-ch FOA + IVs) [ 19 ]	0.62	41.0	23.3	66.3	0.56	37.4	22.8	58.8
GRAM-Clean	Single-channel, acoustic only	0.83 [0.81,0.84]	2.8 [2.3,3.3]	102.1 [99.3,104.9]	34.7 [33.0,36.2]	0.83 [0.81,0.86]	2.5 [1.6,3.4]	147.8 [146.2,149.3]	16.6 [15.0,18.2]
GRAM-Clean	+ Bearings (ours)	0.55 [0.53,0.56]	50.4 [48.2,52.7]	19.3 [18.3,20.3]	66.8 [65.0,68.5]	0.51 [0.46,0.55]	38.9 [34.6,43.8]	22.0 [18.3,25.7]	55.5 [49.3,61.4]
Dasheng-Base	Single-channel, acoustic only	0.83 [0.82,0.85]	3.2 [2.6,3.9]	87.1 [84.1,90.3]	43.2 [41.5,45.0]	0.82 [0.80,0.86]	2.9 [2.3,3.5]	146.1 [144.6,147.5]	18.9 [17.1,20.7]

Table 1: Downstream SELD performance. Bold marks the best value per column.

Training data	Embeddings	$\mathrm{ER}_{20^{\circ}}\downarrow$	$\mathrm{F}_{20^{\circ}}\uparrow$	$\mathrm{LE_{CD}}\downarrow$	$\mathrm{LR_{CD}}\uparrow$
25%	Bearings (ours)	0.67	30.6	24.4	49.4
25%	Learned-In Domain	0.74	21.7	40.5	45.2
50%	Bearings (ours)	0.57	46.6	20.5	64.2
50%	Learned-In Domain	0.66	31.7	24.8	56.5
100%	Bearings (ours)	0.55	50.4	19.3	66.8
100%	Learned-In Domain	0.60	39.0	22.6	64.3

Table 2: Label efficiency on TAU-NIGENS 2021.

	SELD				Probe
Configuration	$\mathrm{ER}_{20^{\circ}}\downarrow$	$\mathrm{F}_{20^{\circ}}\uparrow$	$\mathrm{LE_{CD}}\downarrow$	$\mathrm{LR_{CD}}\uparrow$	$\mathrm{DOA}^{\circ}\downarrow$	$\mathrm{RT}_{60}$ $r\uparrow$
Full model	0.55	50.4	19.3	66.8	16.1	0.53
Random init	0.70	25.8	28.2	55.4	30.4	0.26
No conditioning	0.57	46.4	20.6	65.2	17.5	0.50
w/o $\mathcal{L}_{\Psi}$	0.57	48.3	19.8	65.9	16.1	0.52
w/o $\mathcal{L}_{q}$	0.68	29.5	27.7	63.7	49.2	0.55

Table 3: Ablation studies (TAU-NIGENS 2021, Matterport3D).

Encoder initialization, downstream	$\mathrm{ER}_{20^{\circ}}\downarrow$	$\mathrm{F}_{20^{\circ}}\uparrow$	$\mathrm{LE_{CD}}\downarrow$	$\mathrm{LR_{CD}}\uparrow$
Single-channel, acoustic only	0.83	2.8	102.1	34.7
Bearings: random, frozen	0.70	25.8	28.2	55.4
Bearings: random, trained end-to-end	0.65	35.7	23.7	63.6
Bearings: Pre-trained, frozen	0.55	50.4	19.3	66.8
Bearings: Pre-trained, fine-tuned	0.54	51.2	18.1	65.5

Table 4: Impact of spatial pre-training (TAU-NIGENS 2021).

Explore similar work

Probing Spatial Structure in Pretrained Audio Representations
Jun 4, 2026Chuyang Chen, Sivan Ding, Adrian S. Roman +1Audio Representation Learning Representation Probing
Music and Audio Research Laboratory, New York University, USA
SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays
Oct 4, 2026Sonal Kumar, Sinan Hersek, Artem Dementyev +6Direction-of-Arrival Estimation Audio-Language Models
University of Maryland, College Park · Google · Google DeepMind
Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners
Aug 20, 2026Umberto Cappellazzo, Xubo Liu, Stavros Petridis +1Self-Supervised Speech Representation Learning Autoregressive Modeling
Imperial College London · University of Surrey

Jun 4, 2026J/K move · Enter open · S save

Probing Spatial Structure in Pretrained Audio Representations

Chuyang Chen, Sivan Ding, Adrian S. Roman, Juan P. Bello

Music and Audio Research Laboratory, New York University, USA

Audio Representation Learning Representation Probing Spatial Audio

Open paper AskPDF arXiv

Pretrained spatial audio encoders are increasingly used as general-purpose representations for perceptual tasks, yet their spatial encoding capabilities remain poorly understood. We introduce the Spatial Audio Representation Learning (SARL) benchmark, a controlled framework for evaluating spatial information in pretrained audio models. SARL probes source-level factors (azimuth, elevation, distance, class) and room-level factors (RT60, volume, shape). Experiments across diverse encoders reveal three patterns: input configuration and training paradigm shape spatial encoding; source factors are consistently easier to decode than room factors; and sensitivity analysis under controlled perturbations shows heterogeneous responses to source and room variation. These results reveal systematic biases in current pretrained audio representations. SARL is released as an open-source benchmark for reproducible evaluation of spatial audio representations.

Abstract

Explore similar work

Probing Spatial Structure in Pretrained Audio Representations

SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners