EquiSELD: Efficient training of equivariant sound event localization and detection networks

Goksenin Yuksel; Marcel van Gerven; Kiki van der Heijden

cs.SDSep 19, 2026

EquiSELD: Efficient training of equivariant sound event localization and detection networks

Authors: Goksenin Yuksel, Marcel van Gerven, Kiki van der Heijden

Organizations: Dept. of Machine Learning and Neural Computing, Donders Institute, Radboud University, Nijmegen, the Netherlands · Mortimer B. Zuckerman Institute, Columbia University, New York, USA

Abstract

First-order Ambisonics (FOA) signals exhibit exact O(3) symmetry: The rotation or reflection of the FOA signal modifies the direction of arrival of the sound sources, while preserving the sound sources themselves. Prior attempts to utilize this spatial symmetry of FOA to improve the efficiency and robustness of sound event detection and localization (SELD) systems either learned only an approximation of the symmetry through rotation-based augmentation or relied on computationally expensive methods to integrate equivariance. Furthermore, prior work focused exclusively on SO(3) equivariance , leaving the potential of incorporating O(3) equivariance for SELD tasks unclear. To address these limitations, we developed EquiSELD. This equivariant attention network processes first-order Ambisonics as paired streams of O(3)-invariant scalars and equivariant intensity vectors, producing an invariant activity magnitude and an equivariant DOA with a Multi-ACCDOA readout. To compare the impact of O(3) versus SO(3)-equivariance, we designed a matched SO(3)-only variant. EquiSELD outperforms prior equivariant networks on both simulated scenes with measured RIRs and recordings of real-world sound scenes at a fraction of the training cost. EquiSELD additionally surpasses the performance of non-equivariant SELD networks of a similar size on the simulated real-world sound scenes and achieves competitive performance on the real-world sound scenes.

Figures & tables

Figure 1: EquiSELD architecture. Five invariant scalar features (orange) and two equivariant intensity vectors (blue) are processed with frequency attention, attention pooling, and attention across time and tracks. The network produces multi-ACCDOA outputs for $C$ sound classes for $K$ tracks of which direction estimates are equivariant. ISAB = induced set attention block; PMA = pooling by multihead attention [ 10 ] .

Model	Par.	GPU h $\downarrow$	ER $\downarrow$	F $\uparrow$	LE (det.) $\downarrow$	LR $\uparrow$	Det. $\uparrow$	$\mathcal{E}_{\mathrm{S}}\downarrow$	ER $\downarrow$	F $\uparrow$	LE $\downarrow$	LR $\uparrow$	$\mathcal{E}_{\mathrm{S}}$ $\downarrow$
			STARSS23						TAU2021
SELDNet	0.75M	0.16	0.74	13.7	$55.7^{\circ}$ ( $33.1^{\circ}$ )	26.5	11	0.66 [0.61, 0.69]	0.68	27.5	$35.8^{\circ}$	42.8	0.55 [0.53, 0.56]
SELDNet	2.25M	0.18	0.75	16.1	$51.9^{\circ}$ ( $28.6^{\circ}$ )	29.5	11	0.65 [0.62, 0.68]	0.68	29.8	$22.3^{\circ}$	46.6	0.51 [0.50, 0.52]
SELDNet +	2.25M	0.83	0.57	25.4	$68.2^{\circ}$ ( $18.5^{\circ}$ )	30.0	0 9	0.60 [0.57, 0.63]	0.61	40.5	$32.3^{\circ}$	49.4	0.47 [0.46, 0.49]
CGNet-STS-MultiACCDOA	6.53M	14.5	0.80	11.3	$78.3^{\circ}$ ( $33.1^{\circ}$ )	24.5	0 9	0.72 [0.68, 0.74]	0.73	30.1	$25.3^{\circ}$	52.0	0.51 [0.50, 0.53]
CGNet-STS	6.28M	12.1	–	–	–	–	–	–	0.79	26.2	$31.9^{\circ}$	55.1	0.54 [0.52, 0.55]

Table 1: SELD performance on STARSS23 and TAU2021, with training cost on TAU2021 (single A100). For STARSS23, “Det.” is the number of the 13 classes the model detects and the parenthesised LE is the class-averaged localization error restricted to those classes. We show the jackknife estimates of the

\mathcal{E}_{\mathrm{S}}

under the test-set recordings.

Model	Mean	Std	Min	Max
SELDNet	0.76	$\pm$ 0.03	0.65	0.82
SELDNet +	0.60	$\pm$ 0.01	0.60	0.62
AttnSELD	0.75	$\pm$ 0.04	0.70	0.86
AttnSELD +	0.58	$\pm$ 0.01	0.57	0.59
EquiSELD (ours)	0.56	$\pm$ 0.00	0.56	0.56

Table 2: SELD score over O(3) elements on STARSS23.

Figure 2: Data scaling. Shaded areas are 95% confidence intervals.

Explore similar work

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection
Jun 26, 2026Stefano Giacomelli, Stefano Damiano, Claudia Rinaldi +2Audio Representation Learning Direction-of-Arrival Estimation
DISIM Department, University of L’Aquila, Italy · ESAT Department, KU Leuven, Belgium · CNIT, Research Unit of L’Aquila, Italy +1
Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonics
Sep 19, 2026Goksenin Yuksel, Marcel van Gerven, Kiki van der HeijdenSpatial Audio Audio Representation Learning
Dept. of Machine Learning and Neural Computing, Donders Institute, Radboud University, Nijmegen, the Netherlands · Mortimer B. Zuckerman Institute, Columbia University, New York, USA
Velocity Potential Neural Field for Efficient Ambisonics Impulse Response Modeling
Mar 23, 2026Yoshiki Masuyama, Francois G. Germain, Gordon Wichern +2Spatial Audio Physics-Informed ML
Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA

Jun 26, 2026J/K move · Enter open · S save

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection

Stefano Giacomelli, Stefano Damiano, Claudia Rinaldi, Fabio Graziosi+1

DISIM Department, University of L’Aquila, Italy · ESAT Department, KU Leuven, Belgium · CNIT, Research Unit of L’Aquila, Italy+1

Audio Representation Learning Direction-of-Arrival Estimation Sound Event Detection Spatial Audio

Open paper AskPDF arXiv

This report investigates the extension of pretrained General-Purpose Audio Tagging (GP-AT) models toward spatially grounded Sound Event Localization and Detection (SELD). The proposed AT2SELD framework couples a pretrained AT backbone with compact First-Order Ambisonics (FOA) spatial processing, track-wise SED and Cartesian DOA estimation, permutation aware supervision, and calibration. It characterizes how semantic audio priors support localization-aware scene analysis under data, computation, and deployment constraints. The framework is developed through informed multi-stage Neural Architecture Search (NAS). Stage 1 shows that spectral FOA descriptors, based on magnitude, phase, and Intensity Vectors (IVs), provide the most reliable interface for semantic-to-spatial transfer. Stage 2 identifies early residual spatial encoding as the main capacity-sensitive component, while late track-wise abstraction and recurrent smoothing act mainly as refinement stages. Stage 3 shows that late cross-stitch coupling improves semantic-spatial interaction, whereas early fusion is costlier and less effective. Diagnostic evaluation analyzes the selected architecture under class balancing, focal loss, activity-conditioned DOA supervision, threshold calibration, and transfer across STARSS23, TAU2019, TAU-NIGENS2020, and TAU-NIGENS2021. Focal loss improves the activity point, active-only DOA supervision mitigates inactive target dominance, and validation-selected thresholds recover calibration without replacing spatial learning. Cross-dataset and oracle-activity analyses indicate strong fixed source localization on TAU2019, transferable representations from TAU NIGENS2021, and meaningful but uncertain behavior on STARSS23. Overall, GP-AT priors appear promising for SELD design when embedded in spatial-aware architectures and optimized through integrated calibration and deployment oriented strategies.

Abstract

Explore similar work

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection

Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonics

Velocity Potential Neural Field for Efficient Ambisonics Impulse Response Modeling