cs.SDNov 6, 2026

AudioSphere: Towards Self-Supervised Spatial Audio Representation Models

Authors: Goksenin Yuksel, Marcel van Gerven, Kiki van der Heijden

Organizations: Donders Institute, Radboud University, Nijmegen, the Netherlands · Mortimer B Zuckerman Institute, Columbia University, New York, United States

Abstract

Recent audio embedding models learn representations that transfer across clip- and frame-level tasks, yet they operate almost exclusively on single-channel signals and are blind to spatial structure. Spatial audio models capture that structure but rely on clip-level supervision or contrastive objectives, yielding embeddings unsuitable for frame-level tasks like sound event detection and localization (SELD). We argue this dichotomy is not fundamental: patch-level spatial self-supervision can jointly learn acoustic and spatial features at both temporal granularities. We introduce AudioSphere, a multi-channel masked autoencoder trained on Ambisonics recordings that reconstructs both spectral content and directional cues via a joint spatiotemporal masking strategy that masks both time-frequency patches and intensity vectors. AudioSphere supports frame-level predictions while producing strong clip-level representations via aggregation. Alongside the model, we release RealSELD, a standardized benchmark for evaluating frozen backbones on real-world multi-channel SELD. AudioSphere enables, for the first time, SELD with a frozen self-supervised backbone, while matching or exceeding prior self-supervised audio embedding models on clip-level classification tasks spanning environmental sound, speech, and music, and on frame-level sound event detection. All code and data are available at https://github.com/labhamlet/AudioSphere and https://huggingface.co/labhamlet.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Bearings: Self-Supervised Soundfield Embeddings from First-Order Ambisonics

    Sep 19, 2026Goksenin Yuksel, Marcel van Gerven, Kiki van der HeijdenSpatial AudioAudio Representation Learning

  2. Self-Supervised Object-Centric Representation for Multi-Task Audio Analysis

    Oct 30, 2026Parthasaarathy Sudarsanam, Tuomas VirtanenAudio Representation LearningAudio Source Separation

  3. AudioFM: Towards General Purpose Audio Representation Learning

    Oct 25, 2026Yamin Li, Ruidong Zhang, Anran Wang +19Audio Representation LearningTemporal Representation Learning