Geo-ATBench: A Benchmark for Geospatial Audio Tagging with Geospatial Semantic Context

Yuanbo Hou; Yanru Wu; Qiaoqiao Ren; Shengchen Li; Stephen Roberts; Dick Botteldooren

eess.ASMar 11, 2026

Geo-ATBench: A Benchmark for Geospatial Audio Tagging with Geospatial Semantic Context

Authors: Yuanbo Hou, Yanru Wu, Qiaoqiao Ren, Shengchen Li, Stephen Roberts, Dick Botteldooren

Organizations: Machine Learning Research Group, Engineering Science, University of Oxford, UK · Department of Intelligent Science, Xi’an Jiaotong-Liverpool University, China · EECS, KTH Royal Institute of Technology, Sweden · WAVES Research Group, Information Technology, Ghent University, Belgium

Abstract

Environmental sound understanding in computational auditory scene analysis (CASA) is often formulated as an audio-only recognition problem. This formulation leaves a persistent drawback in multi-label audio tagging (AT): acoustic similarity can make certain events difficult to separate from waveforms alone. In such cases, disambiguating cues often lie outside the waveform. Geospatial semantic context (GSC), derived from geographic information system data, e.g., points of interest (POI), provides location-tied environmental priors that can help reduce this ambiguity. A systematic study of this direction is enabled through the proposed geospatial audio tagging (Geo-AT) task, which conditions multi-label sound event tagging on GSC alongside audio. To benchmark Geo-AT, Geo-ATBench is introduced as a polyphonic audio benchmark with geographical annotations, containing 10.71 hours of audio across 28 event categories; each clip is paired with a GSC representation from 11 semantic context categories. GeoFusion-AT is proposed as a unified geo-audio fusion framework that evaluates feature-, representation-, and decision-level fusion on representative audio backbones, with audio- and GSC-only baselines. Results show that incorporating GSC improves AT performance, especially on acoustically confounded labels, indicating geospatial semantics provide effective priors beyond audio alone. A crowdsourced listening study with 10 participants on 579 samples shows that there is no significant difference in performance between models on Geo-ATBench labels and aggregated human labels, supporting Geo-ATBench as a human-aligned benchmark. The Geo-AT task, benchmark Geo-ATBench, and reproducible geo-audio fusion framework GeoFusion-AT provide a foundation for studying AT with geospatial semantic context within the CASA community. Dataset, code, models are on homepage (https://github.com/WuYanru2002/Geo-ATBench).

Figures & tables

Figure 1: The number of recordings with GPS information uploaded to Freesound each year.

Class	Cnt.	Dur.(s)	Class	Cnt.	Dur.(s)	Class	Cnt.	Dur.(s)
Natural Sounds			Human Sounds			Sounds of Things
Bird sounds	1024	8191	Speech	794	5133	Car	463	3068
Crickets	343	3091	Footsteps	288	2225	Plane	340	3092
Falling water	325	2922	Music Instru.	188	1593	Train	165	1291
Flowing water	319	2774	Music	144	1330	Bell	121	835
Waves	307	2754	Singing	81	624	Boat	115	927

Table 1: Sound classes in Geo-ATBench, grouped by Natural, Human, and Thing. Dur. denotes the total duration (in seconds) of each class, and Cnt denotes sample count. Musical instru. abbreviates Musical instrument, and Falling water denotes Falling water/Rain.

Figure 2: Summary of sound classes and acoustic similarity. (Left) Distribution of three coarse-grained sound classes. (Right) Intra-class similarity across 28 sound event classes computed from log-Mel spectrogram features.

Figure 3: Sankey diagram summarizing co-occurrence links from left to right: 3 coarse-grained sound classes, 28 fine-grained sound event classes, GSC types, and the Geo-ATBench dataset. Flow width indicates co-occurrence strength. This diagram represents the distribution of audio events and GSC types within the dataset, and is not intended to imply precise real-world relationships, as sound occurrences can vary significantly depending on the specific geographical context (e.g., residential roads vs highways).

Figure 4: Overall architecture of the GeoFusion-AT framework for Geo-AT task.

Figure 5: Instantiations of GeoFusion-Early (feature-level fusion).

Figure 6: Instantiations of GeoFusion-Inter (representation-level fusion).

Figure 7: Instantiations of GeoFusion-Late (decision-level fusion).

Figure 8: Average Precision (AP) for GSC-only multi-label tagging under different POI extraction ranges. POI-derived GSC is constructed from OSM entities retrieved with the same 11 OSM feature keys (e.g., land use, amenities) and encoded into the fixed-length GSC vector $\mathbf{g}$ . mAP is computed on the test set over 5 independent runs.

Figure 9: ROC curves for zero-shot audio-only tagging inference on Geo-ATBench labels. Micro and macro ROC AUC are reported for AudioSet-pretrained PANNs, AST, and CLAP after AudioSet-to-Geo-ATBench label mapping. Overall performance: PANNs (Micro AUC 0.8576, Macro AUC 0.8409), AST (0.6672, 0.6443), and CLAP (0.8325, 0.8022).

Figure 10: t-SNE visualization of audio embeddings from zero-shot inference for PANNs, AST, and CLAP on the 28 Geo-ATBench event classes; clusters show effective separation, while overlaps highlight acoustically similar classes.

Strategy	Fine-grained (28 classes)			Coarse-grained (3 classes)
Strategy	PANNs	AST	CLAP	PANNs	AST	CLAP
Audio-Only	0.770±0.006	0.820±0.015	0.824±0.008	0.961±0.006	0.904±0.012	0.966±0.005
GSC-Only	0.767±0.010			0.867±0.009
GeoFusion-Early	0.812±0.010	0.846±0.010	0.826±0.010	0.954±0.004	0.914±0.009	0.950±0.008
Gain ( $\Delta$ )	+0.042	+0.026	+0.002	-0.007	+0.010	-0.016
GeoFusion-Inter	0.824±0.010	0.829±0.003	0.842±0.006	0.964±0.008	0.912±0.011	0.968±0.005

Table 2: The mean average precision (mAP) of different models on the Geo-ATBench dataset. The rows labeled Gain (

\Delta

) represent the performance difference relative to the audio-only baseline. All metrics are averaged across 5 independent experimental runs.

Figure 11: Per-class average precision across 28 classes for the audio-only AST, the GSC-only baseline, and GeoFusion-Early-AST, the integration of geospatial information improves performance for multiple classes.

Figure 12: Per-class AP change (fusion minus audio-only) for the GeoFusion-Early-AST exemplar on the 28-class Geo-AT task.

Event	F1 (Label)	F1 (Human)	Dur.	Event	F1 (Label)	F1 (Human)	Dur.
Bird sounds	0.861	0.856	8191	Falling water	0.872	0.786	2922
Speech	0.827	0.869	5133	Flowing water	0.774	0.667	2774
Plane	0.779	0.514	3092	Waves	0.756	0.677	2754
Crickets	0.883	0.836	3091	Footsteps	0.629	0.607	2225
Car	0.491	0.404	3068	Musical instru.	0.651	0.667	1593

Table 3: Top-10 event classes (total = 34,843 s

\approx

9.68 h) by descending total duration in Geo-ATBench. F1 (Label) and F1 (Human) are F1 scores of GeoFusion-Inter-CLAP predictions evaluated against Geo-ATBench labels and aggregated human consensus labels, respectively. Dur. denotes total event duration (seconds). Musical instru. denotes Musical instrument, and Falling water corresponds to Falling water/Rain.

Audio Classification

Audio classification assigns recordings to categories such as speech, music, animal calls, acoustic scenes, and health-related events.

See all 147 papers

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection
Jun 26, 2026Stefano Giacomelli, Stefano Damiano, Claudia Rinaldi +2Audio Representation Learning Direction-of-Arrival Estimation
DISIM Department, University of L’Aquila, Italy · ESAT Department, KU Leuven, Belgium · CNIT, Research Unit of L’Aquila, Italy +1
Geo2Sound: A Scalable Geo-Aligned Framework for Soundscape Generation from Satellite Imagery
Apr 16, 2026Kunlin Wu, Yanning Wang, Haofeng Tan +6Cross-Modal Alignment Audio Generation
The Hong Kong University of Science and Technology (Guangzhou), China · University of South Carolina, USA · University of Canterbury, New Zealand +2
Interpretable Perception and Reasoning for Audiovisual Geolocation
Mar 5, 2026Yiyang Su, Xiaoming LiuAudio-Visual Understanding
Michigan State University, East Lansing, MI 48824, USA

Jun 26, 2026J/K move · Enter open · S save

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection

Stefano Giacomelli, Stefano Damiano, Claudia Rinaldi, Fabio Graziosi+1

DISIM Department, University of L’Aquila, Italy · ESAT Department, KU Leuven, Belgium · CNIT, Research Unit of L’Aquila, Italy+1

Audio Representation Learning Direction-of-Arrival Estimation Sound Event Detection Spatial Audio

Open paper AskPDF arXiv

This report investigates the extension of pretrained General-Purpose Audio Tagging (GP-AT) models toward spatially grounded Sound Event Localization and Detection (SELD). The proposed AT2SELD framework couples a pretrained AT backbone with compact First-Order Ambisonics (FOA) spatial processing, track-wise SED and Cartesian DOA estimation, permutation aware supervision, and calibration. It characterizes how semantic audio priors support localization-aware scene analysis under data, computation, and deployment constraints. The framework is developed through informed multi-stage Neural Architecture Search (NAS). Stage 1 shows that spectral FOA descriptors, based on magnitude, phase, and Intensity Vectors (IVs), provide the most reliable interface for semantic-to-spatial transfer. Stage 2 identifies early residual spatial encoding as the main capacity-sensitive component, while late track-wise abstraction and recurrent smoothing act mainly as refinement stages. Stage 3 shows that late cross-stitch coupling improves semantic-spatial interaction, whereas early fusion is costlier and less effective. Diagnostic evaluation analyzes the selected architecture under class balancing, focal loss, activity-conditioned DOA supervision, threshold calibration, and transfer across STARSS23, TAU2019, TAU-NIGENS2020, and TAU-NIGENS2021. Focal loss improves the activity point, active-only DOA supervision mitigates inactive target dominance, and validation-selected thresholds recover calibration without replacing spatial learning. Cross-dataset and oracle-activity analyses indicate strong fixed source localization on TAU2019, transferable representations from TAU NIGENS2021, and meaningful but uncertain behavior on STARSS23. Overall, GP-AT priors appear promising for SELD design when embedded in spatial-aware architectures and optimized through integrated calibration and deployment oriented strategies.

Abstract

Audio Classification

Explore similar work

From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection

Geo2Sound: A Scalable Geo-Aligned Framework for Soundscape Generation from Satellite Imagery

Interpretable Perception and Reasoning for Audiovisual Geolocation