Large-scale bioacoustic detection using semantic segmentation: a deep learning framework applied to fin whale calls in ocean-bottom seismometer recordings

Jocelyn Japnanto; Alex A. Saoulis; Miriam Romagosa; Rita Leitão; Gabrielle Arrieta; Mónica A. Silva; Matthew Graham; Ana M. G. Ferreira

physics.geo-phSep 8, 2026

Large-scale bioacoustic detection using semantic segmentation: a deep learning framework applied to fin whale calls in ocean-bottom seismometer recordings

Authors: Jocelyn Japnanto, Alex A. Saoulis, Miriam Romagosa, Rita Leitão, Gabrielle Arrieta, Mónica A. Silva, Matthew Graham, Ana M. G. Ferreira

Organizations: Department of Earth Sciences, University College London, London, United Kingdom · Department of Physics and Astronomy, University College London, London, United Kingdom · Instituto de Investigação em Ciências do Mar (OKEANOS), Universidade dos Açores, Horta, Portugal · Marine Geosciences Department, Institute of Marine Sciences, CSIC, 08003 Barcelona, Spain ·

Abstract

Ocean-bottom seismometers (OBS), originally deployed for geophysical research, continuously record low-frequency sound for months to years across broad areas of ocean, offering a largely untapped resource for passive acoustic monitoring (PAM) of baleen whales. Realising this potential requires automated detection methods that operate reliably across the varied conditions in large sensor networks. We present a deep learning semantic segmentation framework that detects the 20-Hz notes of fin whales (Balaenoptera physalus) in OBS spectrograms, assigning each pixel a probability of belonging to a call and converting the resulting probability maps into time-frequency bounding boxes describing individual detections. We trained the model on hydrophone data from one OBS deployment in the Azores-Madeira-Canaries region and applied it without retraining to vertical-component seismometer data from a second, geographically distinct deployment, showing that a single trained model generalises across sensor types and recording environments. Applied to 378,912 h of recordings from 46 OBS sites, the detector identified 6.3 million calls, forming the largest fin whale call catalogue assembled to date, with high precision (~97%) across both deployments. The resulting catalogue resolves call timing and spectral structure accurately enough to support ecological analyses, revealing coherent seasonal shifts in three persistent inter-note interval (INI) groups across the singing season and basin-scale patterns in calling activity. By transforming existing geophysical infrastructure into a scalable sensing network, our approach substantially expands the spatial and temporal reach of PAM without new hardware investment, offering a transferable framework for tracking other low-frequency vocalising species and informing conservation planning, marine spatial management, and abundance estimation across large scales.

Figures & tables

Figure 1: Map of the São Jorge (triangles) and UPFLOW (squares) OBS arrays located across the Azores-Madeira-Canaries region. Each symbol marks an individual OBS station, with those coloured red being used in this study. Hydrophone-component data from São Jorge (August 2022 to February 2023), and vertical-component data from UPFLOW (June 2021 and August 2022), were used to detect fin whale calls.

Panel of Figure 2 — Figure 2: A 5-minute recording showing part of a fin whale song sequence at station SJO03, located west of the São Jorge Island. a) Waveform time series and b) spectrogram of the hydrophone component. Instrument response was removed and the trace was band-pass filtered between 12 to 35 Hz with a Butterworth filter. The spectrogram was calculated using a short-time Fourier transform with a window length of 2.0 s and 75% overlap. Manual annotations of 20-Hz notes are shown as white boxes.

Figure 3: Overview of the data processing and detection pipeline. a) Raw waveforms are segmented into five-minute windows. b) Greyscale spectrograms, computed via short-time Fourier transforms, serve as input to the model. c) The encoder-decoder machine learning (ML) model. d) Post-processing extracts individual calls from the model’s predictions. e) Resulting detections (white boxes) form the call catalogue.

Panel of Figure 4 — Figure 4: Comparison of post-processing approaches to extract calls, applied to a 5-minute sample from the test dataset from station SJO03 (05-Dec-2022). a) Manual annotations of 20-Hz notes (pink). b) Corresponding 20-Hz note probability map, inferred by the detector. ci) A simple thresholding approach, where pixels exceeding a fixed probability threshold are grouped into connected components to form detections, cii) and the resulting calls after filtering, evaluated against annotations, showing true positives (TP; green), false positives (FP; red), and false negatives (FN; orange). di) Our method, where local maxima (“seeds”) in a smoothed response map initialise the formation of our candidate bounding boxes prior to non-maximum suppression (NMS), dii) and the resulting detections, processed and evaluated similarly. Around 03:02:30, simple thresholding fragments and discards noisy calls that our method recovers.

Network	Station	Total recording hours	Number of detections
São Jorge	SJO01	3,792	58,478
	SJO02	3,792	248,155
	SJO03	3,768	197,509
	SJO04	3,768	217,934
	SJO05	3,768	215,590
	SJO06	3,768	216,157

Table 1: Number of 20-Hz notes detected at each station in the São Jorge and UPFLOW arrays.

Panel of Figure 5 — Figure 5: Recall as a function of minimum SNR threshold applied to a) an aggregation of all UPFLOW annotations, and b) the held-out test set from station SJO03. The threshold sets the minimum SNR an annotated call must exceed to enter the evaluation set, and the grey shading shows the number of annotations retained at each threshold.

Network	Station	TP	FP	FN	Precision	Recall	F1 Score	mIoU
São Jorge	SJO03	4583	267	1137	0.94	0.80	0.87	0.61
UPFLOW	OBS17	114	2	196	0.98	0.37	0.53	0.13
	UP06	93	2	247	0.98	0.27	0.43	0.13
	UP09	187	1	250	0.99	0.43	0.60	0.12
	UP11	46	1	447	0.98	0.09	0.17	0.13
	UP16	199	2	275	0.99	0.42	0.59	0.13

Table 2: Detection performance on the held-out SJO03 dataset and independent validation dataset from 16 UPFLOW stations.

Figure 6: Weekly fin whale call counts across all 46 stations in this study. The heatmap shows the number of detected calls per station aggregated by week. Grey intervals correspond to periods where the instrument was not recording, while white intervals indicate weeks with no detections. Elevated call abundance is concentrated between October and February across most stations. North-western (located farther offshore in the central North Atlantic) and south-eastern -most stations (located closer to continental Portugal and Morocco) are highlighted in red and yellow, respectively.

Panel of Figure 7 — Figure 7: INI distributions of 20-Hz notes at the a) São Jorge array (6 stations, August 2022 to February 2023) and the b) UPFLOW array (40 stations, June 2021 to August 2022). Three distinct INI groups are observed in both arrays.

Figure 8: INI group composition of fin whale calls on the UPFLOW array during peak singing season. Detections are clustered into one of three INI groups ( $\mu$ ). Stations with fewer than 1,000 INIs for a given month are excluded from the plot.

Panel of Figure 9 — Figure 9: a) Number of calls detected and b) mean INI over time in the UPFLOW array, aggregated by week. Only peak singing months are included in the latter. North-western and south-eastern -most stations are highlighted in red and yellow, respectively. Grey dashed vertical lines indicate the two weeks that contrast the start and end of the singing season. These are the weeks used in c) , which shows a bubble plot of the number of calls detected at each station (indicated by the size of the bubble), and the mean INI (indicated by the bubble’s colour), in a single week beginning in ci) early December and cii) mid-February.

Figure 10: Heatmap of weekly mean INI values across all 46 stations in this study. Grey intervals correspond to periods where the instrument is not recording, while white intervals show insufficient INIs as only weeks with more than 100 valid INIs were considered. North-western and south-eastern -most stations are highlighted in red and yellow, respectively.

Network	Station	Deployment start	Recovery date	Sensor used	Sampling rate (Hz)	Station depth (km)
São Jorge	SJO01	26/08/2022	30/01/2023	HighTech HTI-96-Min hydrophone	250	1.24
	SJO02	26/08/2022	30/01/2023		250	1.36
	SJO03	26/08/2022	29/01/2023		250	1.24
	SJO04	26/08/2022	29/01/2023		250	1.16
	SJO05	26/08/2022	29/01/2023		250	1.16
	SJO06	26/08/2022	29/01/2023		250	1.13

Table S1 : Recording characteristics of each station from the São Jorge and UPFLOW arrays used in this study.

Figure S1 : Samples from the training set. Top: Samples from station SJO03 with annotated calls (pink). Bottom: Background samples from multiple stations, showing various sources of noise.

Figure S2 : Example of spectrogram-based SNR estimation for a single detection from station UP09 (03-Feb-2022, 20:16:27). Left : Power spectrogram centred on the detection (yellow vertical line). SNR is computed within a predefined frequency band $[f_{\min},f_{\max}]$ (magenta lines). The signal window (green) is defined as a duration $T=1.5~s$ immediately following the detection onset, while the noise window (cyan) is the preceding interval of equal duration. A narrow frequency band (orange) is constructed symmetrically about the dominant frequency (orange dotted line) at the onset time. The signal level $S_{\text{sig}}$ is taken as the median power within this narrow band over the signal window, whereas the local and global noise level $S_{\text{noise,local}}$ and $S_{\text{noise,global}}$ is the median power across the full frequency band within the noise window, and context window, respectively. Right : The power spectrum at the onset time and includes the peak frequency and the corresponding narrow-band selection used for SNR estimation.

Figure S3 : Recall as a function of minimum SNR threshold applied to ground-truth annotations, with results broken down at each annotated UPFLOW station.

Panel of Figure 17 — Figure S4 : Comparisons of INIs derived from model detections $N_{d}$ (green) and annotations $N_{a}$ (yellow). These songs show a a) singlet INI pattern centred at 16.5 s, b) doublet INI pattern alternating between 12.5 s and 16.5 s, and c) predominantly singlet INI pattern centred at 11.5 s.

Figure S5 : Example of anomalous noise signals in station UP28 masking fin whale vocalisations around $\sim$ 22 Hz. This noise appeared at multiple distinct frequency bands and was observed throughout the whole deployment.

Figure S6 : Broadband noise in station UP38 observed throughout the deployment, which may have affected the detectability of calls (white boxes)

Figure S7 : Broadband noise in station UP39, similar to those in station UP38.

Panel of Figure 21 — Figure S8 : Detected 20-Hz notes (white boxes) overlaid on spectrograms of OBS recordings illustrating sources of missed detections. a) Calls at station SJO03 masked by a regional earthquake, visible as a broadband transient signal. b) A dense chorusing event at station UP25 in which overlapping calls from multiple individuals reduce call separability. c) Masking due to shipping noise at station UP41.

Bioacoustics

Bioacoustics uses sound to study animal activity, communication, biodiversity, and ecological change.

See all 60 papers

An End-to-End Workflow for Fin Whale Song Detection, Note Characterization, and Localization with Distributed Acoustic Sensing
Aug 3, 2026Dídac Diego-Tortosa, Miriam Romagosa, Arantza Ugalde +4Distributed Acoustic Sensing Sound Event Detection
Institut de Ciències del Mar (ICM-CSIC), Barcelona · Institut de Física Corpuscular (IFIC), Centre Mixte Universitat de València (UV-CSIC), Paterna
Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade
Sep 1, 2026Daniela Ruiz, Manuel Castellote, Zhongqi Miao +5Sound Event Detection Bioacoustics
Microsoft AI for Good Research Lab, Redmond, Washington, USA · Universidad de los Andes, Bogota, Colombia
A Self-Supervised Approach for Minimal-Annotation Hydroacoustic Data Exploration
Jul 7, 2026Pierre-Yves Raumer, Axel Marmoret, Dorian Cazau +6Audio Representation Learning Self-Supervised Learning
Laboratoire de G´eologie, Ecole Normale Sup´erieure/CNRS UMR 8538, PSL Research University, Paris 75005, France · Universit´e de Brest, CNRS, Ifremer, UMR6538 Geo-Ocean, 29280 Plouzan´e, France · Lab-STICC – UMR 6285 CNRS, ENSTA IP Paris, Brest, France +5

Aug 3, 2026J/K move · Enter open · S save

An End-to-End Workflow for Fin Whale Song Detection, Note Characterization, and Localization with Distributed Acoustic Sensing

Dídac Diego-Tortosa, Miriam Romagosa, Arantza Ugalde, Hugo Latorre+3

Institut de Ciències del Mar (ICM-CSIC), Barcelona · Institut de Física Corpuscular (IFIC), Centre Mixte Universitat de València (UV-CSIC), Paterna

Distributed Acoustic Sensing Sound Event Detection Bioacoustics Underwater Acoustics

Open paper AskPDF arXiv

Submarine fiber-optic cables instrumented with distributed acoustic sensing (DAS) provide an effective approach for large-scale monitoring of fin whales. We present an end-to-end workflow for detecting, characterizing, and localizing fin whale notes, tested on two submarine telecom cables in the Strait of Gibraltar and western Alboran Sea. The workflow applies a kurtosis-value picker adapted to narrow-band fin whale notes. Channel-wise detections are grouped into individual notes using density-based spatio-temporal clustering, cluster agglomeration, and hyperbolic fitting to reject incoherent picks. The retained clusters are characterized through temporal, spectral, and energy-related descriptors that support note-type discrimination and estimation of inter-note intervals. Relative arrival times across DAS channels are then used in a grid-search procedure to estimate candidate source locations. Evaluation against manually annotated detections from six fin whale songs yielded median pick-level precision of 0.990 and recall of 0.744, and median cluster-level precision of 0.880 and recall of 0.806. Representative applications demonstrate separation of overlapping vocalizations, characterization of type-A and type-B notes, and the inference of apparent source movement. By transforming dense DAS recordings into compact note-level bioacoustic information, the workflow provides an integrated framework for fin whale monitoring and a basis for adaptation to other synchronized acoustic receiver arrays.

Abstract

Bioacoustics

Explore similar work

An End-to-End Workflow for Fin Whale Song Detection, Note Characterization, and Localization with Distributed Acoustic Sensing

Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade

A Self-Supervised Approach for Minimal-Annotation Hydroacoustic Data Exploration