Dual-Channel Contrastive Encoding of Sperm Whale Coda

João Victor Quintanilha; Emma Virnelli; Shuo Zhang

cs.SDOct 21, 2026

Dual-Channel Contrastive Encoding of Sperm Whale Coda

Authors: João Victor Quintanilha, Emma Virnelli, Shuo Zhang

Organizations: Tufts University, Medford, MA, USA

Abstract

Sperm whales (Physeter macrocephalus) communicate through short click sequences called codas, whose structure includes the rhythmic pattern of inter-click intervals (ICI) and the spectral composition of the clicks. In this paper, we propose the Dual-Channel Contrastive Encoder (DCCE), which explicitly encodes rhythm and spectral information with separate encoders before fusing them. We evaluate DCCE on three coda-understanding tasks: social unit, coda type, and individual identification. We compare DCCE against strong baselines from literature under a common linear-probe protocol and analyze the role of each channel and training objective through ablations. On 1,383 codas, shared partitions and validation-only layer selection support three random splits and three recording-date-disjoint folds. DCCE with unit pairing and type supervision achieves mean random-split macro-F1 of 0.901/0.865/0.597 for unit/type/identity. Raw ICI remains strongest for type (0.987), and ICI plus temporal mel statistics is a strong identity reference (0.724). An auxiliary identity objective improves spectral-branch identity prediction but does not improve the mean fused-identity score in this configuration. Date-grouped evaluation and four year holdouts expose generalization limits. The results support task-dependent combinations of timing and spectral information, without establishing independent biological channels or general architectural superiority. Code, notebooks, and complete results are available at https://dcce-research-site.vercel.app.

Figures & tables

Fig. 1: DCCE: separate input encoders, fusion, and optional branch objectives. Positive-pair construction varies across conditions. The linear probe is trained after freezing the representation.

	Random split			Recording-date grouped
Model	Unit	Type	ID	Unit	Type	ID
Raw ICI	$0.570\pm 0.023$	$0.987\pm 0.009$	$0.439\pm 0.007$	$0.540\pm 0.036$	$0.956\pm 0.036$	$0.379\pm 0.103$
Temporal mel	$0.773\pm 0.007$	$0.202\pm 0.025$	$0.429\pm 0.045$	$0.569\pm 0.094$	$0.069\pm 0.012$	$0.130\pm 0.074$
ICI + temporal mel	$0.837\pm 0.013$	$0.804\pm 0.016$	$0.724\pm 0.039$	$0.608\pm 0.150$	$0.668\pm 0.051$	$0.255\pm 0.062$
WhAM (frozen)	$0.906\pm 0.011$	$0.258\pm 0.046$	$0.450\pm 0.052$	$0.667\pm 0.155$	$0.066\pm 0.016$	$0.101\pm 0.075$
AVES (frozen)	$0.910\pm 0.010$	$0.253\pm 0.028$	$0.491\pm 0.029$	$0.700\pm 0.100$	$0.061\pm 0.022$	$0.074\pm 0.051$

Table 1: Main comparison: macro-F1 mean

\pm

sample SD over three runs. Fixed/frozen rows train only the downstream probe locally; + ICI adds annotated timing. DCCE uses unit pairing (U), type loss (T), and optional ID loss (I), with uniform sampling. AVES FT uses unit/type/ID supervision while freezing the waveform frontend. All learned representations are frozen before probing. Primary scores exclude unsupported test classes; grouped ID coverage differs from random splits.

Fig. 2: Random versus recording-date-grouped evaluation for four representative models. Points show mean macro-F1 over three runs; sample SD and the full model comparison appear in Table 1 . Lines connect protocol means. Grouped identity evaluation covers different supported classes and examples, so these differences are not isolated estimates of a recording-context penalty.

Model	R Type	R ID	G ID
Rhythm 110	$.978\pm.021$	$.477\pm.037$	$.352\pm.134$
Rhythm 111	$.973\pm.013$	$.454\pm.061$	$.331\pm.143$
Spectrum 110	$.202\pm.021$	$.358\pm.103$	$.054\pm.047$
Spectrum 111	$.199\pm.037$	$.444\pm.092$	$.092\pm.016$
Joint concat. 110	$.939\pm.062$	$.636\pm.015$	$.275\pm.117$
Joint concat. 111	$.927\pm.060$	$.696\pm.055$	$.239\pm.043$

Table 2: Matched architecture controls: macro-F1 mean

\pm

sample SD, three runs. R/G denote random/date-grouped; digits denote U/T/I. All use uniform sampling, 50 epochs, and whole-coda pairing. Enabled auxiliary heads act on the available branch; representations are frozen before probing. Joint concatenation is 128-dimensional; other outputs are 64-dimensional. Fused DCCE anchors appear in Table 1 .

Audio Representation Learning

Audio representation learning builds reusable features that capture speech, music, and sound for recognition, retrieval, generation, and analysis.

See all 211 papers

Jun 15, 2026

Rhythm of the Deep: Two-tier acoustic organization of sperm-whale codas from click waveforms to coda sequences

Sperm-whale codas are usually described by the number and timing of their clicks. These measures have revealed patterns in whale communication, but leave open how click sound contributes beyond timing and whether the order of whole codas carries additional structure. We investigate these questions through two tiers of acoustic organization, first within codas and then across coda sequences. Frozen pretrained audio encoders provide a framework for measuring acoustic variation beyond timing. We use them to identify recurring forms independently for clicks and whole codas in 1,483 codas, then validate their differences through direct waveform measurements. Within codas, click sound provides information beyond timing alone, while timing remains informative when click composition is held fixed. At the level of complete codas, we also find recurring acoustic forms whose differences are not fully captured by conventional timing-based types. With these forms identified, we examine how ordering contributes at each tier. Within codas, click-form order adds little predictive information beyond composition. Across codas, knowing the last two codas improves prediction over knowing only the last one. Together, these findings support two tiers with different organizing rules: acoustic composition and rhythmic timing within codas, and predictive relationships among whole codas across recorded sequences.

Mudit Sinha, Sanika Chavan

Independent Researcher, San Bruno, California 94066, United States

Bioacoustics

Ask PDF

Oct 25, 2026

Music of the Deep: An acoustic blueprint of sperm whale codas

Sperm-whale codas are usually described by the number and timing of their clicks. Earlier acoustic studies showed that individual clicks and longer coda sequences contain additional organization. We ask how that organization compares with music, whose smaller sounds also form larger units and ordered sequences. We compare codas with performed violin music, drum grooves, and sung melodies. Their sequences resemble music in some measures of predictability, yet codas show a different relationship between the smaller sounds within a unit and the order of units across a sequence. This combination was not observed in the musical references tested. Coda timing also varies coherently and carries information about individual callers. The recordings show another striking property: broad, reproducible variation along acoustic dimensions associated with human ratings of emotion, with a range comparable to human audio in most eligible tests. These findings motivate an acoustic blueprint that describes natural codas beyond their click timing. Its importance is apparent in synthetic audio: WhAM produces recognizable coda-like recordings, yet the tested clips remain distinguishable from natural codas in five of six acoustic comparisons after strict matching. The blueprint provides a fuller way to study natural codas and to evaluate how faithfully synthetic recordings represent them.

Mudit Sinha, Sanika Chavan

Independent Researcher, San Bruno, California 94066, United States

Bioacoustics Underwater Acoustics

Ask PDF

Sep 15, 2026

Structure Across Voices: Comparing acoustic-event type accumulation and sequence dependence across four vocal corpora using frozen audio encoders

Comparing acoustic structure across very different vocal corpora is difficult because the recordings use different native event units, contain different amounts of sequence, and were collected under different conditions. We develop a matched procedure for comparing such corpora without forcing them into the same event definition. Frozen audio encoders provide a common acoustic representation for recorded events, while matched sampling gives each corpus the same amount of evidence for acoustic and sequence analysis. We apply this procedure to sperm-whale codas, human speech phones, Bengalese-finch syllables, and common-marmoset calls. The comparison reveals different patterns across the four corpora. Sperm-whale recordings stand out broadly: new acoustic forms continue to appear, while their sequences also show strong organization and repetition. Finch recordings show the strongest organization of sounds over time, especially in how neighboring sounds and repeated patterns are arranged. Direct temporal and spectral measurements show that these differences are also present in the recorded sounds themselves. The comparison provides a practical way to measure acoustic diversity and temporal organization across very different vocal corpora by combining a common acoustic representation with matched observation opportunity, without treating their native sounds as equivalent.

Mudit Sinha, Sanika Chavan

Independent Researcher, San Bruno, California 94066, United States

Bioacoustics

Ask PDF

Abstract

Audio Representation Learning

Explore similar work

Rhythm of the Deep: Two-tier acoustic organization of sperm-whale codas from click waveforms to coda sequences

Music of the Deep: An acoustic blueprint of sperm whale codas

Structure Across Voices: Comparing acoustic-event type accumulation and sequence dependence across four vocal corpora using frozen audio encoders