eess.ASNov 9, 2026

SCLAP: Temporal structural alignment improves localization in long-form audio

Authors: Yacouba Kaloga, Ina Kodrasi

Organizations: Signal Processing for Communication, Idiap Research Institute, Martigny, Switzerland

Abstract

Audio-text models such as Contrastive Language-Audio Pretraining (CLAP) typically pool frame and token sequences uniformly into global embeddings. This discards temporal structure and impairs both retrieval and localization, especially for short events in long recordings. We introduce SCLAP, a sequence-level contrastive framework that aligns audio frames and text tokens using monotone one-dimensional optimal transport. Its learned marginals and transport plan provide fine-grained temporal supervision from clip-level captions alone. On short-form audio (i.e., Clotho and TACOS datasets with 15-30s recordings), SCLAP remains competitive for retrieval while improving zero-shot localization over CLAP and Text-aware Attention Pooling. On long-form audio (i.e., CASTELLA dataset with up to 300s recordings), SCLAP improves retrieval performance and its localization advantage grows as the event-to-context ratio decreases. At inference, SCLAP requires no optimal transport computation since retrieval can use pooled embeddings and localization can use inexpensive frame-text cosine scores.

Figures & tables

Explore similar work

CardsList
  1. FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining

    Apr 1, 2026Xiquan Li, Xuenan Xu, Ziyang Ma +4Audio-Language ModelsAudio Representation Learning

  2. Encode Once, Decode Never: Reusing Audio LM Internals for Efficient Temporal Localization

    Feb 10, 2026Joseph An, Phillip Keung, Jiaqi Wang +2Temporal Audio GroundingAudio-Language Models

  3. Query-Conditioned Liquid Temporal Grounding for Long-Form Audio

    Oct 22, 2026Yuelan Cheng, Yonggang Zhu, Yongqiang Chen +3Temporal Audio Grounding