cs.SDOct 25, 2026

STFT Frontend Mismatch in Pretrained Audio Spectrogram Transformers: Analysis and Adaptation

Authors: Julia Wilkins, Wei-Ching Lin, Abinaya Kumar, Shabnam Ghaffarzadegan, Juan Pablo Bellow

Organizations: New York University, USA · Bosch Research, USA

Abstract

Pretrained audio spectrogram transformers are widely used as backbones of machine listening systems for sound event classification. However, they are typically trained using spectrograms with a fixed Short-Time Fourier Transform (STFT) configuration, implicitly binding the model to the time-frequency resolution seen during pretraining. In practice, adapting these systems to new domains or computational constraints often requires different STFT frontend parameters. In this work, we conduct a controlled study of the sensitivity of the foundational Audio Spectrogram Transformer to variations in STFT window and hop size, showing that audio classification performance degrades substantially as frontend parameters deviate from pretraining settings. As a mitigation strategy, we introduce a lightweight conditioning mechanism using FiLM that enables a pretrained model to adapt to variable input configurations using limited additional training data. Our method generalizes well to unseen STFT configurations, improving model robustness without sacrificing performance in the original setting.

Explore similar work

CardsList
  1. LAST: Looped Audio Spectrogram Transformer

    Oct 1, 2026Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina +1Audio UnderstandingRecurrent Transformers

  2. From Masking to Merging: Rethinking SpecAugment for Efficient Audio Spectrogram Transformer

    Sep 6, 2026Minhee Park, Hyowon Ahn, Chanwoo KimToken MergingData Augmentation

  3. Spectrogram features for audio and speech analysis

    Mar 16, 2026Ian McLoughlin, Lam Pham, Yan Song +7Audio Representation LearningFrequency-Domain Feature Learning