cs.SDNov 10, 2026

Exploring Post-Hoc Disentanglement of Audio Foundation Models for Music Similarity

Authors: Lucas de Souza Brandão, Gabriel da Costa Merlin, Diego Furtado Silva

Organizations: Universidade de São Paulo

Abstract

Audio foundation models like CLAP, MuQ, and MERT excel at capturing general musical similarity. On the other hand, their entangled latent spaces limit interpretability and fine-grained attribute control. Conditional Similarity Networks (CSN) address this by pairing a backbone representation model with learned masks that gate the embedding space into attribute-specific subspaces. However, existing methods train the backbone from scratch, rather than leveraging pretrained foundation models. We instead present a systematic study of post-hoc disentanglement, adapting CSN to gate frozen pretrained embeddings. We evaluate adding trainable parameters to adapt the embedding under different masking and pooling regimes across two datasets. Our findings show that minimal adaptation capacity is sufficient for post-hoc gating to improve both attribute-specific disentanglement and overall retrieval performance, while pooling, masking, and depth choices yield limited additional gains.

Figures & tables

Explore similar work

CardsList
  1. MERIT: Learning Disentangled Music Representations for Audio Similarity

    May 26, 2026Abhinaba Roy, Junyi Liang, Dorien HerremansDisentangled Representation LearningAudio Representation Learning

  2. Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings

    Jul 4, 2026Héctor Martel, Joe Hennessy-Priest, Taemin ChoRepresentation GeometryAudio Representation Learning

  3. What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models

    Aug 14, 2026Angelos-Nikolaos Kanatas, Yuexuan Kong, Pablo Alonso-Jiménez +2Audio Representation LearningMusic Information Retrieval