cs.SDNov 8, 2026

LAION-Audio-XL: A Web-Scale Corpus and Scaling Studies for Contrastive Language-Audio Pretraining

Authors: Gijs Wijngaard, Felix Förster, Ross Wightman, Marianna Nezhurina, Christoph Schuhmann, Jenia Jitsev, A. Sophia Koepke

Organizations: Jülich Supercomputing Center (JSC), Forschungszentrum Jülich (FZJ) · LAION · Technical University of Munich, MCML · University of Tübingen, Tübingen AI Center

Abstract

Recent progress on contrastive audio-language models can be attributed mainly to the scale and quality of their training data. Larger training datasets expose models to a broader range of sounds and corresponding audio captions. To exploit this, we collect 1.48 billion audio samples from academic datasets, internet datasets, web collections, Common Crawl, and YouTube, and process them with a multi-stage curation pipeline. Furthermore, we generate short and long captions for samples in our corpus that lack them. We use our dataset to study the scaling behavior of contrastive audio-language models based on the NaFlex architecture for the audio encoder. Our experiments identify trade-offs between model size and training data for a given compute budget. Moreover, our resulting models, CLAP2, outperform prior audio-text models of comparable size on the MAEB and HEAR benchmarks. We release our dataset, captions, model checkpoints, and code.

Figures & tables

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation

    Nov 20, 2025Wei-Cheng Tseng, Xuanru Zhou, Mingyue Huo +3Audio-Language Model EvaluationAudio Representation Learning

  2. FineLAP: Taming Heterogeneous Supervision for Fine-grained Language-Audio Pretraining

    Apr 1, 2026Xiquan Li, Xuenan Xu, Ziyang Ma +4Audio-Language ModelsAudio Representation Learning

  3. Adaptive Perturbation Selection for Contrastive Audio Decoding

    Jun 30, 2026Aaron Isidore Grace, Zhouyuan Huo, Weiran WangAudio-Language Model EvaluationAudio-Language Model Hallucination Detection