Audio-Text Retrieval

Ask about this topic

Overview

Audio-text retrieval studies how to find sounds, music, or speech from text queries—and how to retrieve matching text from audio. Common approaches use dual encoders trained with contrastive learning on audio-caption pairs; newer systems adapt multimodal language models, add instruction-aware embeddings, or align words with local audio segments. Benchmarks such as AudioCaps and Clotho test broad semantic matching, while newer evaluations probe speech intent, music relations, long-audio moments, and queries involving negation or duration. A central challenge is that models can match overall meaning yet miss acoustic details, user intent, or reasoning about what is absent, so robust retrieval across varied queries remains unresolved.

Latest papers

CardsList

25 papers1 paper in the last four weeks, down 83% on the four weeks before. 0.0% of all new papers.

  1. SSEmb: Training a Generalizable Speech Retrieval Model Using Only Automatic Speech Recognition Data

    Oct 13, 2026Sangkwon Park, Dongwook Lee, Youngho Cho +3Audio-Text Retrieval

    ECE, Seoul National University · IPAI, Seoul National University · University of Seoul +1

  2. CORA: A Protocol for Diagnosing Boundary Robustness in Text-to-Audio Retrieval under Query Reformulations

    Sep 27, 2026Jae Min Woo, Kyongmin Kong, Bogyung Jeong +3Retrieval RobustnessQuery Reformulation

    Sogang University

  3. Nearest but Not Dearest: Shared Curator-Feedback Infrastructure for Content-Only Search and Recommendation

    Sep 24, 2026Matt SandlerRecommender SystemsMusic Information Retrieval

    Feed.fm San Francisco, California, USA

  4. VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval

    Sep 16, 2026Aaron Yee, Fengjie Lu, Jiarui Hai +5Audio-Text RetrievalHybrid Retrieval

    Zhejiang University · Johns Hopkins University · independent researcher

  5. MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions

    Sep 14, 2026Zhancheng Guo, Congren Dai, Shangda Wu +4Audio-Language Model EvaluationAudio-Text Retrieval

    Central Conservatory of Music, Beijing, China · Tsinghua University, Beijing, China

  6. Overview and Meta-Analysis of DCASE 2026 Challenge Task 6: Audio Moment Retrieval from Long Audio

    Sep 11, 2026Hokuto Munakata, Tatsuya Komatsu, Keisuke Imoto +3Audio-Text Retrieval

    LY Corporation, Japan · Kyoto University, Japan · Sony Interactive Entertainment, Japan +2

  7. SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval

    Sep 2, 2026Zineb Lahrichi, Marc Ferras, Gaël Richard +1Audio CaptioningAudio-Language Models

    Sony CTC, France · LTCI, Telecom Paris, Institut polytechnique de Paris, France

  8. AllMusicCaps: Album Reviews as Complementary Supervision for Music CLAP

    Aug 26, 2026Pablo Alonso-Jiménez, Xavier Lizarraga-Seijas, Xavier Serra +1Audio-Language ModelsAudio-Text Retrieval

    Music Technology Group, Universitat Pompeu Fabra, Barcelona

  9. INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval

    Aug 17, 2026Chen-An Li, Hung-yi LeeAudio-Text RetrievalAudio-Language Model Evaluation

    National Taiwan University · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)

  10. Steering dense music retrieval with open-vocabulary concept discovery

    Aug 9, 2026Julien Guinot, Alain Riou, Elio Quinton +1Feature AttributionSparse Autoencoders

    Music & Audio Machine Learning Lab, Universal Music Group, London, U.K. · Centre for Digital Music, Queen Mary University of London, U.K.

  11. The Sound of Absence: Audio-Language Embedding Models Struggle with Negation

    Jul 14, 2026Chun-Yi Kuan, Hung-yi LeeAudio-Language Model EvaluationAudio-Language Models

    ♡Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · ♣Artificial Intelligence Center of Research Excellence (AI-CoRE), National Taiwan University, Taiwan

  12. ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models

    Jun 27, 2026Fengjie Lu, Chenang Jiang, Jiarui Hai +2Audio Representation LearningAudio-Language Models

    Zhejiang University · Johns Hopkins University

  13. ATCCaps: A Call-Sign-Aware Speech Dataset for Air Traffic Control Recognition

    Jun 21, 2026Dongdong Li, Jianwei Song, Jianwei Wang +1ASR EvaluationAutomatic Speech Recognition

    Department of Computer Science and Technology, East China University of Science and Technology, 130 Meilong Road, Shanghai 200237, China

  14. MixProLAP: Mixture-Induced Uncertainty Modeling for Probabilistic Language-Audio Pretraining

    Jun 18, 2026Yu Nakagome, Jaesong Lee, Soo-Whan ChungAudio-Text AlignmentMultimodal Pretraining

    LINE WORKS Corporation, Japan · NAVER Cloud Corporation, South Korea

  15. FIGMA: Towards FIne-Grained Music retrievAl

    Jun 4, 2026Nishit Anand, Ashish Seth, Sreyan Ghosh +2Audio UnderstandingAudio-Text Retrieval

    University of Maryland, College Park, USA

  16. Forgive or forget: Understanding the context of hate in audio retrieval systems

    Jun 4, 2026Arghya Pal, Sailaja Rajanala, Raphael C. -W. Phan +1Content ModerationAudio-Text Retrieval

    School of Information Technology, Monash University, Malaysia campus · University of Groningen, the Netherlands

  17. OmniRetriever: Any-to-Any Audio-Video-Text Retrieval via Fusion-as-Teacher Distillation

    May 26, 2026Yunze Liu, Chi-Hao Wu, Enmin Zhou +1Cross-Modal LearningCross-Modal Knowledge Distillation

    Memories.ai Research

  18. ReasonAudio: A Benchmark for Evaluating Reasoning Beyond Matching in Text-Audio Retrieval

    May 5, 2026Honglei Zhang, Yuting Chen, Chenpeng Hu +2Audio ReasoningAudio Understanding

    School of Software, Nanjing University Nanjing, China · School of Software, Northwestern Polytechnical University Xi’an, China · College of Computing and Data Science, Nanyang Technological University Singapore, Singapore

  19. Robust Audio-Text Retrieval via Cross-Modal Attention and Hybrid Loss

    Apr 25, 2026Meizhu Liu, Matthew Rowe, Amit Agarwal +8Multimodal RobustnessMultimodal Embedding

  20. ATIR: Towards Audio-Text Interleaved Contextual Retrieval

    Apr 22, 2026Tong Zhao, Chenghao Zhang, Yutao Zhu +1Multimodal IRMultimodal Large Language Models

    Gaoling School of Artificial Intelligence, Renmin University of China

  21. Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

    Apr 20, 2026HaeJun Yoo, Yongseop Shin, Insung Lee +2Audio-Language Model EvaluationMultimodal IR

    Sogang University

  22. ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining

    Mar 30, 2026Anuj Diwan, Eunsol Choi, David HarwathAudio-Language ModelsAudio-Text Retrieval

    Department of Computer Science, The University of Texas at Austin, USA · Computer Science and Data Science, New York University, USA

  23. CLAR: CIF-Localized Alignment for Retrieval-Augmented Speech LLM-Based Contextual ASR

    Mar 26, 2026Shangkun Huang, Huan Shen, Wei Zou +1Contextual ASRAutomatic Speech Recognition

    BRVoice Team, Bairong, Inc., China

  24. Cross-lingual Matryoshka Representation Learning across Speech and Text

    Feb 23, 2026Yaya Sy, Dioula Doucouré, Christophe Cerisara +1Representation LearningAudio-Text Retrieval

    LORIA, CNRS · Soynade Research

  25. Re-purposing Multimodal Large Language Models for Audio-Text Retrieval

    Feb 20, 2026Jilan Xu, Carl Thomé, Danijela Horak +2Multimodal Large Language ModelsAudio Understanding

    Visual Geometry Group, University of Oxford · Epidemic Sound · School of Artificial Intelligence, Shanghai Jiao Tong University