Overview

Audio-language models learn to connect speech, music, and environmental sounds with language so they can answer questions, describe events, retrieve audio, and generate speech or music. Common approaches align audio and text with contrastive dual encoders, or connect audio encoders to language models through projectors, discrete tokens, or continuous representations; training often combines large-scale pretraining with instruction tuning and preference or reinforcement learning. Benchmarks increasingly test temporal grounding, overlapping sounds, multilingual speech, and conversational or clinical settings, alongside standard captioning and question-answering tasks. A central challenge is ensuring answers actually depend on acoustic evidence rather than text priors, while improving robustness, faithful reasoning, and efficiency.

Latest papers

CardsList

325 papers34 papers in the last four weeks, down 43% on the four weeks before. 0.3% of all new papers.

  1. Think2Listen: Evidence-Centric Learning For Agentic Audio Understanding With LALMS

    Nov 16, 2026Jin Wang, Liang Lin, Kaiwen Luo +7Audio UnderstandingAudio Reasoning

  2. Speech Tokenizers are Vulnerable: Transferable Semantic Attack and Robust Tokenizer

    Nov 11, 2026Zhisheng Zhang, Yifan Mi, Yixuan Zhou +6Transfer-Based Adversarial AttacksAdversarial Robustness of ASR

  3. Harmony: Bridging Audio Understanding and Editing through Agentic Collaboration

    Nov 10, 2026Zhongyi Tong, Yapeng TianAudio EditingAudio Understanding

  4. Walrus Technical Report

    Nov 10, 2026Alkis Koudounas, Osamu Take, Hayato Futami +2Speech Foundation ModelsAudio-Language Models

  5. AURA: Auditory Spatial Intelligence via a Unified Representation from a General-Audio Encoder through Complete Alignment

    Nov 10, 2026Boyan Han, Xiaoyan Wang, Zhe Sun +6Audio-Language ModelsSpatial Reasoning in Language Models

  6. SCLAP: Temporal structural alignment improves localization in long-form audio

    Nov 9, 2026Yacouba Kaloga, Ina KodrasiAudio-Language ModelsAudio-Text Alignment

  7. LAION-Audio-XL: A Web-Scale Corpus and Scaling Studies for Contrastive Language-Audio Pretraining

    Nov 8, 2026Gijs Wijngaard, Felix Förster, Ross Wightman +4Audio-Language PretrainingAudio-Language Models

  8. Beyond the Transcript: Measuring and Improving the Use of Acoustic Information in Audio Language Models

    Nov 8, 2026Linkai Peng, Baorian Nuchged, Yuyang YaoAudio-Language ModelsAudio-Language Model Evaluation

  9. LongCap: Hour-Scale Timestamped Audio Captioning via Chunk-wise Processing

    Nov 8, 2026Jinchuan Tian, Sen Chen, Wenbin Huang +7Audio CaptioningAudio-Language Models

  10. Machine Hearing: Can Audio LLMs Generalize Beyond Human Auditory Perception?

    Nov 7, 2026Liwei Liu, Jun Tao, Xingyuan Li +8Audio-Language ModelsAudio Understanding

  11. Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval

    Nov 6, 2026Chen-An Li, Cheng-Han Chiang, Hung-yi LeeAudio-Text RetrievalAudio Representation Learning

  12. FoleyForge: Free-form, Controllable, Rectified-flow Editing of Sound Effects

    Nov 5, 2026Giovana Morais, Hugo Flores García, Prem Seetharaman +4Audio EditingAudio-Language Models

  13. Confidence-Aware Aggregation for Multimodal Audio Reasoning

    Nov 3, 2026Michel Olvera, Paraskevas Stamatiadis, Ga{ë}l RichardAudio-Language ModelsAudio Reasoning

  14. ReListen: Training-Free Evidence Localization in Hour-Long Audio with Frozen Audio Language Models

    Nov 3, 2026Ming-Hao Hsu, Shujie Liu, Yuxuan Hu +4Audio QAAudio-Language Models

  15. Spatial Transfusion: From Structured Spatial Scene Understanding to Generation and Editing

    Nov 2, 2026Jaeyeon Kim, Sebastian Braun, Hannes GamperSpatial AudioAudio-Language Models

  16. One Waveform, Two Stories: Benchmarking Speech-Acoustic Conflict in Audio-Language Models

    Oct 30, 2026Wenbin Wang, Yuheng Li, Koichiro YoshinoAudio-Language Model EvaluationAudio-Language Models

  17. Hierarchical Dual-Graph Retrieval for Long-form Audio Reasoning

    Oct 30, 2026Eunsang Lee, Taein Ju, Jiyoung LeeAudio ReasoningGraph Retrieval-Augmented Generation

  18. Developing a Speech Language Model for Basque

    Oct 27, 2026Andoni Sudupe, Inma Hernaez, Ander Arriandiaga +3Speech Language ModelsAudio-Language Models

  19. TAC2: Timestamped Audio Captioning as composable functions

    Oct 27, 2026Swapnil Bhosale, Prem Seetharaman, Ke Chen +3Audio-Language ModelsAudio Captioning

  20. Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks

    Oct 25, 2026Jihoon Jeong, Mirco Ravanelli, Zhepei Wang +1Audio ReasoningAudio-Language Models

  21. In-Context Adapter Generation for Compositional Generalization in Large Audio Language Models

    Oct 25, 2026Yiwei Guo, Siqi Huang, Bohan Li +4Compositional GeneralizationAudio-Language Models

  22. MusicTwins: Training Audio-Language Models to Listen like Their Persona

    Oct 23, 2026Milan Liessens Dujardin, Kevin MiaoAudio-Language ModelsPersonalized Language Model Alignment

  23. Concentrated Causal Influence, Distributed Diagnostic Information: Cognitive Impairment Decisions in Large Audio-Language Models

    Oct 23, 2026Xiutian Zhao, Luqi Sun, Björn Schuller +1Audio-Language ModelsCircuits in Language Models

  24. MixJudge: Learning a Language-Queryable Critic for Vocal Mixing

    Oct 22, 2026Kuan-Hsun Ho, Yuan-Chiao Cheng, Yen-Tung Yeh +1Audio-Language ModelsAudio Editing

  25. AT-Differ: Dataset for Audio-Text Difference Captioning

    Oct 21, 2026Akihiro Fujii, Yuki Okamoto, Yuki Saito +1Audio-Text AlignmentAudio-Language Model Evaluation