Audio-Language Models

Ask about this topic

Overview

Audio-language models learn to connect speech, music, and environmental sounds with language so they can answer questions, describe events, retrieve audio, and generate speech or music. Common approaches align audio and text with contrastive dual encoders, or connect audio encoders to language models through projectors, discrete tokens, or continuous representations; training often combines large-scale pretraining with instruction tuning and preference or reinforcement learning. Benchmarks increasingly test temporal grounding, overlapping sounds, multilingual speech, and conversational or clinical settings, alongside standard captioning and question-answering tasks. A central challenge is ensuring answers actually depend on acoustic evidence rather than text priors, while improving robustness, faithful reasoning, and efficiency.

Latest papers

CardsList

303 papers55 papers in the last four weeks, up 22% on the four weeks before. 0.3% of all new papers.

  1. Developing a Speech Language Model for Basque

    Oct 27, 2026Andoni Sudupe, Inma Hernaez, Ander Arriandiaga +3Speech Language ModelsAudio-Language Models

    HiTZ Basque Center for Language Technology - Aholab Signal Processing Laboratory, University of the Basque Country UPV/EHU, Spain

  2. AudioRSI: Recursively Self-Improving Audio Reasoners

    Oct 27, 2026Jiajun Fan, Roger Ren, Jingyuan Li +6Audio ReasoningAudio-Language Models

    UIUC

  3. TAC2: Timestamped Audio Captioning as composable functions

    Oct 27, 2026Swapnil Bhosale, Prem Seetharaman, Ke Chen +3Audio-Language ModelsAudio Captioning

    University of Surrey, UK · Adobe Research, USA

  4. Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks

    Oct 25, 2026Jihoon Jeong, Mirco Ravanelli, Zhepei Wang +1Audio ReasoningAudio-Language Models

    Mila-Québec AI Institute · Université Laval · Concordia University +1

  5. In-Context Adapter Generation for Compositional Generalization in Large Audio Language Models

    Oct 25, 2026Yiwei Guo, Siqi Huang, Bohan Li +4Compositional GeneralizationAudio-Language Models

    X-LANCE Lab, MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University · Jiangsu Key Lab of Language Computing · Xi’an Jiaotong University

  6. MusicTwins: Training Audio-Language Models to Listen like Their Persona

    Oct 23, 2026Milan Liessens Dujardin, Kevin MiaoAudio-Language ModelsPersonalized Language Model Alignment

    Bryel Labs, UC Berkeley · Bryel Labs

  7. Concentrated Causal Influence, Distributed Diagnostic Information: Cognitive Impairment Decisions in Large Audio-Language Models

    Oct 23, 2026Xiutian Zhao, Luqi Sun, Björn Schuller +1Audio-Language ModelsCircuits in Language Models

    Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Group on Language, Audio & Music (GLAM), Imperial College London, UK

  8. MixJudge: Learning a Language-Queryable Critic for Vocal Mixing

    Oct 22, 2026Kuan-Hsun Ho, Yuan-Chiao Cheng, Yen-Tung Yeh +1Audio-Language ModelsAudio Editing

    Independent Researcher · Music Informatics Group, Georgia Institute of Technology · Music and AI Lab, National Taiwan University +1

  9. AT-Differ: Dataset for Audio-Text Difference Captioning

    Oct 21, 2026Akihiro Fujii, Yuki Okamoto, Yuki Saito +1Audio-Text AlignmentAudio-Language Model Evaluation

    The University of Tokyo, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan

  10. Heard But Not Said: Prompting Understates Temporal Information of Musical Moments in Frozen Large Audio-Language Models

    Oct 21, 2026Eunsik ShinAudio-Language ModelsAudio-Language Model Evaluation

    Music and Audio Research Group (MARG), Department of Intelligence and Information, Seoul National University

  11. XingChen-UniFE: A Unified Generative Speech Front-End via Complementary Acoustic-Semantic Modeling

    Oct 21, 2026Kelan Kuang, Qihan Hu, Jian Kang +3Audio-Language ModelsTarget Speaker Extraction

    China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd.

  12. Qwen-Audio-3.1-ASR Technical Report

    Oct 20, 2026Chuanmeng Bian, Daren Chen, Peixin Chen +45Audio-Language ModelsAutomatic Speech Recognition

  13. Uncovering Audio-Text Alignment in Large Audio-Language Models through Cross-Modal Concept Neurons

    Oct 20, 2026Tzu-Quan Lin, Chih-Kai Yang, Yi-Cheng Lin +2Audio-Text AlignmentAudio-Language Models

    Centre for Speech Technology Research, University of Edinburgh, United Kingdom · Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan

  14. AudioScorer: Learning Question-Aware Token Relevance for Large Audio Language Models

    Oct 19, 2026Marwan Farag, Bin YangToken PruningAudio-Language Models

    Institute of Signal Processing and System Theory (ISS), University of Stuttgart, Stuttgart, Germany

  15. Text Dominance in Paralinguistic Conflict: Tracing Suppression Through Audio LLM Circuits

    Oct 19, 2026Ankur Banerjee, Vishnu Guntakala, Danity Pike +3Audio-Language ModelsAudio-Language Model Evaluation

    Independent Researcher · Brown University · University of Birmingham

  16. DF-PALM: Dynamic Fusion Prompt Learning for Audio--Language Models

    Oct 18, 2026Yulin Sun, Shenghua Li, Yong Dou +3Few-Shot Audio ClassificationPrompt Learning

    College of Computer Science and Technology, National University of Defense Technology, Changsha, China. · State Key Laboratory of Complex & Critical Software Environment

  17. Long Listening Thoughts: Eliciting Open Auditory Reasoning with Deliberative Perception and Cognitive Refinement

    Oct 17, 2026Jaeyeon Kim, Chao-Han Huck Yang, Luoyi Zhang +5Audio ReasoningAudio-Language Models

    Carnegie Mellon University · NVIDIA

  18. VocalGrad: Evaluating Acoustic Perception in Audio Language Models

    Oct 16, 2026Koki Ryu, Hitomi YanakaAudio-Language Model EvaluationAudio Understanding

    The University of Tokyo · Riken · Tohoku University

  19. ConvexLLM: Scaling Convex-Hull Speech Interfaces to General-Purpose Audio Intelligence

    Oct 16, 2026Ming-Hao Hsu, Shujie Liu, Yuxuan Hu +4Audio-Language ModelsSpeech Language Models

    School of Data Science, The Chinese University of Hong Kong, Shenzhen, China. · Microsoft Research Asia, Hong Kong, China. · Microsoft Research, Redmond, WA, USA.

  20. TRACE: Trustworthy Reasoning from Acoustic Cues and Evidence

    Oct 15, 2026Giovana Morais, Yujin Kim, Atilay Kucukoglu +1Audio-Language Model EvaluationAudio-Language Models

    Music and Audio Research Lab, NYU · Integrated Design and Media, NYU

  21. From Signals to Evidence: Interpretable Underwater Acoustic Target Sensing with Large Language Models - Part I: Framework and Recognition

    Oct 15, 2026Yuge Zhang, Jie Chen, Zhengqiao Zhao +4Underwater AcousticsAudio Representation Learning

    School of Artificial Intelligence, Northwestern Polytechnical University, Xi’an, Shaanxi 710129, China · School of Ocean Engineering and Technology, Sun Yat-sen University, Zhuhai, 519000, Guangdong, China · School of Marine Science and Technology, Northwestern Polytechnical University, Xi’an, Shaanxi 710129, China +1

  22. Bias Mitigation in Audio-Enabled LLMs through Adversarial Training and Distillation Methods

    Oct 15, 2026Adnan Al Ali, Jindřich Libovický, Anne LauscherAdversarial DebiasingLanguage Model Debiasing

    Charles University, Faculty of Mathematics and Physics, Czech Republic · Trustworthy AI Lab, University of Hamburg, Germany

  23. Qwen-Audio-Jev: A Unified Interface for Audio Decisions

    Oct 15, 2026Peixin Chen, Zhigao Chen, Zhiyun Fan +19Audio-Language ModelsAudio-Language Model Evaluation

    Token Foundry, Alibaba Group

  24. MedSonor: Learning to Use Clinical Audio in Medical LLMs

    Oct 14, 2026Geyang Yu, Junying Chen, Benyou WangMedical LLMsAudio-Language Models

    The Chinese University of Hong Kong, Shenzhen

  25. TVPara: Paralinguistic Captioning of Expressive TV-Series Speech in Complex Acoustic Environments

    Oct 14, 2026Ziyang Zhang, Wenyi Yu, Wen Wu +2Audio-Language ModelsSpeech Prosody

    Tsinghua University, China · Shanghai Artificial Intelligence Laboratory, China

  26. What Does LALM Post-training Learn? Rethinking Audio Understanding and Reasoning Capabilities

    Oct 14, 2026Hua-Hsuan Chen, Xuanjun Chen, Wei-Chung Lu +5Audio-Language ModelsAudio-Language Model Evaluation

    National Taiwan University · NVIDIA · Independent Researcher +1

  27. ATTEST: Evidence-Grounded Reinforcement Learning for Long-Audio Temporal Understanding

    Oct 12, 2026Ashutosh Chaubey, Jinxi Guo, Minh Tran +2Temporal Audio GroundingAudio Understanding

    Meta Superintelligence Labs · University of Southern California

  28. SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning

    Oct 12, 2026Chenlong Wei, Hanling Wang, Mingjin Lou +6Spatial AudioSpatial Reasoning Benchmarks

    Xi’an Jiaotong-Liverpool University · University of California, San Diego · Imperial College London +2

  29. SpectralBridge: Adding High-Frequency Capability to Large Audio-Language Models

    Oct 12, 2026Sergei Polezhaev, Saba Anjum, Michael GormishAudio-Language ModelsAudio Understanding

    Nebius AI

  30. PitchIn: Natural-Language Vocal Coaching from Unstructured Community Feedback

    Oct 12, 2026Hayeon Bang, Hounsu Kim, Joonyong Park +2Audio-Language ModelsAudio Understanding

    KAIST, Republic of Korea · The University of Tokyo, Japan · ETRI, Republic of Korea