Audio-Language Models

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 414% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 184

All topics

Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models
Jul 13, 2026Yu-Han Huang, Chih-Kai Yang, Ke-Han Lu +2Audio Understanding Audio-Language Models
National Taiwan University, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan
GigaChat Audio: Time-aware Large Audio Language Model
Jul 11, 2026Aleksandr Kutsakov, Mariia Sadovina, Georgii Gospodinov +4Audio QA Audio Understanding
SaluteDevices, Russia
Dual-BEATs: Unlocking Zero-Shot Stereo Audio Perception in Audio Large Language Models via Dithering
Jul 9, 2026Shuo-Chun Lin, Hen-Hsen HuangAudio-Language Models Spatial Audio
Institute of Information Science, Academia Sinica, Taiwan
Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models
Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Audio Representation Learning Representation Learning
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · ASUS AICS, Taipei, Taiwan
Unified Audio Intelligence Without Regressing on Text Intelligence
Jul 6, 2026Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim +17TTS Synthesis Unified Multimodal Models
NVIDIA
CARD: Cross-component Audio Representation Distillation for Encoder-Free Audio Captioning
Jul 6, 2026Ganesh Pavan Kartikeya Bharadwaj Kolluri, Yuchen Zhang, Michael Kampouridis +1Audio Representation Learning Audio-Language Models
School of Computer Science and Electronic Engineering, University of Essex · Institute for Analytics and Data Science, University of Essex
Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding
Jul 5, 2026Zihan Zhang, Xize Cheng, Wenhao Yan +5Audio Understanding Audio-Language Models
Zhejiang University · Tsinghua University
Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings
Jul 4, 2026Héctor Martel, Joe Hennessy-Priest, Taemin ChoRepresentation Geometry Audio Representation Learning
BandLab Technologies, Singapore
Reinforcement Learning for Data-Efficient Code-Switched ASR
Jul 2, 2026Ziwei Ye, Peter VickersCode-Switching Speech Recognition Audio-Language Models
Independent Researcher · Spotify Canada
Adaptive Perturbation Selection for Contrastive Audio Decoding
Jun 30, 2026Aaron Isidore Grace, Zhouyuan Huo, Weiran WangAudio-Language Model Evaluation Audio-Language Model Hallucination Detection
Department of Computer Science University of Iowa Iowa City, IA, USA · Google Mountain View, CA, USA
ZEBRA: Zero-Shot Entropy-Regularized Prompt Learning for Base-to-Novel Generalization in Audio-Language Models
Jun 30, 2026Asif Hanif, Mohammad YaqubZero-Shot Learning Prompt Learning
Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models
Jun 30, 2026Yujun Lee, Joonhyeok Shin, Hyoeun Kim +1Audio-Language Model Evaluation Audio QA
Sungkyunkwan University
ALM2Vec: Learning Audio Embeddings for Universal Audio Retrieval with Large Audio-Language Models
Jun 27, 2026Fengjie Lu, Chenang Jiang, Jiarui Hai +2Audio Representation Learning Audio-Language Models
Zhejiang University · Johns Hopkins University
WQ-Fusion: Dynamic Gated Attention for Cross-Domain Audio Representation
Jun 25, 2026Mingda Lin, Lei Ding, Xinyue Zhou +6Audio Representation Learning Gated Attention
School of Electronic Information, Wuhan University, Wuhan, Hubei, China · Tencent AI Lab Seattle, Seattle, USA · CIAIC, Northwestern Polytechnical University, Xi’an, Shaanxi, China +1
From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models
Jun 24, 2026Pengfei Zhang, Hoang H Nguyen, Kazi Shaharair Sharif +6Audio-Language Model Evaluation Audio Understanding
University of California Irvine · University of Illinois Chicago · Kennesaw State University
Comparative Reasoning: Making an Audio Language Model Better at Comparing Emotions
Jun 23, 2026Abinay Reddy Naini, Jaeyeon Kim, Chao-Han Huck Yang +2Audio-Language Model Evaluation Pairwise Comparison
Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, US · NVIDIA
Unlocking In-Context Learning in Audio-Language Models from Decentralized Medical Audio
Jun 22, 2026Ran Piao, Tsai-Ning Wang, Martijn den Dekker +4Few-Shot Audio Classification Multimodal ICL
Eindhoven University of Technology, The Netherlands · Erasmus MC, The Netherlands · Rijnstate Hospital, The Netherlands +1
Learning from Audio-Dependency Errors: Data Curation Strategies Based on Model Confusion Patterns in Audio Question Answering
Jun 20, 2026Hyeonuk NamAudio-Language Model Evaluation Audio QA
Independent Researcher · Seoul, South Korea
MixProLAP: Mixture-Induced Uncertainty Modeling for Probabilistic Language-Audio Pretraining
Jun 18, 2026Yu Nakagome, Jaesong Lee, Soo-Whan ChungMultimodal Pretraining Audio-Language Models
LINE WORKS Corporation, Japan · NAVER Cloud Corporation, South Korea
Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs
Jun 17, 2026Hyebin Cho, Suho Yoo, Jaehyuk Jang +2Audio-Language Models Mechanistic Interpretability
School of Electrical Engineering, KAIST
SubT: Subspace Tuning for Few-shot Generalization of Audio-Language Models
Jun 17, 2026Jaehyuk Jang, Kangwook Ko, Wonjun Lee +1Few-Shot Audio Classification Zero-Shot Learning
KAIST
A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
Jun 16, 2026Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3Audio-Language Model Evaluation Audio-Language Models
University of Maryland, College Park, USA
Are you speaking my languages? On spoken language adherence in multimodal LLMs
Jun 15, 2026Hyungwon Kim, Kandarp Joshi, Lillian Zhou +2Audio-Language Model Evaluation LLM Prompting
Google DeepMind
Transductive Zero-Shot Audio Classification with Audio-Language Models
Jun 15, 2026Jingwen Zhou, Mingzhe WangAudio-Language Model Evaluation Few-Shot Audio Classification
Xidian University, Xi’an, China
Bridging the SEA Gap: An Initial Benchmark for Neural Audio Codec-Synthesized Speech Deepfakes in South-East Asian Languages
Jun 14, 2026Orchid Chetia Phukan, Girish, Mohd Mujtaba Akhtar +1Neural Audio Codecs Audio Deepfake Detection
IIIT-Delhi, India · UPES, India · VBSPU, India
Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models
Jun 14, 2026Hyebin Cho, Jaehyuk Jang, Changick Kim +1Few-Shot Audio Classification Audio Representation Learning
Korea Advanced Institute of Science and Technology, South Korea
EChO-Agent: Evidence Chain Orchestration Agent for Audio Reasoning
Jun 13, 2026Siyuan Zhang, Jian Zong, Junyu Wang +7Audio QA Tool-Augmented Language Model Agents
School of Artificial Intelligence, Tianjin University, Tianjin, China
When the Same Musical Knowledge Forgets Differently: A Clean Probe of Pathway-Dependent Forgetting
Jun 13, 2026Yu Liu, Zhiwei Yang, Wenxiao Zhang +8Audio-Language Models Multimodal Learning
Institute of Information Engineering, CAS · School of Cyber Security, UCAS · The University of Western Australia +1
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
Jun 12, 2026Hui Geng, Yi Su, Han Yin +7Audio Reasoning Audio-Language Models
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China · Korea Advanced Institute of Science and Technology (KAIST) Korea · Shanghai Jiaotong University Shanghai, China
Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources
Jun 12, 2026Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida +6Audio-Language Model Evaluation Audio Representation Learning
1POSTECH · 2Sony AI · 4Sungkyunkwan University +2

Jul 13, 2026, cs.SDJ/K move · Enter open · S save

Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models

Yu-Han Huang, Chih-Kai Yang, Ke-Han Lu, An-Yu Cheng+1

National Taiwan University, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taiwan · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE), Taiwan

Audio Understanding Audio-Language Models Activation Steering

Open paper AskPDF arXiv

Large audio-language models (LALMs) often underperform on fine-grained, non-semantic attributes of speech, such as a speaker's emotion, despite strong performance on speech content. Improving this without the cost of retraining calls for an effective inference-time intervention, yet most existing methods intervene only after the audio encoder and operate at a relatively coarse granularity. The encoder itself, where acoustic information is first extracted from the waveform, remains largely unexplored, especially at the level of individual neurons. We introduce IAAN, Identifying and Amplifying Acoustic Neurons, a training-free and label-free method that scores each feed-forward neuron in the audio encoder by contrasting its activation on the real waveform with that on a noise reference lacking the real audio's acoustic information. IAAN then amplifies a small set of the highest-scoring neurons at inference. Across ten non-semantic speech attributes, IAAN improves average accuracy by 25.7 points on Audio-Flamingo-3, 21.4 on Qwen2.5-Omni, and 9.7 on Kimi-Audio. It also improves a model already explicitly fine-tuned to prioritize acoustic evidence. In controlled comparisons, both the encoder locus and neuron-level selectivity prove necessary for this gain. Intervening after the encoder, at the decoding side or inside the language model, yields little to no improvement, or even deteriorates accuracy. The improvement also depends on which specific neurons are amplified, not merely on their number, confirming that IAAN's acoustic score succeeds in identifying the neurons that matter. These results show that a small, precisely targeted intervention inside the audio encoder is an effective and largely untapped way to strengthen the acoustic understanding of LALMs, opening a new direction for inference-time methods that improve acoustic perception through neuron-level access to the encoder.

Momentum

Related topics

Latest papers 184