cs.SDNov 7, 2026

Machine Hearing: Can Audio LLMs Generalize Beyond Human Auditory Perception?

Authors: Liwei Liu, Jun Tao, Xingyuan Li, Xuenan Xu, Chen Zhang, Junyi Wang, Boya Zhang, Xiaoyu Yang, 

Organizations: Shanghai Artificial Intelligence Laboratory · Harbin Institute of Technology · University of Science and Technology of China · Fudan University · 

Abstract

Multimodal LLMs have made remarkable progress in perceiving the physical world, while their perceptual scope remains largely human-centric, focusing on what humans can directly see or hear. This progress raises the prospect of perception beyond human limits, such as extending hearing to sounds that are inaudible to humans. More broadly, the physical world is full of waves, vibrations, and oscillations, only a small fraction of which are directly audible to humans. This invites the extension of multimodal LLMs toward a broader form of machine hearing, which remains largely underexplored. To investigate this direction, we introduce the Machine Hearing Benchmark (MHB), comprising 6,128 instances across 13 tasks and seven domains. MHB spans highly heterogeneous input regimes, with sampling rates from approximately 10−510^{-5} to 10610^6 Hz, sequence lengths from 10110^1 to 10610^6 samples, and both single- and multi-channel inputs. Evaluations of current open-source and proprietary Audio LLMs (ALLMs) reveal limited zero-shot transfer to machine hearing. Supervised adaptation on machine-hearing data alone does not close this gap, suggesting that the challenge extends beyond data exposure to a systematic mismatch between existing audio front ends and machine-hearing inputs. To address this mismatch, we introduce MachEar, a lightweight encoder extension that complements the pretrained Audio LLM for machine-hearing inputs. The results show that MachEar equips a pretrained Audio LLM with stronger machine-hearing capabilities while largely preserving its existing audio understanding performance.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. EXAM2: Extending Audio Understanding in Multilingual and Multimodal Analysis

    Aug 24, 2026Jiawen Wang, Xiaoxue Gao, Zi Haur Pang +1Audio UnderstandingAudio-Language Model Evaluation

  2. How Auditory Knowledge in LLM Backbones Shapes Audio Language Models: A Holistic Evaluation

    Mar 19, 2026Ke-Han Lu, Szu-Wei Fu, Chao-Han Huck Yang +13Audio UnderstandingAudio-Language Model Evaluation

  3. MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Models

    Mar 10, 2026Chih-Kai Yang, Yun-Shao Tsai, Yu-Kai Guo +7Audio-Language Model EvaluationAudio Understanding