MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness
Organizations: Xiaomi Inc., China
Abstract
Large audio-language models (LALMs) have achieved strong performance in general audio understanding, yet most are designed for monaural input and discard the inter-channel cues essential for spatial perception. In contrast, existing spatial audio-language models are purpose-built for spatial tasks and fail to capitalize on the general understanding capabilities of monaural LALMs. We present MiDashengLM-Spatial, the first open-source end-to-end unified audio-language model, to our knowledge, which supports both general audio understanding and spatial awareness within a single architecture. It extends MiDashengLM with a spatial audio encoder, Spatial-Dasheng, integrated through a hierarchical semantic-to-spatial conditioning module that injects intermediate semantic representations into the spatial branch at multiple depths while preserving the original semantic pathway. To provide spatial audio-language supervision at scale, we develop a data synthesis pipeline that renders diverse spatial acoustic scenes with scene-level spatial descriptions and question-answer pairs. Experiments show that Spatial-Dasheng achieves strong performance on sound event localization and detection in real-world scenes, and that MiDashengLM-Spatial substantially outperforms existing LALMs on spatial understanding and reasoning benchmarks. Meanwhile, it remains competitive with state-of-the-art 8B-scale LALMs on diverse monaural benchmarks, demonstrating that spatial awareness can be acquired without compromising general audio understanding. The source code and model checkpoint are available at https://github.com/xiaomi-research/midashenglm-spatial and https://huggingface.co/mispeech/midashenglm-spatial.
Figures & tables
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Question-Answer Template |
| Source Localization | |
| Source-to-Direction Localization | Q : From which direction is {sound} coming? / From which direction do you hear the person who says {speech} ? A : {dir} Q : Is {sound} above or below the listener? A : above / below / Level with the listener |
| Direction-to-Source Identification | Q : Which sound comes from the {dir} ? A : {sound} Q : What does the person on your {dir} say? A : {speech} |
| Motion Trajectory Perception | |
| Trajectory Tracking | Q : Does {sound} / the person who says {speech} move or stay in place? A : It moves / It stays in place / It cannot be determined Q : How does {sound} / the person who says {speech} move? A : Clockwise / Counter-clockwise / Stays in place Q : Where does {sound} / the person who says {speech} start out / end up? A : {dir} (the start/end position of the movement path) |
| Distance Change Tracking | Q : Does {sound} / the person who says {speech} move closer to or farther from the listener (you)? A : Getting closer / Moving away / Distance unchanged |