Spatial Audio Question Answering and Reasoning on Dynamic Source Movements

Arvind Krishna Sridhar; Yinyi Guo; Erik Visser

cs.SDFeb 18, 2026

Spatial Audio Question Answering and Reasoning on Dynamic Source Movements

Authors: Arvind Krishna Sridhar, Yinyi Guo, Erik Visser

Organizations: Qualcomm Technologies Inc. San Diego, CA

Abstract

Spatial audio understanding aims to enable machines to interpret complex auditory scenes, particularly when sound sources move over time. In this work, we study Spatial Audio Question Answering (Spatial AQA) with a focus on movement reasoning, where a model must infer object motion, position, and directional changes directly from stereo audio. First, we introduce a movement-centric spatial audio augmentation framework that synthesizes diverse motion patterns from isolated mono audio events, enabling controlled and scalable training data generation. Second, we propose an end-to-end multimodal finetuning approach with a thinking mode, which allows audio-language models to produce explicit intermediate reasoning steps before predicting an answer. Third, we investigate the impact of query-conditioned source separation as a preprocessing stage and compare three inference regimes: no masking, an audio grounding model (AGM), and ground-truth masks. Our results show that reasoning amplifies the benefits of source separation, with thinking mode showing significant improvement of +5.1% when a single event is present in the question. These findings highlight the interplay between movement modeling, reasoning, and separation quality, offering new insights for advancing spatial audio understanding.

Figures & tables

Figure 1: Our proposed spatial audio language model architecture with thinking mode. AGM refers to audio grounding model.

Figure 2: Spatial audio dataset generation

Metric	Score
Answer Accuracy	$4.93$
Reasoning Quality	$4.94$
Groundedness	$4.97$

Table 1: Human evaluation of the Spatial AQA dataset (1-5 scale; higher is better) across 200 randomly selected samples.

Category	Question Type	Example Question	Answer
Trajectories	yes_no	Is the Croak sound moving from left to right?	Yes
Radial changes	yes_no	Are the waves moving closer to the listener as they sweep across?	Yes
Comparative diagnostics	multiple_choice	Which source sweeps across a wider left $\rightarrow$ right range? Options: [Blender, Car, Both equally, Neither]	Blender
Temporal sequence	open	Narrate the overall choreography: where does each source start, how does it move, and where does it finish?	The whale begins on the left and moves a moderate amount further left while approaching the listener; the lawn mower starts on the right, makes a wide sweep across the center to the left while coming closer and then moving away, and finishes on the left — resulting in the lawn mower crossing through the middle and the whale remaining on the left throughout.
Relative motion between events	yes_no	Do scissors and cattle sound move toward the same region (converge) as they progress?	Yes
Natural perception	open	Which source feels most noticeable near the middle of the scene and why?	The Applause feels most noticeable near the middle because it crosses the center with a moderate sweep, passing through that region, whereas the Camera stays on the left and does not pass through the middle.

Table 2: Example QA pairs by category

	BAT	Thinking			Non-Thinking			$\boldsymbol{\Delta}$ Interaction (pp)
Question Type		NoMask	AGM	GT	NoMask	AGM	GT	AGM	GT
Yes_No	41.7	72.1	73.8	76.3	72.6	72.8	72.3	+1.5	+4.5
Multiple Choice	0.3	53.1	52.9	52.8	50.4	50.3	50.3	$-$ 0.1	$-$ 0.2
Open	0.00	41.0	41.4	41.8	41.2	41.0	41.5	+0.6	+0.5
Overall	14.7	54.3	55.0	56.1	54.1	54.0	54.1	+0.8	+1.8

Table 3: Accuracy ( %).

\Delta\text{Interaction}

measures how much more (or less) thinking helps under a method compared to NoMask:

\Delta\text{Interaction}=(\mathrm{Acc}_{T}^{\text{method}}-\mathrm{Acc}_{N}^{\text{method}})-(\mathrm{Acc}_{T}^{\text{NoMask}}-\mathrm{Acc}_{N}^{\text{NoMask}})

. Acc is accuracy, T refers to thinking and N refers to non-thinking

Overlap	NoMask	AGM	GT
No overlap	55.0	56.4	57.9
Overlap	53.7	53.7	54.3

Table 4: Accuracy by overlap category (%) for thinking mode.

Audio Events	NoMask	AGM	GT
Complete match	48.4	48.6	49.2
1 Event	64.3	66.2	68.7
2 events	54.1	53.7	53.8

Table 5: Accuracy by audio events (%) for thinking mode.

Error Type	Non-Thinking	Thinking
Lateral Direction	18.5	15.4
Attribute Swapping	15.7	14.4
Source Misidentification	14.5	13.3
Distance / Depth	14.2	10.3
Crossing Center	11.7	8.9
Temporal Sequence	7.8	5.5

Table 6: Error category comparison for GT masking: Thinking vs Non-Thinking. Values are in %. The lower error percentage the better.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Thinking			Non-Thinking
Metric	NoMask	AGM	GT	NoMask	AGM	GT
Accuracy (%)	54.3	55.0	56.1	54.1	54.0	54.1
Thinking Avg (1-5)	3.507	3.522	3.549	0.0	0.0	0.0
Rationale Avg (1-5)	4.156	4.168	4.179	4.182	4.182	4.180

Appendix

Table 7: Overall core metrics. Accuracies are %; Thinking/Rationale are 1–5 scales. GT refers to ground - truth masking based source separation, NoMask refers to without using source separation, AGM refers to audio grounding model based source separation.

		Thinking			Non-Thinking			$\boldsymbol{\Delta}$ Interaction (pp)
QType	Audio Events	NoMask	AGM	GT	NoMask	AGM	GT	AGM	GT
Multiple Choice	Complete Match	53.2	53.0	53.9	50.6	50.0	51.0	+0.4	+0.3
Multiple Choice	1 Event	57.0	56.3	52.7	52.7	54.2	50.2	$-$ 2.2	$-$ 1.8
Multiple Choice	2 Events	51.1	50.9	47.7	48.7	50.0	47.0	$-$ 1.5	$-$ 1.7
Open	Complete Match	35.6	36.1	36.2	36.1	35.5	36.3	+1.1	+0.4
Open	1 Event	52.8	53.2	54.2	52.3	52.3	52.7	+0.4	+1.0

Appendix

Table 8: Question Type

\times

Audio Events. Accuracies are %;

\Delta

Interaction is defined as

(\mathrm{Acc}_{T}^{\text{method}}-\mathrm{Acc}_{N}^{\text{method}})-(\mathrm{Acc}_{T}^{\text{NoMask}}-\mathrm{Acc}_{N}^{\text{NoMask}})

		Thinking			Non-Thinking			$\boldsymbol{\Delta}$ Interaction (pp)
QType	Overlap	NoMask	AGM	GT	NoMask	AGM	GT	AGM	GT
Yes No	No Overlap	72.2	74.4	78.3	72.6	73.0	72.2	+1.8	+6.5
Yes No	Overlap	72.0	73.2	74.4	72.6	72.6	72.4	+1.2	+2.6
Multiple Choice	No Overlap	53.2	55.3	54.2	49.2	47.6	48.4	+3.7	+1.8
Multiple Choice	Overlap	53.0	50.5	51.4	51.6	53.0	52.2	$-$ 3.9	$-$ 2.2
Open	No Overlap	42.5	43.0	43.8	41.8	41.2	41.9	+1.1	+1.2

Appendix

Table 9: Question Type

\times

Overlap. Accuracies are %;

Question	Model Responses
Which sound spans a wider left–right range? Choices : [Whimper (dog), Tick-tock, They span the same range]	Ground truth answer: Whimper (dog) NoMask without thinking mode
	Thinking:
	Answer: Tick-tock
	Rationale: Tick-tock is described as a wide sweep while the dog’s movement is moderate, so Tick-tock spans more left–right range. NoMask with thinking mode
	Thinking: Step 1: Compare the span categories of both sounds. Solution: Both the whimper and tick-tock are described with a wide lateral span. Step 2: Decide which is wider based on those descriptions. Solution: Since both are labeled wide, they span the same wide left–right range. Final reasoning: Both sources have wide sweeps, so neither spans a wider left–right range than the other.
	Answer: They span the same range

Appendix

Table 10: Selected samples to illustrate model reasoning and responses across masking regimes

Explore similar work

Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources
Jun 12, 2026Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida +6Audio-Language Model Evaluation Audio Representation Learning
1POSTECH · 2Sony AI · 4Sungkyunkwan University +2
Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models
Aug 10, 2026Zhi Zeng, Cheng Zhang, Zesheng Yang +9Unified Multimodal Models Audio-Visual QA
Xi’an Jiaotong University · The Hong Kong Polytechnic University · National University of Singapore +2
SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning
Oct 12, 2026Chenlong Wei, Hanling Wang, Mingjin Lou +6Spatial Audio Spatial Reasoning Benchmarks
Xi’an Jiaotong-Liverpool University · University of California, San Diego · Imperial College London +2

Jun 12, 2026J/K move · Enter open · S save

Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources

Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida, Kim Sung-Bin+5

1POSTECH · 2Sony AI · 4Sungkyunkwan University+2

Audio-Language Model Evaluation Audio Representation Learning Audio Reasoning Audio Understanding Audio-Language Models Spatial Audio

Open paper AskPDF arXiv

Sound events are entities with semantic identities, locations, and trajectories, but current audio-language models usually reason about clips as global event content. Conversely, sound event localization models track source directions over time but offer limited semantic coverage for language reasoning. To address this gap, we introduce ST-AudioQA, a spatio-temporal audio QA dataset and benchmark built from first-order ambisonic (FOA) renderings of static and moving sound sources. Each scene provides source identity, activity, direction, distance, and motion metadata, enabling dense trajectory supervision and questions about what is sounding, where it is, how it moves, and how sources relate. We further propose ST-Audio Encoder, a time-resolved FOA audio encoder that learns event semantics together with source trajectories, and ST-AudioLM, which connects the audio tokens from the encoder to an LLM for spatio-temporal audio QA. Experiments show that this representation improves the semantic-localization tradeoff and yields stronger reasoning performance than static spatial and localization-oriented baselines.

Abstract

Explore similar work

Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

SonicWorld: Source-Centric Latent Dynamics for Spatial Audio Reasoning