Audio phase relations encode source locations, yet existing large audio-language models (LALM) reduce audio to single-channel content to stabilise general audio understanding, leaving spatial information for direction, distance, and source-relation questions unused. We present SpheraAudio, which accepts recordings with any channel count -- mono, stereo, First-Order Ambisonics (FOA), or higher -- and maps them into a single token stream. A spatial encoder turns complex multichannel spectra into spatial tokens interleaved with audio tokens at corresponding temporal positions. Spatial Multimodal RoPE (SMRoPE) makes each token's modality identity and temporal slot visible to rotary attention without altering the language model's original RoPE. Training runs in three stages -- spatial-encoder pretraining, projector alignment, and joint supervised-and-reinforcement learning -- progressing from geometric supervision to language-based spatial decisions, with answers trained as explicit spatial chain-of-thought: step-by-step geometric reasoning over the heard scene. Current benchmarks tie questions to a fixed channel-count format and score answers without spatial reasoning evidence. We introduce SpheraQA, which spans mono, stereo, and FOA recordings and organises questions hierarchically from single-attribute perception through cross-source relations to multi-step reasoning, scored programmatically against simulated scene geometry and real-recording annotations with step-by-step solution references. Across SpheraQA and public spatial-audio benchmarks, SpheraAudio sets a new performance standard for spatial audio understanding. It supports high-precision localisation, tracking, and reasoning over static and moving sound sources while preserving strong general-audio capabilities. We will release code, SpheraQA, and evaluation metrics to support reproducibility.
Figures & tables
Figure 1: SpheraAudio overview. The audio encoder AuT and spatial encoder Spatial AuT extract complementary features from the same recording. Separate projectors produce audio tokens and spatial tokens, which are paired in time and interleaved. SMRoPE encodes their temporal correspondence and modality identities for the language model.
Figure 2: Schematic training framework. Spatial-encoder pretraining learns temporally resolved geometry. Projector alignment connects spatial features to the pretrained language representation. Joint SFT & RL combines spatial and acoustic evidence through supervised answer learning and group-relative optimization with verifiable outcome rewards.
Figure 3: SpheraQA. The task taxonomy spans single-attribute perception, cross-source relations, and spatial reasoning. Examples connect source observations to comparisons and geometric operations.
Task / benchmark
N
Spatial- Omni-7B
Gemini 3.1 Pro
SpheraAudio
Δ (pp)
(a) SpheraQA by question type
Azimuth
1,563
22.39
9.72
55.60
+33.21
Elevation
1,556
35.86
32.46
69.34
+33.48
Distance
1,564
49.55
55.75
74.81
+25.26
Source count
1,107
46.52
54.29
60.70
+14.18
Lateral position
1,315
29.66
44.64
41.52
+11.86
Table 1: Spatial understanding (%). SpheraQA reports whole-answer accuracy, with Spatial Reasoning pooling six reasoning question types. SO-Bench reports the mean per-question score. Gemini receives only W . N counts questions and Δ is SpheraAudio minus Spatial-Omni-7B in percentage points before rounding. Best scores are in bold.
Spatial MAE ↓
Speech
Spatial Acc. (%) ↑
Variant
Azimuth ( ∘ )
Elevation ( ∘ )
Distance (m)
AISHELL CER (%) ↓
Left/right
Motion state
Trajectory
Element-wise addition
13.47
7.93
1.32
5.34
91.19
82.85
64.60
Token interleaving
4.79
5.97
1.20
2.81
94.82
87.23
65.49
Table 4: Component ablations. (a) Token fusion in single-source scenes: spatial mean absolute error (MAE), AISHELL CER, and task accuracy. (b) Named-source direction under token-stream shifts. (c) Timed and untimed queries under SMRoPE and sequential RoPE. Panels (b,c) evaluate a fixed SFT checkpoint on identical confirmation recordings, with N questions per task (Appendix C.1 ).
Task
N
SFT
RL
Δ
TWNM / stereo transfer: Acc. (%)
Distance estimation
43
41.86
62.79
+20.93
Distance comparison
60
46.67
51.67
+5.00
Azimuth direction
43
83.72
86.05
+2.33
Lateral position (stereo)
73
91.78
93.15
+1.37
Motion (stereo)
272
37.13
37.87
+0.74
Table 5: Task-level gains after multiple-choice RL. Each supervised policy is compared with its direct RL continuation. Values are TWNM and stereo accuracy or SO-Bench mean per-question score (%); Δ is the percentage-point gain before rounding.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Recording
Front-end operation
AuT signal
Spatial AuT signals
Tokens / chunk
Mono
Audio pathway
a
—
13
Stereo
Mid–side transform
Wm
[Wm,Ym,0,0]
26
FOA
Native first-order components
W
[W,Y,Z,X]
26
Higher-order Ambisonics
External first-order extraction
W
[W,Y,Z,X]
26
Calibrated array
External array-to-Ambisonics encoding
W
[W,Y,Z,X]
26
Appendix
Table 6: Input routing and projected token counts. Waveform signals are shown before spectrogram conversion; a denotes the mono waveform. Counts refer to a complete one-second chunk and exclude text and boundary tokens.
Setting
Value
Updates
1000
Questions per update
32
Responses per question
8
Learning rate (constant)
5×10−7
Adam coefficients
(0.9,0.999)
Weight decay
0.01
Appendix
Table 10: GRPO configuration for the single outcome-based RL stage. The final SFT policy initializes training and remains fixed as the KL reference.
Source
Recordings
N
Simulated scenes
6,828
7,047
LOCATA
63
1,140
STARSS23
168
12,305
Appendix
Table 11: Recording sources in SpheraQA. Recordings are counted uniquely within each source, and N denotes the number of questions.
Question type
Answer format
N
L1 Perception
Azimuth
Numeric angle
1,563
Elevation
Numeric angle
1,556
Distance
Numeric distance
1,564
Source count
Integer
1,107
Lateral position
Direction category
1,315
Appendix
Table 12: SpheraQA question types, answer formats, and counts across the L1–L3 hierarchy, corresponding to Table 5.1 .
Benchmark
N
Metric
Result (%)
SpheraQA
25
Whole-answer Acc.
28.00
SO-Bench
25
Mean Score
34.47
TWNM
25
Option Acc.
60.00
Stereo
25
Acc.
0.00
Appendix
Table 17: Waveform-analysis diagnostic on 25 selected questions per benchmark.
Figure 4: Responses to sound-field transformations on 223 fixed azimuth questions. (a) MAE across all nine yaw rotations. (b) Left–right reflection and removal of directional channels. Rotation and reflection use transformed reference directions; channel removal uses original reference directions. The same supervised model is evaluated across all conditions.