Spatial DJ: Spatial Audio Control for Arbitrary Loudspeaker Layouts
Authors: Anastasia Geleta, Margarita Geleta
Organizations: AVINCE, San Francisco, CA 94115, USA · Department of Electrical Engineering and Computer Science, University of California, Berkeley, Berkeley, CA 94720, USA
Object-based spatial audio promises render-anywhere playback, yet deploying it in a new venue still requires two tasks that are rarely addressed together: acquiring the physical loudspeaker geometry and enabling creators manipulate spatial audio while listening through the target loudspeaker system rather than a headphone proxy. We present Spatial DJ, a system for on-site loudspeaker layout acquisition and real-time spatial audio rendering and control, using head-worn AR in a coordinate frame aligned to the physical loudspeakers. In an 18-loudspeaker room, we evaluate calibration accuracy, repeatability, and control latency, and conduct an 18-participant listener study comparing diffuse and orbiting point-source rendering configurations applied to vocal and instrumental layers. Orbiting point-source conditions received higher ratings of perceived immersion, energy, and novelty than the diffuse baseline. Beyond this controlled setting, four representative deployments illustrate reuse of the same system across heterogeneous loudspeaker venues.
Figures & tables
Figure 1: Spatial DJ . A Arbitrary loudspeaker layouts are registered in a room-aligned coordinate frame by aligning AR markers with physical loudspeakers. B The Spatial DJ backend connects the AR controller, database, audio interface, and active or passive loudspeakers, while supporting external sources such as DJ controllers and DAWs. C The AR interface enables creators to spatially position and manipulate virtual audio objects during live performance. D Interaction workflow from scene calibration and channel arrangement to spatial mixing.
Figure 2: Spatial DJ frontend UIs . A The desktop frontend supports project configuration, audio source management and stem separation, loudspeaker layout configuration, speaker level calibration, 3D room visualization, and control of virtual audio object position, spread, flag states, and live mixer inputs. B The AR controller’s interface exposes corresponding controls for individual virtual audio objects, enabling users to locate the virtual audio objects in their environment and toggle solo and mute flags.
Figure 3: Controlled evaluation setup and key measurements . A Schematic of the 18-loudspeaker room with arrows indicating the measured distances from the room axes to the loudspeaker position coordinates. B Panoramic view with a subset of loudspeaker locations, arrows indicate additional speakers outside the field of view. C Between-operator calibration discrepancy per loudspeaker. The dashed line marks the smallest between-operator discrepancy. D Normalized per-speaker energy as the radius parameter increases, illustrating the transition from point-source to ambient rendering in a stereo setup.
Figure 4: Control latency measurement pipeline . The AR controller records t0 when an interaction commits an object-state update and transmits it via OSC. Then, the host records t1 when the packet is received and parsed, and t2 when the update is first applied inside the audio callback.
Metric
n
Median [p25, p75]
Mean (SD)
p95
Δ0→1 (network+parse)
6164
2.18 [1.30, 3.18]
4.55 (9.38)
22.54
Δ1→2 (engine+block)
6164
5.15 [2.63, 7.87]
5.33 (4.52)
9.82
Δ0→2 (control)
6164
8.07 [5.07, 11.00]
9.88 (10.08)
27.40
Δ0→2 by number of active speakers N
N=2
1596
8.20 [5.34, 11.04]
9.96 (8.52)
27.89
N=4
1629
8.11 [4.96, 10.73]
9.01 (6.89)
22.36
Table 1: Control latency (ms) for block size B=256 at 44.1 kHz. Values report median [p25, p75], mean (SD), and p95.
Figure 5: Spatial audio conditions used in the listener study, illustrated in a representative top view of the 18-loudspeaker room where the study was conducted . A C 1 , diffuse baseline: the vocal and instrumental layers are rendered as ambient audio objects. B C 2 , vocal orbit: the vocal layer follows a clockwise elliptical orbit as a point-source object while the instrumental layer remains ambient. C C 3 , instrumental orbit: the instrumental layer follows a clockwise elliptical orbit as a point-source object while the vocal layer remains ambient. D C 4 , dual orbit: the vocal and instrumental layers follow the same clockwise elliptical orbit with the same angular velocity and a 180∘ phase offset. Blue represents the instrumental layer and pink represents the vocal layer, while A and P denote ambient and point-source rendering, respectively.
Perceived motion type
C 1
C 2
C 3
C 4
Diffuse or scattered
16 (44.4%)
7 (19.4%)
10 (27.8%)
10 (27.8%)
Elliptical movement
2 (5.6%)
26 (72.2%)
28 (77.8%)
25 (69.4%)
Horizontal movement
8 (22.2%)
9 (25.0%)
18 (50.0%)
8 (22.2%)
Vertical movement
7 (19.4%)
1 (2.8%)
0 (0.0%)
1 (2.8%)
Two moving layers or objects
5 (13.9%)
9 (25.0%)
9 (25.0%)
23 (63.9%)
Not sure
5 (13.9%)
0 (0.0%)
0 (0.0%)
0 (0.0%)
Table 2: Perceived motion types by spatial condition . Values indicate the number of trials and percentage of trials within each condition ( N=144 trials, 18 participants × 8 trials, 36 observations per condition). Participants could select more than one motion type per trial, so percentages within a condition do not sum to 100%.
Figure 6: Model-adjusted perceptual ratings across study conditions . Points show estimated marginal means and vertical error bars indicate 95% confidence intervals, adjusted for musical excerpt, music block, and within-block presentation position. C 1 : Diffuse baseline; C 2 : Vocal orbit; C 3 : Instrumental orbit; C 4 : Dual orbit. (* p<.05 , ** p<.01 , *** p<.001 .)
Figure 7: Representative deployments across heterogeneous loudspeaker configurations . Left for A–C, top for D : schematic layouts (not to scale) showing loudspeaker placement and room dimensions for four tested setups: A 2.0 stereo baseline with two active loudspeakers. B 7.1 laboratory surround system routed through an A/V receiver (planar layout). C 7.1.4 home-cinema system with passive loudspeakers (periphonic layout with height channels). And D nightclub venue with eight non-uniformly spaced loudspeakers and an asymmetric performer/DJ position. Right for A–C, bottom for D : photographs from each deployment showing on-site spatial audio control with the AR interface in the corresponding physical space.
Pretrained spatial audio encoders are increasingly used as general-purpose representations for perceptual tasks, yet their spatial encoding capabilities remain poorly understood. We introduce the Spatial Audio Representation Learning (SARL) benchmark, a controlled framework for evaluating spatial information in pretrained audio models. SARL probes source-level factors (azimuth, elevation, distance, class) and room-level factors (RT60, volume, shape). Experiments across diverse encoders reveal three patterns: input configuration and training paradigm shape spatial encoding; source factors are consistently easier to decode than room factors; and sensitivity analysis under controlled perturbations shows heterogeneous responses to source and room variation. These results reveal systematic biases in current pretrained audio representations. SARL is released as an open-source benchmark for reproducible evaluation of spatial audio representations.
Chuyang Chen, Sivan Ding, Adrian S. Roman +1
Music and Audio Research Laboratory, New York University, USA
Spatial audio has an accessibility problem. Largescale spatial and immersive audio installations remain costly and exclusive, with high barriers to entry. We propose ''Ananas,'' a networked audio system consisting of network protocols, control software and microcontroller firmware, with the aim of rendering spatial audio applications accessible to a broad cohort of potential users. Based on a network of low-cost microcontroller development boards programmed in the Faust programming language, synchronised to nanosecond order via IEEE 1588 Precision Time Protocol, and controlled by approachable, digital audio workstation-based tooling, Ananas represents an opportunity to disrupt and democratise the spatial audio landscape. In this paper we demonstrate Ananas's protocols for audio and control data transmission and device discovery, its methods for media clock and presentation time recovery, and its applications to wave field synthesis and higher order ambisonics in a modular, scalable form.
Dynamic spatial audio reasoning requires a model to associate changing acoustic observations with the correct source and recover its spatial state at a queried time or interval, a challenge intensified by overlapping speech and source motion. To study this problem, we introduce Sonic-Bench, comprising 2.8K first-order Ambisonics clips and 28.8K target-specific question-answer pairs across 480 simulated indoor scenes. The benchmark covers localization, distance estimation, and motion reasoning in single- and dual-source settings. We further propose SonicWorld, which integrates physical, learned acoustic, and semantic features into persistent source states through learned transitions and recurrent updates, and projects them into an Omni LLM for spatial reasoning. Compared with Spatial-Omni on the Sonic-Bench test split, SonicWorld improves direction accuracy from 31.98% to 34.69% and reduces azimuth error from 57.22° to 48.01°. In zero-shot transfer to Spatial LibriSpeech, SonicWorld retains strong localization performance, achieving 73.98% direction accuracy and 19.29° azimuth error, compared with 27.14% and 70.74° for Spatial-Omni.
Chenlong Wei, Hanling Wang, Mingjin Lou +6
Xi’an Jiaotong-Liverpool University · University of California, San Diego · Imperial College London +2