Pitch-based sonification of quantitative data increases the accessibility of data visualizations that are otherwise inaccessible for blind and low-vision (BLV) individuals. We argue that, although pitch representations can reveal the coarse-grained information of data, such as data trend and value comparison, they cannot effectively convey the fine-grained details like the sign and exact value of individual data points. Informed by existing sound perception research, we propose a spatial audio-based approach by representing data values as the sound direction in the azimuth plane to achieve accessible fine-grained data representation. We conducted a user study with 26 participants (including 10 BLV participants) on four data perception tasks. The results show our approach significantly outperforms pitch representation on fine-grained data perception tasks like recognizing data signs and exact values, and performs similarly on data trend identification, despite its inferior accuracy on data value comparison.
Figures & tables
Figure 1: Illustration of data encoding in SpaudioData . The encoding space forms a semicircle in front of the participant, spanning 180∘ . Sounds from the left, straight ahead, and right represent data -10, 0, and 10 respectively. Negative values are encoded as leftward spatial audio, while positive values are encoded as rightward. The angular interval is 9∘ .
Figure 2: Pilot Study Results. The mean error for the 9∘ interval angle was smaller than that observed for the 3∘ configuration. For the 9∘ condition, errors for the one-playback and three-playback conditions were similar and lower than those in the five-playback condition.
ID
Gender
Age
Vision Level
Education
Exp.
P1
Male
26
Sighted
Master
None
P2
Female
23
Sighted
Bachelor
None
P3
Male
25
Sighted
Bachelor
None
P4
Male
28
Sighted
Master
Train
P5
Female
28
Sighted
Master
None
P6
Female
24
Sighted
Bachelor
Train
Table 1: Details of Participants (“Exp.” refers to the participant’s musical training experience).
Figure 3: Results of four tasks for BLV participants. Statistical significance is indicated by stars, where ∗∗∗ denotes p<0.001 , ∗∗ denotes p<0.01 , and ∗ denotes p<0.05 , based on the Wilcoxon rank-sum test.
Figure 4: Performance comparison between SpaudioData and the pitch representations for sighted participants across four tasks, showing consistent results with the overall participant group, with notably high accuracy and low variance in the data trend recognition tasks using SpaudioData .
Figure 5: Distribution of accuracy by different gaps for data value comparison task. For the Comparison, Trend, Sign, and Exact Match tasks, higher values indicate better performance, whereas for the Value Difference task, lower values indicate better performance.
Figure 6: Distribution of accuracy by different gaps for data trend recognition task.
Figure 7: Accuracy distribution across ground truth values for BLV participants.
Figure 8: Accuracy distribution across ground truth values for sighted participants.
Current scene perception tools for Blind and Low Vision (BLV) individuals rely on spoken descriptions but lack engaging representations of visually pleasing distant environmental landscapes (Vista spaces). Our proposed Scene2Audio framework generates comprehensible and enjoyable nonverbal audio using generative models informed by psychoacoustics, and principles of scene audio composition. Through a user study with 11 BLV participants, we found that combining the Scene2Audio sounds with speech creates a better experience than speech alone, as the sound effects complement the speech making the scene easier to imagine. A mobile app "in-the-wild" study with 7 BLV users for more than a week further showed the potential of Scene2Audio in enhancing outdoor scene experiences. Our work bridges the gap between visual and auditory scene perception by moving beyond purely descriptive aids, addressing the aesthetic needs of BLV users.
Chitralekha Gupta, Jing Peng, Ashwin Ram +3
Augmented Human Lab, Dept. of Computer Science, National University of Singapore · Augmented Human Lab, National University of Singapore · Saarland University, Saarland Informatics Campus Saarbrücken, Germany +2
Pretrained spatial audio encoders are increasingly used as general-purpose representations for perceptual tasks, yet their spatial encoding capabilities remain poorly understood. We introduce the Spatial Audio Representation Learning (SARL) benchmark, a controlled framework for evaluating spatial information in pretrained audio models. SARL probes source-level factors (azimuth, elevation, distance, class) and room-level factors (RT60, volume, shape). Experiments across diverse encoders reveal three patterns: input configuration and training paradigm shape spatial encoding; source factors are consistently easier to decode than room factors; and sensitivity analysis under controlled perturbations shows heterogeneous responses to source and room variation. These results reveal systematic biases in current pretrained audio representations. SARL is released as an open-source benchmark for reproducible evaluation of spatial audio representations.
Chuyang Chen, Sivan Ding, Adrian S. Roman +1
Music and Audio Research Laboratory, New York University, USA
Affective visualization can influence how users perceive, interpret, and engage with data by embedding and conveying emotion through visual design. While sound is widely used in media to evoke emotions, little is known about how sonification can support affective visualization. In this work, we investigate how sonification can communicate emotion in uncertainty visualizations through a co-design study. Participants created two sonifications to accompany a visualization: one conveying the affective component of uncertainty and one conveying neutrality. Our findings show that uncertainty was commonly associated with wavy auditory qualities related to an ominous sentiment. On the other hand, neutrality was associated with clear and relaxing auditory qualities. These results provide insights for the design of visualizations that integrate sonification to communicate the affective component of uncertainty.
Marcel-Simon Dutt, Sita A. Vriend, Elias Elmquist +1