Affective visualization can influence how users perceive, interpret, and engage with data by embedding and conveying emotion through visual design. While sound is widely used in media to evoke emotions, little is known about how sonification can support affective visualization. In this work, we investigate how sonification can communicate emotion in uncertainty visualizations through a co-design study. Participants created two sonifications to accompany a visualization: one conveying the affective component of uncertainty and one conveying neutrality. Our findings show that uncertainty was commonly associated with wavy auditory qualities related to an ominous sentiment. On the other hand, neutrality was associated with clear and relaxing auditory qualities. These results provide insights for the design of visualizations that integrate sonification to communicate the affective component of uncertainty.
Figures & tables
Figure 1: The study tool consists of seven main parts: 1.) The visualization showing the yearly average German temperatures since 1881, including predicted temperatures up to 2100. 2.) Four white buttons for selecting a musical instrument (labeled as Sound 1-4 to reduce bias). 3.) One looping button for looping the sonification. 4.) Sliders to change the envelope length and wait time of the looping sonification. 5.) Sliders that allow changing the auditory parameters for the currently selected musical instrument. 6.) A green start button and drop-down menus allowing participants to map the pitch to the given data on the left side. The middle drop-down menu lets participants choose their pitch range based on pre-set options. The right drop-down menu allows for the change of the instrument, while the green button plays a sonification based on the options selected in the blue, orange, and red boxes. 7.) A pink button to play a pre-definded sonification, also called the baseline. The visualization is publicly available from Deutscher Wetterdienst ( \url https://www.dwd.de/DE/klimaumwelt/klimaatlas/klimaatlas_node.html).
Figure 2: The study procedure started with a tutorial of the tool, and continued with the creation of an uncertain and neutral sonification. This was followed up with filling in two questionnaires.
Figure 3: The distribution of musical inclination ratings of the 21 participants, which was recorded in a demographics questionnaire.
Figure 4: Number of participants mentioning the code.