Understanding the relationship between retrospective soundscape attributes and continuous pleasantness and eventfulness ratings
Authors: Yuanbo Hou, Kening Guo, Francesco Aletta, Woon-Seng Gan, Dick Botteldooren, Jian Kang
Organizations: Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, United Kingdom · UCL Institute for Environmental Design and Engineering, The Bartlett, University College London, London WC1H 0NN, United Kingdom · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore · Department of Information Technology, Faculty of Engineering and Architecture, Ghent University, 9052 Ghent, Belgium
Soundscape studies ask listeners to rate perceived affective quality (PAQ) attributes, such as pleasant and vibrant, after listening to recordings or experiencing acoustic environments. These retrospective ratings summarise an experience during which impressions may change. International Organization for Standardization (ISO) procedures convert these PAQ ratings into Pleasantness and Eventfulness scores. Pleasantness and Eventfulness (P/E) can also be rated directly and continuously during listening, capturing changes in reported perception over time. Retrospective PAQ ratings and continuous direct P/E describe the same soundscape experience from different perspectives. We examine how PAQs relate to direct P/E and how ISO-derived P/E corresponds to direct judgements, using 1,782 paired 30-s trials involving audiovisual recordings from 18 streets. The results show that PAQs retain a predominantly two-dimensional structure. Vibrant and Monotonous lie closer to the Eventfulness-Uneventfulness axis in the direct P/E plane than in the ISO 2025 reference geometry. The full 30-s direct P/E mean has lower calibrated error against ISO-derived P/E than three 10-s means. An empirical mapping using the same 8 PAQs reduces held-out joint root-mean-square error by 4.9% relative to affine-calibrated ISO 2025 scoring. The findings show that retrospective and continuous assessments offer related but distinct soundscape descriptions.
Figures & tables
Figure 1: Paired retrospective and continuous assessments of the same 30-s soundscape exposure. During listening, participants report Pleasantness and Eventfulness (direct P/E) continuously by moving a cursor within a two-dimensional P/E display, producing time-resolved ratings. After listening, participants rate the 8 perceived affective quality (PAQ) attributes, which are converted into ISO-derived P/E coordinates. The continuous records are summarised over time for trial-level comparison with the retrospective scores.
Retrospective PAQ
Direct P/E
PAQ attributes
8
P/E dimensions
2
Rating scale
1–5
Duration / trial
30 s
Paired PAQ sets
1,782
Sampling rate
8 Hz
Sets / condition
594
Samples / trial
240
Complete trajectories
1,724
≥ 80% valid trajectories
1,777
Table 1: Configuration of the paired retrospective PAQ and continuous direct P/E measurements.
Measure
Overall
S
SV
WV
Retrospective PAQ ratings (1–5)
Pleasant
2.79(0.92)
2.73(0.92)
2.79(0.90)
2.85(0.93)
Vibrant
3.13(0.84)
3.11(0.84)
3.13(0.86)
3.15(0.83)
Eventful
3.26(0.89)
3.26(0.87)
3.25(0.90)
3.27(0.91)
Chaotic
3.10(0.97)
3.15(0.97)
3.05(0.97)
3.10(0.96)
Annoying
3.25(0.97)
3.29(0.96)
3.22(0.99)
3.23(0.97)
Table 2: Distributions of retrospective PAQ ratings, direct P/E, and ISO-derived P/E. Values are mean (SD); n=1,782 overall and n=594 for each visual condition (S, SV, WV).
Figure 2: Internal 2D geometry of the 8 PAQs. Blue circles indicate the empirical PAQ directions, orange crosses indicate the corresponding ISO reference directions, and green arcs indicate the shortest angular differences between them. Radial offsets are used only to separate the markers visually; the comparison concerns angular direction. Grey diameters indicate the ISO Pleasantness and ISO Eventfulness axes.
ISO 2019
ISO 2025
PAQ
Ref.
Est.
Δ
Ref.
Est.
Δ
Pleasant
0
334.4
−25.6
0
339.6
−20.4
Vibrant
45
65.7
+20.7
46
70.8
+24.8
Eventful
90
94.0
+4.0
94
99.1
+5.1
Chaotic
135
132.7
−2.3
138
137.8
−0.2
Annoying
180
152.6
−27.4
177
157.7
−19.3
Table 3: Empirical PAQ directions compared with the ISO 2019 and ISO 2025 reference geometries (degrees). Ref./Est. gives the ISO reference angle and the PCA-derived estimate after alignment. Δ is the shortest signed difference (estimate minus reference); positive values indicate counterclockwise displacement.
Reference
Mean ∣Δ∣ ( ∘ )
Median ∣Δ∣ ( ∘ )
Maximum ∣Δ∣ ( ∘ )
ISO 2019
14.35
12.33
32.99
ISO 2025
15.44
19.87
32.13
Table 4: Overall angular fit of the empirical PAQ geometry to the ISO 2019 and ISO 2025 reference geometries.
ISO
Direct P/E
Fitted angle
Signed Δ
95% CI of Δ
2019
Pleasantness
340.16∘
−19.84∘
[−26.03∘,−13.52∘]
Eventfulness
83.50∘
−6.50∘
[−10.61∘,−2.45∘]
2025
Pleasantness
343.27∘
−16.73∘
[−23.41∘,−9.72∘]
Eventfulness
88.24∘
−1.76∘
[−5.58∘,1.95∘]
Table 5: Fitted directions of direct P/E relative to the ISO 2019 and ISO 2025 PAQ reference geometries. Fitted angles are measured counterclockwise from the positive P axis; signed Δ denotes the shortest angular difference from the nominal P ( 0∘ ) or E ( 90∘ ) axis. The 95% confidence intervals (CIs) of Δ are estimated from 2,000 participant-level bootstrap resamples.
Figure 3: Direct-P/E-anchored geometry of the 8 PAQ attributes. The red endpoint vectors show the estimated PAQ directions, with their lengths scaled by the largest regression-vector magnitude for visualisation. Circles and crosses indicate the ISO 2019 ideal and ISO 2025 adjusted-English reference positions, respectively.
Figure 4: Temporal correspondence between direct and retrospective ISO 2025-derived P/E. Shading and error bars indicate 95% participant-bootstrap confidence intervals.
Figure 5: LOPO agreement between calibrated ISO 2025-derived and direct P/E. (a,b) Paired Pleasantness and Eventfulness coordinates with lines of equality. (c,d) Differences between calibrated ISO 2025-derived and direct P/E, calculated as ISO-derived minus direct, plotted against their paired means for Pleasantness and Eventfulness, respectively.
Agreement metric
LOPO
LOSO
Pleasantness CCC
0.7986
0.7953
Eventfulness CCC
0.6885
0.6880
Joint 2D RMSE
0.356371
0.356420
Quadrant agreement (%)
70.37
70.54
Median absolute angle ( ∘ )
20.5447
20.5356
Table 6: Trial-level agreement between calibrated ISO 2025-derived and direct P/E.
Figure 6: LOPO quadrant confusion matrix (left) and distribution of absolute angular differences (right) between calibrated ISO 2025-derived and direct P/E.
Validation
Dimension
ISO
Relaxed
Reduction
projection
model
LOPO
P
0.2458
0.2263
7.9%
E
0.2401
0.2350
2.1%
Joint 2D
0.3481
0.3311
4.9%
LOSO
P
0.2554
0.2367
7.3%
E
0.2455
0.2398
2.3%
Table 7: Fold-averaged held-out RMSE for the ISO 2025 projection and relaxed model.
Objective audio quality metrics typically provide point estimates, whereas listening tests yield score distributions from which mean opinion scores (MOS), confidence intervals (CIs), and significance decisions are derived. We propose a lightweight intrusive metric that combines a PEAQ-style perceptual front-end (ITU-R BS.1387) with a bagging ensemble of regressors. Calibration with subjective data aligns ensemble outputs with listener scores. The resulting item-dependent score distributions enable uncertainty assessment, panel-size-matched CIs, and identification of less conclusive predictions. Calibration improves agreement with subjective distributions and CI coverage across all evaluated datasets while preserving MOS accuracy. Using only 11 fixed PEAQ features, low-capacity regressors, and public training data, the method performs comparably to more data-intensive end-to-end approaches. Its output can support uncertainty-aware assessment and target listening tests towards uncertain conditions. The distributions also enable approximate pairwise comparisons, but not yet reliable significance inference.
Pablo M. Delgado, Andreas Brendel, Konstantin Schmidt +1
Fraunhofer Institute for Integrated Circuits IIS, Erlangen, Germany · International Audio Laboratories Erlangen †, Germany
Audio systems aim to maximise the listener's perceived quality, traditionally evaluated through mean opinion scoring (MOS), a subjective process that is slow, costly, and collected only after playback. Machine learning now predicts MOS directly from the audio, but in generalising across participants it tends to discard the subject variance that defines individual perception. Neurophysiological measures may offer a complementary signal taken directly from the listener, objective, implicit, and continuous, though prior work has relied on laboratory-grade EEG and fNIRS. We ask how practical consumer-grade devices compare, and whether they might recover population-level audio-quality trends and, when calibrated per listener, track that listener's effort. Recording 32-channel laboratory EEG alongside wearable EEG, fNIRS, and eye tracking from thirty listeners, our results suggest commercial wearable and optical sensing can decode audio quality at statistically significant levels and recover listening effort per listener. These findings may point towards platforms tuned not only to perceived quality but to the effort a clip demands, read from the listener's own response as they listen.
Niall McGuire, Dimitra Emmanouilidou
University of Strathclyde, Glasgow, UK · Microsoft Research, Redmond, WA, USA
Evaluating generative spatial audio for First-Order Ambisonics (FOA) remains challenging due to a limited understanding of how metrics respond to changes in spatial parameters such as azimuth and elevation. We propose a framework to analyze metric sensitivity along continuous spatial trajectories, drawing on principles of sensitivity analysis in parametric sound synthesis. Using controlled FOA scenes with increasing scene complexity, we define three desiderata for metric behavior: Responsiveness, Smoothness, and Symmetry. We assess standard distribution-based and sample-based metrics, including Fréchet Audio Distance (FAD), intensity vectors, and acoustic maps. Our findings show that FAD using localization-specific embeddings and acoustic maps yield high Responsiveness and robust Smoothness and Symmetry across conditions, while intensity vectors degrade with increasing scene complexity. This is the first step towards investigating the sensitivity of metrics for generative spatial audio.
Purnima Kamath, Adrian S. Roman, Koichi Saito +2
New York University, New York, USA · Sony AI, New York, USA · Sony Group Corporation, Tokyo, Japan