cs.SDSep 20, 2026

Misrecognition or Abstraction? Rethinking Outputs of Sound Event Recognition

Authors: Naoya Tomida, Yuki Okamoto, Keisuke Imoto

Organizations: Kyoto University, Japan · The University of Tokyo, Japan

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Conventional general sound recognition systems typically output deterministic sound event labels, implicitly assuming that the target sound class can be correctly identified from the input audio. However, in real listening situations, the sound event class is not always clearly identifiable. Human listeners may nevertheless understand their surroundings from an ambiguous sound without identifying its exact sound event class. This motivates a discussion of how the outputs of sound recognition systems should be redesigned under such uncertainty. As a basis for this discussion, this paper proposes an output representation for sound event recognition that combines a sound event class, its confidence score, and an onomatopoeic description of the sound. The proposed representation preserves conventional class-based recognition while providing an additional onomatopoeic description of acoustic characteristics that can remain informative even when the class prediction is uncertain. Experiments using ESC-50 and ESC-50-Onomatopoeia show that the proposed method achieves sound recognition performance comparable to that of a conventional recognition-only system. In addition, an LLM-as-a-judge evaluation and subjective listening experiments indicate that the proposed output is preferred over conventional deterministic outputs based on the sound event label, particularly when used to support understanding of the surrounding environment. These results suggest that such output representations can make sound event recognition more informative and communicative under uncertainty.

Figures & tables

Fig. 1: Conventional output of SER system
Fig. 2: Example of proposed output representation of SER system
Fig. 3: Overview of implementation of proposed method
Network for MLP and classifier3 dense layers
# units in MLP and classifier layers512, 256, 128
λ\lambda1.0
OptimizerAdamW [ 17 ]
Table 1: Detailed experimental conditions
MethodMicro-F scoreMacro-F score
CLAP + classifier97.81 ±\pm 0.6697.80 ±\pm 0.66
CLAP + onoma.98.54 ±\pm 0.5198.52 ±\pm 0.51
(Proposed)
Table 2: SER performance on 31-class subset of ESC-50 (%)
Ground-truthCLAP + classifierCLAP + classifierCLAP + onoma.
label(event label)(LLM)(template)
cowcowDid you hear that, too? It sounded like cow .I heard a " m o o: " sound, which sounded like cow .
enginechainsawSomething just made a sound around us,I heard a sound nearby, but I can only
and I think it may have been chainsaw .describe it as something like " g a g a g a g a g a ".
Table 3: Examples of system outputs under high- and low-confidence predictions. Onomatopoeias are written in an IPA-like format.
Fig. 5: Pairwise preference scores for SER output representations, obtained using LLM-as-a-judge
​​ Method​​ Naturalness​​ Richness​​ Usefulness
CLAP + classifier (event label)2.91 ±\pm 0.311.78 ±\pm 0.473.96 ±\pm 1.08
CLAP + classifier (LLM)3.71 ±\pm 0.552.04 ±\pm 0.254.06 ±\pm 0.27
CLAP + onoma. (template)3.87 ±\pm 0.363.97 ±\pm 0.614.39 ±\pm 0.87
Table 4: Mean LLM-as-a-judge scores for utterance naturalness, richness of sound description, and communicative usefulness. Values are means ±\pm standard deviations.

Explore similar work

CardsList
May 5, 2026cs.SD

Towards Open World Sound Event Detection

Sound Event Detection (SED) plays a vital role in audio understanding, with applications in surveillance, smart cities, healthcare, and multimedia indexing. However, conventional SED systems operate under a closed-world assumption, limiting their effectiveness in real-world environments where novel acoustic events frequently emerge. Inspired by the success of open-world learning in computer vision, we introduce the Open-World Sound Event Detection (OW-SED) paradigm, where models must detect known events, identify unseen ones, and incrementally learn from them. To tackle the unique challenges of OW-SED, such as overlapping and ambiguous events, we propose a 1D Deformable architecture that leverages deformable attention to adaptively focus on salient temporal regions. Furthermore, we design a novel Open-World Deformable Sound Event Detection Transformer (WOOT) framework incorporating feature disentanglement to separate class-specific and class-agnostic representations, together with a one-to-many matching strategy and a diversity loss to enhance representation diversity. Experimental results demonstrate that our method achieves marginally superior performance compared to existing leading techniques in closed-world settings and significantly improves over existing baselines in open-world scenarios.

P. H. Hai, L. T. Minh, L. H. Son

VNU University of Engineering and Technology, Hanoi, 100000, Vietnam · Artificial Intelligence Research Center, VNU Information Technology Institute, Hanoi, 100000, Vietnam

Sound Event DetectionAudio Understanding
AskPDF
Sep 28, 2026cs.SD

Probing Large Audio-Language Models for Compositional Understanding of Sounding Actions

Large audio-language models (LALMs) excel at understanding and reasoning tasks over atomic sound events, yet their ability to infer higher-level human activities from such fine-grained events remains largely unexamined. Everyday human actions and activities, such as setting a table, cleaning the house, or preparing a breakfast emerge compositionally from temporally distributed sound events, requiring abstraction beyond the event-centric granularity that dominates current training and evaluation paradigms. Our benchmark evaluates a wide set of LALMs under a principled framework that tests how language-based reasoning, grounded in acoustic perception, structures sound abstractions into higher-level understanding. By systematically varying exemplar typicality and distractor similarity, our evaluation exposes \added{that current models do not reliably perform compositional inference from atomic acoustic events to higher-level human activities solely from audio.} All data, taxonomies, and evaluation scripts are publicly available on our companion website: https://alm-sounding-actions.onrender.com/

Michel Olvera, Paraskevas Stamatiadis, Changhong Wang +1

S2A, LTCI, IDS · S2A, IDS

Large Audio Language ModelsFine-Grained ActionsAcoustic
AskPDF
Jun 5, 2026cs.SD

Towards Event-Robust Acoustic Scene Classification

This paper introduces the Event-Shifted Acoustic Scene (ESAS) dataset, a novel benchmark for evaluating the robustness of Acoustic Scene Classification (ASC) systems against unknown sound events. Existing ASC datasets typically contain recordings of clean and consistent audio, while real-world environments often include diverse and unexpected sound events. To bridge this gap, ESAS simulates real-world acoustic variability by injecting foreground sound events into background scenes with the assistance of large language models. In this work, we present the construction methodology, dataset statistics, and evaluation protocols. Furthermore, a comprehensive evaluation of state-of-the-art ASC systems is conducted using the ESAS benchmark. Experimental results reveal that existing ASC models suffer significant performance degradation when facing the event-shift challenge. The introduction of the ESAS dataset aims to drive future research toward event-robust ASC.

Yiqiang Cai, Bohan Hu, Yu Yang +3

Xi’an Jiaotong-Liverpool University, Suzhou, China · Zhongdian Zhiheng Information Technology Service Co., Ltd, Nanjing, China · China Telecom Jiangsu Branch, Nanjing, China +1

Sound Event DetectionAcousticAutomatic Speech Recognition Evaluation
AskPDF