Bridging Neural Representations and Symbolic Semantics in Audio Classification

Avishek Das; Pascal Hitzler

cs.SDOct 29, 2026

Bridging Neural Representations and Symbolic Semantics in Audio Classification

Authors: Avishek Das, Pascal Hitzler

Organizations: Kansas State University, Manhattan, Kansas

Abstract

Deep audio classifiers can achieve strong predictive performance, but their hidden representations remain difficult to interpret. Existing audio explainability methods often focus on input-level attribution, such as waveform segments, spectrogram regions, or acoustic features, and provide limited insight into what hidden neurons encode. Interpreting hidden neurons at the concept level can provide a more meaningful view of what an audio model learns internally. This paper investigates whether hidden neurons in audio classifiers can be associated with human-understandable symbolic acoustic concepts. A model-agnostic neurosymbolic framework is presented for analyzing the final hidden layer of audio classifiers, containing 32 neurons, through symbolic concept induction. The framework is evaluated on two audio classification settings: music genre classification using GTZAN dataset and musical instrument classification using IRMAS dataset. Results show that several hidden neurons can be associated with meaningful induced concepts, and most confirmed neuron-concept pairs are statistically supported using the Mann-Whitney U test. These findings suggest that symbolic concept induction is a promising direction for explaining latent audio representations beyond input-level attribution.

Figures & tables

Figure 1: Audio to symbolic tag construction. Raw audio recordings are segmented, acoustic features are extracted, and statistical thresholding is used to map feature values into human-understandable symbolic tags.

GTZAN		IRMAS
Class	Count	Class	Count
Jazz	1000	Electric guitar (gel)	760
Classical	998	Piano (pia)	721
Hip-hop	998	Organ (org)	682
Rock	998	Acoustic guitar (gac)	637
Country	997	Cello (cel)	388

Table 1: Class-wise distribution of audio files.

Feature Descriptor	Low Tags	High Tags	Purpose	Tags Interpretation	Source
Tempo ( tempo )	Slow	Fast	Estimates pace in beats per minute	Lower tempo = slower musical pacing, higher tempo = faster pacing.	( McFee et al., 2015 ; Costa, 2026 )
Root Mean Square Energy ( rms_mean )	Soft	Loud	Measures average signal energy	Lower RMS energy is treated as a proxy for softer sound, while higher RMS energy for louder sound.	( McFee et al., 2015 ; Rawlinson et al., 2015 ; Constantinescu and Brad, 2023 )
Spectral Centroid ( spectral_centroid _mean )	Dark	Bright	Measures the center of mass of the spectrum	Lower centroid refers to darker or lower frequency dominant timbre, and higher centroid to brighter timbre.	( Racharla et al., 2020 ; McFee et al., 2015 ; Rawlinson et al., 2015 )
Spectral Bandwidth ( spectral_bandwidth _mean )	Pure	Rich	Measures the spread of spectral energy around the spectral centroid.	Lower bandwidth = more focused spectrum, higher bandwidth = broader spectral spread and richer overtone content.	( Antunes et al., 2019 ; Encyclopaedia Britannica, 2025 ; Cusack and Roberts, 1999 ; Jannereth and Esch, 2021 )
Spectral Rolloff ( rolloff_mean )	Mellow	Crisp	Measures the frequency below which most spectral energy is contained.	Lower rolloff means most energy is concentrated in lower frequencies and interpreted as more mellow; higher rolloff indicates stronger high-frequency contribution, thus more crisp.	( Reymore et al., 2022 ; McFee et al., 2015 )
Zero-Crossing Rate ( zero_crossing_rate _mean )	Smooth	Noisy	Measures how frequently the waveform changes sign over time.	Lower zero-crossing rate is associated with smoother or more periodic signals, while higher zero-crossing rate is with noisier or more rapidly varying signals.	( Gouyon et al., 2000 ; Bogdanov et al., 2013 )

Table 2: Mapping acoustic features to symbolic tags used for concept induction. Tag names are proxy labels for relative acoustic behavior, not exact psychoacoustic measurements.

Figure 2: Overview of the proposed neuron-level symbolic explanation pipeline. A trained audio classifier is analyzed by extracting hidden-layer activations, constructing positive and negative activation sets, and applying concept induction over mapped symbolic tags to generate and validate neuron-level explanations.

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_22	80	0	Noisy and Soft
neuron_10	70	0	Pure and Soft
neuron_12	70	10	Crisp and Smooth
neuron_31	60	0	Pure and Soft
neuron_9	60	10	Bright and (not (Rich))
neuron_21	60	0	Noisy and Pure

Table 3: Confirmed neurons based on neuron trigger counts with best matching expressions for Genre Classification task. The trigger % are based on 10 data for each neuron. Full version of the Table with all the neurons is shown in Appendix A Table 8 .

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_30	80	0	NonMelodic and Pure
neuron_7	80	20	Rich and Soft
neuron_9	70	10	Rich and Smooth
neuron_22	60	0	Loud and Noisy
neuron_15	60	10	Crisp and Smooth
neuron_0	50	0	Bright and Loud

Table 4: Confirmed neurons based on neuron trigger counts with best matching expressions for Instrument Classification task. The trigger% are based on 10 data for each neuron. Full version of the Table with all the neurons is shown in Appendix A Table 9 .

Table 7

Study / Task	Domain	Analyzed Neurons	Validated	%
Dalal et al. ( Dalal et al., 2024 )	Image	64	19	29.7
Sarma et al. ( Sarma et al., 2026 )	Image	64	29	45.3
Das et al. ( Das et al., 2025 )	Text	64	23	35.9
Genre classification (this paper)	Audio	32	7	21.9
Instrument classification (this paper)	Audio	32	9	28.1

Table 7: Comparison of effectively labeled neurons across hidden-neuron analysis studies.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_22	80	0	Noisy and Soft
neuron_10	70	0	Pure and Soft
neuron_12	70	10	Crisp and Smooth
neuron_31	60	0	Pure and Soft
neuron_9	60	10	Bright and (not (Rich))
neuron_21	60	0	Noisy and Pure

Appendix

Table 8: Neuron trigger counts with best matching expressions.

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_30	80	0	NonMelodic and Pure
neuron_7	80	20	Rich and Soft
neuron_9	70	10	Rich and Smooth
neuron_22	60	0	Loud and Noisy
neuron_15	60	10	Crisp and Smooth
neuron_0	50	0	Bright and Loud

Appendix

Table 9: Neuron trigger counts with best matching expressions.

Neuron-Level Interpretability

See all 22 papers

Feb 17, 2026

What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model

In this paper, we analyze the internal representations of a general-purpose audio self-supervised learning (SSL) model from a neuron-level perspective. Despite their strong empirical performance as feature extractors, the internal mechanisms underlying the robust generalization of SSL audio models remain unclear. Drawing on the framework of mechanistic interpretability, we identify and examine class-specific neurons by analyzing conditional activation patterns across diverse tasks. Our analysis reveals that SSL models foster the emergence of class-specific neurons that provide extensive coverage across novel task classes. These neurons exhibit shared responses across different semantic categories and acoustic similarities, such as speech attributes and musical pitch. We also confirm that these neurons have a functional impact on classification performance. To our knowledge, this is the first systematic neuron-level analysis of a general-purpose audio SSL model, providing new insights into its internal representation.

Takao Kawamura, Daisuke Niizumi, Nobutaka Ono

Graduate School of Systems Design, Tokyo Metropolitan University, Tokyo, Japan

Audio Representation Learning Neural Network Interpretability Self-Supervised Learning

Ask PDF

May 11, 2026

APEX: Audio Prototype EXplanations for Classification Tasks

Explainable AI (XAI) has achieved remarkable success in image classification, yet the audio domain lacks equally mature solutions. Current methods apply vision-based attribution techniques to spectrograms, overlooking fundamental differences between visual and acoustic signals. While prototype reasoning is promising, acoustic similarity remains multidimensional. We introduce APEX (Audio Prototype EXplanations), a post-hoc framework for interpreting pre-trained audio classifiers. Crucially, APEX requires no fine-tuning of the original backbone and strictly preserves output invariance. APEX disentangles explanations into four perspectives: Square-based prototypes to localize transient events, Time-based for temporal patterns, Frequency-based highlighting spectral bands, and Time-Frequency-based integrating both. This yields intuitive, example-based explanations that respect acoustic properties, providing greater semantic clarity than standard gradient-based methods.

Piotr Kawa, Kornel Howil, Piotr Borycki +3

Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland · IDEAS Research Institute, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland +1

Explainable Artificial Intelligence Prototype-Based Explanation Audio Understanding

Ask PDF

Feb 24, 2026

AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs

Despite strong performance in audio perception tasks, large audio-language models (AudioLLMs) remain opaque to interpretation. A major factor behind this lack of interpretability is that individual neurons in these models frequently activate in response to several unrelated concepts. We introduce the first mechanistic interpretability framework for AudioLLMs, leveraging sparse autoencoders (SAEs) to disentangle polysemantic activations into monosemantic features. Our pipeline identifies representative audio clips, assigns meaningful names via automated captioning, and validates concepts through human evaluation and steering. Experiments show that AudioLLMs encode structured and interpretable features, enhancing transparency and control. This work provides a foundation for trustworthy deployment in high-stakes domains and enables future extensions to larger models, multilingual audio, and more fine-grained paralinguistic features. Project URL: https://townim-faisal.github.io/AutoInterpret-AudioLLM/

Townim Faisal Chowdhury, Ta Duc Huy, Siqi Pan +2

Australian Institute for Machine Learning, University of Adelaide · Dolby Laboratories · School of Computer and Mathematical Sciences, University of Adelaide, Australia

Multimodal Model Interpretability Mechanistic Interpretability Representation Disentanglement

Ask PDF

Abstract

Neuron-Level Interpretability

Explore similar work

What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model

APEX: Audio Prototype EXplanations for Classification Tasks

AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs