Bridging Neural Representations and Symbolic Semantics in Audio Classification

Avishek Das; Pascal Hitzler

cs.SDOct 29, 2026

Bridging Neural Representations and Symbolic Semantics in Audio Classification

Authors: Avishek Das, Pascal Hitzler

Organizations: Kansas State University, Manhattan, Kansas

Abstract

Deep audio classifiers can achieve strong predictive performance, but their hidden representations remain difficult to interpret. Existing audio explainability methods often focus on input-level attribution, such as waveform segments, spectrogram regions, or acoustic features, and provide limited insight into what hidden neurons encode. Interpreting hidden neurons at the concept level can provide a more meaningful view of what an audio model learns internally. This paper investigates whether hidden neurons in audio classifiers can be associated with human-understandable symbolic acoustic concepts. A model-agnostic neurosymbolic framework is presented for analyzing the final hidden layer of audio classifiers, containing 32 neurons, through symbolic concept induction. The framework is evaluated on two audio classification settings: music genre classification using GTZAN dataset and musical instrument classification using IRMAS dataset. Results show that several hidden neurons can be associated with meaningful induced concepts, and most confirmed neuron-concept pairs are statistically supported using the Mann-Whitney U test. These findings suggest that symbolic concept induction is a promising direction for explaining latent audio representations beyond input-level attribution.

Figures & tables

Figure 1: Audio to symbolic tag construction. Raw audio recordings are segmented, acoustic features are extracted, and statistical thresholding is used to map feature values into human-understandable symbolic tags.

GTZAN		IRMAS
Class	Count	Class	Count
Jazz	1000	Electric guitar (gel)	760
Classical	998	Piano (pia)	721
Hip-hop	998	Organ (org)	682
Rock	998	Acoustic guitar (gac)	637
Country	997	Cello (cel)	388

Table 1: Class-wise distribution of audio files.

Feature Descriptor	Low Tags	High Tags	Purpose	Tags Interpretation	Source
Tempo ( tempo )	Slow	Fast	Estimates pace in beats per minute	Lower tempo = slower musical pacing, higher tempo = faster pacing.	( McFee et al., 2015 ; Costa, 2026 )
Root Mean Square Energy ( rms_mean )	Soft	Loud	Measures average signal energy	Lower RMS energy is treated as a proxy for softer sound, while higher RMS energy for louder sound.	( McFee et al., 2015 ; Rawlinson et al., 2015 ; Constantinescu and Brad, 2023 )
Spectral Centroid ( spectral_centroid _mean )	Dark	Bright	Measures the center of mass of the spectrum	Lower centroid refers to darker or lower frequency dominant timbre, and higher centroid to brighter timbre.	( Racharla et al., 2020 ; McFee et al., 2015 ; Rawlinson et al., 2015 )
Spectral Bandwidth ( spectral_bandwidth _mean )	Pure	Rich	Measures the spread of spectral energy around the spectral centroid.	Lower bandwidth = more focused spectrum, higher bandwidth = broader spectral spread and richer overtone content.	( Antunes et al., 2019 ; Encyclopaedia Britannica, 2025 ; Cusack and Roberts, 1999 ; Jannereth and Esch, 2021 )
Spectral Rolloff ( rolloff_mean )	Mellow	Crisp	Measures the frequency below which most spectral energy is contained.	Lower rolloff means most energy is concentrated in lower frequencies and interpreted as more mellow; higher rolloff indicates stronger high-frequency contribution, thus more crisp.	( Reymore et al., 2022 ; McFee et al., 2015 )
Zero-Crossing Rate ( zero_crossing_rate _mean )	Smooth	Noisy	Measures how frequently the waveform changes sign over time.	Lower zero-crossing rate is associated with smoother or more periodic signals, while higher zero-crossing rate is with noisier or more rapidly varying signals.	( Gouyon et al., 2000 ; Bogdanov et al., 2013 )

Table 2: Mapping acoustic features to symbolic tags used for concept induction. Tag names are proxy labels for relative acoustic behavior, not exact psychoacoustic measurements.

Figure 2: Overview of the proposed neuron-level symbolic explanation pipeline. A trained audio classifier is analyzed by extracting hidden-layer activations, constructing positive and negative activation sets, and applying concept induction over mapped symbolic tags to generate and validate neuron-level explanations.

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_22	80	0	Noisy and Soft
neuron_10	70	0	Pure and Soft
neuron_12	70	10	Crisp and Smooth
neuron_31	60	0	Pure and Soft
neuron_9	60	10	Bright and (not (Rich))
neuron_21	60	0	Noisy and Pure

Table 3: Confirmed neurons based on neuron trigger counts with best matching expressions for Genre Classification task. The trigger % are based on 10 data for each neuron. Full version of the Table with all the neurons is shown in Appendix A Table 8 .

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_30	80	0	NonMelodic and Pure
neuron_7	80	20	Rich and Soft
neuron_9	70	10	Rich and Smooth
neuron_22	60	0	Loud and Noisy
neuron_15	60	10	Crisp and Smooth
neuron_0	50	0	Bright and Loud

Table 4: Confirmed neurons based on neuron trigger counts with best matching expressions for Instrument Classification task. The trigger% are based on 10 data for each neuron. Full version of the Table with all the neurons is shown in Appendix A Table 9 .

Table 7

Study / Task	Domain	Analyzed Neurons	Validated	%
Dalal et al. ( Dalal et al., 2024 )	Image	64	19	29.7
Sarma et al. ( Sarma et al., 2026 )	Image	64	29	45.3
Das et al. ( Das et al., 2025 )	Text	64	23	35.9
Genre classification (this paper)	Audio	32	7	21.9
Instrument classification (this paper)	Audio	32	9	28.1

Table 7: Comparison of effectively labeled neurons across hidden-neuron analysis studies.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_22	80	0	Noisy and Soft
neuron_10	70	0	Pure and Soft
neuron_12	70	10	Crisp and Smooth
neuron_31	60	0	Pure and Soft
neuron_9	60	10	Bright and (not (Rich))
neuron_21	60	0	Noisy and Pure

Appendix

Table 8: Neuron trigger counts with best matching expressions.

Neuron	Pos. Trigger(%)	Neg. Trigger(%)	Induced Concepts
neuron_30	80	0	NonMelodic and Pure
neuron_7	80	20	Rich and Soft
neuron_9	70	10	Rich and Smooth
neuron_22	60	0	Loud and Noisy
neuron_15	60	10	Crisp and Smooth
neuron_0	50	0	Bright and Loud

Appendix

Table 9: Neuron trigger counts with best matching expressions.

Neuron-Level Interpretability

See all 22 papers

What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model
Feb 17, 2026Takao Kawamura, Daisuke Niizumi, Nobutaka OnoAudio Representation Learning Neural Network Interpretability
Graduate School of Systems Design, Tokyo Metropolitan University, Tokyo, Japan
APEX: Audio Prototype EXplanations for Classification Tasks
May 11, 2026Piotr Kawa, Kornel Howil, Piotr Borycki +3Explainable Artificial Intelligence Prototype-Based Explanation
Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland · IDEAS Research Institute, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland +1
AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs
Feb 24, 2026Townim Faisal Chowdhury, Ta Duc Huy, Siqi Pan +2Multimodal Model Interpretability Mechanistic Interpretability
Australian Institute for Machine Learning, University of Adelaide · Dolby Laboratories · School of Computer and Mathematical Sciences, University of Adelaide, Australia

Feb 17, 2026J/K move · Enter open · S save

What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model

Takao Kawamura, Daisuke Niizumi, Nobutaka Ono

Graduate School of Systems Design, Tokyo Metropolitan University, Tokyo, Japan

Audio Representation Learning Neural Network Interpretability Self-Supervised Learning

Open paper AskPDF arXiv

In this paper, we analyze the internal representations of a general-purpose audio self-supervised learning (SSL) model from a neuron-level perspective. Despite their strong empirical performance as feature extractors, the internal mechanisms underlying the robust generalization of SSL audio models remain unclear. Drawing on the framework of mechanistic interpretability, we identify and examine class-specific neurons by analyzing conditional activation patterns across diverse tasks. Our analysis reveals that SSL models foster the emergence of class-specific neurons that provide extensive coverage across novel task classes. These neurons exhibit shared responses across different semantic categories and acoustic similarities, such as speech attributes and musical pitch. We also confirm that these neurons have a functional impact on classification performance. To our knowledge, this is the first systematic neuron-level analysis of a general-purpose audio SSL model, providing new insights into its internal representation.

Abstract

Neuron-Level Interpretability

Explore similar work

What Do Neurons Listen To? A Neuron-level Dissection of a General-purpose Audio Model

APEX: Audio Prototype EXplanations for Classification Tasks

AR&D: A Framework for Retrieving and Describing Concepts for Interpreting AudioLLMs