cs.LGDec 8, 2025

HOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability

Authors: Sudhanva Manjunath Athreya, Paul Rosen

Organizations: University of Utah

Abstract

Deep learning models have achieved remarkable success across various domains, yet their learned representations and decision-making processes remain largely opaque and hard to interpret. This work introduces HOLE (Homological Observation of Latent Embeddings), a method for analyzing and interpreting discriminative neural networks through persistent homology. HOLE extracts topological features from intermediate activations and presents them using a suite of visualization techniques, including cluster flow diagrams, blob graphs, and heatmap dendrograms. These tools facilitate the examination of representation structure and quality across layers. We evaluate HOLE using a range of discriminative models, focusing on representation quality, interpretability across layers, and robustness to input perturbations and model compression. The results indicate that topological analysis reveals patterns associated with class separation, feature disentanglement, and model robustness, providing a complementary perspective for understanding and improving deep learning systems.

Figures & tables

Appendix figures & tables34 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 29, 2026

DUNE: Disentangling Polysemantic Neurons Without Losing Predictive Performance

Deep neural networks (DNNs) are widely used, but understanding what they actually learn remains difficult. A major obstacle is that individual neurons often respond to multiple unrelated concepts, obscuring the network's internal decision process. Existing approaches, such as sparse autoencoders, can disentangle these mixed signals into more meaningful, ``monosemantic'' features, but typically introduce approximation errors that may degrade downstream performance. Consequently, these methods exhibit a Pareto frontier between monosemanticity and functional faithfulness. To address this, we introduce DUNE (disentangling units with no error), a method that decomposes DNN neurons into more monosemantic subunits while preserving their original function by design. DUNE identifies concept-specific contributions from the preceding layer and re-routes them into separate subunits whose aggregation exactly recovers the original activation. DUNE can be applied directly to pretrained models, without labels or training of an auxiliary model. Across several models, including DINOv2, DUNE achieves state-of-the-art or competitive monosemanticity while exactly preserving functional faithfulness, substantially advancing the existing Pareto frontier between these properties. DUNE operates efficiently and enables fine-grained interventions such as concept-specific steering.
Oct 22, 2026

Post-Hoc Feature Selection Layer for Neural Networks Interpretability

The interpretability of complex neural networks remains a critical challenge, especially for models already deployed in high-stakes domains. To address this, we adapted the Feature Selection Layer (FSL) to optimize feature weights post-training, a variant we term post-hoc FSL. Our approach reframes the FSL as a lightweight, trainable module that integrates with already frozen pre-trained models on tabular datasets to highlight the features the original model considers most important. This post-hoc FSL learns feature relevance by fine-tuning its weights based on the original model's learned outputs. Crucially, this process is non-invasive, operating without altering the original model's architecture or its learned parameters. We conducted our experiments using both statistical and visual metrics, including accuracy, F1 score, recall, precision, weighted t-SNE and silhouette score, and also analyzed the stability of the post-hoc FSL on high-dimensional synthetic and real-world tabular datasets. We compared the post-hoc FSL feature weighting method using these metrics against the original embedded FSL and other post-hoc interpretability methods, such as Integrated Gradients, Noise Tunnel, DeepLIFT, Gradient SHAP, and Feature Ablation. Experimental results demonstrate that the post-hoc FSL feature weighting method successfully identified relevant features across the different datasets, maintaining the predictive power of the original neural network while enhancing its interpretability.
Oct 8, 2026

The Polytopal Neural Network

Understanding how deep neural networks process information remains a central challenge. Existing interpretability methods often compromise structural fidelity, rely on prespecified corpora, or explain models post-hoc. We propose Polytopal Neural Networks (PNNs), a framework that extracts distinct layer-wise aspects by enforcing a polytope-based structure that is used directly in subsequent information processing. We scale our approach using learned corpus representations and an amortized simplex inference procedure and highlight how the framework also gives a direct route to vector quantized (VQ) training. In PNNs, observations are explicitly described by their alignment with layer-specific aspects. Empirical results show that imposing polytopal constraints on neural network representations preserves meaningful structures in the latent space with minimal degradation in performance, favorable compressed representations when compared to VQ representations in unsupervised learning, while also providing a performant new approach to VQ deep learning training. Our findings suggest that deep networks can enforce interpretable polytope-based representations, offering a principled path toward more transparent AI systems with minimal performance compromise.