The Polytopal Neural Network

A. Emilie J. Wedenborg; Anders V. Nørskov; Teresa Dorszewski; Kristoffer Wickstrøm; Morten Mørup

cs.LGOct 8, 2026

The Polytopal Neural Network

Authors: A. Emilie J. Wedenborg, Anders V. Nørskov, Teresa Dorszewski, Kristoffer Wickstrøm, Morten Mørup

Organizations: Technical University of Denmark · UiT The Arctic University of Norway

Abstract

Understanding how deep neural networks process information remains a central challenge. Existing interpretability methods often compromise structural fidelity, rely on prespecified corpora, or explain models post-hoc. We propose Polytopal Neural Networks (PNNs), a framework that extracts distinct layer-wise aspects by enforcing a polytope-based structure that is used directly in subsequent information processing. We scale our approach using learned corpus representations and an amortized simplex inference procedure and highlight how the framework also gives a direct route to vector quantized (VQ) training. In PNNs, observations are explicitly described by their alignment with layer-specific aspects. Empirical results show that imposing polytopal constraints on neural network representations preserves meaningful structures in the latent space with minimal degradation in performance, favorable compressed representations when compared to VQ representations in unsupervised learning, while also providing a performant new approach to VQ deep learning training. Our findings suggest that deep networks can enforce interpretable polytope-based representations, offering a principled path toward more transparent AI systems with minimal performance compromise.

Figures & tables

Figure 1: Two PathMNIST test images traced through two PNN layers ( $K=10$ ). Each layer passes a convex combination of archetypes to the next layer. Numbers denote mixture weights; each archetype is illustrated by its two nearest training images.

Figure 2: The Polytopal Neural Network: Points in the input space (left) are used to learn data driven corpus points ( $\times$ ) used to span the polytopes in the layers of the PNN. Each layer maps the observations into a low-dimensional simplex-structured latent space (polytope), where convex combinations of the encoded corpus points define the layer specific archetypal vertices. A subsequent latent transformation refines the geometric organization forming a refined polytope. Finally, points are decoded into the output space (right), where the resulting representation exhibits improved separation and structure. Colored points indicate different data groups, and black vertices denote learned archetypes.

Figure 3: Performance as a function of the number of components across datasets. Shaded regions indicate

\pm

one standard deviation over runs. Top row: VQ reconstruction MSE, with improved reconstruction quality reflected by lower values. Middle row: Classification accuracy. Bottom row: Autoencoder reconstruction MSE.

Setting	Classes	Unconstr.	$K$	PNN	Kept (%)
IN-100, ConvNeXt-T	100	94.6 $\pm$ 0.1	25	91.7 $\pm$ 0.5	96.9
IN-1k, ConvNeXt-T	1000	81.2 $\pm$ 0.0	128	78.6 $\pm$ 0.2	96.9
CIFAR-10, compact ViT	10	94.47 $\pm$ 0.21	10	94.46 $\pm$ 0.23	100.0

Table 1: Larger backbones and label spaces. Accuracy (%) of the unconstrained model and of the PNN, and the share of unconstrained accuracy the PNN retains. IN-100 and IN-1k are ImageNet-100 and ImageNet-1k, whose backbones are fine-tuned. Full results in Appendix F .

Figure 4: PNN-AE ( $K=3$ ) on FashionMNIST. Left: latent space projected via MDS; crosses indicate archetypes. Right: the four training images with the largest weight on each archetype, showing coherent groups (upper body, lower body, footwear).

Figure 5: Token-level PNN on ImageNet-100 with ConvNeXt-T and 200 archetypes. Each spatial token is represented as a convex combination of shared archetypes. A linear classifier reads the mean of the reconstructed token features. Two archetypes associated with the predicted and competing classes are illustrated using training images. The outlined tokens have the largest weights on the displayed archetypes.

Figure 6: Token predictions and archetype weights for two ImageNet-100 images. Each row shows the input, token predictions, and spatial weights of selected archetypes associated with the predicted and competing classes. Training examples are used to illustrate the archetypes. Blue denotes the predicted class, orange the competing class, and gray other classes. The counts below the token-prediction panels show how many tokens favor each displayed class. The image prediction depends on the averaged class scores.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Test MSE of the PNN-AE with the exact SMO projections vs. the Amortized version.

Figure 8: Share of PNN-CNN test predictions that change when the highest-weight archetype is removed and when a single archetype is removed at random, against the number of archetypes

K

. Mean

\pm

standard deviation for five seeds.

Figure 9: Increase in test MSE of the PNN-AE when the highest-weight archetype is removed and when a single archetype is removed at random, against the number of archetypes

K

. Mean

\pm

standard deviation for five seeds.

	CNN classifier	Autoencoder
Latent dimension	64	64
Corpus size	2048	2048
Baseline batch size	256	2048
Phase 2 and Phase 3 batch size	512	512
Unconstrained baseline, epochs	200	200
Phase 2 epochs	60	100

Appendix

Table 2: Hyperparameters of the main CNN and AE grids. Epoch counts are maxima and learning rates are initial values.

	MedMNIST	ImageNet-100	ImageNet-1k	Frozen head
Backbone	two-site CNN	ConvNeXt-T	ConvNeXt-T	compact ViT
Backbone initialization	baseline, same seed	ImageNet-1k	ImageNet-1k	CIFAR-10, same seed
Input resolution	$64\times 64$	$224\times 224$	$224\times 224$	$32\times 32$
PNN site	two FC sites	every token	every token	pooled feature
PNN input dimension	128	256	256	256
$K$	10, 25	25, 50, 100, 150, 200	128, 256	3, 5, 10, 20, 40

Appendix

Table 3: Settings of the experiments beyond the main CNN and AE grids. Unconstrained models use the same backbone, data and optimization as the PNN of their column. The ImageNet-100 PNN has three seeds and its unconstrained model five.

Training time (minutes per 100 epochs)
	Unconstrained	PNN $K=3$	PNN $K=10$	PNN $K=25$	PNN-PL $K=25$	PNN-VQ $K=25$
MNIST	5.3	10.3	14.9	24.8	7.7	6.7
FMNIST	5.3	10.3	14.9	25.5	7.8	6.6
CIFAR-10	4.5	9.0	12.8	21.1	6.6	5.9
SVHN	6.5	14.2	18.8	31.4	9.7	8.5
EuroSAT	2.1	5.4	7.0	10.5	4.2	3.7

Appendix

Table 4: Measured training cost of the CNN models on one NVIDIA RTX A5000. Each model trains for 100 epochs from its seed-42 checkpoint with its own training step.

Figure 10: Cost of the PNN-CNN (CIFAR-10,

K=10

) against the corpus size

N^{c}

, on one NVIDIA RTX A5000. Left: time per batch of 512 for a Phase-3 training step and for inference with the archetypes cached. Right: peak GPU memory of the same. Using the full training set as the corpus (

N^{c}=45,000

) costs 4.7

\times

the training time and 9.3

\times

the memory of the learned corpus (

N^{c}=2048

); inference time does not depend on

N^{c}

	Unconstrained	$K{=}25$	$K{=}50$	$K{=}100$	$K{=}150$	$K{=}200$
ConvNeXt-T	94.6 $\pm$ 0.1	91.7 $\pm$ 0.5	92.0 $\pm$ 0.3	91.9 $\pm$ 0.7	92.5 $\pm$ 0.2	92.0 $\pm$ 0.7

Appendix

Table 5: ImageNet-100 at

224{\times}224

with a fine-tuned ConvNeXt-T. Top-1 accuracy (%) on the 5,000 validation images at the final epoch, mean

\pm

std.

	Top-1	Seeds
Unconstrained, 256-d linear projection	81.2 $\pm$ 0.0	2
Unconstrained, 128-d linear projection	81.1 $\pm$ 0.1	2
PNN, $K=128$	78.6 $\pm$ 0.2	2
PNN, $K=256$	78.9 $\pm$ 0.1	2

Appendix

Table 6: ImageNet-1k with a fine-tuned ConvNeXt-T. Top-1 accuracy (%) on the 50,000 validation images at the final epoch, mean

\pm

std over two seeds.

	$K$	CVX		QP		SMO		Amortized
		Batch	$\neg$ Batch	Batch	$\neg$ Batch	Batch	$\neg$ Batch	Batch	$\neg$ Batch
Breast	5	96.1 $\pm$ 2.3	97.1 $\pm$ 1.3	96.4 $\pm$ 1.5	96.4 $\pm$ 1.8	95.4 $\pm$ 4.1	96.8 $\pm$ 1.5	96.3 $\pm$ 2.2	96.3 $\pm$ 1.8
	10	97.3 $\pm$ 1.2	97.1 $\pm$ 1.2	96.4 $\pm$ 1.2	97.2 $\pm$ 1.4	96.7 $\pm$ 1.1	97.0 $\pm$ 1.2	96.7 $\pm$ 1.7	96.3 $\pm$ 1.7
	25	97.4 $\pm$ 1.0	96.9 $\pm$ 1.5	96.5 $\pm$ 0.0	97.4 $\pm$ 2.0	96.9 $\pm$ 2.5	96.9 $\pm$ 1.5	96.0 $\pm$ 0.7	96.3 $\pm$ 1.4
Iris	5	93.7 $\pm$ 4.0	94.7 $\pm$ 2.8	94.3 $\pm$ 3.5	95.0 $\pm$ 3.6	93.7 $\pm$ 4.0	93.0 $\pm$ 5.1	94.0 $\pm$ 5.3	84.7 $\pm$ 15.1
	10	95.3 $\pm$ 2.8	95.3 $\pm$ 2.8	95.3 $\pm$ 2.8	95.3 $\pm$ 2.8	95.7 $\pm$ 2.7	96.0 $\pm$ 3.1	94.0 $\pm$ 3.9	94.0 $\pm$ 3.9

Appendix

Table 7: Test accuracy (%) comparison between batched and non-batched training across optimization methods. Results show mean

\pm

std. Batched training with SMO achieves statistically equivalent performance to exact methods while enabling scalability.

	K	Pretrained	PNN-PL-MLP		PNN-VQ-MLP		PNN-MLP
			Polytope	Full	Polytope	Full	Polytope	Full
Breast	3	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	96.0 $\pm$ 2.9	86.0 $\pm$ 13.9	85.6 $\pm$ 13.7	85.5 $\pm$ 15.6	92.0 $\pm$ 10.5
	5	96.3 $\pm$ 2.3	96.5 $\pm$ 2.2	96.1 $\pm$ 2.6	93.3 $\pm$ 4.7	93.0 $\pm$ 4.6	97.1 $\pm$ 2.0	96.8 $\pm$ 1.9
	7	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	89.5 $\pm$ 6.4	89.3 $\pm$ 6.4	96.2 $\pm$ 2.0	95.7 $\pm$ 2.6
	10	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	93.2 $\pm$ 3.4	92.6 $\pm$ 3.1	96.9 $\pm$ 2.2	96.8 $\pm$ 1.9
	15	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	96.3 $\pm$ 2.3	94.4 $\pm$ 1.8	92.6 $\pm$ 5.1	97.0 $\pm$ 1.9	96.9 $\pm$ 2.1

Appendix

Table 8: MLP test classification accuracy (%) across training phases for PNN variants

	Classes	Unconstrained	PNN $K=10$	PNN $K=25$
PathMNIST	9	90.4 $\pm$ 1.2	88.5 $\pm$ 1.5	89.1 $\pm$ 1.5
DermaMNIST	7	78.6 $\pm$ 0.7	76.4 $\pm$ 0.6	77.4 $\pm$ 0.7
BloodMNIST	8	98.3 $\pm$ 0.0	98.1 $\pm$ 0.2	98.1 $\pm$ 0.1

Appendix

Table 9: CNN test accuracy (%) on three MedMNIST datasets at

64{\times}64

, mean

\pm

std over five seeds. Each PNN run starts from the unconstrained network of the same seed and uses the deployed coordinate map.

	$K$	Unconstrained	One site	Two sites
MNIST	3	99.6 $\pm$ 0.0	98.4 $\pm$ 1.3	91.7 $\pm$ 12.8
	10		99.4 $\pm$ 0.0	99.4 $\pm$ 0.1
	25		99.5 $\pm$ 0.0	99.5 $\pm$ 0.0
FMNIST	3	93.2 $\pm$ 0.1	92.1 $\pm$ 0.1	91.6 $\pm$ 0.6
	10		92.8 $\pm$ 0.3	92.8 $\pm$ 0.2
	25		93.0 $\pm$ 0.1	92.7 $\pm$ 0.1

Appendix

Table 10: CNN test accuracy (%) with one constrained site or two, deployed coordinates (the amortizer followed by

K

rounds of warm-started batched SMO), mean

\pm

std over the same three seeds for every column. The configuration is otherwise identical.

Panel of Figure 11 — Figure 11: PNN-AE reconstruction by K for SVHN, MNIST, CIFAR-10, EuroSAT

Panel of Figure 12 — Figure 12: Reconstruction by model for $K=25$ for SVHN, MNIST, CIFAR-10 and EuroSAT.

	$K$	Softmax	Sparsemax
MNIST	3	0.0399 $\pm$ 0.0003	0.0400 $\pm$ 0.0005
	10	0.0147 $\pm$ 0.0005	0.0148 $\pm$ 0.0001
	25	0.0073 $\pm$ 0.0001	0.0075 $\pm$ 0.0003
FMNIST	3	0.0266 $\pm$ 0.0002	0.0268 $\pm$ 0.0003
	10	0.0127 $\pm$ 0.0001	0.0127 $\pm$ 0.0001
	25	0.0099 $\pm$ 0.0002	0.0100 $\pm$ 0.0004

Appendix

Table 11: PNN-AE reconstruction MSE with a softmax or sparsemax amortizer output. Values are mean

\pm

std over the same three seeds.

	$K$	VQ-VAE	PNN-VQ
MNIST	3	0.0613 $\pm$ 0.0005	0.0614 $\pm$ 0.0006
	5	0.0568 $\pm$ 0.0006	0.0580 $\pm$ 0.0007
	10	0.0530 $\pm$ 0.0006	0.0527 $\pm$ 0.0009
	15	0.0502 $\pm$ 0.0005	0.0493 $\pm$ 0.0004
	20	0.0478 $\pm$ 0.0001	0.0474 $\pm$ 0.0003
	25	0.0464 $\pm$ 0.0004	0.0458 $\pm$ 0.0007

Appendix

Table 12: Reconstruction MSE of a standard VQ-VAE (EMA codebook, commitment loss, straight-through estimator) and of PNN-VQ (direct gradient updates of a corpus-grounded codebook, reconstruction loss only), mean

\pm

std over five seeds.

Explore similar work

Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance
May 29, 2026Doğukan Bağcı, Bernt Schiele, Simone Schaub-Meyer +2Disentangled Representation Learning Representation Disentanglement
Max Planck Institute for Informatics, SIC · Department of Computer Science, TU Darmstadt
Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks
May 26, 2025Safa Hamreras, Sukhbinder Singh, Román OrúsTensor Networks Neural Network Interpretability
Donostia International Physics Center, Paseo Manuel de Lardizabal 4, E-20018 San Sebastián, Spain · Multiverse Computing, Spadina Ave., Toronto, ON M5T 2C2, Canada · Multiverse Computing, Paseo de Miramón 170, E-20014 San Sebastián, Spain
AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks
May 7, 2026Yi Wei, Xuan Qi, Furao Shen +3Neural Network Interpretability Neural Network Approximation Theory
State Key Laboratory of Novel Software Technology School of Intelligence Science and Technology Nanjing University, Jiangsu, China · AI for Good Istituto Italiano di Tecnologia, Genoa, Italy · DITEN University of Genoa, Genoa, Italy +2

May 29, 2026, cs.LGJ/K move · Enter open · S save

Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance

Doğukan Bağcı, Bernt Schiele, Simone Schaub-Meyer, Jonas Fischer+1

Max Planck Institute for Informatics, SIC · Department of Computer Science, TU Darmstadt

Disentangled Representation Learning Representation Disentanglement Neural Network Interpretability

Open paper AskPDF arXiv

Deep neural networks (DNNs) are widely used, but interpreting what they actually learn remains difficult. A major obstacle is that individual neurons often encode multiple unrelated concepts, obscuring the decision process of the network. While prior work, such as sparse autoencoders, can separate these mixed signals into more meaningful, "monosemantic" features, this typically requires altering the model in ways that can degrade downstream performance. To overcome this, we introduce ELUDe (explicit, lossless, unsupervised disentanglement), a method for improving the interpretability of DNNs while preserving their functional equivalence. ELUDe breaks latent representations into clear, inspectable sub-units that behave like interpretable features, while guaranteeing that the model's outputs remain exactly the same. It requires no explicit training, no labels, and can be applied to pretrained models. ELUDe works by reorganizing how information flows between layers, re-routing concept-specific contributions while preserving the original computation by construction. Across several vision models, including DINOv2 and supervised ViT-B/16, ELUDe improves interpretability, keeps downstream accuracy unchanged, runs efficiently, and supports practical uses such as steering model representations. In short, ELUDe offers interpretability (almost) without a tradeoff: clearer, scalable, and actionable model insights with no loss in performance.

Abstract

Explore similar work

Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks