TropNNC: Structured Neural Network Compression Using Tropical Geometry

We propose a computational tropical geometry framework for the symbolic analysis of neural networks with tropical activations. The number of linear regions of a neural network has been actively studied as a measure of the expressivity of a given architecture. To study these, we work in the setting of tropical geometry---a combinatorial and polyhedral variant of algebraic geometry---where there are known connections between tropical rational maps and feedforward neural networks. We expand this connection by developing concrete computational tools for studying the linear regions of neural networks. We present an algorithm, together with a proof of correctness, which computes the linear regions of a neural network as explicit unions of polyhedra. We further relate the computation of the number of linear regions of a tropical expression to the number of monomials that appear in it, and show how tropical expressions can often be pruned to remove redundant monomials. We introduce the Hoffman constant of a neural network's tropical expression, a geometric quantity that controls the distance from any point in the input space to the farthest linear region. We provide the open source Julia library TropicalNN.jl, which is built on top of the OSCAR computer algebra system and implements the algorithms mentioned above to analyze neural networks symbolically using their tropical representations. We present a set of proof-of-concept computational examples to demonstrate how our tropical geometric theory can be applied to reveal insights on the expressivity of a network architecture.

Neural network compression is commonly achieved by pruning parameters based on local importance scores, e.g., magnitude-based pruning. We propose a complementary approach that compresses models by aggregating neurons with similar functional behavior rather than removing weights independently. Our method encodes a trained network as a polynomial ODE system and applies a lumping method called Approximate Forward Differential Equivalence to identify neurons with approximately matching induced dynamics. A single tolerance parameter, $\varepsilon$, controls the compression level and induces a smooth trade-off between model size and predictive accuracy. We evaluate the method on synthetic datasets derived from nonlinear dynamical systems with known ground-truth behavior and on public regression benchmarks. Across both settings, the proposed approach achieves substantial parameter reduction while preserving accuracy, and consistently compares favorably with magnitude-based pruning and Wanda at similar compression levels. These results suggest that differential equivalence-based aggregation is a principled and effective alternative to conventional weight-centric pruning.

Kolmogorov--Arnold Networks (KANs) replace scalar edge weights with learnable univariate functions parameterized by multiple basis coefficients. This introduces a source of redundancy that conventional neural-network compression does not directly expose. We present SparseKAN, a unified approach that compresses KANs along three complementary axes: basis functions, neurons/channels, and numerical precision. SparseKAN equips the base branch, nonlinear basis branch, and individual basis terms with hierarchical learnable gates trained under a differentiable active-cost objective. The learned importance structure is subsequently hardened under explicit basis and width budgets, recovered in full or low precision, and physically compacted into smaller dense tensors rather than retained as sparse masks. Experiments on MNIST, CIFAR-10, and CIFAR-100 across spline, polynomial, RBF, wavelet, and convolutional KAN variants show that the structural axes compose predictably in cost. We also find strong basis-dependent differences in term importance: coefficient-based selection outperforms matched low-order truncation by up to 15.25 accuracy points in the evaluated Gram-polynomial settings. Eight-bit quantization is broadly robust, whereas 4-bit convolutional KANs require quantization-aware adaptation. Physical compaction removes up to 73.0% of parameters without accuracy loss on MNIST and reduces large-batch CUDA latency to as little as $0.51\times$ dense execution. On a ZCU104 FPGA, the resulting sparse low-bit models achieve up to $23.63\times$ lower inference latency, demonstrating that SparseKAN converts functional redundancy into measurable software and hardware efficiency. The SparseKAN implementation is available at https://github.com/OSU-STARLAB/SparseKAN.