cs.LGNov 6, 2026

Projected Neural Additive Models as Universal Approximators

Authors: Nhon N. Phan, Qiang Du, John D. Clayton, WaiChing Sun

Organizations: Terminal Effects Division, DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD 21005, USA · Department of Civil Engineering & Engineering Mechanics, and Data Science Institute, Columbia University, New York, NY 10027, USA

Abstract

Neural additive models (NAMs) offer interpretability but lack expressivity due to their rigid additive structure, whereas multi-layer perceptrons (MLPs) achieve universal approximation at the cost of transparency. We establish the first theoretical foundation for projected neural additive models (PNAMs), which augment NAMs with a learnable linear transformation T\boldsymbol{T} to capture cross-variable interactions. While standard NAMs cannot approximate even the simplest interaction term χ1χ2χ_1 χ_2 because their cross-derivatives are identically zero, we prove that PNAMs are universal approximators and characterize the minimum number of projections required for exact polynomial representation. Furthermore, we derive an approximation rate of O(M−s/(N−1))\mathcal{O}(M^{-s / (N - 1)}) for Cs\mathcal{C}^s functions (i.e., functions with ss continuous derivatives), where MM denotes the number of projections and NN the input dimension; this rate is known to be optimal for ridge function approximation. To recover interpretability from the projected inputs, we introduce structured sparsity regularizers and post-hoc symbolic regression techniques that enable global input ranking, parameter pruning, and the conversion of learned bases into compact analytical expressions. Experiments on knot theory invariants, MNIST, and phase field fracture mechanics demonstrate that PNAMs match or surpass similar architectures in accuracy while discovering salient features and yielding symbolic models with orders of magnitude fewer parameters.

Figures & tables

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Do Feature Interactions Break Explainability? Provable Explanations for Any-Order Neural Additive Models

    Nov 8, 2026Idan Refaeli, Yizhak Yisrael Elboher, Guy Katz +1Neural Network InterpretabilityFeature Interaction Modeling

  2. Provably Explaining Neural Additive Models

    Feb 19, 2026Shahaf Bassan, Yizhak Yisrael Elboher, Tobias Ladner +4Interpretable MLNeural Network Interpretability

  3. Neural Additive and Basis Models with Feature Selection and Interactions

    Jun 18, 2026Yasutoshi Kishimoto, Kota Yamanishi, Takuya Matsuda +1Feature Interaction ModelingNeural Network Interpretability