Polynomial Network

Polynomial networks (PNs) are a class of neural networks that utilize polynomial activation functions instead of traditional activation functions like ReLU or sigmoid, offering potential advantages in interpretability and theoretical analysis. Current research focuses on improving PN performance to match or exceed that of standard deep neural networks, exploring architectures like multilinear operator networks and employing regularization techniques to enhance accuracy and robustness. This research aims to bridge the performance gap between PNs and established deep learning methods, potentially leading to more explainable and theoretically grounded machine learning models with applications in diverse fields such as image recognition and click-through rate prediction.

Papers

November 17, 2024

RPN 2: On Interdependence Function Learning Towards Unifying and Advancing CNN, RNN, GNN, and Transformer
Jiawei Zhang
Transformer Based Gene Level GNN CNN Network \Sigma}{\Delta}$ Low Pas RNN Backbone Fine Tuning Interdependent Network Polynomial Network Structure Function Racah Polynomial

October 21, 2024

Karush-Kuhn-Tucker Condition-Trained Neural Networks (KKT Nets)
Shreya Arvind, Rishabh Pomaje, Rajshekhar V Bhat
Neural Network Loss Function Convex Optimization Dual Formulation Polynomial Network

July 5, 2024

RPN: Reconciled Polynomial Network Towards Unifying PGMs, Kernel SVMs, MLP and KAN
Jiawei Zhang
Kolmogorov Arnold Network Markov Chain Probabilistic Graphical Model Multi Layer Perceptron Novel Deep Learning Model Polynomial Network Deep Function

January 31, 2024

Multilinear Operator Networks
Yixin Cheng, Grigorios G. Chrysos, Markos Georgopoulos, Volkan Cevher
Deep Neural Network Image Recognition Deep Operator Polynomial Network

March 24, 2023

Regularization of polynomial networks for image recognition
Grigorios G Chrysos, Bohan Wang, Jiankang Deng, Volkan Cevher
Deep Neural Network Image Recognition Neural Network Baseline Polynomial Network

January 3, 2023

September 16, 2022

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study
Yongtao Wu, Zhenyu Zhu, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
Neural Network Neural Tangent Kernel Neural Network Architecture Extrapolation Framework Spectral Bias Standard Deep Polynomial Neural Network Hadamard Layer Polynomial Network

September 15, 2022

Sound and Complete Verification of Polynomial Networks
Elias Abad Rocamora, Mehmet Fatih Sahin, Fanghui Liu, Grigorios G Chrysos, Volkan Cevher
Verification Task Sound Design Global Convergence Guarantee Polynomial Network

February 10, 2022

Controlling the Complexity and Lipschitz Constant improves polynomial nets
Zhenyu Zhu, Fabian Latorre, Grigorios G Chrysos, Volkan Cevher
Adversarial Perturbation Complexity Matter Robustness Guarantee Lipschitz Bound Polynomial Network CP Decomposition

November 4, 2021

Rate of Convergence of Polynomial Networks to Gaussian Processes
Adam Klukowski
Gaussian Process Early Stage Convergence Convergence Rate Universal Rate Random Weight Polynomial Activation Hidden Neuron Polynomial Network

Polynomial Network

Papers

RPN 2: On Interdependence Function Learning Towards Unifying and Advancing CNN, RNN, GNN, and Transformer

Karush-Kuhn-Tucker Condition-Trained Neural Networks (KKT Nets)

RPN: Reconciled Polynomial Network Towards Unifying PGMs, Kernel SVMs, MLP and KAN

Multilinear Operator Networks

Regularization of polynomial networks for image recognition

xDeepInt: a hybrid architecture for modeling the vector-wise and bit-wise feature interactions

DynInt: Dynamic Interaction Modeling for Large-scale Click-Through Rate Prediction

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Sound and Complete Verification of Polynomial Networks

Controlling the Complexity and Lipschitz Constant improves polynomial nets

Rate of Convergence of Polynomial Networks to Gaussian Processes