Sparse Polynomial

Sparse polynomial approximation focuses on efficiently representing complex functions using only a small subset of polynomial terms, aiming to reduce computational cost and improve generalization. Current research explores efficient algorithms for constructing these sparse representations, including techniques based on sketching, neural networks (particularly two-layer networks and deep Gaussian processes), and evolutionary methods. These advancements are impacting diverse fields, from accelerating large language model inference and improving scientific simulations (e.g., of spallation targets) to enhancing robust statistical methods like mean estimation. The overall goal is to develop methods that achieve high accuracy with significantly reduced computational complexity compared to dense polynomial representations.

Papers

August 16, 2024

Optimal Sketching for Residual Error Estimation for Matrix and Vector Norms
Yi Li, Honghao Lin, David P. Woodruff
Low Rank Approximation Sparse Matrix Terminator Economy P$ Norm Matrix Sketching Residual Error Sparse Polynomial Rank $K$ Approximation

May 24, 2024

Sparse Expansion and Neuronal Disentanglement
Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir Shavit
Sparsity Increase Sparse Expert Sparse Polynomial

April 4, 2024

Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks
Ben Adcock, Simone Brugiapaglia, Nick Dexter, Sebastian Moraga
Deep Neural Network High Dimension Approximation Theory Smooth Function Sparse Approximation Dimensional Approximation Data Driven Approximation Sparse Polynomial

February 14, 2024

Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input
Ziang Chen, Rong Ge
Stochastic Gradient Descent Mean Field Sparse Polynomial Gaussian Input

June 8, 2022

Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials
Eshaan Nichani, Yu Bai, Jason D. Lee
Neural Tangent Kernel Interpretable Direction Polynomial System Sparse Polynomial

May 28, 2022

List-Decodable Sparse Mean Estimation
Shiwei Zeng, Jie Shen
Sample Complexity List Decodable Robust Mean Estimation Sparse Polynomial

February 18, 2022

Model Calibration of the Liquid Mercury Spallation Target using Evolutionary Neural Networks and Sparse Polynomial Expansions
Majdi I. Radaideh, Hoang Tran, Lianshan Lin, Hao Jiang, Drew Winder, Sarma Gorti, Guannan Zhang, Justin Mach, Sarah Cousineau
Neural Network Model Calibration Constitutive Model Target Spacecraft Sparse Polynomial Cavitation Intensity Recognition

December 11, 2021

A Sparse Expansion For Deep Gaussian Processes
Liang Ding, Rui Tuo, Shahin Shahrampour
Gaussian Process Deep Gaussian Process Sparse Polynomial