Sparse Approximation

Sparse approximation focuses on representing data using a minimal number of non-zero components, aiming to reduce computational complexity and improve efficiency in various applications. Current research emphasizes developing faster algorithms like improved Orthogonal Matching Pursuit and leveraging sparse structures within models such as Gaussian Processes and neural networks (including Binary Neural Networks and Vision Transformers) for enhanced scalability and accuracy. This field is crucial for addressing computational bottlenecks in machine learning, signal processing, and scientific computing, enabling the analysis of high-dimensional data and the deployment of complex models on resource-constrained devices.

Papers

May 27, 2024

Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization
Jianting Yang, Srećko Ðurašinović, Jean-Bernard Lasserre, Victor Magron, Jun Zhao
Adversarial Attack Native Robustness Binary Neural Network Sparse Approximation Semidefinite Relaxation

April 4, 2024

Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks
Ben Adcock, Simone Brugiapaglia, Nick Dexter, Sebastian Moraga
Deep Neural Network High Dimension Approximation Theory Smooth Function Sparse Approximation Dimensional Approximation Data Driven Approximation Sparse Polynomial

March 29, 2024

Fast OMP for Exact Recovery and Sparse Approximation
Huiyuan Yu, Jia He, Maggie Cheng
Sparse Representation Sparse Signal Sparse Approximation Orthogonal prOjection Exact Community Recovery Orthogonal Matching Pursuit

February 20, 2024

Order-Optimal Regret in Distributed Kernel Bandits using Uniform Sampling with Shared Randomness
Nikola Pavlovic, Sudeep Salgia, Qing Zhao
Kernel Hilbert Space Centralized Learning Uniform Sampling Sparse Approximation Kernel Bandit Order Optimal Regret Distributed Stochastic

January 20, 2024

Accelerating Sinkhorn Algorithm with Sparse Newton Iterations
Xun Tang, Michael Shavlovsky, Holakou Rahmanian, Elisa Tardini, Kiran Koshy Thekumparampil, Tesi Xiao, Lexing Ying
Hessian Matrix Sinkhorn Algorithm Sparse Approximation Linear Convergence Rate Semismooth Newton

November 28, 2023

Gaussian Processes for Monitoring Air-Quality in Kampala
Clara Stoddart, Lauren Shrack, Richard Sserunjogi, Usman Abdul-Ganiy, Engineer Bainomugisha, Deo Okure, Ruth Misener, Jose Pablo Folch, Ruby Sedgwick
Gaussian Process Air Quality Kernel Function Air Pollution Sparse Approximation Efficient Sparse

October 21, 2023

Fast Approximation of Similarity Graphs with Kernel Density Estimation
Peter Macgregor, He Sun
Kernel Density Efficient Approximation Sparse Approximation Graph Similarity

August 25, 2023

Network Embedding Using Sparse Approximations of Random Walks
Paula Mercurio, Di Liu
Stochastic Gradient Descent Network Programming Node Embeddings Random Walk Low Dimensional Representation Sparse Approximation Wavelet Diffusion

July 28, 2023

Weighted variation spaces and approximation by shallow ReLU networks
Ronald DeVore, Robert D. Nowak, Rahul Parhi, Jonathan W. Siegel
Average Approximation Shallow ReLU Neural Network Approximation ReLU Neural Network Sparse Approximation Variation Space

June 20, 2023

LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Yixiao Li, Yifan Yu, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao
Language Understanding Transformer Model Model Compression Low Rank Approximation Sparse Approximation

June 12, 2023

Generative Plug and Play: Posterior Sampling for Inverse Problems
Charles A. Bouman, Gregery T. Buzzard
Inverse Problem Posterior Sampling Plug and Play Tomographic Reconstruction Play Style Sparse Approximation

April 25, 2023

Explain like I am BM25: Interpreting a Dense Model's Ranked-List with a Sparse Approximation
Michael Llordes, Debasis Ganguly, Sumit Bhatia, Chirag Agarwal
Local Explanation Dense Model Query Expansion Neural Information Retrieval Ranking Task Sparse Approximation Sparse Retrieval

November 9, 2022

ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attention
Jyotikrishna Dass, Shang Wu, Huihong Shi, Chaojian Li, Zhifan Ye, Zhongfeng Wang, Yingyan Lin
Vision Transformer Low Rank Sparse Attention Attention Matrix Sparse Approximation

November 1, 2022

Batch Active Learning from the Perspective of Sparse Approximation
Maohao Shen, Bowen Jiang, Jacky Yibo Zhang, Oluwasanmi Koyejo
Active Learning Visual Perspective Model Training Sparsity Constraint Sparse Approximation Batch Active Learning

August 23, 2022

Sparse Polynomial Optimization: Theory and Practice
Victor Magron, Jie Wang
Theoretical Understanding Practice Mode Nonconvex Optimization Sparse Approximation Polynomial Optimization Polynomial Bound

July 30, 2022

Learning robot inverse dynamics using sparse online Gaussian process with forgetting mechanism
Wei Li, Zhiwen Li, Yiqi Liu, Yongping Pan
Functional Mechanism Sparse Gaussian Process Sparse Approximation Inverse Dynamic Sparse GP

April 10, 2022

Gaussian Processes for Missing Value Imputation
Bahram Jafrasteh, Daniel Hernández-Lobato, Simón Pedro Lubián-López, Isabel Benavente-Fernández
Gaussian Process Imputation Algorithm Missing Value Sparse Approximation Missing Value Imputation

March 18, 2022

Convolutional Simultaneous Sparse Approximation with Applications to RGB-NIR Image Fusion
Farshad G. Veshki, Sergiy A. Vorobyov
Financial Application Deep Convolutional Neural Network Image Fusion Sparse Approximation Sparse Vector Convolutional Sparse Sparsity Structure NIR Fusion

February 8, 2022

Improved Convergence Rates for Sparse Approximation Methods in Kernel-Based Learning
Sattar Vakili, Jonathan Scarlett, Da-shan Shiu, Alberto Bernacchia
Gaussian Process Convergence Rate Kernel Ridge Regression Kernel Based Sparse Approximation Sparse Variational Gaussian Process

February 3, 2022

Variational Nearest Neighbor Gaussian Process
Luhuan Wu, Geoff Pleiss, John Cunningham
Gaussian Process Variational Method Variational Approximation Sparse Approximation Variational Gaussian Process