Input Sparsity

Input sparsity research focuses on efficiently processing large datasets by leveraging the inherent sparsity—the presence of many zero or negligible values—within the data. Current efforts concentrate on developing algorithms that achieve input sparsity time complexity, meaning computational cost scales linearly with the number of non-zero entries, for tasks like tensor low-rank approximation and solving regularized regression problems arising in large language models and video question answering. This is achieved through techniques such as adaptive input selection, leveraging score sampling, and approximate Newton methods. These advancements promise significant improvements in computational efficiency and scalability for various machine learning applications, particularly those involving high-dimensional data.

Papers

November 27, 2023

Characterizing Video Question Answering with Sparsified Inputs
Shiyuan Huang, Robinson Piramuthu, Vicente Ordonez, Shih-Fu Chang, Gunnar A. Sigurdsson
Video Question Answering Video Question VideoQA Benchmark Video Language Task Sparse Motion Input Sparsity

April 13, 2023

Solving Tensor Low Cycle Rank Approximation
Yichuan Deng, Yeqi Gao, Zhao Song
Large Language Model Low Rank Approximation Probabilistic Context Free Grammar Tensor Rank Tensor Approximation Input Sparsity

March 28, 2023

Solving Regularized Exp, Cosh and Sinh Regression Problems
Zhihang Li, Zhao Song, Tianyi Zhou
Regularization Model Regression Problem Input Sparsity

February 9, 2022

Leverage Score Sampling for Tensor Product Matrices in Input Sparsity Time
David P. Woodruff, Amir Zandieh
Kronecker Product Tensor Product Gram Matrix Leverage Score Sampling Polynomial Kernel Input Sparsity

Input Sparsity

Papers

Characterizing Video Question Answering with Sparsified Inputs

Solving Tensor Low Cycle Rank Approximation

Solving Regularized Exp, Cosh and Sinh Regression Problems

Leverage Score Sampling for Tensor Product Matrices in Input Sparsity Time