Input Compression

Input compression aims to reduce the computational burden of machine learning models by decreasing the size of input data without significant performance loss. Current research focuses on developing efficient compression techniques, including methods based on quasi-Monte Carlo point sets, dynamic networks with early exiting and quantization, and data augmentation strategies like positional consistency for transformers. These advancements are crucial for deploying large models on resource-constrained devices and improving training efficiency, impacting fields ranging from edge computing to large-scale model training.

Papers

September 20, 2024

Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning
Michael Gnewuch, Kumar Harsha, Marcin Wnuk
Machine Learning Loss Function Parameter Estimation Data Compression Distributive Lattice Quasi Monte Carlo Input Compression

June 20, 2024

Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE
Florence Regol, Joud Chataoui, Bertrand Charpentier, Mark Coates, Pablo Piantanida, Stephan Gunnemann
Quantization Operator Error Feedback Dynamic Network Early eXit Post Training Predictive Probability Input Compression

November 22, 2023

Input Compression with Positional Consistency for Efficient Training and Inference of Transformer Neural Networks
Amrit Nagarajan, Anand Raghunathan
Scientific Inference Transformer Architecture Efficient Training Temporal Resolution Positional Information Input Compression

November 8, 2022

Efficient Compressed Ratio Estimation Using Online Sequential Learning for Edge Computing
Hiroki Oikawa, Hangli Ge, Noboru Koshizuka
Edge Computing Compressed Sensing Compression Technique Sequential Learning Extreme Learning Machine Compression Efficiency Input Compression

August 5, 2022

Compressing (Multidimensional) Learned Bloom Filters
Angjela Davitkova, Damjan Gjurovski, Sebastian Michel
Deep Learning Model Bloom Filter Input Compression

July 19, 2022

Bounding generalization error with input compression: An empirical study with infinite-width networks
Angus Galloway, Anna Golubeva, Mahmoud Salem, Mihai Nica, Yani Ioannou, Graham W. Taylor
Deep Neural Network Empirical Study Generalization Error DNN Architecture Generalization Error Bound Infinite Width Neural Network Width Network Input Compression

November 7, 2021

Use of 1D-CNN for input data size reduction of LSTM in Hourly Rainfall-Runoff modeling
Kei Ishida, Ali Ercan, Takeyoshi Nagasato, Masato Kiyama, Motoki Amagasaki
Long Short Term Memory Greater Public Use CNN LSTM Input Compression

Input Compression

Papers

Data Compression using Rank-1 Lattices for Parameter Estimation in Machine Learning

Predicting Probabilities of Error to Combine Quantization and Early Exiting: QuEE

Input Compression with Positional Consistency for Efficient Training and Inference of Transformer Neural Networks

Efficient Compressed Ratio Estimation Using Online Sequential Learning for Edge Computing

Compressing (Multidimensional) Learned Bloom Filters

Bounding generalization error with input compression: An empirical study with infinite-width networks

Use of 1D-CNN for input data size reduction of LSTM in Hourly Rainfall-Runoff modeling