Fast Inference

Fast inference in machine learning aims to accelerate the process of obtaining predictions from complex models, addressing the computational bottleneck hindering the deployment of powerful models like large language models and vision transformers. Current research focuses on techniques such as speculative decoding, model compression (including pruning and quantization), and architectural innovations like mixture-of-experts and hierarchical attention mechanisms to achieve speedups. These advancements are crucial for deploying sophisticated AI models in resource-constrained environments and real-time applications, impacting fields ranging from natural language processing and computer vision to astrophysics and robotics.

Papers

May 31, 2022

SymFormer: End-to-end symbolic regression using transformer-based architecture
Martin Vastl, Jonáš Kulhánek, Jiří Kubalík, Erik Derner, Robert Babuška
Gradient Descent Symbolic Regression Transformer Based Approach Fast Inference

May 26, 2022

Consistent and fast inference in compartmental models of epidemics using Poisson Approximate Likelihoods
Michael Whitehouse, Nick Whiteley, Lorenzo Rimella
Fast Inference Disease Outbreak Compartmental Model Approximate Likelihood

May 25, 2022

Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization
Sungryull Sohn, Hyunjae Woo, Jongwook Choi, lyubing qiang, Izzeddin Gur, Aleksandra Faust, Honglak Lee
Formality Transfer Hierarchical Reinforcement Learning Meta Reinforcement Learning Fast Inference Task Inference Task Graph Shot Generalization Task Structure

January 21, 2022

Unity Smoothing for Handling Inconsistent Evidence in Bayesian Networks and Unity Propagation for Faster Inference
Mads Lindskou, Torben Tvedebrink, Poul Svante Eriksen, Søren Højsgaard, Niels Morling
Bayesian Network Fast Inference Tree Based Inconsistency Detection Laplacian Learning Hierarchical Propagation

January 8, 2022

Low-Rank Constraints for Fast Inference in Structured Models
Justin T. Chiu, Yuntian Deng, Alexander M. Rush
Neural Network Participation Constraint Fast Inference Probabilistic Context Free Grammar Structured Distribution Probabilistic Representation Structured Model

November 11, 2021

On Efficient Uncertainty Estimation for Resource-Constrained Mobile Applications
Johanna Rock, Tiago Azevedo, René de Jong, Daniel Ruiz-Muñoz, Partha Maji
Uncertainty Estimation Deep Ensemble Predictive Uncertainty Fast Inference Efficient Uncertainty Estimation Robust Uncertainty

Fast Inference

Papers

SymFormer: End-to-end symbolic regression using transformer-based architecture

Consistent and fast inference in compartmental models of epidemics using Poisson Approximate Likelihoods

Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

Unity Smoothing for Handling Inconsistent Evidence in Bayesian Networks and Unity Propagation for Faster Inference

Low-Rank Constraints for Fast Inference in Structured Models

On Efficient Uncertainty Estimation for Resource-Constrained Mobile Applications