Model Inference

Model inference, the process of using a trained machine learning model to make predictions, is a critical area of research focusing on improving efficiency, accuracy, and robustness. Current efforts concentrate on mitigating issues like hallucinations in large language and vision-language models, optimizing resource allocation for both inference and retraining (especially in edge computing scenarios), and enhancing privacy and security during inference. These advancements are crucial for deploying machine learning models effectively in various applications, ranging from real-time IoT systems to large-scale data analysis, while addressing challenges related to computational cost, data heterogeneity, and model interpretability.

Papers

August 8, 2023

Federated Inference with Reliable Uncertainty Quantification over Wireless Channels via Conformal Prediction
Meiyi Zhu, Matteo Zecchin, Sangwoo Park, Caili Guo, Chunyan Feng, Osvaldo Simeone
Conformal Prediction Exact Inference Model Inference Wireless Channel Noisy Communication

June 30, 2023

The Integer Linear Programming Inference Cookbook
Vivek Srikumar, Dan Roth
Integer Programming Inference Problem Model Inference

May 10, 2023

Fast Distributed Inference Serving for Large Language Models
Bingyang Wu, Yinmin Zhong, Zili Zhang, Shengyu Liu, Fangyue Liu, Yuanhang Sun, Gang Huang, Xuanzhe Liu, Xin Jin
Large Language Model Scientific Inference LLM Inference Model Inference Inference Service

April 7, 2023

Profiling the news spreading barriers using news headlines
Abdul Sittar, Dunja Mladenic, Marko Grobelnik
News Article Knowledge Barrier Semantic Structure Important News Model Inference Commonsense Inference Traditional Text Classification

March 16, 2023

PyVBMC: Efficient Bayesian inference in Python
Bobby Huggins, Chengkun Li, Marlon Tobaben, Mikko J. Aarnos, Luigi Acerbi
Model Inference Variational Monte Carlo Bayesian Model Selection Efficient Bayesian Inference

March 2, 2023

Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decision
Siyuan Yan, Zhen Yu, Xuelin Zhang, Dwarikanath Mahapatra, Shekhar S. Chandra, Monika Janda, Peter Soyer, Zongyuan Ge
Deep Neural Network Full Model Black Box Model Skin Lesion Decision Relevant Information Model Inference Melanoma Diagnosis

December 20, 2022

Improving the Robustness of Summarization Models by Detecting and Removing Input Noise
Kundan Krishna, Yao Zhao, Jie Ren, Balaji Lakshminarayanan, Jiaming Luo, Mohammad Saleh, Peter J. Liu
Native Robustness Summarization Model Model Inference Abstractive Summarization Model Noisy Input

November 30, 2022

A Probabilistic-Logic based Commonsense Representation Framework for Modelling Inferences with Multiple Antecedents and Varying Likelihoods
Shantanu Jaiswal, Liu Yan, Dongkyu Choi, Kenneth Kwok
Knowledge Graph Commonsense Knowledge Key Factor Model Inference Marginal Likelihood Large Knowledge Graph Commonsense Knowledge Graph

September 28, 2022

July 23, 2022

RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
Baolin Li, Rohan Basu Roy, Tirthak Patel, Vijay Gadepally, Karen Gettings, Devesh Tiwari
Scientific Inference Inference Cost Cost Effective Multiple Instance Model Inference Inference Service Diverse Body

May 26, 2022

Deep Active Learning with Noise Stability
Xingjian Li, Pengkun Yang, Yangcheng Gu, Xueying Zhan, Tianyang Wang, Min Xu, Chengzhong Xu
Active Learning Uncertainty Estimation Deep Active Learning Data Uncertainty Model Inference

November 29, 2021

SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning
Sixing Yu, Phuong Nguyen, Waqwoya Abebe, Wei Qian, Ali Anwar, Ali Jannesari
Transfer Learning Federated Learning Model Convergence Heterogeneous Client Model Inference Parameter Selection Local Inference

Model Inference

Papers

Federated Inference with Reliable Uncertainty Quantification over Wireless Channels via Conformal Prediction

The Integer Linear Programming Inference Cookbook

Fast Distributed Inference Serving for Large Language Models

Profiling the news spreading barriers using news headlines

PyVBMC: Efficient Bayesian inference in Python

Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decision

Improving the Robustness of Summarization Models by Detecting and Removing Input Noise

A Probabilistic-Logic based Commonsense Representation Framework for Modelling Inferences with Multiple Antecedents and Varying Likelihoods

MLink: Linking Black-Box Models from Multiple Domains for Collaborative Inference

InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference

RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances

Deep Active Learning with Noise Stability

SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning