Inference Scheme

Inference schemes encompass methods for deriving conclusions from data or model outputs, with key objectives including improving model efficiency, enhancing reasoning capabilities, and addressing privacy concerns in machine learning. Current research focuses on developing novel inference algorithms, such as contrastive learning for improved inductive reasoning in dialogue and self-speculative decoding for faster large language model processing, alongside optimization techniques like convex and bilevel optimization for neuro-symbolic systems. These advancements are significant for improving the performance and efficiency of various machine learning applications, while also mitigating risks associated with data privacy and model vulnerabilities.

Papers

October 31, 2024

Inclusive KL Minimization: A Wasserstein-Fisher-Rao Gradient Flow Perspective
Jia-Jie Zhu
Visual Perspective KL Divergence Fisher Rao Gradient Flow Inference Scheme

June 4, 2024

Inference Attacks: A Taxonomy, Survey, and Promising Directions
Feng Wu, Lei Cui, Shaowen Yao, Shui Yu
Timely Survey Comprehensive Taxonomy Interpretable Direction Inference Attack Privacy Inference Privacy Breach Inference Scheme

April 12, 2024

AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees
William Fleshman, Aleem Khan, Marc Marone, Benjamin Van Durme
Large Language Model Scientific Inference Fine Grained Control Continual Training Efficient Continual Learning Long Short Range Adapter Data Removal Inference Scheme

January 17, 2024

Convex and Bilevel Optimization for Neuro-Symbolic Inference and Learning
Charles Dickens, Changyu Gao, Connor Pryor, Stephen Wright, Lise Getoor
LeArning Abstract Gradient Based Bilevel Optimization Block Coordinate Descent Inference Scheme Gradient Based Parameter Inference Module

October 19, 2023

Contrastive Learning for Inference in Dialogue
Etsuko Ishii, Yan Xu, Bryan Wilie, Ziwei Ji, Holy Lovenia, Willy Chung, Pascale Fung
Contrastive Learning Scientific Inference Dialogue Utterance Inference Task Inductive Reasoning Semantic Gap Inductive Inference Inference Scheme

September 15, 2023

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, Sharad Mehrotra
Large Language Model Speculative Decoding Inference Scheme K$ Draft

September 8, 2023

Unsupervised Multi-document Summarization with Holistic Inference
Haopeng Zhang, Sangwoo Cho, Kaiqiang Song, Xiaoyang Wang, Hongwei Wang, Jiawei Zhang, Dong Yu
Multi Document Summarization Inference Scheme

June 12, 2023

Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models
Soochan Lee, Gunhee Kim
Language Model Thought Reasoning Recursive Algorithm Divide and Conquer Context Reasoning Multi Step Reasoning Capability Inference Capability Inference Scheme Contextual Relationship

May 22, 2023

SCL(FOL) Can Simulate Non-Redundant Superposition Clause Learning
Martin Bromberger, Chaahat Jain, Christoph Weidenbach
First Order Logic First Order Superposition Inference Scheme Conflict Driven Clause Learning

December 15, 2022

Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability
Parvin Malekzadeh, Konstantinos N. Plataniotis
Reinforcement Learning Markov Decision Process Action Space Active Inference Partial Observability Reward Maximization Continuous POMDP Inference Scheme

November 24, 2022

Data Origin Inference in Machine Learning
Mingxue Xu, Xiang-Yang Li
Machine Learning Data Source Model Development Inference Scheme

November 20, 2022

UniMASK: Unified Inference in Sequential Decision Problems
Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin
Language Model Sequential Decision Sequential Decision Making Problem Single Task Inference Scheme