Inference Framework

Inference frameworks encompass methods for efficiently extracting information and making predictions from complex models, primarily focusing on optimizing computational resources and improving accuracy. Current research emphasizes scaling inference compute through techniques like repeated sampling, sparse attention mechanisms, and efficient model architectures such as Mixture-of-Experts (MoE), aiming to balance speed and accuracy across diverse applications. These advancements are crucial for deploying large language models and other computationally intensive AI systems in resource-constrained environments and for improving the efficiency and reliability of AI-driven decision-making.

40papers

Papers

April 1, 2025

When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning
Inference Cost LLM Reasoning Optimal Algorithm Inference Framework

March 21, 2025

Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models
Full Model Inference Framework Large Language Large Language Model

March 6, 2025

AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services
Optimization Purpose Language Model Large Language Model General Analysis Driven Inference LLM Inference Inference Framework

February 27, 2025

Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners
Pre Trained Transformer Inference Framework Reasoning Datasets Chain of Thought Scalable Inference Contrastive Reasoner Teacher Model

February 22, 2025

Inference Computation Scaling for Feature Augmentation in Recommendation Systems
Large Language Model Effective Recommendation Recommendation System Recommendation Performance System Description Inference Framework Inference Efficiency Feature Augmentation

February 16, 2025

Diversified Sampling Improves Scaling LLM inference
Class Disjoint Data Sample Diversity LLM Inference Large Language Model Inference Framework

February 12, 2025

Inference Scaling Reshapes AI Governance
Inference Framework Artificial Intelligence Governance Strong Scaling

February 11, 2025

Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design
Bridging Text Scientific Inference Experimental Design Inference Framework Mutual Information Variational Inference Simulation Based Inference

February 8, 2025

ARIES: Stimulating Self-Refinement of Large Language Models by Iterative Preference Optimization
Inference Framework Self Refinement Large Language Model Preference Optimization Iterative Preference Learning

January 12, 2025

Unifying Two Types of Scaling Laws from the Perspective of Conditional Kolmogorov Complexity
Lossless Compression Kolmogorov Complexity Inference Performance Conditional Reasoning Visual Perspective Different Type Scaling Law Inference Framework

December 22, 2024

Multi-Agent Sampling: Scaling Inference Compute for Data Synthesis with Tree Search-Based Agentic Collaboration
Single Agent Scalable Inference Inference Framework Multi Agent Collaboration Model Collaboration Data Synthesis

December 19, 2024

OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving
Multimodal Large Language Model End to End Autonomous Driving Inference Framework End to End Model Autonomous Driving

December 6, 2024

November 30, 2024

QuAKE: Speeding up Model Inference Using Quick and Approximate Kernels for Exponential Non-Linearities
Computational Efficiency Kernel Approximation Seismic Activity Inference Framework Model Inference

October 25, 2024

Efficient Biological Data Acquisition through Inference Set Design
Active Learning Uncertainty Based Active Learning Drug Discovery Inference Framework Data Acquisition

October 12, 2024

Inference and Verbalization Functions During In-Context Learning
Scientific Inference Inference Framework Context Learning Large Language Model

October 6, 2024

September 13, 2024

An Intent Modeling and Inference Framework for Autonomous and Remotely Piloted Aerial Systems
Intent Prediction Intent Classifier Human Intention Legal Autonomy Intent Detection Inference Framework

Inference Framework

Papers

When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning

Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models

AOLO: Analysis and Optimization For Low-Carbon Oriented Wireless Large Language Model Services

Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners

Inference Computation Scaling for Feature Augmentation in Recommendation Systems

Diversified Sampling Improves Scaling LLM inference

Inference Scaling Reshapes AI Governance

Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design

ARIES: Stimulating Self-Refinement of Large Language Models by Iterative Preference Optimization

Unifying Two Types of Scaling Laws from the Perspective of Conditional Kolmogorov Complexity

Multi-Agent Sampling: Scaling Inference Compute for Data Synthesis with Tree Search-Based Agentic Collaboration

OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving

GUIDE: A Global Unified Inference Engine for Deploying Large Language Models in Heterogeneous Environments

Slicing Vision Transformer for Flexible Inference

QuAKE: Speeding up Model Inference Using Quick and Approximate Kernels for Exponential Non-Linearities

Efficient Biological Data Acquisition through Inference Set Design

Inference and Verbalization Functions During In-Context Learning

Distributed Inference on Mobile Edge and Cloud: An Early Exit based Clustering Approach

Inference Scaling for Long-Context Retrieval Augmented Generation

An Intent Modeling and Inference Framework for Autonomous and Remotely Piloted Aerial Systems