Overview

Cost-aware inference studies how to deliver useful predictions while limiting the resources spent on each request. Common approaches route easy cases to smaller models and escalate uncertain ones, adapt reasoning or verification effort to task difficulty and budget, and optimize caching, precision, batching, and hardware placement. Researchers evaluate these methods using quality-cost frontiers, latency and energy measurements, and deployment workloads, with growing attention to agents whose costs accumulate across many steps. A central challenge is that savings depend on workload and infrastructure: fewer tokens do not always mean lower billed cost, and routing gains can disappear when quality estimates, calibration, or operating conditions shift.

Latest papers

CardsList

308 papers24 papers in the last four weeks, down 49% on the four weeks before. 0.2% of all new papers.

  1. ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition

    Jan 7, 2026Muyang Zhao, Chunhao Tian, Qi Qi +1Adaptive Reasoning Resource Allocation for Language ModelsCost-Aware Inference

  2. Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement

    Dec 5, 2025Nils Strassenburg, Boris Glavic, Tilmann RablCost-Aware InferenceSmall Language Models

  3. PAC Reasoning: Controlling the Performance Loss for Efficient Reasoning

    Oct 10, 2025Hao Zeng, Jianguo Huang, Bingyi Jing +2Efficient Language Model ReasoningAdaptive Reasoning in Language Models

  4. Contrastive Nonmyopic Objective Cost-Tradeoff Acquisition for Longitudinal Data

    Jul 16, 2025Dzung Dinh, Boqi Chen, Yunni Qu +2Active Feature AcquisitionCost-Aware Inference

  5. PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution

    Jul 2, 2025Omkar Shende, Gayathri Ananthanarayanan, Marcello TraiolaEfficient Neural Network InferenceAdaptive Inference

  6. Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

    Mar 31, 2025Rui Wang, Hongru Wang, Boyang Xue +9LLM Inference EfficiencyCost-Aware Inference

  7. The Revenue Cost of Inference in Online Assortment Optimization

    Jan 31, 2025Jierui Zuo, Hanzhang QinAssortment OptimizationCost-Aware Inference