Adaptive Inference

Follow this topic Ask about this topic

Adaptive inference studies how to change the amount or form of computation according to each input, the system’s current resources, or the data collected so far. Common approaches include early-exit networks, confidence-based cascades and model routing, dynamic depth or resolution, and adaptive reasoning or sampling under a compute budget. In statistics, related work develops calibration, conformal prediction, and asymptotic guarantees for estimates produced by adaptive experiments such as contextual bandits. The central tension is whether signals like confidence or difficulty reliably identify when extra computation helps, especially under distribution shift and when efficiency gains must not compromise accuracy, coverage, or safety.

Latest papers

95 papers10 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning
Jan 12, 2026Chenjie Hao, Weyl Lu, Yuko Ishiwaka +3Adaptive Inference Confidence Estimation in Language Models
University of California, Davis · SoftBank Corp. · Aizip
Blind Quality Enhancement of Compressed Video via Fine-Grained Degradation-Guided Sequential Inference
Nov 20, 2025Li Yu, Yingbo Zhao, Shiyu Wu +3Adaptive Inference Representation Learning
School of Computer Science, Nanjing University of Information Science & Technology, Nanjing, China, and also with Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science & Technology, Nanjing, China · School of Computer Science, Nanjing University of Information Science & Technology, Nanjing, China · School of Software, Nanjing University of Information Science & Technology, Nanjing, China +3
Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation
Oct 30, 2025Georgios Kamaras, Craig Innes, Subramanian RamamoorthySimulation-Based Inference Adaptive Inference
School of Informatics, The University of Edinburgh, EH8 9AB, UK. · Heriot-Watt University · UKRI Turing AI World Leading Researcher Fellowship on AI for Person-Centred and Teachable Autonomy
PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution
Jul 2, 2025Omkar Shende, Gayathri Ananthanarayanan, Marcello TraiolaEfficient Neural Network Inference Adaptive Inference
Department of Computer Science and Engineering, Indian Institute of Technology Dharwad, Karnataka, India · University of Rennes, CNRS, Inria, IRISA, Rennes, France
No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference
May 26, 2025Pranav Mani, Peng Xu, Zachary C. Lipton +1Adaptive Inference Statistical Learning Theory
Abridge AI, San Francisco, CA, USA · Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

Jan 12, 2026J/K move · Enter open · S save

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

Chenjie Hao, Weyl Lu, Yuko Ishiwaka, Zengyi Li+2

University of California, Davis · SoftBank Corp. · Aizip

Adaptive Inference Confidence Estimation in Language Models Training Data Curation Model Calibration Uncertainty Calibration

Open paper AskPDF arXiv

When a model knows when it does not know, many possibilities emerge. The first question is how to enable a model to recognize that it does not know. A promising approach is to use confidence, computed from the model's internal signals, to reflect its ignorance. Prior work in specific domains has shown that calibration can provide reliable confidence estimates. In this work, we propose a simple, effective, and universal training-free method that applies to both vision and language models, performing model calibration, cascading, and data cleaning to better exploit a model's ability to recognize when it does not know. We first highlight two key empirical observations: higher confidence corresponds to higher accuracy within a single model, and models calibrated on the validation set remain calibrated on a held-out test set. These findings empirically establish the reliability and comparability of calibrated confidence. Building on this, we introduce two applications: (1) model cascading with calibrated advantage routing and (2) data cleaning based on model ensemble. Using the routing signal derived from the comparability of calibrated confidences, we cascade large and small models to improve efficiency with almost no compromise in accuracy, and we further cascade two models of comparable scale to achieve performance beyond either model alone. Leveraging multiple experts and their calibrated confidences, we design a simple yet effective data-cleaning method that balances precision and detection rate to identify mislabeled samples in ImageNet and Massive Multitask Language Understanding (MMLU) datasets. Our results demonstrate that enabling models to recognize when they do not know is a practical step toward more efficient, reliable, and trustworthy AI.

Latest papers

When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning

Blind Quality Enhancement of Compressed Video via Fine-Grained Degradation-Guided Sequential Inference

Posterior-driven Heuristic Support Adaptation in a Probabilistic Treatment of Real2Sim2Real for Vision-Driven Deformable Linear Object Manipulation

PERTINENCE: Input-based Opportunistic Neural Network Dynamic Execution

No Free Lunch: Non-Asymptotic Analysis of Prediction-Powered Inference