cs.RONov 12, 2026

Maximizing LLM Utility While Managing LLM Fallibility

Authors: Ekta U. Samani, Christopher G. Atkeson

Organizations: E. U. Samani and C. G. Atkeson are with the Robotics Institute at Carnegie Mellon University

Abstract

Large language models (LLMs) can supply autonomous systems with plans, predictions, and judgments, but these contributions are sometimes wrong. This review asks how a system can maximize an LLM's utility while managing its fallibility. Our answer is to adapt the system's reliance on the LLM as evidence about the LLM's performance accumulates. Systems built from imperfect components, such as noisy sensors and faulty software, have long worked in this way, and the classical methods they use can be applied to LLM-based systems as well. A system can change its reliance on an LLM in several ways, depending on the LLM's role and on the other components in the system. Each change needs an estimate of the LLM's reliability, and the LLM's role and the kind of contribution it makes determine what to estimate. Classical methods from decision theory, estimation, and online learning then turn these estimates into reliance decisions. These decisions can reduce how often LLM errors affect the system. However, some errors matter even when they are rare. An independently verified mechanism can prevent such errors from being executed. Without one, adapting reliance using reliability estimates can still keep their risk within a bound. Studies on adapting reliance on LLMs are emerging but limited, and the reviewed comparisons suggest that adapting reliance can improve system performance.

Figures & tables

Explore similar work

CardsList
  1. Reliability is a user-specific property of large language models

    Nov 18, 2026Xingyu ZhaoLLM ReliabilityLLM Evaluation

  2. Sequential statistical inference for Large Language Models: Representation, validity, and monitoring

    May 30, 2026Yao XieLanguage Model CalibrationLLM Reliability

  3. Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

    Oct 15, 2026Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe +1Confidence Estimation in Language ModelsLanguage Model Calibration