cs.CLNov 16, 2026

Stability-Compensated Threshold Calibration for Selection-Conditioned Risk Control in Large Language Models

Authors: Zhiyuan Wang, Yuanchang Ye, Jiahao Yuan, Jinhao Duan, Xiaoshuang Shi, Weijun Xie, Ye Wei, Kaidi Xu

Organizations: City University of Hong Kong · National University of Singapore · East China Normal University · University of North Carolina at Chapel Hill · 

Abstract

Large language models often produce hallucinated answers, motivating selective prediction policies that abstain on uncertain queries. However, heuristic uncertainty thresholds alone do not provide statistical control of errors among accepted predictions. Linear Expectation Constraints (LEC) reduce selection-conditioned error rate (SCER) calibration to a nonpositive expected linear-loss constraint, but the associated loss is non-monotonic in the acceptance threshold. We construct an exact counterexample under i.i.d. sampling demonstrating that LEC's empirical threshold-selection rule can violate the population SCER target. To address this risk-transfer gap, we introduce Stability-Compensated Linear Expectation Control (SC-LEC), which uses the negative linear risk of a theoretical full-data reference to compensate for threshold-selection instability. For local expansion beyond a safe anchor, we derive two finite-sample certificates: a stability-based certificate exploiting rank crossings and a direct certificate using the structure of threshold expansion to reduce conservatism. With a valid population anchor margin, our certificate conditions ensure finite-sample marginal SCER control, and independent margin certification further provides a 1−δ1-δ high-probability guarantee for this calibration-averaged control. Experiments on CommonsenseQA, ARC-Challenge, and TriviaQA show useful retention gains beyond the same certified anchor, while revealing conservative behavior in small-margin regimes.

Figures & tables

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Asymptotic Risk Calibration for Selective Question Answering

    Aug 12, 2026Shufan Lin, Sijin DongQuestion AnsweringSelective QA

  2. Reinforcement Learning Done Right for Explicit Confidence Targets

    Oct 26, 2026Zefeng He, Yichen He, Yuan Lin +2LLM AbstentionSelective Prediction