Effective automation hinges on deciding when to act and when to escalate. We model this as a decision under uncertainty: an LLM forms a prediction, estimates its probability of being correct, and compares the expected costs of acting and escalating. Using this framework across five domains of recorded human decisions-demand forecasting, content recommendation, content moderation, loan approval, and autonomous driving-and across multiple model families, we find marked differences in the implicit thresholds models use to trade off these costs. These thresholds vary substantially and are not predicted by architecture or scale, while self-estimates are miscalibrated in model-specific ways. We then test interventions that target this decision process by varying cost ratios, providing accuracy signals, and training models to follow the desired escalation rule. Prompting helps mainly for reasoning models. SFT on chain-of-thought targets yields the most robust policies, which generalize across datasets, cost ratios, prompt framings, and held-out domains. These results suggest that escalation behavior is a model-specific property that should be characterized before deployment, and that robust alignment benefits from training models to reason explicitly about uncertainty and decision costs.
Figures & tables
Figure 1: The escalation decision. A human delegates a task to an LLM agent, which produces a prediction and then decides whether to implement (incurring error cost cw if wrong) or escalate (incurring labor cost cℓ ). The agent escalates when its estimated probability of being correct p^ falls below a threshold τ .
Figure 2: The multi-turn prompting protocol, illustrated with a LendingClub example. In Turn 1, the agent receives the scenario and signal, then produces a prediction. In Turn 2, the agent sees its own prediction and decides whether to implement or escalate. The signal provides the feature split and predictive accuracy from a decision tree trained on the dataset.
Figure 3: Each model exhibits a distinct escalation profile. Escalation rate versus predictive accuracy for all eight models across four datasets (with signals, no thinking). Same-family models are vertically aligned. Models vary in both overall escalation level and sensitivity to the signal. Error bars show ±1 standard error.
Figure 4: Most models overestimate their own accuracy. Actual versus self-estimated accuracy for each model across four datasets. Points above the diagonal indicate overconfidence (self-estimated > actual). Across 304 model × condition pairs, 201 (66%) are overconfident. The direction of miscalibration varies by model: some are predominantly overconfident, others underconfident.
Intervention
Accuracy
Δ
Qwen 9B baseline
62.0%
—
+ cost framing
63.9%
+1.9
+ thinking
61.9%
−0.1
+ thinking + cost framing
78.8%
+16.8
GPT-5-mini baseline
64.7%
—
+ cost framing
75.8%
+11.1
Table 1: Sample-level escalation accuracy under different interventions, evaluated against the optimal policy for cost ratio R=4 ( τ∗=75% , cw=4cℓ ). For each sample, the correct action is to escalate if the condition’s predictive accuracy is below 75% and implement if above. MoralMachine is excluded from all rows. GPT-5-mini further excludes MovieLens due to content filter restrictions.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Model
With signal
Without signal
Pred. acc
Esc. rate
Pred. acc
Esc. rate
Qwen3.5-9B
61.2%
68.3%
52.1%
70.6%
Qwen3.5-397B
70.4%
93.4%
69.3%
91.3%
Llama 4 Maverick
65.8%
84.0%
65.2%
94.6%
Llama 3.3 70B
67.5%
97.7%
71.4%
99.9%
Mixtral 8x7B
65.1%
96.5%
39.5%
93.5%
Appendix
Table 2: MoralMachine escalation behavior. All models escalate at high rates, recognizing the inherent uncertainty of ethical dilemmas.
Dataset
With signal
Without signal
HotelBookings
100%
84.7%
LendingClub
100%
84.0%
Appendix
Table 3: SFT model evaluated with and without the signal, on the original cost framing.
Dataset
Original
Dollar
Wording
Trained?
HotelBookings
100%
98.3%
100%
Yes
LendingClub
100%
100%
100%
Yes
WikipediaToxicity
100%
100%
100%
Yes
MovieLens
100%
100%
100%
No
Appendix
Table 4: SFT with chain-of-thought reasoning on full scenario prompts. The model is trained on three datasets and evaluated on all four main datasets, including the held-out MovieLens . Accuracy is measured against the Bayes-optimal policy across six cost ratios ( R∈{2,4,8,10,20,50} ), with 300 samples per dataset (50 per R ). MoralMachine results are reported in Appendix A .
Figure 5: Implicit escalation threshold p∗ (left) and self-estimated accuracy a^ (right) for each model. The threshold p∗ varies widely (53% to over 100%), while self-estimated accuracy ranges from 76% to 97%. The dotted lines show the optimal threshold τ∗=75% at cost ratio R=4 (left) and average actual accuracy (right).
Deciding when to escalate a query from a small language model to a larger one requires a cheap signal that predicts, before the large model is called, whether escalating would help. Semantic entropy, originally developed to detect hallucinations, is a natural candidate: it measures how much a model's sampled answers disagree in meaning, and high disagreement often signals an unreliable answer. We test it across three benchmarks and two model families. On GSM8K, with a small/large pair about twelve times apart in size, semantic entropy reliably distinguishes the small model's mistakes (AUROC 0.871) and improves routed accuracy over random escalation by up to nine points at matched cost. An earlier strong-looking result on a synthetic benchmark proved misleading: a simple rule based only on question difficulty, with no model involved, matched semantic entropy almost exactly. This paper's main contribution is a set of checks that catch this before it is reported as real. We show that scoring a cheap, question-only difficulty estimate alongside any signal reveals whether the signal adds real information or just tracks how hard a question looks; that two reasonable definitions of "escalation worked" can produce very different results on the same data; that a benchmark can leave almost no room for any signal to beat simply always using the large model; and that the true cost of live sampling can make routing more expensive than calling the large model directly. For a cheaper alternative that reuses cached past outcomes, we show how to predict whether it will work on a new dataset -- confirmed by correctly forecasting a collapse from AUROC 0.908 to chance level (0.518) ahead of time. We offer these as a general checklist for evaluating escalation signals.
Ramin Pishehvar, Andrea Morandi, Mahesh Viswanathan
Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior -- an online estimate of the agent's eventual task success whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove that the optimal policy is a time-varying threshold with no shape assumption on the raw signal. We further prove exponential separation of the oracle belief at the Chernoff-information rate of the signal, a regret bound governed by the calibration of the posterior, and a finite-sample guarantee: with n labelled calibration trajectories the deployed plug-in policy's regret decays as 1/sqrt(n). A controlled simulation study confirms each prediction of the theory, including the predicted 1/sqrt(n) rate. We additionally report a real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade (MBPP, 257 tasks), confirming two of three pre-registered predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief's discrimination rises over generation.
LLM cascades, in which a cheap model defers to an expensive one on low-confidence queries, are widely used to reduce inference cost. Given a pool of models, a practitioner must decide how many models to include and where to set each deferral threshold. We derive first-order optimality conditions showing that, at an optimum, the ratio of expected accuracy gain to expected downstream cost is equal across deferral boundaries. A local search based on these conditions closely matches exhaustive search. We also derive an identity that decomposes the accuracy gain of score-based escalation over random escalation into two AUROC terms. Across five benchmarks and nine deferral scores, with model sequences and thresholds optimized from a pool of eight models, two-model cascades improve mean test-set accuracy over single-model selection by 2.1 to 8.2 percentage points. However, allowing more than two models does not improve mean test-set accuracy in 118 of 135 comparisons across scorers, datasets, and depth caps, and adds at most 0.43 percentage points. To understand the role of deferral scores in depth gains, we conduct counterfactual experiments with simulated confidence scores. When these scores have high AUROC and reflect only whether the current model answered correctly, allowing more than two models improves test-set accuracy on four of five benchmarks. However, these gains do not persist when the scores also reflect query difficulty shared across models, even at the same AUROC. These results suggest that gains from additional depth depend on how well the confidence score separates correct from incorrect answers for the current model compared with later models.