Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents

Nadeem Shaikh

cs.LGAug 25, 2026

Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agents

Authors: Nadeem Shaikh

Organizations: Independent Researcher Melbourne, Australia

Abstract

Current LLM agent systems decide delegation before reasoning begins (a router picks a model) or after a response is complete (a verifier scores it and may retry). We study a third regime: an agent that recognises, during its own reasoning, that it is unlikely to succeed and transfers control to a stronger model. We formulate intra-generation delegation as a Bayesian optimal-stopping problem over a learned competence posterior -- an online estimate of the agent's eventual task success whose sufficient statistics are learned from labelled trajectories, not read off raw entropy. We derive the myopic escalation threshold in closed form, characterise the optimal policy via dynamic programming, and prove that the optimal policy is a time-varying threshold with no shape assumption on the raw signal. We further prove exponential separation of the oracle belief at the Chernoff-information rate of the signal, a regret bound governed by the calibration of the posterior, and a finite-sample guarantee: with n labelled calibration trajectories the deployed plug-in policy's regret decays as 1/sqrt(n). A controlled simulation study confirms each prediction of the theory, including the predicted 1/sqrt(n) rate. We additionally report a real-model validation on a Qwen2.5-Coder 1.5B->7B code cascade (MBPP, 257 tasks), confirming two of three pre-registered predictions: the escalation frontier dominates post-hoc routing at equal cost, and the cumulative competence belief's discrimination rises over generation.

Figures & tables

Policy	Accuracy	Compute/query	Esc. rate
Junior only	$0.601$	$0.080$	$0.00$
Senior only	$0.901$	$0.150$	$1.00$
Fixed-rule (entropy, tuned)	$0.910$	$0.114$	—
Selective (post-hoc, tuned)	$0.959$	$0.140$	—
Bayesian, myopic schedule (Eq. ( 4 ))	$0.934$	$0.129$	$0.68$
Bayesian, constant threshold (tuned)	$0.958$	$0.111$	—

Table 1: Matched-compute comparison (

\approx 0.11

cost/query) on the simulation model. Accuracy is fraction correct; “esc.” is escalation rate. Rows marked “tuned” sweep one operating parameter and are reported at the compute-matched point; the myopic-schedule and optimal-stopping rows are parameter-free. Numbers are produced by the accompanying simulation.

Figure 1: Cost–accuracy frontier on the simulation model. The Bayesian policies (teal) dominate the fixed-rule (orange) and post-hoc selective (grey) baselines; the optimal-stopping point (red star) beats unconditional escalation on both axes.

Figure 2: Posterior success-belief trajectories

B_{t}

for twelve queries. Eventual successes (teal) separate from eventual failures (orange) within a few tokens; the dashed line is an illustrative threshold.

Figure 3: Sensitivity to signal miscalibration. As confidently-wrong contamination grows, accuracy degrades and—counter-productively—the escalation rate falls, because miscalibrated confidence suppresses the very escalations that are needed.

Observation model	Policy	Accuracy	Compute	Total cost
Beta, i.i.d. (baseline)	Fixed-rule	$0.956$	$0.117$	$0.161$
	Bayesian myopic	$0.960$	$0.111$	$\mathbf{0.151}$
Gaussian, i.i.d.	Fixed-rule	$0.958$	$0.117$	$0.160$
	Bayesian myopic	$0.958$	$0.111$	$\mathbf{0.153}$
Gaussian AR(1), $\varphi{=}0.6$	Fixed-rule	$0.954$	$0.120$	$0.166$
(misspecified update)	Bayesian myopic	$0.956$	$0.114$	$\mathbf{0.158}$

Table 2: Robustness across observation models. Each policy is tuned to its best total-cost operating point per model; total cost

=

compute

+

expected error cost. AR(1) uses the i.i.d. belief update on correlated signals (misspecified). Numbers produced by the accompanying robustness script.

Policy	Accuracy	Compute ( $\times$ junior)	Esc. rate
Junior only	$0.623$	$1.00$	$0.00$
Senior only	$0.809$	—	$1.00$
Post-hoc routing (tuned)	$0.750$	$1.41$	—
Streaming (tuned $\tau$ )	$0.747$	$\mathbf{0.98}$	$0.37$

Table 3: Real-model comparison on MBPP (sanitized test,

257

tasks; Qwen2.5-Coder

1.5

\to

7

B). Compute is total generated tokens, normalised to junior-only. Post-hoc and streaming are reported at the operating point reaching

75\%

accuracy; the belief is cross-validated. Numbers produced by the accompanying harvest and analysis scripts.

Figure 4: Real-model cost–accuracy frontier. Streaming escalation (purple) dominates post-hoc routing (teal) everywhere: it climbs from junior to senior accuracy at near-constant compute, because aborting doomed generations offsets the added senior calls. Dotted lines mark junior-only and senior-only accuracy.

Figure 5: Discrimination of the cumulative belief

B_{t}

versus generation fraction (AUROC of

B_{t}

against eventual success, cross-validated). The curve rises near-monotonically (Spearman

\rho=0.93

) and plateaus —rather than declining—through the shaded

25

–

55\%

band where the instantaneous signal collapses, confirming prediction (b).

Explore similar work

Mar 31, 2026

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models

Effective automation hinges on deciding when to act and when to escalate. We model this as a decision under uncertainty: an LLM forms a prediction, estimates its probability of being correct, and compares the expected costs of acting and escalating. Using this framework across five domains of recorded human decisions-demand forecasting, content recommendation, content moderation, loan approval, and autonomous driving-and across multiple model families, we find marked differences in the implicit thresholds models use to trade off these costs. These thresholds vary substantially and are not predicted by architecture or scale, while self-estimates are miscalibrated in model-specific ways. We then test interventions that target this decision process by varying cost ratios, providing accuracy signals, and training models to follow the desired escalation rule. Prompting helps mainly for reasoning models. SFT on chain-of-thought targets yields the most robust policies, which generalize across datasets, cost ratios, prompt framings, and held-out domains. These results suggest that escalation behavior is a model-specific property that should be characterized before deployment, and that robust alignment benefits from training models to reason explicitly about uncertainty and decision costs.

Matthew DosSantos DiSorbo, Harang Ju

Harvard Business School · Johns Hopkins Carey Business School

LLM Decision-Making AI-Assisted Decision Making Selective Prediction Language Model Calibration

Ask PDF

Jun 6, 2026

Bayesian-Agent: Posterior-Guided Skill Evolution for LLM Agent Harnesses

LLM agents increasingly rely on external inference conditions: prompts, tools, memory, SOPs, skills, and harness feedback. These assets can improve task execution without changing model weights, but they are often revised by heuristic reflection or by reusing observed successes and failures as if counts alone were reliable belief. We introduce Bayesian-Agent, a native and cross-harness framework that treats reusable skills and SOPs as hypotheses about whether a frozen model will succeed under a particular prompt, context, and harness environment. Bayesian-Agent records verified trajectory evidence, maintains a feature-conditioned categorical posterior over each skill, and maps posterior state into inspectable actions such as patch, split, compress, retire, and explore. Model-facing prompts receive executable guardrails and failure-mode patches, while posterior summaries remain available for audit. With deepseek-v4-flash, incremental repair improves SOP-Bench from 80% to 95%, Lifelong AgentBench from 90% to 100%, and RealFin-Bench from 45% to 65%. We further evaluate Bayesian-Agent's native backend and optional GenericAgent, mini-swe-agent, and Claude Code backends. The results include positive, negative, saturated, and case-study settings, suggesting that agent skill evolution is best viewed as posterior-guided harness optimization rather than uncalibrated prompt accumulation. The source code is available at https://github.com/DataArcTech/Bayesian-Agent.

Xiaojun Wu, Cehao Yang, Honghao Liu +7

1IDEA Research · 2The Hong Kong University of Science and Technology (Guangzhou) · 3DataArcTech Ltd.

Agent Harness Optimization LLM Agent Workflow Optimization LLM Agent Self-Improvement LLM Agent Skill Learning LLM Agent Harnesses

Ask PDF

May 1, 2026

Position: agentic AI orchestration should be Bayes-consistent

LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to consult, or how many resources to invest. While the usefulness and feasibility of Bayesian approaches remain unclear for LLM inference, this position paper argues that the control layer of an agentic AI system (that orchestrates LLMs and tools) is a clear case where Bayesian principles should shine. Bayesian decision theory provides a framework for agentic systems that can help to maintain beliefs over task-relevant latent quantities, to update these beliefs from observed agentic and human-AI interactions, and to choose actions. Making LLMs themselves explicitly Bayesian belief-updating engines remains computationally intensive and conceptually nontrivial as a general modeling target. In contrast, this paper argues that coherent decision-making requires Bayesian principles at the orchestration level of the agentic system, not necessarily the LLM agent parameters. This paper articulates practical properties for Bayesian control that fit modern agentic AI systems and human-AI collaboration, and provides concrete examples and design patterns to illustrate how calibrated beliefs and utility-aware policies can improve agentic AI orchestration.

Theodore Papamarkou, Pierre Alquier, Matthias Bauer +27

PolyShape, Greece. · NTUA, Greece. · ESSEC Business School, Singapore. +19

LLM Agent Orchestration AI Control Bayesian Decision Theory Human-AI Decision Making Decision-Making under Uncertainty

Ask PDF