ReDAct: Uncertainty-Aware Deferral for LLM Agents

Dzianis Piatrashyn; Nikita Kotelevskii; Kirill Grishchenkov; Nikita Glazkov; Ivan Nasonov; Ilya Makarov; Timothy Baldwin; Preslav Nakov

cs.CLApr 8, 2026

ReDAct: Uncertainty-Aware Deferral for LLM Agents

Authors: Dzianis Piatrashyn, Nikita Kotelevskii, Kirill Grishchenkov, Nikita Glazkov, Ivan Nasonov, Ilya Makarov, Timothy Baldwin, Preslav Nakov,

Organizations: Mohamed bin Zayed University of Artificial Intelligence · Ocapital · National University of Science and Technology (NUST) MISIS · AXXX ·

Abstract

Recently, LLM-based agents have become increasingly popular across many applications, including complex sequential decision-making problems. However, they inherit the tendency of LLMs to hallucinate, leading to incorrect decisions. In sequential settings, even a single mistake can irreversibly degrade the trajectory, making hallucinations an even bigger problem. Although larger LLMs hallucinate less, they incur a significantly higher per-token cost. In this paper, we address this tradeoff by proposing ReDAct (Reason-Defer-Act). In ReDAct, an agent is equipped with two LLMs: a small, cheap model used by default, and a large, more reliable but expensive model. When the predictive uncertainty of the small model exceeds a calibrated threshold, the decision is deferred to the large model. We evaluate our approach in text-based embodied environments such as ALFWorld and MiniGrid and show that deferring only about 15% of decisions to the large model can match the quality of using it exclusively, while significantly reducing inference costs.

Figures & tables

Figure 1: Overview of the proposed ReDAct framework. The agent uses the small model to propose an action $\tilde{a}^{s}_{t}$ and estimates its uncertainty $u_{t}$ . If $u_{t}$ exceeds the threshold $\tau$ , the step is deferred to the large model, which proposes $\tilde{a}^{l}_{t}$ as the final decision.

	Action		Reasoning
UQ metric	PRR	ROC-AUC	PRR	ROC-AUC
Mean Token Entropy	0.424	0.710	0.168	0.596
Perplexity	0.399	0.684	0.193	0.616
Sequence Probability	0.417	0.704	0.279	0.682

Table 1: Performance comparison of uncertainty metrics estimated over action selection and reasoning trace.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	0.683 $\pm$ 0.025
	+ GPT-5.2	0.788 $\pm$ 0.020	0.798 $\pm$ 0.025	0.808 $\pm$ 0.011	0.770 $\pm$ 0.026
	+ Qwen3-480B	0.743 $\pm$ 0.020	0.763 $\pm$ 0.014	0.790 $\pm$ 0.023	0.760 $\pm$ 0.029
Llama3.3-70B	Base	0.635 $\pm$ 0.018
	+ GPT-5.2	0.768 $\pm$ 0.017	0.763 $\pm$ 0.025	0.800 $\pm$ 0.028	0.750 $\pm$ 0.025
	+ Qwen3-480B	0.755 $\pm$ 0.020	0.760 $\pm$ 0.028	0.773 $\pm$ 0.017	0.690 $\pm$ 0.026

Table 2: Success rate on ALFWorld with bootstrapped standard deviation. For the small model without deferral, all methods coincide.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	0.855 $\pm$ 0.024
	+ GPT-5.2	0.930 $\pm$ 0.015	0.950 $\pm$ 0.013	0.950 $\pm$ 0.013	0.930 $\pm$ 0.019
	+ Qwen3-235B	0.930 $\pm$ 0.013	0.915 $\pm$ 0.017	0.940 $\pm$ 0.020	0.900 $\pm$ 0.012
Llama4	Base	0.220 $\pm$ 0.024
	+ GPT-5.2	0.370 $\pm$ 0.050	0.325 $\pm$ 0.030	0.420 $\pm$ 0.023	0.380 $\pm$ 0.039
	+ Qwen3-235B	0.330 $\pm$ 0.021	0.355 $\pm$ 0.047	0.370 $\pm$ 0.031	0.295 $\pm$ 0.027

Table 3: Success rate on MiniGrid full-view with bootstrapped standard deviation. For the small model without deferral, all methods coincide.

Figure 2: Pareto front of success rate vs. large model calls on ALFWorld with perplexity-based deferral. ReDAct enables small models to approach large model performance while using only 15% of large model calls.

Panel of Figure 3 — Figure 3: Qwen3-80B + GPT-5.2. Large model invocation frequency by step in ALFWorld (top row) and MiniGrid (bottom row). At each step, the frequency is defined as the number of large model calls divided by the number of episodes that reached that step.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	9.22
Qwen3-80B	+ GPT-5.2	15.43	16.39	16.25	14.15
	+ Qwen3-480B	16.3	16.15	15.39	13.84
Llama3.3-70B	Base	29.24
Llama3.3-70B	+ GPT-5.2	31.56	30.72	30.76	29.27
	+ Qwen3-480B	33.05	32.18	33.21	31.82

Table 4: Costs for inference per model in

for ALFWorld. Total is

512

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Success Rate
Qwen3-80B	0.630
Llama4-Maverick	0.105
Llama3.3-70B	0.000
GPT-OSS-20B	0.000
Mistral-8x-7B	0.000
Mistral-Small	0.000

Appendix

Table 5: Success rate on MiniGrid partial-view across small and large models.

Figure 4: Prediction-Rejection Ratio (PRR) Curve. The curve illustrates the quality of the non-rejected predictions as a function of the rejection rate. Oracle represents the optimal rejection strategy, Random is a random rejection, and UE is rejection based on the evaluated uncertainty estimation method.

UQ metric	PRR	ROC-AUC
Number Semantic Sets	0.144	0.545
Sum of Eigenvalue	0.213	0.575
Eccentricity	0.218	0.578
Lexical Similarity	0.219	0.579
Sentence SAR	0.260	0.603
Cocoa PPL	0.300	0.628

Appendix

Table 6: Performance comparison of uncertainty metrics estimated over action selection.

Model	Input Tokens Price	Output Tokens Price
Qwen3-80B	0.15	1.50
Llama3.3-70B	0.88	0.88
Llama4-Maverick	0.27	0.85
GPT-5.2	1.75	14.00
Qwen3-480B	2.00	2.00

Appendix

Table 7: Input and Output token price by model in USD per million tokens.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	0
	+ GPT-5.2	1331	1447	1447	1487
	+ Qwen3-480B	1187	1193	1140	1141
Llama3.3-70B	Base	0
	+ GPT-5.2	1446	1349	1426	1099
	+ Qwen3-480B	1445	1383	1481	1139

Appendix

Table 8: Number of large model calls on ALFWorld. For the small model without deferral, all methods coincide.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	0
	+ GPT-5.2	765	763	697	892
	+ Qwen3-235B	870	895	803	920
Llama4-Maverick	Base	0
	+ GPT-5.2	914	915	900	901
	+ Qwen3-235B	965	1019	942	932

Appendix

Table 9: Number of large model calls on MiniGrid. For the small model without deferral, all methods coincide.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	19.58 $\pm$ 11.72
	+ GPT-5.2	20.31 $\pm$ 12.04	20.87 $\pm$ 12.61	21.49 $\pm$ 13.41	20.36 $\pm$ 12.10
	+ Qwen3-480B	21.39 $\pm$ 12.61	20.88 $\pm$ 13.24	21.32 $\pm$ 12.78	19.96 $\pm$ 12.42
Llama3.3-70B	Base	20.83 $\pm$ 13.27
	+ GPT-5.2	20.78 $\pm$ 12.68	20.47 $\pm$ 12.09	21.32 $\pm$ 12.78	20.35 $\pm$ 12.43
	+ Qwen3-480B	21.12 $\pm$ 12.34	19.44 $\pm$ 11.91	21.09 $\pm$ 12.54	19.35 $\pm$ 11.45

Appendix

Table 10: Average number of steps before success with standard deviation for ALFWorld.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	Base	26.47 $\pm$ 8.62
	+ GPT-5.2	24.12 $\pm$ 7.09	25.02 $\pm$ 7.72	24.08 $\pm$ 6.81	25.09 $\pm$ 7.61
	+ Qwen3-235B	25.65 $\pm$ 7.55	25.34 $\pm$ 7.49	25.64 $\pm$ 8.06	25.38 $\pm$ 7.81
Llama4-Maverick	Base	33.41 $\pm$ 9.79
	+ GPT-5.2	33.53 $\pm$ 7.03	33.83 $\pm$ 8.40	32.40 $\pm$ 8.76	33.93 $\pm$ 8.89
	+ Qwen3-235B	33.17 $\pm$ 8.99	32.44 $\pm$ 8.99	32.53 $\pm$ 8.80	33.49 $\pm$ 9.18

Appendix

Table 11: Average number of steps before success with standard deviation for MiniGrid.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	+ GPT-5.2	3.328 $\pm$ 5.270	3.618 $\pm$ 5.453	3.618 $\pm$ 5.369	3.717 $\pm$ 2.852
Qwen3-80B	+ Qwen3-480B	2.967 $\pm$ 4.747	2.982 $\pm$ 4.406	2.850 $\pm$ 4.405	2.853 $\pm$ 2.328
Llama3.3-70B	+ GPT-5.2	3.615 $\pm$ 3.990	3.373 $\pm$ 3.947	2.850 $\pm$ 4.405	2.748 $\pm$ 2.195
Llama3.3-70B	+ Qwen3-480B	3.612 $\pm$ 4.343	3.458 $\pm$ 4.633	3.703 $\pm$ 4.638	2.848 $\pm$ 2.346

Appendix

Table 12: Mean number of large model calls with standard deviation per episode for ALFWorld.

Base	Setting	MTE	SP	PPL	Random
Qwen3-80B	+ GPT-5.2	3.825 $\pm$ 3.449	3.815 $\pm$ 3.211	3.485 $\pm$ 2.421	4.460 $\pm$ 2.414
Qwen3-80B	+ Qwen3-235B	4.350 $\pm$ 3.594	4.475 $\pm$ 3.703	4.015 $\pm$ 3.126	4.600 $\pm$ 2.532
Llama4-Maverick	+ GPT-5.2	4.570 $\pm$ 2.673	4.575 $\pm$ 2.708	4.500 $\pm$ 2.706	4.505 $\pm$ 2.121
Llama4-Maverick	+ Qwen3-235B	4.825 $\pm$ 2.786	5.095 $\pm$ 2.898	4.710 $\pm$ 2.531	4.660 $\pm$ 2.125

Appendix

Table 13: Mean number of large model calls with standard deviation per episode for MiniGrid.

	MTE	SP	PPL	Random
Small Model Input Tokens	25,638,669	25,850,196	26,166,777	25,463,015
Small Model Output Tokens	2,530,446	2,522,110	2,546,153	2,579,383
Large Model Input Tokens	3,905,271	4,372,542	4,236,147	3,209,024
Large Model Output Tokens	68,350	76,765	78,514	60,285
Large Model Calls	1,331	1,447	1,447	1,487
Small Model price $	7.64	7.66	7.74	7.69

Appendix

Table 14: ALFWorld with Qwen3-80B + GPT-5.2. Token usage and cost comparison across different uncertainty metrics.

	MTE	SP	PPL	Random
Small Model Input Tokens	27,982,528	26,776,457	25,846,244	26,049,938
Small Model Output Tokens	2,986,414	2,886,570	2,828,277	2,823,600
Large Model Input Tokens	3,582,041	3,660,392	3,411,146	2,652,560
Large Model Output Tokens	228,614	239,339	229,266	196,881
Large Model Calls	1,187	1,193	1,140	1,141
Small Model price $	8.68	8.35	8.12	8.14

Appendix

Table 15: ALFWorld with Qwen3-80B + Qwen3-480B. Token usage and cost comparison across different uncertainty metrics.

	MTE	SP	PPL	Random
Small Model Input Tokens	25,724,005	25,400,456	24,894,956	26,120,373
Small Model Output Tokens	1,381,591	1,382,268	1,367,765	1,458,582
Large Model Input Tokens	3,827,339	3,554,058	3,833,118	2,480,882
Large Model Output Tokens	72,139	66,605	67,572	47,121
Large Model Calls	1,446	1,349	1,426	1,099
Small Model price $	23.85	23.57	23.11	24.27

Appendix

Table 16: ALFWorld with Llama3.3-70B + GPT-5.2. Token usage and cost comparison across different uncertainty metrics.

	MTE	SP	PPL	Random
Small Model Input Tokens	26,235,704	25,074,274	25,778,042	27,914,399
Small Model Output Tokens	1,613,201	1,543,982	1,596,770	1,679,948
Large Model Input Tokens	3,994,820	4,108,710	4,278,414	2,692,680
Large Model Output Tokens	275,371	271,067	282,763	197,577
Large Model Calls	1,445	1,383	1,481	1,139
Small Model price $	24.51	23.42	24.09	26.04

Appendix

Table 17: ALFWorld with Llama3.3-70B + Qwen3-480B. Token usage and cost comparison across different uncertainty metrics.

Figure 10: Pareto front of success rate vs. large model calls on MiniGrid with perplexity-based deferral. ReDAct enables small models to approach large model performance while using only 15% of large model calls.

Panel of Figure 11 — Figure 11: Pareto front of success rate vs. cost on ALFWorld with perplexity-based deferral. Each plot shows the trade-off between performance and computational cost. (a): Qwen3-80B small model. (b): Llama3.3-70B small model.

Panel of Figure 12 — Figure 12: Pareto front of success rate vs. cost on MiniGrid with perplexity-based deferral. Each plot shows the trade-off between performance and computational cost. (a): Qwen3-80B small model. (b): Llama4-Maverick small model.

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 681 papers

Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents
Feb 18, 2026Wenxuan Ding, Nicholas Tomlin, Greg DurrettLLM Agents Decision-Making under Uncertainty
New York University
Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Jul 29, 2026Amirmohammad Farzaneh, Osvaldo SimeoneLLM Inference Efficiency AI Agent Reliability
Institute for Intelligent Networked Systems (INSI) Northeastern University London London, UK
Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds
Jun 28, 2026Mengdie Flora Wang, Haochen Xie, Guanghui Wang +2AI Agent Reliability Multi-Agent LLM Systems
1AWS Generative AI Innovation Center, USA · General Motors (GM), USA

Feb 18, 2026J/K move · Enter open · S save

Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents

Wenxuan Ding, Nicholas Tomlin, Greg Durrett

New York University

LLM Agents Decision-Making under Uncertainty LLM Agent Planning

Open paper AskPDF arXiv

LLM agents are deployed in environments where they must interact to acquire information. In these scenarios, the agent must reason about inherent cost-uncertainty tradeoffs in how to act, such as when to stop exploring and commit to an answer. For instance, on a programming task, an agent might run the code it generates, or it might generate tests for that code snippet; the cost of writing and running a test is nonzero, but typically lower than the cost of running buggy code. In this work, we show that we can induce LLM agents to explicitly reason about balancing these cost-uncertainty tradeoffs, then act more optimally in their environments. We formalize multiple tasks, including retrieval-augmented QA and a file reading coding task, as sequential decision-making problems under uncertainty. Each problem has latent environment state that impacts the agent's performance. We introduce a framework called Calibrate-Then-Act (CTA), where we pass the agent an inferred prior about this environment state to enable it to act more optimally. This information qualitatively changes agent behavior, and adds environment sensitivity to the agent which is not learned via standard RL training. Our results on a synthetic task, QA, and file reading show that making cost-benefit tradeoffs explicit with CTA helps agents discover more optimal decision-making strategies.

Abstract

LLM Agents

Explore similar work

Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents

Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds