As Expected? Efficient LLM Planning via Expectation-Observation Consistency Verification

Qingfu Zhu; Huanyang Zheng; Ting Liu; Wanxiang Che

cs.CLNov 2, 2026

As Expected? Efficient LLM Planning via Expectation-Observation Consistency Verification

Authors: Qingfu Zhu, Huanyang Zheng, Ting Liu, Wanxiang Che

Organizations: Harbin Institute of Technology

Abstract

LLM planning often relies on reactive executors (e.g., ReAct) that generate a single action at each step to remain grounded in the latest observations, incurring substantial inference cost. Batched executors reduce this cost by predicting multiple actions in a single model invocation, yet they are prone to compounding errors, as intermediate observations are ignored. We propose an expectation-observation consistency verification (EOCV) mechanism for batched executors, preserving grounding while improving efficiency. The mechanism generates expectations of action outcomes and compares them with the resulting observations. Consistency permits continued execution, whereas mismatches trigger early termination and replanning from the current observation. Experiments on text-based and multimodal benchmarks across multiple backbones show broadly comparable average task performance between EOCV and reactive executors, with a 29-35% reduction in average weighted token usage. Moreover, the mechanism is model-agnostic and remains effective when integrated into state-of-the-art planners.

Figures & tables

Figure 1: Executor paradigms for LLM planning. Given the query “Put apple in fridge” and initial observation $O_{1}$ : “The fridge is closed. You have an apple”, the reactive executor remains grounded but inefficient, whereas the batched executor is efficient but error-prone. Ours verifies expectations against observations to continue or replan batched actions selectively. Intermediate reasoning steps are omitted for clarity.

Figure 2: An overview of EOCV. The executor generates future action batches with expectations, executes them sequentially, and verifies expectations against the resulting observations. Consistency permits continued execution within the batch, whereas a mismatch triggers early termination and replanning from the latest observation.

Method	ALFWorld			ScienceWorld		EB-Habitat
Method	Reward	Tokens	Time	Reward	Tokens	Reward	Tokens
Reactive	88.81	1.00 $\times$	1.00 $\times$	89.37	1.00 $\times$	66.94	1.00 $\times$
Batched	83.58	0.57 $\times$	0.53 $\times$	82.56	0.41 $\times$	56.56	0.38 $\times$
Ours	88.56	0.65 $\times$	0.59 $\times$	88.87	0.69 $\times$	67.77	0.71 $\times$

Table 1: Average executor results on ALFWorld, ScienceWorld, and EB-Habitat. Reward is the mean score on

[0,100]

; Tokens denotes weighted token usage; Time is summed per-task wall-clock time. Token and time ratios are normalized to Reactive (

1.00\times

) within each backbone before averaging. ALFWorld time averages cover the locally deployed Qwen3.5-4B and Qwen3.5-9B models.

Verifier	ALFWorld				ScienceWorld				EB-Habitat
Verifier	Prec.	Rec.	Reward	Tokens	Prec.	Rec.	Reward	Tokens	Prec.	Rec.
Embedding	100.00	100.00	94.78	1.00 $\times$	96.46	100.00	85.42	1.00 $\times$	–	–
LLM/VLM	100.00	100.00	94.78	1.06 $\times$	97.32	100.00	85.93	1.15 $\times$	89.38	87.07

Table 2: Verifier comparison. Precision (Prec.) and recall (Rec.) use 300 randomly sampled, manually labeled pairs per benchmark. Pairs are collected with DeepSeek-V3.2 on the text benchmarks, while Qwen3.6-Flash is used for EB-Habitat. Tokens are normalized to embedding verification within each text benchmark; dashes mark inapplicable entries.

Figure 3: EOCV with different planner controllers. The x-axis shows the proportional reduction in weighted token usage, and the y-axis shows the change in reward, both relative to the corresponding planner with a reactive executor. Colors indicate controllers, and marker shapes indicate backbone models. Hollow, filled, and half-filled markers indicate ALFWorld, ScienceWorld, and EB-Habitat, respectively.

Figure 4: Executor comparisons on (a) ALFWorld with Qwen3.5-9B and (b) ScienceWorld with DeepSeek-V4.1-Flash. Reward is the mean score on

[0,100]

; Tokens denotes weighted token usage normalized to ReAct (

1.00\times

) within each setting. AdaPlanner is evaluated only on ALFWorld. Dashed lines mark the corresponding ReAct baseline.

	Batched Executor		Ours
Step	Action	Observation	Action	Expectation	Observation
$t$	move(glass cup, oven)	That can’t be moved there, because the oven isn’t open.	move(glass cup, oven)	You move the glass cup to the oven.	That can’t be moved there, because the oven isn’t open.
$t{+}1$	activate(oven)	The oven is now activated.	close(oven)	The oven is now closed.	Terminate & Replan

Table 3: ScienceWorld case study with DeepSeek-V4.1-Flash. Steps are aligned at each executor’s failed move. Red denotes batched actions executed after the failure; gray denotes actions withheld by EOCV. The full batches are shown in Table 12 .

Panel of Table 4 — Table 4: Multimodal EB-Habitat case study with Qwen3.6-Flash. The batched executor continues with a stale plan after the shared pick-up step, whereas EOCV detects an expectation–observation mismatch and triggers replanning. Annotations describe visual observations for human readers and are not accessible to the agent. The full trajectory is reported in Table 15 ; dashes indicate not applicable.

Setting	Reward $\uparrow$	Tokens (M) $\downarrow$
Full Model	94.78	2.45
w/o Expectation Skipping	94.78	3.96
w/o Verification	90.30	1.73

Table 5: Progressive ablation on ALFWorld with DeepSeek-V3.2. Components are removed cumulatively; w/o Verification omits both expectation generation and verification.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: ALFWorld token-cost decomposition without caching. Bar segment heights show price-weighted token costs, not raw token counts. Input, RT, and AT have weights 0.125, 1, and 1. Each model’s ReAct total cost is normalized to 100%.

Model	Input Weight	Output Weight
Qwen3.5-4B/9B	$1/8$	$1$
GPT-5.5	$1/6$	$1$
DeepSeek-V4.1-Flash	$1/4$	$1$
DeepSeek-V3.2	$2/3$	$1$
Qwen3.6-Plus/Flash	$1/6$	$1$

Appendix

Table 6: Token weights used to compute weighted token usage. The output-token weight is normalized to 1; each input-token weight is the model’s input-to-output price ratio.

Figure 6: ALFWorld token-cost decomposition with caching. Bar segment heights show price-weighted token costs, not raw token counts. Cached, uncached Input, RT, and AT have weights 0.025, 0.125, 1, and 1. Each model’s ReAct total cost is normalized to 100

Strategy	Prec.	Rec.	Reward	Tokens
Predefined	–	–	94.78	1.00 $\times$
Automatic	99.68	99.36	94.03	0.92 $\times$

Appendix

Table 7: Predefined and automatic expectation skipping on ALFWorld with DeepSeek-V3.2. Precision (Prec.) and recall (Rec.) are computed on 400 labeled actions for the automatic action-is-skippable tag. Token usage is normalized to predefined skipping; dashes indicate that a metric is not applicable.

Backbone	Skip	Pass	Fail
ALFWorld
DeepSeek-V3.2	75.55	11.60	12.85
Qwen3.5-4B	80.39	8.64	10.96
Qwen3.5-9B	77.00	10.78	12.23
ScienceWorld
DeepSeek-V3.2	49.56	33.06	17.38

Appendix

Table 8: Distribution (%) of EOCV execution outcomes. Skip denotes actions for which expectation prediction and verification are omitted.

Figure 7: Analysis of verification threshold

\tau

. Token usage is measured in millions (M).

Figure 8: Analysis of batch budget

B

. Token usage is measured in millions (M).

Backbone	Executor	Reward	Input	Output	Weighted Tokens		Time
Backbone	Executor	Reward	(M)	(M)	(M)	Ratio	(s)	Ratio
ALFWorld
Qwen3.5-4B	Reactive	79.85	7.15	1.50	2.39	1.00 $\times$	17 394.41	1.00 $\times$
	Batched	73.88	5.44	1.03	1.71	0.71 $\times$	9 603.92	0.55 $\times$
	Ours	79.10	6.12	0.92	1.68	0.70 $\times$	11 170.33	0.64 $\times$
Qwen3.5-9B	Reactive	91.79	6.15	1.95	2.72	1.00 $\times$	25 984.66	1.00 $\times$

Appendix

Table 9: Executor results by benchmark and backbone. Reward is on

[0,100]

; token totals are in millions (M). Qwen3.5 times are summed task durations in seconds. Ratios are relative to Reactive within each backbone. EB-Habitat totals sum the six categories in Table 10 . Dashes denote unavailable values. DS-V4.1-Flash denotes DeepSeek-V4.1-Flash.

Category	Reactive		Batched			Ours
Category	Reward	Tokens (M)	Reward	Tokens (M)	Ratio	Reward	Tokens (M)	Ratio
Qwen3.6-Flash
Base	87.25	0.56	79.75	0.14	0.25 $\times$	89.75	0.27	0.49 $\times$
Common Sense	44.00	1.17	31.47	0.47	0.40 $\times$	45.00	0.69	0.59 $\times$
Complex Instr.	54.00	1.06	52.07	0.30	0.28 $\times$	56.00	0.61	0.58 $\times$
Long Horizon	33.42	1.36	19.71	0.32	0.24 $\times$	34.50	0.74	0.55 $\times$

Appendix

Table 10: Detailed EB-Habitat results by task category and vision-language backbone. Rewards are rescaled to

[0,100]

; tokens are price-weighted totals in millions (M). Ratios are relative to Reactive within each category. Average rows report category means; their token ratios divide each mean by the corresponding Reactive mean.

Controller	Executor	Reward	Input	Output	Weighted Tokens		Time
Controller	Executor	Reward	(M)	(M)	(M)	Ratio	(s)	Ratio
ALFWorld
Qwen3.5-4B
ReflAct	Reactive	81.34	7.89	3.48	4.46	1.00 $\times$	40 540.94	1.00 $\times$
ReflAct	Ours	81.34	5.45	2.22	2.90	0.65 $\times$	25 778.94	0.64 $\times$
MPO	Reactive	88.06	5.11	1.54	2.18	1.00 $\times$	14 173.30	1.00 $\times$

Appendix

Table 11: Controller composition across three benchmarks with Reactive and EOCV (Ours) executors. Reward is on

[0,100]

; token totals are in millions (M). Qwen3.5 times are summed task durations in seconds. Ratios are relative to Reactive within each backbone and controller. Dashes denote unavailable values. DS-V4.1-Flash denotes DeepSeek-V4.1-Flash.

	Batched Executor		Ours
Step	Action	Observation	Action	Expectation	Observation
$t$	move(cup, oven)	That can’t be moved there, because the oven isn’t open.	move(cup, oven)	You move the glass cup to the oven.	That can’t be moved there, because the oven isn’t open.
$t{+}1$	activate(oven)	The oven is now activated.	close(oven)	The oven is now closed.	Terminate & Replan
$t{+}2$	wait()	You decide to wait for 10 iterations.	activate(oven)	The oven is now activated.	—-
$t{+}3$	look_at(oven)	An oven, which is turned on. The oven door is closed.	wait()	You decide to wait for 10 iterations.	—-

Appendix

Table 12: ScienceWorld case with DeepSeek-V4.1-Flash, expanding Table 3 . Slots preserve the order within each independently generated batch. Red actions continue after the failed move; gray actions are proposed but not executed. Dashes indicate no observation.

	Batched Executor		Ours
Step	Action	Observation	Action	Expectation	Observation
$t$	activate(sink)	The sink is now activated.	activate(sink)	The sink is now activated.	The sink is now activated.
$t{+}1$	dunk(glass cup, sink)	The sink does not contain any liquids to dunk into.	dunk(glass cup, sink)	You dunk the glass cup in the sink.	The sink does not contain any liquids to dunk into.
$t{+}2$	move(sugar, glass cup)	You move the sugar to the glass cup.	move(sugar, glass cup)	You move the sugar to the glass cup.	Terminate & Replan
$t{+}3$	mix(glass cup)	There is only one thing (sugar)	mix(glass cup)	Sugar and water mix to produce sugar water.	—
$t{+}4$	focus_on(sugar water)	No known action matches that input.	focus_on(sugar water)	You focus on the sugar water.	—

Appendix

Table 13: ScienceWorld case with GPT-5.5. Slots preserve the order within each independently generated batch. Red actions continue after the failed move; gray actions are proposed but not executed. Dashes indicate no observation.

	ReAct 5 executor calls; 2,554 weighted tokens		Ours 2 executor calls; 1,432 weighted tokens
Step	Action	Observation	Action	Expectation	Observation
			EOCV executor call 1
$t$	goto(drawer 4)	You arrive at drawer 4. The drawer 4 is closed.	goto(drawer 4)	Skip verification	You arrive at drawer 4. The drawer 4 is closed.
$t{+}1$	open(drawer 4)	You open the drawer 4. The drawer 4 is open. In it, you see nothing.	open(drawer 4)	Skip verification	You open the drawer 4. The drawer 4 is open. In it, you see nothing.
$t{+}2$	goto(drawer 5)	You arrive at drawer 5. The drawer 5 is closed.	goto(drawer 5)	Skip verification	You arrive at drawer 5. The drawer 5 is closed.
$t{+}3$	open(drawer 5)	You open the drawer 5. The drawer 5 is open. In it, you see a watch 1.	open(drawer 5)	Skip verification	You open the drawer 5. The drawer 5 is open. In it, you see a watch 1.

Appendix

Table 14: ALFWorld case with Qwen3.5-4B. Token totals cover five complete ReAct calls and two complete EOCV calls, including EOCV actions omitted from the table.

Panel of Table 15 — Table 15: EB-Habitat case with Qwen3.6-Flash, expanding Table 4 . The batched executor continues executing the red actions after the visual observation contradicts the plan. EOCV detects the mismatch and withholds the gray actions. Annotations describe the images for the reader and are not available to the agent. Dashes indicate that an observation or annotation is not applicable.

Panel of Table 16 — Table 16: EB-Habitat efficiency case study with Qwen3.6-Flash. Token totals cover three complete ReAct calls and one complete EOCV calls.

LLM Planning

Large language model planning studies how models decompose goals, choose actions, and revise plans under constraints.

See all 147 papers

Planning Commitment of LLM under Partial Observability
Oct 15, 2026Cassie Huang, Zhanwei Cao, Yeawon Lee +6LLM Planning LLM Agent Safety
Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution
Sep 29, 2026Subba Reddy Oota, Francisco Herrera, Jordi Cabot Sagrera +2LLM Agents Long-Horizon LLM Agents
ADIA Lab, Abu Dhabi, United Arab Emirates · University of Granada, Granada, Spain · Luxembourg Institute of Science and Technology, Luxembourg +2
Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm
Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM Reasoning Structured Reasoning
1Qizhi Institute · 2Dept. of Computer Science, Tsinghua University · College of AI, Tsinghua University +4

Oct 15, 2026J/K move · Enter open · S save

Planning Commitment of LLM under Partial Observability

Cassie Huang, Zhanwei Cao, Yeawon Lee, Nathalia Gomez+5

LLM Planning LLM Agent Safety LLM Inference Efficiency Partially Observable Markov Decision Processes

Open paper AskPDF arXiv

LLM-based planning agents commit to plans of different lengths before interacting with a partially observable environment. Some aggressively commit to a long plan based on assumptions about unobserved states, while others conservatively plan only over what they have observed. Both behaviors emerge implicitly in pre-trained models, whether the LLM outputs a plan directly or generates a PDDL formulation for a classical solver. Aggressive commitment risks costly failures, while conservative commitment risks excessive LLM calls. This choice pits safety against efficiency, a trade-off that has been neither characterized systematically nor made controllable. We investigate aggressive and conservative LLM planners and formalizers on two partially observable domains, where priors are provided as either explicit statistics or common-sense knowledge. Across three LLMs, both strategies reach similar accuracy. Conservative methods achieve fewer failed attempts (safety) at the cost of more API calls (efficiency). Aggressive methods fail more often but replan efficiently only when priors are informative. Interestingly, analysis shows that LLMs treat the provided priors nearly as facts, which risks failure when priors are unreliable. We also explore hybrid strategies with tunable hyperparameters that adjust the balance between safety and efficiency.

Abstract

LLM Planning

Explore similar work

Planning Commitment of LLM under Partial Observability

Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution

Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm