AquaMend: Minimal Re-probing and Conditional Rollback for Latent-Belief Failures in Embodied Agents

Yufan Liu; Shang Luo; Yang Liu; Haoxuan Jia; Feiyu Han; Qian Li; Chen Li; Yingguang Yang

cs.ROSep 24, 2026

AquaMend: Minimal Re-probing and Conditional Rollback for Latent-Belief Failures in Embodied Agents

Authors: Yufan Liu, Shang Luo, Yang Liu, Haoxuan Jia, Feiyu Han, Qian Li, Chen Li, Yingguang Yang, +4 more

Organizations: University College London · Peking University · School of Electrical and Electronic Engineering, Nanyang Technological University · University of the Chinese Academy of Sciences · Beijing University of Posts and Telecommunications · University of Leeds · Fullive-AI

Abstract

Physical changes or sensing errors can invalidate embodied agents' task-relevant beliefs. AquaMend compares re-probing, rollback, and supported continuation on a probe-belief-action graph under an expected-loss objective covering sensing, physical recovery, and uncorrected failures. A joint posterior guides a one-step policy with conditional detection-power screening. The per-belief three-way optimum requires independence, separability, and fully resolving probes; the general policy has no global optimality guarantee. Across 32 paired scenarios in a self-constructed simulation benchmark, AquaMend recovers in 28/32 cases and reduces mean complete loss by 21.6% versus restart. Its paired loss difference from decision-theoretic troubleshooting (DTT) is not statistically significant after Holm correction. Against the all-candidate ablation, online decision time decreases by 12.3% overall but increases by 3.4% in the uncovered late stage.

Figures & tables

Type	Symbol	Semantics	Cost
Probe	$m\in\mathcal{M}$	one latent-property measurement	redo $c_{\mathrm{re}}(m)\geq 0$
Belief	$\varphi\in\Phi$	latent proposition (e.g., “cup 3 is least full”)	residual $L_{i}\geq 0$
Action	$a\in\mathcal{A}$	e.g., “pour into cup 3”	rollback $c_{\mathrm{rb}}(a)\in[0,\infty]$

Table 1: Node types of the recovery dependency graph.

Mode	Definition	Complexity
Indep	$\tilde{U}=\sum_{i}\hat{q}_{i}B_{i}$ for disjoint charged recovery operations	$O(\|\Phi\|)$
Beam- $K$	approximate $U(x)$ using up to $K$ weighted joint failure hypotheses	$O(KT_{B})$
MC	average $B(S)$ over $N_{\mathrm{mc}}$ posterior samples of the failure set	$O(N_{\mathrm{mc}}T_{B})$

Table 2: Computable proxies for residual rollback risk

U(x)

in the earlier evaluations. Evaluation complexities assume supplied marginal probabilities and local costs, retained hypotheses and weights, or posterior samples.

T_{B}

denotes the time required to evaluate one closure cost

B(S)

Family	Operation	Failed beliefs $\Phi^{\star}$	Attribution
Swap cups	swap two containers’ poses/IDs	water level / target of swapped instances	External
Add water	add $\Delta V$ across a decision boundary	filled cup’s level and ranking	External
Sensor drift	bias / timestamp / mis-binding (world unchanged)	beliefs on the corrupted evidence	Self_Error
False alarm	brief occlusion / noise spike (no task-relevant failure)	$\emptyset$	False_Alarm

Table 3: Perturbation families and ground-truth attribution.

Granularity	Edge semantics	Expectation
Linear chain	stage dependency; rollback = suffix	benefit mainly from zero rollback on false alarms
Fine causal	explicit causal/evidence edges; rollback = minimal upper closure	rollback set itself is smaller

Table 4: Two causal-edge granularities and their expected effects.

Method	$\mathrm{Succ}\uparrow$	$\overline{\mathcal{L}}\downarrow$	$p$ (Holm)
AquaMend (ours)	0.976	12.47	–
Lookahead (depth 2)	0.976	12.47	0.317
Decoupled	0.575	47.83	$<10^{-3}$
Greedy re-probe	0.575	46.14	$<10^{-3}$
Restart	0.333	53.16	$<10^{-3}$

Table 5: Selected comparisons on T4-Stress, the stress extension of discrete LatentProp-Bench task T4. Each method is evaluated on 4,600 paired episodes. Holm-adjusted

p

-values compare each method with AquaMend within the four designated non-oracle comparisons.

Method	$\mathrm{Succ}\uparrow$	$\overline{\mathcal{L}}\downarrow$	$\Delta\mathcal{L}$ [95% CI]	$p$ (Holm)
AquaMend (ours)	28/32	71.356	–	–
DTT	$25/32$	$72.116$	$0.761\;[-7.571,\,7.901]$	$0.465$
No extra re-probe	$21/32$	$84.326$	$12.970\;[1.014,\,27.115]$	$0.233$
Full restart	$24/32$	$90.966$	$19.610\;[12.944,\,26.523]$	$8.09\times 10^{-5}$
Linear chain	$22/32$	$76.391$	$5.035\;[-0.568,\,10.018]$	$0.077$

Table 6: Recovery in the execution-level simulation, with 32 paired scenarios per implementation. Complete loss is measured in simulation-second equivalents.

\Delta\mathcal{L}

is comparator loss minus AquaMend loss, with unadjusted 95% paired-bootstrap intervals; Holm correction covers the four comparisons.

Method	Realized cost $\downarrow$	Expected policy risk $\downarrow$
Exact Adaptive	8.560	9.253
Lookahead (depth 2)	8.565	9.255
AquaMend (ours)	8.594	9.290
Decoupled	11.008	11.756
Greedy re-probe	12.097	12.491

Table 7: Selected comparisons on M1, with 4,000 small graphs per method. Realized cost is evaluated on sampled hidden states; expected policy risk is computed from the public posterior.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Mode	Specification	Control parameter
Indep	Uses posterior marginal belief-failure probabilities. For pairwise-disjoint rollback sets, $\tilde{U}(x)=\sum_{i}\hat{q}_{i}B_{i}$ .	Marginal probabilities $\hat{q}_{i}$
Beam- $K$	Retains up to $K$ highest-probability joint belief-failure hypotheses, with candidate expansion along correlation edges, and approximates rollback risk using the retained hypotheses.	Beam width $K$
MC	Estimates rollback risk from $N_{\mathrm{mc}}$ samples of the joint belief-failure configuration drawn from the posterior.	Sample count $N_{\mathrm{mc}}$

Appendix

Table 8: Approximation families for the residual rollback risk

U(x)

. The displayed Indep expression applies to pairwise-disjoint rollback sets.

Beliefs	Worst-case depth	Policy	$\mathrm{Succ}\uparrow$	Realized cost $\downarrow$	Expected risk $\downarrow$	Time (ms) $\downarrow$	Memory (KiB) $\downarrow$
3	2	AquaMend Myopic	0.800	3.396	3.396	0.512	22.4
3	2	Lookahead (depth 2)	1.000	3.248	3.248	0.705	22.6
3	2	Lookahead (depth 3)	1.000	3.248	3.248	0.744	39.3
3	2	Exact Adaptive	1.000	3.248	3.248	0.533	19.0
4	3	AquaMend Myopic	0.000	4.000	4.000	1.013	28.2
4	3	Lookahead (depth 2)	0.600	3.679	3.679	2.931	54.9

Appendix

Table 9: Controlled planning-depth results on M1-Depth. Worst-case depth is the number of probes required to identify the final candidate. Decision time and peak memory report medians over deterministic policy evaluations.

Method	$\mathrm{Succ}\uparrow$	$\mathbb{E}[\mathcal{L}]\downarrow$	$\mathrm{Save}_{C}\uparrow$	$\#P$	$\#R$	$\mathrm{ZRR}\uparrow$	$p$ (Holm)
Oracle (reference)	1.000	3.31 [3.21, 3.40]	0.768	0.00	1.22	1.00	$<10^{-3}$
Restart	0.799	19.52 [19.23, 19.83]	$-0.370$	3.00	3.00	0.00	$<10^{-3}$
Blind retry	0.252	23.70 [23.26, 24.10]	$-0.663$	2.00	0.00	1.00	$<10^{-3}$
VeriTrace (rollback only)	0.204	39.64 [39.05, 40.21]	$-1.782$	0.00	2.50	0.00	$<10^{-3}$
Troubleshooting	0.555	20.26 [19.69, 20.76]	$-0.422$	1.00	1.50	1.00	$<10^{-3}$
Belief-space replan	0.204	33.60 [32.99, 34.18]	$-1.358$	0.00	0.26	1.00	$<10^{-3}$

Appendix

Table 10: Main comparison on T4 (reversible regime, both graph granularities). Each method is evaluated on 4,900 episodes. Brackets report 95% bootstrap confidence intervals for

\mathbb{E}[\mathcal{L}]

. The final column reports Holm–Bonferroni-adjusted paired Wilcoxon

p

-values against AquaMend. ZRR is evaluated on false-alarm episodes.

Method	$\mathrm{Succ}\uparrow$	$\mathbb{E}[\mathcal{L}]\downarrow$	$\#P$	$\#R$	$\mathrm{OverBudget}\downarrow$	$p$ (Holm)
Oracle (reference)	1.000	7.83	0.00	2.98	0.000	–
AquaMend (ours)	0.976	12.47	2.88	2.97	0.001	–
Lookahead	0.976	12.47	2.88	2.97	0.001	0.317
Decoupled	0.575	47.83	3.61	1.83	0.403	$<10^{-3}$
Greedy re-probe	0.575	46.14	3.61	1.83	0.403	$<10^{-3}$
Restart	0.333	53.16	5.74	3.48	0.000	$<10^{-3}$

Appendix

Table 11: Main comparison on the T4-Stress earlier benchmark. Each method is evaluated on 4,600 frozen paired episodes. The final column reports two-sided paired Wilcoxon

p

-values against AquaMend; Holm correction is applied separately to the four designated non-oracle comparisons and the five supplemental traditional-baseline comparisons.

Method	$\mathrm{Succ}\uparrow$	$\#R/C_{R}\downarrow$	$\#P/C_{P}\downarrow$	$\mathbb{E}[\mathcal{L}]\downarrow$	$\mathrm{Save}_{C}\uparrow$	$\mathrm{ZRR}\uparrow$
AquaMend (ours)	80/80	0.625 / 1.875	0.250 / 0.250	2.125	0.9941	1.000
Decision-theoretic troubleshooting	80/80	0.750 / 2.250	0.075 / 0.075	2.325	0.9936	1.000
Rollback-only	20/80	1.000 / 3.000	0 / 0	14.750	0.9594	0
Restart	80/80	1.000 / 0	0 / 0	362.870	0	0

Appendix

Table 12: Physics-grounded multi-cup execution validation on the combined T0/T1 episodes.

\#R/C_{R}

and

\#P/C_{P}

denote the mean rollback and re-probe counts/costs. Proxy loss is

\mathcal{L}=C_{P}+C_{R}+\sum_{k}L_{k}

. Restart is charged through the separately calibrated full-restart cost. ZRR is evaluated on false-alarm episodes.

Method	$n$	Realized cost $\downarrow$	Expected policy risk $\downarrow$	Gap to oracle $\downarrow$
Oracle (reference)	4,000	7.406	–	0.000
Exact Adaptive	4,000	8.560	9.253	1.154
Lookahead (depth 2)	4,000	8.565	9.255	1.159
AquaMend (myopic)	4,000	8.594	9.290	1.188
Troubleshooting	4,000	8.599	9.557	1.193
Decoupled	4,000	11.008	11.756	3.602

Appendix

Table 13: Exact small-graph comparison on M1. Each method is evaluated on 4,000 graphs across five seeds. Realized cost is charged on the sampled hidden realization, whereas expected policy risk is computed from the public posterior before that realization is revealed. The final column reports the realized-cost gap to the oracle.

Ablation	Removed/ varied	Hypothesis	$\mathbb{E}[\mathcal{L}]$	$\mathrm{Succ}$
No re-probe	rollback only	re-probing is necessary to lower $\mathcal{L}$	38.157	0.213
No attribution	treat all as External	attribution avoids needless rollback	5.625	0.900
No source separation	self and external merged	cross-modal selection matters for drift	5.541	0.901
Detect obj. $C_{s}/c$ vs. VoI	swap criterion	detection needs a falsification objective	5.431 / 10.335	0.901 / 0.708
Chain vs. fine	graph granularity	chain benefit is zero rollback on false alarms	6.830 / 5.360	0.999 / 0.999
$\tilde{U}$ proxy	Indep / Beam / MC	proxy accuracy vs. cost	5.431 / 5.431 / 5.432	0.901 / 0.901 / 0.901

Appendix

Table 14: Ablation results on standard T4. All rows except “Chain vs. fine” use the full 6,100-episode set, including the separately flagged irreversible subset; the graph-granularity row uses the 4,900 reversible episodes. Slash-separated values follow the order in the corresponding “Ablation” or “Removed/varied” entry. Lower

\mathbb{E}[\mathcal{L}]

and higher

\mathrm{Succ}

indicate better performance.

Method	Sensing	Rollback	Continue	Residual	Total
AquaMend	0.463	2.847	53.672	14.375	71.356
DTT	0.450	14.436	47.543	9.688	72.116
No extra re-probe	0.325	4.434	45.191	34.375	84.326
Full restart	0.781	24.495	51.315	14.375	90.966
Linear chain	0.444	11.899	40.923	23.125	76.391

Appendix

Table 15: Mean complete-loss decomposition in the new main batch (

n=32

per method). Physical components are simulation seconds; residual and total losses are simulation-second equivalents. Rounded entries can differ slightly from the rounded total.

Method	Mean-loss CI	$\mathrm{Save}_{C}$	$\#P$	$\#R$	ZRR
AquaMend	[64.833, 78.688]	0.256	0.625	0.312	1.000
DTT	[66.927, 77.555]	0.185	0.563	1.469	1.000
No extra re-probe	[78.399, 90.877]	0.348	0.000	0.469	1.000
Full restart	[83.443, 98.663]	0.000	2.000	2.562	0.000
Linear chain	[70.046, 83.426]	0.305	0.688	1.281	1.000

Appendix

Table 16: Supplementary physical metrics on the 32-scene main batch. Mean-loss CI denotes the unadjusted 95% scene-bootstrap interval for each method’s mean complete loss. ZRR is evaluated on eight predesignated false-alarm scenes and equals ZRR-and-success for the reported methods.

Stage	Scenes	AquaMend	All cand.	All cand. $-$ AquaMend [95% CI]
All	32	2.074	2.364	0.290 [0.212, 0.360]
Early	16	1.703	2.362	0.659 [0.499, 0.795]
Late	16	2.445	2.366	$-$ 0.080 [ $-$ 0.106, $-$ 0.051]

Appendix

Table 17: Complete online decision time in wall-clock seconds. Each scene contributes its median of four calls; intervals describe paired scene-level mean differences and are not adjusted simultaneous intervals.

Method	Stage	Calls	Elig.	Used	Uncov.	Reduced	Enum.	Eval.	Branches
AquaMend	Early	25	24	24	0	20	96	48	7383
AquaMend	Late	27	24	0	24	0	96	96	11094
All cand.	Early	25	0	0	0	0	96	96	11071
All cand.	Late	27	0	0	0	0	96	96	11094
Chain	Early	35	31	31	0	26	110	52	7692
Chain	Late	30	28	0	28	0	102	102	11220

Appendix

Table 18: Actual-trajectory candidate counts in the new main batch. Elig.: screening-eligible; used: screening applied; uncov.: uncovered fallback; reduced: decisions reducing candidates; enum./eval.: candidate totals; branches: predictive branches. Restart has no such decision calls; no-extra-re-probe has 16 decisions per stage and zero candidate evaluations.

Scene	Residual declarations	Safe stop	Residual	Complete loss
s05	Target	Yes	80	94.450
s13	Target	Not recorded	80	146.370
s19	Quantities A/B/C, target, sensing	Yes	150	157.854
s29	Quantities A/B/C, target, sensing	Yes	150	164.362

Appendix

Table 19: The four unsuccessful AquaMend scenes, also shared by the all-candidate ablation. Residual loss is charged only to the listed uncorrected declarations. “Not recorded” does not infer a cause from task failure.

Setting	Sensing	Rollback	Continue	Mean loss	Difference [95% CI]
Original	0.475	2.828	50.611	63.914	0.000 [0.000, 0.000]
Concentrated	0.475	2.828	50.611	63.914	0.000 [0.000, 0.000]
Dispersed	0.525	2.863	50.711	64.099	0.186 [0.023, 0.441]
Negative bias	0.475	2.828	50.611	63.914	0.000 [0.000, 0.000]
Positive bias	0.550	3.889	51.525	65.964	2.051 [0.023, 5.851]

Appendix

Table 20: Closed-loop Q1 results on eight new paired scenes, AquaMend only. Every setting has complete success and pour success of 7/8. Losses are simulation-second equivalents; differences are setting minus original, with exploratory 95% scene-paired bootstrap intervals.

Event	Invalid rate	Mean $\hat{q}$	Brier	ECE
binding_A	0.2500	0.2500	0.0000	0.0000
binding_B	0.2500	0.2500	0.0000	0.0000
binding_C	0	0.0000	0.0000	0.0000
rank_A	0.2500	0.2030	0.1143	0.1085
rank_B	0	0.0622	0.0310	0.0622
rank_C	0	0.0000	0.0000	0.0000

Appendix

Table 21: Old declarations before refresh: descriptive invalidity-probability metrics on 32 main TEST scenes.

Event	Invalid rate	Mean $\hat{q}$	Brier	ECE
binding_A	0	0.0000	0.0000	0.0000
binding_B	0	0.0000	0.0000	0.0000
binding_C	0	0.0000	0.0000	0.0000
rank_A	0.1250	0.2205	0.1106	0.0955
rank_B	0.1250	0.0788	0.0228	0.0462
rank_C	0	0.0000	0.0000	0.0000

Appendix

Table 22: Current declarations at the first refresh: descriptive invalidity-probability metrics on 32 main TEST scenes.

Setting	Changed commands	Mean posterior TV	Mean predicted loss
Original	0/32	0.0000	63.9249
Concentrated	1/32	0.0516	63.7192
Dispersed	8/32	0.0644	64.4001
Negative bias	0/32	0.1450	62.5118
Positive bias	8/32	0.1446	65.2785

Appendix

Table 23: Offline first-command sensitivity at 32 main TEST alarm states. Predicted loss is a model expectation, not realized physical loss. These states and denominators differ from the eight-scene Q1 closed-loop supplement.

Explore similar work

Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents
Apr 19, 2026Hanlin Wang, Chak Tou Leong, Jian Wang +1Embodied Agents Reasoning Skills
Department of Computing, The Hong Kong Polytechnic University · College of Computer Science, Sichuan University
Ask the World Before Acting: Environment Probing for Calibrated Agent World Models
Jun 30, 2026Xinyuan Song, Zekun CaiWorld Models Action Selection
Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan
Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
May 9, 2026Chenyu Zhao, Shenglin Zhang, Yihang Lin +7Model Debugging Diagnosis
Nankai University Tianjin, China · Tsinghua University Beijing, China · Microsoft Redmond, USA

Apr 19, 2026, cs.CLJ/K move · Enter open · S save

Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents

Hanlin Wang, Chak Tou Leong, Jian Wang, Wenjie Li

Department of Computing, The Hong Kong Polytechnic University · College of Computer Science, Sichuan University

Embodied Agents Reasoning Skills

Open paper AskPDF arXiv

Recent advancements in large language models (LLMs) have enabled agents to tackle complex embodied tasks through environmental interaction. However, these agents still make suboptimal decisions and perform ineffective actions, as they often overlook critical environmental feedback that differs from their internal beliefs. Through a formal probing analysis, we characterize this as belief inertia, a phenomenon where agents stubbornly adhere to prior beliefs despite explicit observations. To address this, we advocate active belief intervention, moving from passive understanding to active management. We introduce the Estimate-Verify-Update (EVU) mechanism, which empowers agents to predict expected outcomes, verify them against observations through explicit reasoning, and actively update prior beliefs based on the verification evidence. EVU is designed as a unified intervention mechanism that generates textual belief states explicitly, and can be integrated into both prompting-based and training-based agent reasoning methods. Extensive experiments across three embodied benchmarks demonstrate that EVU consistently yields substantial gains in task success rates. Further analyses validate that our approach effectively mitigates belief inertia, advancing the development of more robust embodied agents. Our code is available at https://github.com/WangHanLinHenry/EVU.

Abstract

Explore similar work

Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents

Ask the World Before Acting: Environment Probing for Calibrated Agent World Models

Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents