Knowing When to Yield: Grounded Arbitration of User Corrections in Text-Based Embodied Agents
Organizations: University of Wisconsin - Madison · Independent Researcher · Brown University · Pinterest Inc
Abstract
How should an embodied agent respond when a person's correction may be wrong? We formulate grounded correction arbitration as a choice among accepting, rejecting, inspecting the world, and asking the speaker. GAVA implements this interface with observation-bounded evidence, legal probes, and a one-step expected-loss rule. In text-only ALFWorld, 162 checkpoints produce 972 paired true and false interventions. Complete local inspections give GAVA and always verify 100 percent correction accuracy, establishing the evidence contract rather than a comparative advantage. In same-episode execution, GAVA reduces interaction cost against always verify but ties a cost threshold under a perfect speaker. An exploratory training-only object-location prior lowers interaction and declared joint cost on 340 unseen scenarios by 0.490 and 0.420 relative to uniform GAVA. After freezing the policy, costs, baselines, and multiplicity plan, the gains replicate on 77 non-overlapping seen checkpoints, covering 308 scenarios: 0.595 and 0.517, with both 95 percent checkpoint-bootstrap confidence intervals excluding zero. Joint cost also improves over an identical-prior fixed policy, while the matched calibrated no-VOI comparison remains inconclusive. Semantic GAVA makes four factual errors in each cohort, corresponding to 98.8 percent and 98.7 percent accuracy, and all methods complete every task. Results support selective information gathering with semantic priors under declared costs, but do not establish a general advantage of environmental value of information over clarification. The study uses normalized claims, complete symbolic observations, and controlled speakers; it evaluates neither human participants, visual input, nor physical robots.
Figures & tables
| Research line | Executable actions | Fallible correction | Active probe | Observable-state boundary |
|---|---|---|---|---|
| Robot language correction ( Thierauf et al., 2024 ; Sharma et al., 2022 ; Cui et al., 2023 ; Liu et al., 2023 ; Shi et al., 2024 ) | Yes | – | – | varies |
| Closed-loop embodied LLM planning ( Huang et al., 2022 ; Bhat et al., 2024 ) | Yes | – | feedback | varies |
| Active HRI query and clarification ( Deits et al., 2013 ; Cakmak and Thomaz, 2012 ) | Yes | – | dialogue | Yes |
| Text/visual correction selectivity ( Sinha, 2026 ; Pi et al., 2025 ; Beigi et al., 2025 ) | – | Yes | – | – |
| This work | Yes | Yes | Yes | Yes |
| Split | All | Pilot/quota | Inspected | Timeout | Other | Accepted |
|---|---|---|---|---|---|---|
| Unseen | 134 | 26 | 108 | 0 | 23 | 85 |
| Seen | 140 | 29 | 111 | 21 | 13 | 77 |
| Split | Checkpoints | Pairs | Interventions |
|---|---|---|---|
| Valid unseen | 85 | 255 | 510 |
| Valid seen | 77 | 231 | 462 |
| Total | 162 | 486 | 972 |
Embodied AI
Embodied AI studies agents that perceive, reason, and act through bodies in physical or simulated environments.
See all 255 papers