cs.AISep 25, 2026

Knowing When to Yield: Grounded Arbitration of User Corrections in Text-Based Embodied Agents

Authors: Yezhou Cheng, Runjia Du, Zeming Liu, Hang Lyu, Zehua Yang, Bojun Lin

Organizations: University of Wisconsin - Madison · Independent Researcher · Brown University · Pinterest Inc

Abstract

How should an embodied agent respond when a person's correction may be wrong? We formulate grounded correction arbitration as a choice among accepting, rejecting, inspecting the world, and asking the speaker. GAVA implements this interface with observation-bounded evidence, legal probes, and a one-step expected-loss rule. In text-only ALFWorld, 162 checkpoints produce 972 paired true and false interventions. Complete local inspections give GAVA and always verify 100 percent correction accuracy, establishing the evidence contract rather than a comparative advantage. In same-episode execution, GAVA reduces interaction cost against always verify but ties a cost threshold under a perfect speaker. An exploratory training-only object-location prior lowers interaction and declared joint cost on 340 unseen scenarios by 0.490 and 0.420 relative to uniform GAVA. After freezing the policy, costs, baselines, and multiplicity plan, the gains replicate on 77 non-overlapping seen checkpoints, covering 308 scenarios: 0.595 and 0.517, with both 95 percent checkpoint-bootstrap confidence intervals excluding zero. Joint cost also improves over an identical-prior fixed policy, while the matched calibrated no-VOI comparison remains inconclusive. Semantic GAVA makes four factual errors in each cohort, corresponding to 98.8 percent and 98.7 percent accuracy, and all methods complete every task. Results support selective information gathering with semantic priors under declared costs, but do not establish a general advantage of environmental value of information over clarification. The study uses normalized claims, complete symbolic observations, and controlled speakers; it evaluates neither human participants, visual input, nor physical robots.

Figures & tables

Explore similar work

CardsList
  1. Correction or Manipulation? Measuring Epistemic Vigilance in LLM Agents Under User Pressure

    Oct 27, 2026Nardine Basta, Venus Haghighi, Dali Kaafar +1LLM Agent EvaluationLLM Agent Safety

  2. ACCORD: Action-Conditioned Contextual Grounding for Language Agents

    Jun 15, 2026Lai Jiang, Cheng Qian, Zhenhailong Wang +3LLM GroundingTool-Augmented Language Model Agents