Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. However, under partial observability, the availability of goal-relevant objects may be uncertain. In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) may generate subgoals that rely on the VLM's prior knowledge without observational support, leading to execution failures or unintended outcomes. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual evidence through VLM-generated exploratory subgoals and TAMP-based execution. EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire further evidence, or halt. Our experiments show that, in cooking tasks with ambiguous object use, EAFG improves recipe completion by discovering task-relevant objects before planning. For instructions requiring an absent object, EAFG promotes appropriate halt decisions and reduces repeated attempts to manipulate that object.
Figures & tables
Figure 1: Overview of Evidence Acquisition and Feasibility Gating (EAFG). The system executes evidence-acquisition subgoals to verify goal-relevant objects and conditions, updates the evidence state from the resulting observations, and uses feasibility gating to decide whether to continue exploration, halt, or proceed to task planning.
Figure 2: Kitchen environment used in the experiments. (a) Initial state of the kitchen. (b) Layout of objects in the kitchen.
Setting
Goal
Research Question
Metrics
S1: Explicit-and-present
make chicken soup with salt and pepper
RQ1: Can EAFG complete each recipe step and the full recipe when explicitly specified objects are present?
Step CR, Recipe CR
S2: Underspecified
make chicken soup
RQ2: Can EAFG acquire the required information through exploration and complete the same recipe under an underspecified goal?
Step CR, Recipe CR
S3: Explicit-but-missing
make chicken soup with salt, pepper, and carrot
RQ3: Can EAFG halt when a specified object is absent while reducing repeated planning attempts involving the missing object?
Halt SR, Missing-object Attempt Count
Table 1: Experimental settings, research questions, and evaluation metrics. CR and SR denote completion rate and success rate, respectively.
EAFG
VLM
Step CR
Recipe CR
Chicken
Water
Pepper
Salt
Stove
GPT-5.5
0.95
0.80
0.75
0.80
0.20
0.20
✓
GPT-5.5
1.00
0.55
1.00
1.00
0.65
0.45
Gemini-3.5-Flash
0.95
0.80
0.90
0.95
0.15
0.15
✓
Gemini-3.5-Flash
1.00
0.85
1.00
1.00
0.10
0.10
Table 2: S1: Explicit-and-present (RQ1) results for the instruction make chicken soup with salt and pepper , comparing runs with and without EAFG across VLMs. Each condition uses 20 runs. Step CR and Recipe CR are reported as rates in [0,1] .
EAFG
VLM
Step CR
Recipe CR
Chicken
Water
Pepper
Salt
Stove
GPT-5.5
1.00
0.95
0.30
0.10
0.40
0.05
✓
GPT-5.5
1.00
0.75
1.00
1.00
0.55
0.40
Gemini-3.5-Flash
0.90
0.90
0.05
0.05
0.30
0.00
✓
Gemini-3.5-Flash
0.95
0.85
0.85
0.95
0.20
0.20
Table 3: S2: Underspecified (RQ2) results for the instruction make chicken soup , comparing runs with and without EAFG across VLMs. Each condition uses 20 runs. Step CR and Recipe CR are reported as rates in [0,1] .
EAFG
VLM
Halt SR
Missing-object Attempt Count
GPT-5.5
0.45
4.00
✓
GPT-5.5
0.90
0.55
Gemini-3.5-Flash
0.40
2.40
✓
Gemini-3.5-Flash
1.00
0.00
Table 4: S3: Explicit-but-missing (RQ3) results for the instruction make chicken soup with salt, pepper, and carrot , where carrot is absent from the environment. Each condition uses 20 runs. Halt SR is reported as a rate in [0,1] and Missing-object Attempt Count is the mean number of execution attempts involving the absent object.