Open-world mobile manipulation requires a robot to find an object, approach it, and move it to a target receptacle from a language instruction. Training vision-language model (VLM) agents for these tasks requires demonstrations of both action selection and active vision. Existing pipelines often convert planned trajectories into high-level action labels, but shortest-path progress provides limited supervision for deciding where to look. We introduce Agent-Interface Data Alignment (AIDA), a data collection pipeline in which a teacher directly selects and executes actions through the deployed agent's skill interface. For mobile manipulation, we design teachers that maintain target visibility, recover lost targets, and approach for picking or placing. We add template-based spatial reasoning supervision and use a shared harness to organize observations, history, and outputs during training and inference. The resulting agent, AIDA-OWMM, achieves 71.76% full-task success on 170 held-out OWMM episodes. Teacher sensitivity analysis shows that active-vision demonstrations improve student success. The simulation-trained agent achieves 38% end-to-end success on a real mobile robot without additional training. Applying AIDA to a navigation subset of EmbodiedBench further demonstrates the pipeline's utility for navigation tasks.
Figures & tables
Figure 1: AIDA-OWMM overview. Left: the teacher generates demonstrations by selecting and executing high-level actions. Right: spatial annotations and a shared agent harness support training and deployment. The real-robot rate shown includes low-level grasp failures among recorded attempts.
Figure 2: OWMM agent interface. The VLM selects search, navigation, turn, pick, and place skills, which fixed controllers execute as robot motions. Scene key frames are provided during search.
Figure 3: AIDA-OWMM pipeline. The teacher collects high-level skill demonstrations, the annotator provides spatial reasoning targets, and the harness constructs training examples and maintains the deployed agent’s context.
Method
Base model
Image Retrieval (Object)
Object Picked
Image Retrieval (Goal)
Robot Close to Goal
Full Task
OWMM-VLM-38B
InternVL-38B
88.56%
38.56%
30.39%
23.53%
21.90%
OWMM-VLM-8B
InternVL-8B
81.43%
17.92%
15.96%
10.42%
9.45%
GPT-4o + PIVOT
GPT-4o
59.15%
0.65%
0.33%
0.00%
0.33%
GPT-4o + RoboPoint
GPT-4o
56.86%
1.31%
0.00%
0.00%
0.33%
AIDA-OWMM (ours)
Qwen3-VL-8B-Instruct
90.59%
82.35%
74.12%
72.35%
71.76%
Table 1: OWMM system comparison. Baseline results are reported by OWMM-Agent; AIDA-OWMM is evaluated on 170 held-out episodes. Intermediate columns report stage completion, and Full Task reports complete object transfer.
Demonstrations
Samples
Success
OWMM, relabeled
152,478
38.8%
AIDA: distance-greedy
28,480
46.3%
AIDA: visibility-aware
28,480
65.5%
AIDA: target recovery
28,480
69.6%
Table 2: Teacher sensitivity. All students use four skills without turn . Sample counts denote training examples.
Key frames
History
Reasoning
Success
Every stage
None
No
55.88%
Search only
None
No
60.00%
Search only
Actions
No
65.88%
Search only
Actions
Yes
71.76%
Table 3: Harness ablations. All rows use the same AIDA demonstrations and five skills. Reasoning denotes spatial reasoning supervision.
Figure 4: Real-robot rollout. The simulation-trained agent turns to locate the object, approaches using pixel-coordinate goals, picks it, and searches for the destination receptacle.
Model
Base
Common
Avg.
Zero-shot VLMs
GPT-5.5
0.78
0.77
0.78
GPT-4o
0.43
0.37
0.40
OpenAI o3
0.53
0.50
0.52
Gemini 3 Flash
0.08
0.15
0.12
Claude Sonnet 4.6
0.27
0.33
0.30
Table 4: EmbodiedBench navigation. The EB-trained policy tests the AIDA pipeline on navigation; the OWMM-trained policy is a zero-shot reference.
Table 5: AIDA-OWMM model, training, and evaluation configurations. The shared prompt and output format is defined in Section 3.5 ; evaluation uses the trained LoRA adapter.