MOTIO: Agentic Skill Engineering for Reality-Grounded Humanoid Loco-Manipulation
Abstract
We present MOTIO, an agentic skill-engineering system that turns ordinary human-action videos, reality-grounded 3D assets, motion references, and reusable robot skills into executable humanoid loco-manipulation programs. Through in-context task and contact reasoning, an agent constructs staged robot-native interactions and improves them using physics execution, multi-frame feedback, and human validation---without task-specific teleoperation or robot demonstration collection. Agentic 3D supplies metric product and environment assets, while existing whole-body, inverse-kinematics, and dexterous-hand interfaces realize the programs. Across 33 timestamped first approvals, the median recorded agent-turn time is 97.0 minutes and request-to-approval duration is 2.80 hours. Simulation-side layout and object variation prepare a long-horizon bottle-disposal skill for a continuous 101.75-second physical G1 execution in a scanned office; physical refinement reduces screw-tightening thread residual by 76.0%. Agent-generated skills additionally supervise BC/DAgger distillation into one shared reference policy, completing 18 simulated tasks under the same checkpoint without inference-time expert intervention. Project page: https://iiiiqiiii.github.io/motio/
Figures & tables
| Study | Variation or sequence | Observed result |
| Simulation preparation | ||
| Full-task layouts | 31 configurations | 31/31 complete sequences |
| Bottle properties | 44 unique settings | 44/44 full-window grasps |
| Paired grasp refinement | Same 12 settings | 11/12 12/12 full-window retention |
| Perception-conditioned execution | 3 whole-body variants | 3 complete eight-stage sequences |
| Physical G1 execution | ||
| Measure | Median | IQR |
|---|---|---|
| Agent-turn (min) | 97.0 | 42.5–170.3 |
| Request-to-approval (h) | 2.80 | 1.33–9.80 |
| Agent turns | 2 | 1–2 |
| Retained dynamic versions | 18 | 7–31 |
| Visual inspections | 10 | 7–18 |
| Human revision prompts | 1 | 0–1 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Product asset | Physical model | Skill interface / event |
|---|---|---|
| Rapesco 159 41 53 mm | Base, lever, magazine, press cap, driver, anvil and contact proxies; articulated lever and magazine. | Right-palm press-cap target, left-hand paper stabilization, contact-gated paper connection and withdrawal. |
| HEUER vise | Fixed seat, translating jaw, screw, moving handle, supported workpiece; screw/jaw coupled at model lead 4 mm/turn. | Hand arc referenced to screw angle; opposing clamp, jaw clearance and retained workpiece measured at task completion. |
| Window | Program phase | Observation |
|---|---|---|
| 0–8 s | Reach and grasp handle bar; workpiece initially supported. | Initial reach/contact. |
| 8–36 s | Four 7-s half-turn cycles with unflex, retreat, reposition, regrasp. | Rotation, contact and jaw travel. |
| 36–41 s | Final tighten, release hand and withdraw support. | Clamp loading and release. |
| 41–48 s | Hold test after both hand and support withdraw. | Workpiece retention and gap (38 to 29.987 mm). |
| Review input | Program revision | Physical result |
| Palm-driven press and natural hand withdrawal. | 13–16 s palm press, 16–17 hold, 17–19 unload, 19–21 paper release, 21–24 elbow retract, 24–29 lower arms. | Staple connection event 14.8125 s; paper joined, hands withdrawn by 31 s; human approval. |
| Study | Variation and criterion | Observed result |
|---|---|---|
| Room layout | 11 named, 12 development-random, 8 post-freeze; full grasp, carry, bin release, return. | 31/31 full-task completions; 61.2–138.6 s. |
| Bottle properties | 58–68 mm diameter, 220–235 mm height, 20–600 g; hand fixture at 8–10 s. | 44/44 opposing-grasp windows. |
| Paired refinement | The same 12 LHS settings, earlier versus refined contact window. | 11/12 to 12/12 full-window retention. |
| Visual whole-body | Measured, narrow-heavy and wide-light parameter conditions. | 3/3 eight-stage completions. |
| ID | Skill | Agent | Calendar | Versions | Visual |
|---|---|---|---|---|---|
| 40 | Mango peeling | 80.6 | 1.67 | 9 | 17 |
| 41 | Caliper measurement | 204.2 | 18.52 | 7 | 18 |
| 43 | Vise clamping | 93.8 | 19.51 | 5 | 6 |
| 44 | Drawer retrieval | 170.3 | 19.53 | 8 | 13 |
| 45 | Dowel insertion | 79.3 | 19.48 | 3 | 9 |
| 46 | Gluing paper | 123.2 | 2.42 | 31 | 20 |
| Component | Shared policy configuration |
|---|---|
| Observation | 176 task/robot state slots, 32 mechanism values with 32 validity-mask bits, skill and phase identifiers, public clock and explicit geometry memory. |
| History | reference_history=False ; current public phase and geometry memory condition reference prediction. |
| Output | Six references s, each 59 channels: 29 body, 12 hand, 6 palm positions and 12 palm 6D rotations. |
| Executor | 50 Hz reference decoder, frozen ScaleBFM tracker, bounded arm IK and task-event/contact feedback. |
| Training | BC initialization, student-state DAgger expert labeling and event-gated recovery corrections. |
| Task | Time (s) | Dev. | Repeat |
|---|---|---|---|
| Bottle pickup | 60 | yes | yes |
| Light switch | 48 | yes | yes |
| Drawer | 66 | yes | yes |
| Stapler | 31 | yes | yes |
| Standing fan | 50 | yes | yes |
| Faucet | 28 | yes | yes |
| Task | Duration | Physical sequence |
|---|---|---|
| Bottle disposal | 101.7543 s | Grasp, turn, carry, bin release, retreat. |
| Door and cart transport | 98.41 s | Door opening and pushing a cart through doorway. |
| Chair sit-to-stand (unsupported) | 24.74 s | Sit, hold and return to stand. |
| Chair sit-to-stand (assisted) | 24.99 s | Sit, hold and return to stand. |
| Light switch | 20.48 s | Wall-switch actuation. |
| Standing mouse drawing | 28.26 s | Desktop drawing with physical mouse. |