Mind2Act World: A Diagnostic Benchmark for Embodied Mind, Physical Action, and Mind-Action Composition in Robotic Manipulation
Abstract
Embodied intelligence aims to let robots perform complex physical tasks with human-like adaptability and reliability, which requires three complementary capabilities: history-based cognitive reasoning, precise and timely physical execution, and the composition of reasoning and execution through environmental feedback. Yet existing benchmarks either evaluate reasoning and execution within the same tasks or assess them separately without their composition, so they reveal little about whether a failure comes from reasoning, execution, or their composition. We introduce Mind2Act World, a diagnostic benchmark of 30 simulated manipulation tasks from five real-world application domains. Each task has three difficulty levels and two scene settings (Clean and Random), yielding 180 configurations. The tasks form three views: Mind tests decisions based on past observations, Act tests precise and timely execution, and Mind2Act combines both in closed-loop tasks where action outcomes guide later decisions. Comparing the three views shows whether a system lacks a capability or fails to compose capabilities it already has. A diagnosis harness further attributes the failed episodes of three representative systems to Mind, Act, or their composition and summarizes each system in a profile card. Our evaluation of multimodal agents, vision-language-action models (VLAs), and world-action models (WAMs) shows that competence in a single view does not carry over to Mind2Act tasks. Even the strongest agent succeeds in 52.7% of Mind and 39.0% of Act episodes but in only 12.0% of Mind2Act episodes. The strongest agent retains task state but struggles with sustained contact, whereas end-to-end policies execute practiced motions but retain little task state. These findings point to composing reasoning and execution through environmental feedback as the main open problem.
Figures & tables
| Application domain | # | Real-world need | Tasks |
| Home & Food Service | 8 | Kitchen prep, meal service, tidying | P1 P2 S1 PR4 CP01 CP02 CP03 CP04 |
| Manufacturing & Logistics | 5 | Part checks, traceability, sorting, interlocks, transport | E1 M1 DR1 DR3 DR4 |
| Instruments & Tool Use | 9 | Lab instruments, control panels, HMIs, office documents | E2 F1 DR2 PR1 PR2 PR3 PR5 CP06 CP09 |
| Assembly & Physical Reasoning | 3 | Building from drawings, limited stock, chain mechanisms | PR6 CP07 CP08 |
| Companion & Edutainment | 5 | Cognitive, music and sports training; puzzle play | F2 M2 S2 CP05 CP10 |
| Allocation | Agent | End-to-end policy |
| Tasks / difficulty levels / scene settings | 30 / 3 / 2 | 30 / 3 / 2 |
| Configurations | 180 | 180 |
| Episodes per configuration | 5 | 50 |
| Clean + Random episodes per task and difficulty | 10 | 100 |
| Episodes per task, across three difficulties | 30 | 300 |
| Episodes per view, ten tasks | 300 | 3,000 |
| Model | Type | Mind | Act | Mind2Act | Overall | |||||||
| All | Clean | Rand. | All | Clean | Rand. | All | Clean | Rand. | ||||
| Agents under one fixed harness | ||||||||||||
| GPT-6 Astra [ 70 ] [0.2mm] OpenAI, 2026 | Agent | All | 53.9/52.7 | 55.9/54.7 | 51.9/50.7 | 40.4/39.0 | 44.6/43.3 | 36.3/34.7 | 13.3/12.0 | 13.3/12.0 | 13.3/12.0 | 30.2/34.6 |
| Easy | 64.7/64.0 | 65.1/64.0 | 64.3/64.0 | 44.9/44.0 | 47.7/46.0 | 42.0/42.0 | 27.3/26.0 | 27.0/26.0 | 27.5/26.0 | 41.0/44.7 | ||
| Med. | 52.1/50.0 | 63.2/62.0 | 40.9/38.0 | 41.0/39.0 | 45.2/44.0 | 36.8/34.0 | 8.5/7.0 | 7.8/6.0 | 9.3/8.0 | 27.5/32.0 | ||
| Hard | 44.9/44.0 | 39.4/38.0 | 50.4/50.0 | 35.5/34.0 | 40.8/40.0 | 30.1/28.0 | 4.2/3.0 | 5.2/4.0 | 3.2/2.0 | 22.2/27.0 | ||
| ID | GPT-6 | DeepSeek | Gemini | KinRT | G0.5 | X-VLA | InternW0 | FastWAM | ID | GPT-6 | DeepSeek | Gemini | KinRT | G0.5 | X-VLA | InternW0 | FastWAM | ||
| Home & Food Service | Manufacturing & Logistics | ||||||||||||||||||
| P1 | 43 | 0 | 0 | 0.7 | 1.0 | 0 | 0 | 0.7 | 0.3 | E1 | 73 | 20 | 6.7 | 6.7 | 1.0 | 0 | 0 | 0 | 1.0 |
| P2 | 0 | 0 | 0 | 15 | 14 | 2.0 | 0 | 1.3 | 0 | M1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| S1 | 23 | 0 | 0 | 0 | 0 | 0 | 0.3 | 0.7 | 0 | DR1 | 77 | 6.7 | 0 | 19 | 13 | 25 | 0 | 23 | 2.0 |
| PR4 | 0 | 0 | 0 | 18 | 10 | 17 | 64 | 6.3 | 0 | DR3 | 80 | 27 | 40 | 79 | 79 | 51 | 76 | 73 | 61 |
| CP01 | 60 | 0 | 0 | 8.0 | 8.7 | 8.0 | 1.3 | 1.7 | 4.3 | DR4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Model | Type | Time | Episode time (min) | API cost (USD) | ||||
| / call | All | Mind | Act | Mind2Act | / episode | / success | ||
| Agents under one fixed harness (900 episodes each) | ||||||||
| GPT-6 Astra | Agent | 11.0 s | 22.2 | 19.1 | 16.8 | 30.7 | 25.13 | 72.7 |
| DeepSeek-V4.1-Flash | Agent | 35.6 s | 96.1 | 91.9 | 93.0 | 103.2 | 1.24 | 18.9 |
| Gemini-4-Argon | Agent | 27.8 s | 18.0 | 15.2 | 15.8 | 22.9 | 1.12 | 24.0 |
| End-to-end policies: VLAs and WAMs (9,000 episodes each) | ||||||||
| Failure group | Evidence in this report | Direction |
| Mind : M-MEMORY , M-SPATIAL , M-RULE | 924 misremembered-information failures of G0.5 and 1,011 of InternW0; VLAs lose 59–72% of Mind successes under Random scenes | Task state across stages : keeping what was seen and recognizing items under a new appearance. Data : varied scenes and longer gaps between when information is shown and when it is needed |
| Act : A-GRASP , A-PRECISION , A-TIMING | 1,324 grasp failures of G0.5 and 1,132 fine-control failures of InternW0; 202 of 300 G0.5 episodes on CP07 never grasp a piece | Grasp, contact and timing : reliable grasps and fine contact, and responses within the windows of DR2 and DR3. Data : demonstrations weighted toward these segments, with more variation in pose, contact and timing. Harness : finer motion tools |
| Composition : C-NO-VERIFY , C-NO-REPAIR , C-PREREQ | 761 precondition failures of InternW0, 271 of them on CP04, which heats the pot before all ingredients are in; agents that stop with budget left (Section 6.2 ) | Checking and repairing outcomes : checking a step before the next one and repairing it when it failed. Data : demonstrations that include failed steps and their repair. Harness : the outcome of each step returned, a check required before moving on, and a retry before stopping |
| All groups , by breakpoint | 299 of 300 G0.5 episodes on CP09 end before the board is lifted | Targeted data for end-to-end policies : demonstrations that start from the states in which episodes stop and from perturbed states near them, instead of new full-length trajectories |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Difficulty | Frames | Duration (h) | Episodes | Avg. frames / episode | Avg. duration / episode (s) |
| Mind | |||||
| Easy | 549,776 | 5.09 | 500 | 1,099.55 | 36.65 |
| Medium | 868,075 | 8.04 | 500 | 1,736.15 | 57.87 |
| Hard | 1,216,463 | 11.26 | 500 | 2,432.93 | 81.10 |
| Total | 2,634,314 | 24.39 | 1,500 | 1,756.21 | 58.54 |
| Act | |||||
| Family | Input | Output | Must disclose |
| VLA / WAM | Instruction, 1–3 RGB views, joint state | Joint or EEF action chunk | Checkpoint, chunk, control period, data |
| Agent | Instruction, RGB, history, tool feedback | Motion-tool calls | Model, harness, tools, budget, latency |
| Hybrid | As above, split across modules | Planner calls and controller actions | Module boundaries, fallbacks, cost |
| Not available to any system: hidden targets and tolerances, gate timing, drift radii, depth and pixel-to-world queries. | |||
| Leaderboard | Physics steps | Decisions | Wall-clock safety limit |
| End-to-end | unlimited | 2 hours | |
| Agent | 150 | 6 hours |
| System | Clean Overall | Clean successes | Random Overall | Random successes |
| GPT-6 Astra | 31.8 | 165/450 | 28.7 | 146/450 |
| DeepSeek-V4.1-Flash | 5.8 | 28/450 | 6.4 | 31/450 |
| Gemini-4-Argon | 4.8 | 24/450 | 3.9 | 18/450 |
| KinRT | 9.1 | 364/4,500 | 6.8 | 264/4,500 |
| 8.9 | 340/4,500 | 6.3 | 234/4,500 | |
| G0.5 | 6.7 | 288/4,500 | 5.8 | 242/4,500 |
| System | Mind | Act | Mind2Act | All episodes |
| Agents (300 episodes per view) | ||||
| GPT-6 Astra | 158 [47.0, 58.2] | 117 [33.7, 44.6] | 36 [8.8, 16.2] | 311 [31.5, 37.7] |
| DeepSeek-V4.1-Flash | 33 [7.9, 15.0] | 26 [6.0, 12.4] | 0 [0.0, 1.3] | 59 [5.1, 8.4] |
| Gemini-4-Argon | 13 [2.5, 7.3] | 29 [6.8, 13.5] | 0 [0.0, 1.3] | 42 [3.5, 6.2] |
| End-to-end policies (3,000 episodes per view) | ||||
| KinRT | 132 [3.7, 5.2] | 408 [12.4, 14.9] | 88 [2.4, 3.6] | 628 [6.5, 7.5] |
| Task | GPT-6 | DeepSeek | Gemini | KinRT | G0.5 | X-VLA | InternW0 | FastWAM | |
| Mind | |||||||||
| E1 | 76.9 80 / 67 | 22.9 13 / 27 | 7.0 7 / 7 | 11.9 7 / 7 | 3.7 1 / 1 | 0.4 0 / 0 | 0.2 0 / 0 | 0.0 0 / 0 | 2.7 0 / 2 |
| E2 | 64.7 60 / 67 | 24.3 20 / 20 | 10.0 7 / 7 | 2.6 1 / 1 | 2.2 0 / 0 | 2.9 0 / 0 | 3.0 0 / 2 | 2.6 0 / 1 | 0.9 1 / 0 |
| P1 | 46.8 53 / 33 | 0.4 0 / 0 | 1.5 0 / 0 | 1.0 1 / 1 | 1.3 1 / 1 | 0.1 0 / 0 | 0.0 0 / 0 | 0.8 1 / 1 | 0.4 1 / 0 |
| P2 | 0.3 0 / 0 | 0.0 0 / 0 | 0.0 0 / 0 | 17.8 30 / 1 | 17.2 27 / 1 | 4.6 3 / 1 | 0.0 0 / 0 | 3.1 3 / 0 | 0.0 0 / 0 |
| S1 | 23.8 33 / 13 | 0.0 0 / 0 | 0.2 0 / 0 | 0.4 0 / 0 | 0.3 0 / 0 | 0.2 0 / 0 | 0.8 1 / 0 | 1.4 1 / 0 | 0.0 0 / 0 |
| G0.5 | |||||||
| ID | Task | Easy | Med. | Hard | Easy | Med. | Hard |
| DR1 | Conveyor pickup | 34 | 5 | 0 | 70 | 4 | 0 |
| DR2 | Light-cued response | 0 | 0 | 0 | 37 | 39 | 34 |
| DR3 | Timed passage through cyclic gates | 76 | 81 | 79 | 51 | 51 | 51 |
| PR1 | Resistance adjustment | 23 | 13 | 13 | 6 | 3 | 0 |
| PR4 | Cloth folding and flat placement | 28 | 1 | 1 | 51 | 0 | 1 |
| Perceive Mind | Execute Act | |||
| M-PERCEIVE | misrecognized an object, color or reading | A-REACH | approach or positioning failed | |
| Reason Mind | A-GRASP | grasp failed or not held | ||
| M-GOAL | misunderstood the goal of the task | A-PLACE | placement inaccurate or unstable | |
| M-MEMORY | forgot or misremembered information | A-PRESS | press, turn or other interaction had no effect | |
| M-RULE | inferred a wrong rule or mapping | A-PRECISION | error in continuous fine control | |
| M-SPATIAL | wrong spatial mapping or layout | A-TIMING | action too early, too late or too slow | |
| ID | Task | GPT-6 | DeepSeek | Gemini | KinRT | G0.5 | X-VLA | InternW0 | FastWAM | |
| CP01 | Conveyor meal assembly | 18 | 0 | 0 | 24 | 26 | 24 | 4 | 5 | 13 |
| CP02 | Rotating-cake decoration | 5 | 0 | 0 | 1 | 2 | 0 | 0 | 1 | 0 |
| CP03 | Moving-rack loading | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CP04 | Temperature-controlled cooking | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CP05 | Piano replay | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CP06 | Produce weighing on a balance | 3 | 0 | 0 | 51 | 62 | 37 | 2 | 48 | 1 |
| System | Failed | Rule layer | LLM judge |
| G0.5 (VLA) | 8,470 | 3,230 (38.1%) | 5,240 |
| InternW0 (WAM) | 8,220 | 2,760 (33.6%) | 5,460 |
| GPT-6 Astra (agent) | 589 | 131 (22.2%) | 458 |
| GPT-6 Astra | DeepSeek-V4.1-Flash | Gemini-4-Argon | ||||
| Stop reason | Clean | Random | Clean | Random | Clean | Random |
| Stopped by the agent | 223 | 237 | 175 | 210 | 409 | 420 |
| Decision budget exhausted | 15 | 14 | 160 | 126 | 0 | 0 |
| Physics budget exhausted | 37 | 38 | 50 | 44 | 4 | 2 |
| Completion flag recorded | 175 | 161 | 64 | 70 | 37 | 28 |
| Adjudicated manually | 0 | 0 | 1 | 0 | 0 | 0 |
| Dimension | Letter | In MBTI, a person | In Mind2Act World, a policy | Measured on |
| 1 Energy | E | turns outward, to action | Act-led: executes better than it reasons | Act view |
| I | turns inward, to thought | Mind-led: reasons better than it executes | Mind view | |
| 2 Information | S | attends to concrete facts | keeps track of what each object is | E1, E2 |
| N | attends to the overall pattern | keeps track of where things are | S1, S2 | |
| 3 Decisions | T | decides by objective criteria | meets a fixed tolerance (precision) | PR1–PR6 |
| F | decides by the situation | acts as the scene changes (reactivity) | DR1–DR4 |
| View | ID | Task | Source | Category or demand pairing; difficulty variables |
| Mind | E1 | Repeated-item recognition | Part inspection | Entity memory; batches and objects |
| E2 | Object-coded keypad arithmetic | Calculation from a reference | Entity memory (object-value bindings); bindings, expression length, repeats | |
| F1 | Unknown button mapping | Unfamiliar control panels | Feedback-rule learning; grid size and route length | |
| F2 | Color-block arrangement by feedback | Mastermind-style trial | Feedback-rule learning (hidden arrangement); blocks and submission budget | |
| P1 | Incremental meal orders | Food-service orders | Process memory; containers, deposits, and new orders | |
| P2 | Steamer-stack order restoration | Dim-sum steamer service | Process memory (layer order); layers and distractor steamers |