ACE-Brain-0.7: Learning Execution-Grounded Reasoning for Embodied Agents
Abstract
Reliable reasoning for long-horizon embodied tasks requires predictions grounded in the observed execution state. We present ACE-Brain-0.7, a 9B vision--language model that combines spatial grounding, task understanding, and progress estimation. To train the model, we curate 3.7 million question--answer pairs, drawing execution supervision from egocentric human videos, robot episodes, and simulation. These questions span different stages of an execution and connect task-level judgments with observed progress and outcomes. We then fine-tune Qwen3.5-9B on this collection together with additional task, spatial-grounding, and progress-estimation data. Across six public task-reasoning benchmarks, ACE-Brain-0.7 achieves an average score of 76.5, ahead of GPT-6 Astra at 72.7 and Claude Opus 5.5 at 73.5. Our model also leads the evaluated models in spatial grounding, averaging 75.90 across seven benchmarks and reaching 91.25 [email protected] on RefCOCOg. Controlled evaluations further indicate that our task-only model makes more effective use of visual history and more accurately distinguishes execution states. Together, these results show that ACE-Brain-0.7 effectively combines spatial perception with temporal understanding, delivering strong execution-grounded reasoning and comparing favorably with closed-source models.
Figures & tables
| Capability | Query on | Reads | Answer derived from | In Figure 1 |
| Plan | A linear extension of | Annotated sub-task sequence | What are the steps? Put the block into the bowl; open the door; … | |
| Monitor | for a named ; object state | Time-stamped steps; state labels | Is the bowl in the microwave? No. | |
| Anticipate | A member of the frontier | , | Step that actually followed | What comes next? Move the right arm to the bowl. |
| Diagnose | , with type and location | Failure annotations | Which sub-task failed? Move the right arm to the bowl. | |
| Interpret | and scene facts, from the observed states | Inverse | Task and scene labels | What task is this? Place the block in the bowl, then the bowl in the microwave. |
| Model | EgoPlan-Bench | Cosmos-Reason1 | RoboBench | LADDER | RoboFAC | ERIQ | Avg6 | MAE |
| ACE-Brain-0.7 (ours) | 74.5 | 83.5 | 67.4 | 79.0 | 69.2 | 85.1 | 76.5 | 0.087 |
| Closed models | ||||||||
| GPT-6 Astra [ 6 ] | 75.9 | 77.5 | 66.0 | 75.7 | 53.2 | 87.9 | 72.7 | 0.125 |
| Claude Opus 5.5 [ 7 ] | 70.1 | 76.5 | 68.9 | 79.5 | 57.7 | 88.4 | 73.5 | 0.118 |
| General-purpose open models | ||||||||
| Qwen3.5-9B [ 4 ] (base) | 48.8 | 62.0 | 55.6 | 68.8 | 35.9 | 71.4 | 57.1 | 0.257 |
| Q1: task decomposition | Q2: next sub-task | Q3: step completion | ||||||||||
| Task family | Ours | Astra | Opus | Ours | Astra | Opus | Ours | Astra | Opus | |||
| Single arm | 235 | 85.7 | 84.5 | 88.3 | 231 | 61.0 | 35.5 | 38.2 | 367 | 78.7 | 76.6 | 78.2 |
| Rigid objects | 76 | 84.9 | 92.5 | 93.6 | 92 | 63.6 | 43.8 | 39.7 | 134 | 65.7 | 70.1 | 67.9 |
| Deformable objects | 114 | 69.0 | 78.6 | 84.3 | 103 | 51.3 | 34.0 | 44.7 | 137 | 78.1 | 73.7 | 79.6 |
| Articulated objects | 152 | 75.2 | 86.2 | 88.5 | 60 | 44.4 | 46.7 | 49.4 | 125 | 76.0 | 65.6 | 65.6 |
| Special materials | 119 | 71.8 | 82.4 | 87.3 | 90 | 53.5 | 32.0 | 38.9 | 119 | 73.9 | 73.1 | 74.8 |
| Input | EgoPlan-Bench | Cosmos-Reason1 | RoboBench | LADDER | RoboFAC |
| Blind (no images) | / | / | / | / | / |
| First frame only | / | – | / | / | / |
| Last frame only | / | – | / | / | / |
| Shuffled frames | / | – | / | / | / |
| Model | Where2Place | PixMo-Count | RoboSpatial | RefSpatial | PointArena | RoboAfford | ShareRobot | Avg7 | RefCOCOg |
| ACE-Brain-0.7 (ours) | 82.89 | 89.98 | 73.13 | 76.17 | 76.17 | 88.66 | 44.31 | 75.90 | 91.25 |
| Closed models | |||||||||
| Claude Opus 5.5 [ 7 ] | 66.44 | 83.74 | 73.30 | 71.10 | 61.20 | 75.10 | 40.20 | 67.30 | 85.89 |
| Gemini 4 Argon [ 43 ] | 71.20 | 87.71 | 80.94 | 51.56 | 66.40 | 71.48 | 29.53 | 65.55 | – |
| GPT-5.5 [ 44 ] | 45.25 | 82.61 | 59.19 | 60.29 | 68.02 | 68.04 | 38.57 | 60.28 | – |
| GPT-5.6-sol [ 45 ] | 46.23 | 83.55 | 64.34 | 54.51 | 69.76 | 71.45 | 36.66 | 60.93 | – |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Recording | Datasets | Annotations used |
| Egocentric human video | EPIC-KITCHENS-100 [ 20 ] , Ego4D [ 21 ] , HoloAssist [ 22 ] ; head-camera recordings annotated by us (Section 4 ) | Time-stamped steps or narrations |
| Third-person robot episodes | DROID [ 23 ] , BridgeData V2 [ 48 ] , Open X-Embodiment [ 24 ] , AgiBot World [ 49 ] ; failure datasets ViFailback [ 50 ] and RoboFAC [ 19 ] | Instructions, trajectories, sub-task lists with boundaries, task labels, failure and success labels |
| Simulation | RoboCasa [ 51 ] | Exact step boundaries and known object states |
| General-purpose replay | Cambrian [ 52 ] | Replayed without conversion |
| Capability | Annotations used | Question types | Answer form |
| Plan | Instruction; Ordered sub-task list of the episode | Decomposition (write the sub-task list from the instruction and first frames); Plan verification (is a candidate plan with a reordered, missing or inserted step valid); Plan continuation (steps remaining after a given moment) | Step list; Yes/No |
| Anticipate | Time-stamped steps or narrations | Next step given the frames up to a moment, with task steps as distractors; Free-text next sub-task on robot episodes | Letter; Sub-task sentence |
| Monitor | Time-stamped steps; Object states (simulation, state annotations) | Last completed step; Step in progress; Is a named step done; Order of a set of steps; Progress comparison between two clips; State of an individual object | Letter; Yes/No; State label |
| Diagnose | Failure labels (type, step or moment, affected state); Successful episodes as negatives | Did the execution fail; Which failure type; At which step or moment; Which object state is wrong | Yes/No; Letter; Location |
| Interpret | Task labels; Gripper or hand state | Which task is being performed (choice or words); What has happened so far; Is the gripper holding an object | Letter; Task name; Description; Yes/No |
| Source | Plan | Anticipate | Monitor | Diagnose | Interpret | General | Total |
| Third-person robot (real) | 198.9 | 595.3 | 930.1 | 124.7 | 326.6 | – | 2,175.6 |
| Egocentric human video | 21.4 | 428.1 | 668.8 | 37.3 | 3.2 | – | 1,158.8 |
| Simulation | 18.9 | 30.9 | 27.9 | – | – | – | 77.7 |
| General-purpose | – | – | – | – | 78.6 | 187.3 | 265.9 |
| Total | 239.3 | 1,054.3 | 1,626.8 | 162.0 | 408.3 | 187.3 | 3,678.0 |
| Capability | Benchmark sub-item | Answer and scoring | |
| Plan | RoboBench Q1: Task decomposition | 1,945 | Free-text plan, judge |
| LADDER: Decomposition | 145 | Step list, exact match | |
| Anticipate | EgoPlan-Bench | 3,343 | 4-way MC |
| RoboBench Q2: Next sub-task | 842 | Free text, judge | |
| LADDER: Next sub-goal | 560 | Sub-goal, validity | |
| Cosmos-Reason1: AgiBot | 100 | MC |
| Capability | Sub-item | ACE-Brain- 0.7 | GPT-6 Astra | Claude Opus 5.5 | Base | |
| Plan | RoboBench Q1: task decomposition | 67.7 | 74.5 | 77.8 | 64.9 | |
| LADDER: decomposition | 93.1 | 86.9 | 93.8 | 92.4 | ||
| Anticipate | EgoPlan-Bench | 74.5 | 75.9 | 70.1 | 48.8 | |
| RoboBench Q2: next sub-task | 52.1 | 34.7 | 39.0 | 22.3 | ||
| LADDER: next sub-goal | 76.8 | 84.6 | 79.5 | 64.3 | ||
| Cosmos-Reason1: AgiBot | 94.3 | 69.0 | 69.5 | 47.7 |
| Group | Subset | ACE-Brain- 0.7 | GPT-6 Astra | Claude Opus 5.5 | |
| Embodiment | Single arm | 235 | 85.7 | 84.5 | 88.3 |
| Dual arm | 157 | 36.5 | 64.4 | 64.9 | |
| Human | 118 | 79.5 | 79.2 | 84.9 | |
| Mobile manipulation | 50 | 45.6 | 66.9 | 72.7 | |
| Object attribute | Shape | 126 | 83.1 | 69.7 | 73.4 |
| Colour | 123 | 88.6 | 81.6 | 81.5 |
| Group | Subset | ACE-Brain- 0.7 | GPT-6 Astra | Claude Opus 5.5 | |
| Source | EPIC-KITCHENS | 2,420 | 74.7 | 75.2 | 70.1 |
| Ego4D | 923 | 74.2 | 77.8 | 70.5 | |
| Progress frames | Present | 2,841 | 75.0 | 76.9 | 71.3 |
| Absent (answer is the first step) | 502 | 71.9 | 69.9 | 64.1 | |
| Correct option is | the shortest | 739 | 69.4 | 73.3 | 66.6 |
| of middle length | 1,755 | 75.4 | 74.7 | 70.3 |
Embodied Reasoning
Embodied reasoning studies how agents use perception, memory, and physical knowledge to plan and act in interactive environments.
See all 86 papers