ABot-ER: Unified Embodied Reasoning through Agentic Data-Model Co-Evolution
Organizations: AMAP CV Lab Alibaba Group
Abstract
We introduce ABot Embodied Reasoning (ABot-ER), a unified embodied reasoning foundation model that breaks the tension between breadth and depth in embodied reasoning. Embodied intelligence demands the joint mastery of perception, planning, and execution, yet reconciling generality with specialization across this spectrum remains an open challenge. Within a single unified model, ABot-ER commands specialist-level depth across all three competencies, spanning spatial, scene, and temporal reasoning in perception; long-horizon action organization in planning; and point grounding and affordance in execution. The Agentic Data-Model Self-Evolution Engine continuously adapts data strategies to the model's evolving capability state. The Task-Conditioned Adaptive Training Strategy uses task tokens to activate lightweight low-rank residuals over a shared backbone, separating updates from heterogeneous output formats while retaining shared embodied knowledge. Across a systematic evaluation on twelve benchmarks covering three core competencies and five capability dimensions, ABot-ER achieves state-of-the-art overall performance across perception, planning, and execution within a unified parametric framework, demonstrating that breadth and depth can coexist in unified embodied reasoning.
Figures & tables
| The robot needs to know | Capability | Example | Answer form |
| Where things are and how they are arranged | Spatial and relational reasoning | Which block is the left operand from the participant’s viewpoint, given that the participant sits across the table and the expression therefore reads right to left in the robot’s camera image | Choice, number, or short text |
| What state the scene is in | Scene and state understanding | How many dots each block carries and which operator symbol is shown: | Short text or choice |
| What just happened, and whether it worked | Temporal and event reasoning | From the frames of the arm’s placement motion, whether the five-dot block was placed on the table or dropped during transfer | Choice: placed, or dropped in transfer |
| What to do, in what order, under which preconditions | Long-horizon action organization | Place the one-dot block that stands for the tens digit, then, once it is down, place the five-dot ones digit to its right, so that the pair reads | Ordered plan in the executor’s schema |
| Exactly where to grasp, place, or act | Point grounding and affordance | A grasp point on the five-dot block, and a placement point inside a free region to the right of the tens digit large enough to accommodate it | Normalized image coordinates in |
| Symptom | Diagnostic check | Fault | Data action |
| Unparseable or off-contract outputs | Output and training labels checked against the frozen renderer and parser | Contract | Labels rebuilt against the frozen renderer and parser when a label mismatch is confirmed; no new samples added |
| Anomalously high EB-ALFRED | Source-level audit confirms overlap with the official evaluation trajectories | Contaminated evaluation | Source-level leakage audit and quarantine of the implicated sources |
| Low EB-ALFRED and EB-Habitat with legal actions | ET-Plan consistent | Missing supervision | Replay-verified multi-step plans with explicit precondition chains and verified terminal states |
| Low Cosmos | Item-level analysis: errors concentrate on temporal-order and failure-recovery items rather than on static-content items | Missing supervision | Train-only video multiple choice balanced across five independent clean domains |
| Low VABench-P with well-formed 8-point output | Where2Place moves in the same direction; training-label inspection identifies incompatible target conventions | Contradictory supervision | Point supervision restricted to verified target-region labels; trajectory endpoints removed |
| A dimension lags while its benchmarks agree and its supervision is already verified | Per-source and per-group exposure counts confirm the dimension is under budget | Under-exposed supervision | Dimension quota raised under the source and exposure caps; no change to the labels themselves |
| Capability dimension | Supervision construction |
| Spatial and relational | Train-split spatial annotation, cleaned, relabelled, and sampled for coverage across fine-grained capability channels |
| Scene and state | Existing question-answer pairs cleaned of answer prefixes and markup, together with indoor question answering constructed from replay-passed scene and state records |
| Temporal and event | Video items that retain the temporal evidence, with determinate answer conversion and sampling across independent clean train domains |
| Point grounding and affordance | Supervision restricted to verified target regions, serialized to the point cardinality each task requires |
| Long-horizon action organization | Planning trajectories with checked preconditions and verified terminal states, together with feedback-conditioned recovery windows |
| VABench-P | Within-run spread | |
| Development history, earlier line | ||
| Pretrained model | 24.85 | — |
| Before unification | 8.58–53.75 | 45.17 |
| After unification and rebuild | 64.54–68.00 | 3.46 |
| Released corpus | ||
| Same configuration, run 1 | 61.67–64.00 | 2.33 |
| Round | Spa. | Sce. | Tem. | Poi. | Lon. |
| Base | 35.26 | 46.45 | 51.64 | 35.25 | 33.07 |
| R1 | 62.32 | 60.13 | 62.87 | 50.58 | 35.23 |
| R2 | 61.41 | 57.48 | 61.87 | 57.39 | 32.98 |
| R3 | 62.70 | 60.89 | 66.56 | 42.15 | 41.86 |
| R4 | 62.11 | 61.88 | 68.64 | 47.80 | 42.96 |
| R5 | 62.88 | 62.21 | 69.16 | 51.73 | 46.89 |
| EQA(9) | ET | ALF | Hab | All(12) | |
| Single pass | 64.44 | 66.34 | 25.00 | 48.67 | 60.00 |
| Six rounds | 63.98 | 63.52 | 30.00 | 51.67 | 60.09 |
| Dimension | Benchmark | Primary Metric | Input Modality |
| Spatial & Relational | EmbSpatial | Exact-choice accuracy | Image |
| VSI | Exact match / MRA | Video / scene sequence | |
| ERQA | Exact-choice accuracy | Ordered multi-image | |
| Scene & State | OpenEQA | Frozen LLM-match | Egocentric observation |
| UniEQA | 12-skill LLM-judged macro | Multimodal scene | |
| RoboVQA | Frozen LLM-match | Robot video |
| Benchmark | ABot-ER 9B | Cosmos Reason 7B | Embodied Brain 7B | Cambrian S 7B | IflyBot VLM 8B | Mimo Embodied 7B | Rynn Brain 8B | Hy-Emb. RxBrain 1.0 | Rynn Brain 1.1 9B | Cosmos3 Nano |
| Spatial & Relational Reasoning | ||||||||||
| EmbSpatial | 81.68 | 66.96 | 74.37 | 69.13 | 75.05 | 77.59 | 81.46 | 80.38 | 82.53 | 82.03 |
| VSI | 63.30 | 27.41 | 37.05 | 63.39 | 30.47 | 43.86 | 67.55 | 0.16 | 71.59 | 48.65 |
| ERQA | 47.31 | 40.28 | 38.27 | 35.23 | 38.38 | 39.50 | 40.40 | 38.58 | 44.02 | 46.09 |
| Scene & State Understanding | ||||||||||
| OpenEQA | 63.33 | 35.17 | 51.79 | 53.19 | 52.95 | 65.85 | 57.06 | 40.90 | 61.60 | 49.22 |
| Backbone | Params | Residual layers | GPUs |
| Qwen3.5-2B | 2B | 11 / 15 / 19 / 23 | 16 |
| Qwen3.5-4B | 4B | 19 / 23 / 27 / 31 | 16 |
| Qwen3.5-9B | 9B | 19 / 23 / 27 / 31 | 32 |
| Qwen3.8-27B | 27B | 51 / 55 / 59 / 63 | 32 |
| Qwen3.6-35B-A3B | 35B (3B active) | 27 / 31 / 35 / 39 | 32 |
| Benchmark | 2B | 4B | 9B | 27B | 35B-A3B |
| Spatial & Relational Reasoning | |||||
| EmbSpatial | 78.87 | 79.53 | 81.68 | 82.39 | 82.69 |
| VSI | 49.85 | 58.85 | 63.30 | 65.62 | 66.85 |
| ERQA | 36.93 | 40.36 | 47.31 | 49.65 | 47.84 |
| Scene & State Understanding | |||||
| OpenEQA | 56.21 | 60.10 | 63.33 | 67.25 | 65.33 |
| Benchmark | Stage A selected | Released | |
| EmbSpatial | 81.70 | 81.68 | |
| VSI | 63.11 | 63.30 | |
| ERQA | 45.39 | 47.31 | |
| OpenEQA | 63.14 | 63.33 | |
| UniEQA | 50.63 | 50.66 | |
| RoboVQA | 72.29 | 71.79 |
| Plain | Task-cond. | ||
| Corpus with unaligned output conventions, matched Stage A step | |||
| EQA (9) | 58.62 | 60.24 | |
| Planning (3) | 31.89 | 32.50 | |
| All (12) | 51.94 | 53.30 | |
| Benchmarks improved | 11 of 12 | ||
| Corpus after protocol unification, 17 matched checkpoints | |||
| Round | Records | GPUs | LR | Weight origin |
| Base | — | — | — | Pretrained model |
| R1 | 70,526 | 80 | Pretrained model | |
| R2 | 288,028 | 128 | Pretrained model | |
| R4 | 463,071 | 64 | Pretrained model | |
| R5 | 153,600 | 64 | Continues R4 | |
| R6 | 320,000 | 32 | Restarts from R4 |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension | Construction method | Unique, R6 |
| Spatial & Relational | Train-only supervision from public spatial corpora, cleaned, relabelled, and sampled under constraints; multiple choice constrained to a single A–D token; numeric answers passed through the benchmark parser; equal-weight channels over 21 fine-grained capabilities correct natural source imbalance; in the constrained rebuild, released records had their capability metadata corrected | |
| Scene & State | Existing train question-answer pairs with answer prefixes and wrapper markup stripped; repetition limited by scene and group quotas. In the targeted release, indoor question answering is additionally constructed deterministically from visible objects, target objects, task instructions, visual state descriptions, and verified action plans of replay-passed episodes, with quotas over seven question types | |
| Temporal & Event | Existing video question answering; strict multiple choice preserved as A–D; yes/no items mapped to A/B only where the mapping is determinate, with no model-generated distractors; frame variants of the same video and question share a group exposure budget; the targeted release draws five independent clean train domains at roughly equal share. In R6 the dimension resolves into multiple-choice items, which include the determinately converted binary items, and open-text proxies | |
| Point & Affordance | Early rounds: target medoid for dataset point sources, endpoint extraction for object trajectories, deletion of non-object trajectories, normalization of free-space points to tuples. After Case A: only labels with an explicit target region are retained. For the eight-point supervision used with the VABench-P contract, where the source provides eight free points and a bounding box, of labels keep the original eight free points with at least four distinct coordinates and use the medoid of valid original points repeated eight times; Where2Place retains its own one-to-three-point protocol. The earlier rebuild rewrote trajectory labels into target-region or endpoint form and dropped non-object trajectories from the candidate pool | |
| Long-Horizon Action Org. | Multi-step and long-horizon windows selected from successful train trajectories with action identifiers and names recorded. For Habitat, each symbolic action is instantiated against the simulator state, its PDDL preconditions are verified, the action is executed in the environment, and only trajectories reaching the target terminal state under replay are retained. For ALFRED, trajectories are drawn from a soft-deduplicated clean split marked as replay-passing. The two source pools hold and decision-window records respectively, of which about a third are feedback-driven recovery windows; R6 samples and of them, alongside general replay-verified planning records |
| Gate | Requirement | Result and round |
| Evaluation contamination | No detected overlap under the specified identifier and hash-matching checks; unindexed media remain documented residual risk | No exact test-sample match on either stage within the checked identifiers and the available hash coverage. The Stage A finding comes from the joint audit with its decontaminated parent, since the format-alignment step changed no media, source or identity. The official point-grounding test anchors were isolated earlier, at source admission |
| Source eligibility | No records drawn from sources ruled ineligible | No prohibited-source record remains on either stage. Three sources had been removed in full, quarantining input records, when the decontaminated parent release was built |
| Output contract | Every label matches the contract template and passes the schema and pattern checks | Stage B: of labels pass, . Stage A: all VABench-P labels are valid eight-point XML, but no round-trip rate under one shared runtime parser was recorded across the remaining benchmarks. A byte-level prompt-match rate was not recorded for either stage |
| Coordinate protocol | No box labels, no non-canonical points, no conflicting protocols for the same question | Zero box outputs and zero non-canonical points on both stages |
| Reference integrity | Every record resolves to an existing media file | Zero media-schema errors on Stage A, and the media check ran and passed on Stage B; neither stage saved a separate count of missing references |
| Capability dimension | Physical | Share |
| Spatial & Relational | 144,902 | 31.34% |
| Scene & State | 150,806 | 32.61% |
| Temporal & Event | 11,905 | 2.57% |
| Point & Affordance | 110,594 | 23.92% |
| Long-Horizon Action Org. | 44,183 | 9.56% |
| Total | 462,390 | 100% |
| Component | Physical | Unique | Share |
| Task planning | 26,240 | 13,120 | 25.00% |
| VABench-P | 16,794 | 8,397 | 16.00% |
| OpenEQA | 12,000 | 6,000 | 11.43% |
| ERQA | 10,496 | 5,248 | 10.00% |
| Where2Place | 10,000 | 5,000 | 9.53% |
| RoboVQA | 8,000 | 4,000 | 7.62% |
| Stage A | Stage B | |
| Cross-Dimension Foundation | Final Joint Post-training | |
| Data | ||
| Dataset | Open-source joint pre-training set | Final Joint closure set |
| Physical training records | 462,390 | 104,960 |
| Unique identities | 294,762 | 52,480 |
| Materialization | on average, 1–4 per identity | Exactly |
| Round | Diagnosis that triggered it | Data revision | Training change | Weight origin |
| Base | Uneven capability across the twelve embodied tasks, weakest on point grounding, video spatial reasoning, and closed-loop planning | None | None | Pretrained model |
| R1 | Validate joint planning and question-answering training and the evaluation pipeline end to end | records, M token proxy; planning to question answering about ; box labels removed and coordinates moved to | Short pilot run, sufficient only to exercise the pipeline | Restart from pretrained |
| R2 | The pilot budget was too small to support any conclusion about capability; the nine question-answering benchmarks need explicit balancing; point format, duplication, and leakage need cleaning | records, M token proxy; constrained sampling over nine benchmarks and fine-grained capabilities; duplicates and conflicting labels removed, test-overlapping records dropped, point data rewritten | Budget raised roughly threefold | Restart from pretrained |
| R3 | Perception improved but ALFRED and Habitat remained low; one point source held of the planning budget while grounded targets numbered records | records, M token proxy; grounded ALFRED and Habitat supervision added; over question answering, general planning, Habitat, ALFRED | Trained in two consecutive segments of one release | Restart from pretrained |
| R4 | R3 reported on EB-ALFRED from a summary covering of episodes, against on recomputation over all ; audit confirmed official test trajectories in the legacy planning pool | records, M token proxy; three high-risk trajectory sources removed in full ( records); R3 proportions and grounded improvements retained; zero box labels, broken media, and cross-task routing errors | Same recipe as the preceding round | Restart from pretrained |
| R5 | R4 improved planning only modestly and VABench-P remained low; targeted increase needed without sacrificing perception | records, M token; VABench , ALFRED , Habitat , question answering , general planning by token proxy; free-space prompt and label conflict rebuilt | Short post-training pass at a reduced learning rate | Continue from the preceding round |
Embodied Reasoning
Embodied reasoning studies how agents use perception, memory, and physical knowledge to plan and act in interactive environments.
See all 78 papers