cs.AIOct 13, 2026
SaveABot-ER: Unified Embodied Reasoning through Agentic Data-Model Co-Evolution
Organizations: AMAP CV Lab Alibaba Group
Abstract
We introduce ABot Embodied Reasoning (ABot-ER), a unified embodied reasoning foundation model that breaks the tension between breadth and depth in embodied reasoning. Embodied intelligence demands the joint mastery of perception, planning, and execution, yet reconciling generality with specialization across this spectrum remains an open challenge. Within a single unified model, ABot-ER commands specialist-level depth across all three competencies, spanning spatial, scene, and temporal reasoning in perception; long-horizon action organization in planning; and point grounding and affordance in execution. The Agentic Data-Model Self-Evolution Engine continuously adapts data strategies to the model's evolving capability state. The Task-Conditioned Adaptive Training Strategy uses task tokens to activate lightweight low-rank residuals over a shared backbone, separating updates from heterogeneous output formats while retaining shared embodied knowledge. Across a systematic evaluation on twelve benchmarks covering three core competencies and five capability dimensions, ABot-ER achieves state-of-the-art overall performance across perception, planning, and execution within a unified parametric framework, demonstrating that breadth and depth can coexist in unified embodied reasoning.
Figures & tables
Figure 2: The embodied reasoning layer between what a robot sees and what it does. It takes observations and an instruction and returns three kinds of answer: semantic assessments, ordered action plans, and normalized image coordinates. The agent runtime uses the assessments and plans to manage task execution, while the coordinates specify spatial targets for the downstream policies; the model itself emits no motor commands. Bottom: one official test case per capability dimension, drawn from the evaluation splits of Section 6 and grouped by the competency it belongs to. Each card pairs the benchmark question with ABot-ER’s own answer. All five answers were scored correct: the planning case succeeded over the full episode, and the predicted point fell inside the official target mask, which is shaded in the image.
| The robot needs to know | Capability | Example | Answer form |
| Where things are and how they are arranged | Spatial and relational reasoning | Which block is the left operand from the participant’s viewpoint, given that the participant sits across the table and the expression therefore reads right to left in the robot’s camera image | Choice, number, or short text |
| What state the scene is in | Scene and state understanding | How many dots each block carries and which operator symbol is shown: | Short text or choice |
| What just happened, and whether it worked | Temporal and event reasoning | From the frames of the arm’s placement motion, whether the five-dot block was placed on the table or dropped during transfer | Choice: placed, or dropped in transfer |
| What to do, in what order, under which preconditions | Long-horizon action organization | Place the one-dot block that stands for the tens digit, then, once it is down, place the five-dot ones digit to its right, so that the pair reads | Ordered plan in the executor’s schema |
| Exactly where to grasp, place, or act | Point grounding and affordance | A grasp point on the five-dot block, and a placement point inside a free region to the right of the tens digit large enough to accommodate it | Normalized image coordinates in |
Table 1: The five capability dimensions and the form of each answer. Examples follow one task, the Block Arithmetic demonstration of Section 6.6 : a participant seated across the table lays out with blocks whose value is given by the number of dots on them, oriented toward the participant, and the robot places two blocks that read as the result. Directional descriptions are stated from the participant’s viewpoint, whereas coordinates follow the normalized robot-camera image protocol of Section 4 .
Figure 3: Model architecture.
Figure 4: Closed-loop pipeline of the Agentic Data-Model Self-Evolution Engine. Each round begins with evaluation-driven capability diagnosis (Section 5.1 ), revises the training corpus through three evolution mechanisms, namely content (Section 5.2 ), protocol (Section 5.3 ), and mixture (Section 5.4 ), under a data governance layer (Section 5.5 ), and closes with training and checkpoint selection (Section 5.6 ); the released model uses the two-stage recipe described there. The evaluation result of the new model version determines the revision applied in the next round.
| Symptom | Diagnostic check | Fault | Data action |
| Unparseable or off-contract outputs | Output and training labels checked against the frozen renderer and parser | Contract | Labels rebuilt against the frozen renderer and parser when a label mismatch is confirmed; no new samples added |
| Anomalously high EB-ALFRED | Source-level audit confirms overlap with the official evaluation trajectories | Contaminated evaluation | Source-level leakage audit and quarantine of the implicated sources |
| Low EB-ALFRED and EB-Habitat with legal actions | ET-Plan consistent | Missing supervision | Replay-verified multi-step plans with explicit precondition chains and verified terminal states |
| Low Cosmos | Item-level analysis: errors concentrate on temporal-order and failure-recovery items rather than on static-content items | Missing supervision | Train-only video multiple choice balanced across five independent clean domains |
| Low VABench-P with well-formed 8-point output | Where2Place moves in the same direction; training-label inspection identifies incompatible target conventions | Contradictory supervision | Point supervision restricted to verified target-region labels; trajectory endpoints removed |
| A dimension lags while its benchmarks agree and its supervision is already verified | Per-source and per-group exposure counts confirm the dimension is under budget | Under-exposed supervision | Dimension quota raised under the source and exposure caps; no change to the labels themselves |
Table 2: Representative mappings from evaluation symptom to attributed fault and data action. The symptom triggers the investigation, and the diagnostic check is the evidence available when the fault is assigned: contaminated evaluation and contradictory supervision are confirmed by the audits named there. The fault class determines which mechanism primarily handles the symptom.
| Capability dimension | Supervision construction |
| Spatial and relational | Train-split spatial annotation, cleaned, relabelled, and sampled for coverage across fine-grained capability channels |
| Scene and state | Existing question-answer pairs cleaned of answer prefixes and markup, together with indoor question answering constructed from replay-passed scene and state records |
| Temporal and event | Video items that retain the temporal evidence, with determinate answer conversion and sampling across independent clean train domains |
| Point grounding and affordance | Supervision restricted to verified target regions, serialized to the point cardinality each task requires |
| Long-horizon action organization | Planning trajectories with checked preconditions and verified terminal states, together with feedback-conditioned recovery windows |
Table 3: Supervision construction by capability dimension. Released record counts, historical processing volumes, and the exact label recipes are reported in Appendix A .
| VABench-P | Within-run spread | |
| Development history, earlier line | ||
| Pretrained model | 24.85 | — |
| Before unification | 8.58–53.75 | 45.17 |
| After unification and rebuild | 64.54–68.00 | 3.46 |
| Released corpus | ||
| Same configuration, run 1 | 61.67–64.00 | 2.33 |
Table 4: VABench-P around the coordinate-protocol unification, and its reproduction on the released corpus. The upper block is an earlier development line, where point supervision carried three coordinate conventions at once; its scores are development history, not results of the released model. The lower block is the format-aligned corpus the released model trains on, run twice under one configuration. Spread is the range over the later evaluated checkpoints of each run, excluding the early training ramp; over that range the two runs differ by at most points at matched checkpoints.
| Round | Spa. | Sce. | Tem. | Poi. | Lon. |
| Base | 35.26 | 46.45 | 51.64 | 35.25 | 33.07 |
| R1 | 62.32 | 60.13 | 62.87 | 50.58 | 35.23 |
| R2 | 61.41 | 57.48 | 61.87 | 57.39 | 32.98 |
| R3 | 62.70 | 60.89 | 66.56 | 42.15 | 41.86 |
| R4 | 62.11 | 61.88 | 68.64 | 47.80 | 42.96 |
| R5 | 62.88 | 62.21 | 69.16 | 51.73 | 46.89 |
Table 5: Five-dimension profile at a representative checkpoint of each round. R3 is reported for completeness only and is excluded from all performance claims, since its EB-ALFRED score is affected by the contamination of Section 5.5 .
| EQA(9) | ET | ALF | Hab | All(12) | |
| Single pass | 64.44 | 66.34 | 25.00 | 48.67 | 60.00 |
| Six rounds | 63.98 | 63.52 | 30.00 | 51.67 | 60.09 |
Table 6: A single post-training pass against a chain of six. Both start from the same ancestor checkpoint and use the same corpus for their final pass, and are compared at the same point within it; the chain has accumulated five times as much post-training.
| Dimension | Benchmark | Primary Metric | Input Modality |
| Spatial & Relational | EmbSpatial | Exact-choice accuracy | Image |
| VSI | Exact match / MRA | Video / scene sequence | |
| ERQA | Exact-choice accuracy | Ordered multi-image | |
| Scene & State | OpenEQA | Frozen LLM-match | Egocentric observation |
| UniEQA | 12-skill LLM-judged macro | Multimodal scene | |
| RoboVQA | Frozen LLM-match | Robot video |
Table 7: Primary metrics for each evaluation benchmark.
| Benchmark | ABot-ER 9B | Cosmos Reason 7B | Embodied Brain 7B | Cambrian S 7B | IflyBot VLM 8B | Mimo Embodied 7B | Rynn Brain 8B | Hy-Emb. RxBrain 1.0 | Rynn Brain 1.1 9B | Cosmos3 Nano |
| Spatial & Relational Reasoning | ||||||||||
| EmbSpatial | 81.68 | 66.96 | 74.37 | 69.13 | 75.05 | 77.59 | 81.46 | 80.38 | 82.53 | 82.03 |
| VSI | 63.30 | 27.41 | 37.05 | 63.39 | 30.47 | 43.86 | 67.55 | 0.16 | 71.59 | 48.65 |
| ERQA | 47.31 | 40.28 | 38.27 | 35.23 | 38.38 | 39.50 | 40.40 | 38.58 | 44.02 | 46.09 |
| Scene & State Understanding | ||||||||||
| OpenEQA | 63.33 | 35.17 | 51.79 | 53.19 | 52.95 | 65.85 | 57.06 | 40.90 | 61.60 | 49.22 |
Table 8: Main results across five capability dimensions and twelve benchmarks. Bold marks the best listed score; -- denotes a valid zero and is included in the means. All means are equally weighted, and every listed model has valid results on all twelve benchmarks, so the denominators are identical across models. Perception (7) and Execution (2) together make up EQA (9). All (12) mixes score-like and success-rate metrics and should be read together with the rows above it.
| Backbone | Params | Residual layers | GPUs |
| Qwen3.5-2B | 2B | 11 / 15 / 19 / 23 | 16 |
| Qwen3.5-4B | 4B | 19 / 23 / 27 / 31 | 16 |
| Qwen3.5-9B | 9B | 19 / 23 / 27 / 31 | 32 |
| Qwen3.8-27B | 27B | 51 / 55 / 59 / 63 | 32 |
| Qwen3.6-35B-A3B | 35B (3B active) | 27 / 31 / 35 / 39 | 32 |
Table 9: Backbones trained with the one-pass recipe: one Stage A run, then one joint post-training run from the selected Stage A checkpoint. Every model uses the same two releases and the same task-conditioned architecture.
| Benchmark | 2B | 4B | 9B | 27B | 35B-A3B |
| Spatial & Relational Reasoning | |||||
| EmbSpatial | 78.87 | 79.53 | 81.68 | 82.39 | 82.69 |
| VSI | 49.85 | 58.85 | 63.30 | 65.62 | 66.85 |
| ERQA | 36.93 | 40.36 | 47.31 | 49.65 | 47.84 |
| Scene & State Understanding | |||||
| OpenEQA | 56.21 | 60.10 | 63.33 | 67.25 | 65.33 |
Table 10: Twelve-benchmark results of the one-pass recipe across backbone sizes, each at its best checkpoint. Bold marks the best score in each row.
| Benchmark | Stage A selected | Released | |
| EmbSpatial | 81.70 | 81.68 | |
| VSI | 63.11 | 63.30 | |
| ERQA | 45.39 | 47.31 | |
| OpenEQA | 63.14 | 63.33 | |
| UniEQA | 50.63 | 50.66 | |
| RoboVQA | 72.29 | 71.79 |
Table 11: What the joint post-training stage adds. The first column is the Stage A checkpoint the released model starts from, the second is the released checkpoint, and is the difference. Both are the task-conditioned open-source line; no unconditioned run enters this comparison. Bold marks the higher of the two.
| Plain | Task-cond. | ||
| Corpus with unaligned output conventions, matched Stage A step | |||
| EQA (9) | 58.62 | 60.24 | |
| Planning (3) | 31.89 | 32.50 | |
| All (12) | 51.94 | 53.30 | |
| Benchmarks improved | 11 of 12 | ||
| Corpus after protocol unification, 17 matched checkpoints | |||
Table 12: Task-conditioning ablation on Qwen3.6-35B-A3B under two corpora. Each pair differs only in whether the route tokens and conditional residuals are enabled, with data, optimization, and evaluation held fixed. The first pair is a single Stage A comparison at a matched training step on a corpus whose point supervision still carried unaligned coordinate conventions, which is why its VABench-P scores sit near . The second pair is measured after the protocol unification of Section 5.3 , over nine matched pre-training and eight matched post-training checkpoints, each post-training run continuing from its own pre-training checkpoint at the same step.
| Round | Records | GPUs | LR | Weight origin |
| Base | — | — | — | Pretrained model |
| R1 | 70,526 | 80 | Pretrained model | |
| R2 | 288,028 | 128 | Pretrained model | |
| R4 | 463,071 | 64 | Pretrained model | |
| R5 | 153,600 | 64 | Continues R4 | |
| R6 | 320,000 | 32 | Restarts from R4 |
Table 13: Training configuration of the versions compared in this subsection, named as in Figures 5 and 6 . R3 is omitted: its planning score is affected by the contamination described in Section 5.5 . Records are physical training records of the round’s release.
Figure 5: Evolution of reported scores across five capability dimensions. Curves show cross-version changes; radar plots show multidimensional profiles. Dimension scores equally average fixed benchmark groups. The rounds are those listed in Table 13 ; R6 is not the final model. retains UniEQA judge-error zeros; includes historical VABench-P protocol changes. The figure describes trends, not directly comparable absolute abilities or a radar-area total score.
Figure 6: Evolution across twelve benchmarks at two scales. (a) Score change at each transition between the versions of Figure 5 ; colour is clipped at so that the transitions after the first remain legible, and the printed values are unclipped. (b) Checkpoint trajectories inside R1 and R2 for the three trajectory shapes that occur, with the R1 endpoint as reference. The outcome for all twelve benchmarks is the R1 R2 column of (a), where five of twelve end above the preceding version. All markers are measured results; VABench-P observations in (b) predate the frozen point contract.
Figure 7: Real-world interaction with ABot-ER on the Galaxea R1 Pro. Left: task requirements; middle: physical demonstrations; right: head-camera views annotated with visual recognition results. Top: Block Arithmetic requires expression validation, dot-based number recognition, and result-digit readout, illustrated by and the answer blocks representing . Bottom: Rock-Paper-Scissors requires gesture validity assessment and classification, illustrated by a valid paper gesture. ABot-ER supplies visual state information to ABot-AgentOS, which coordinates decisions and downstream actions.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Dimension | Construction method | Unique, R6 |
| Spatial & Relational | Train-only supervision from public spatial corpora, cleaned, relabelled, and sampled under constraints; multiple choice constrained to a single A–D token; numeric answers passed through the benchmark parser; equal-weight channels over 21 fine-grained capabilities correct natural source imbalance; in the constrained rebuild, released records had their capability metadata corrected | |
| Scene & State | Existing train question-answer pairs with answer prefixes and wrapper markup stripped; repetition limited by scene and group quotas. In the targeted release, indoor question answering is additionally constructed deterministically from visible objects, target objects, task instructions, visual state descriptions, and verified action plans of replay-passed episodes, with quotas over seven question types | |
| Temporal & Event | Existing video question answering; strict multiple choice preserved as A–D; yes/no items mapped to A/B only where the mapping is determinate, with no model-generated distractors; frame variants of the same video and question share a group exposure budget; the targeted release draws five independent clean train domains at roughly equal share. In R6 the dimension resolves into multiple-choice items, which include the determinately converted binary items, and open-text proxies | |
| Point & Affordance | Early rounds: target medoid for dataset point sources, endpoint extraction for object trajectories, deletion of non-object trajectories, normalization of free-space points to tuples. After Case A: only labels with an explicit target region are retained. For the eight-point supervision used with the VABench-P contract, where the source provides eight free points and a bounding box, of labels keep the original eight free points with at least four distinct coordinates and use the medoid of valid original points repeated eight times; Where2Place retains its own one-to-three-point protocol. The earlier rebuild rewrote trajectory labels into target-region or endpoint form and dropped non-object trajectories from the candidate pool | |
| Long-Horizon Action Org. | Multi-step and long-horizon windows selected from successful train trajectories with action identifiers and names recorded. For Habitat, each symbolic action is instantiated against the simulator state, its PDDL preconditions are verified, the action is executed in the environment, and only trajectories reaching the target terminal state under replay are retained. For ALFRED, trajectories are drawn from a soft-deduplicated clean split marked as replay-passing. The two source pools hold and decision-window records respectively, of which about a third are feedback-driven recovery windows; R6 samples and of them, alongside general replay-verified planning records |
Table 14: Supervision construction by capability dimension. The construction column describes the methods used across rounds; historical processing volumes named there belong to the rounds that performed them. The count column is the R6 development release, whose rows sum to its unique identities, each materialized twice for physical records. It is not the released model, whose Stage A composition is given in Table 16 .
| Gate | Requirement | Result and round |
| Evaluation contamination | No detected overlap under the specified identifier and hash-matching checks; unindexed media remain documented residual risk | No exact test-sample match on either stage within the checked identifiers and the available hash coverage. The Stage A finding comes from the joint audit with its decontaminated parent, since the format-alignment step changed no media, source or identity. The official point-grounding test anchors were isolated earlier, at source admission |
| Source eligibility | No records drawn from sources ruled ineligible | No prohibited-source record remains on either stage. Three sources had been removed in full, quarantining input records, when the decontaminated parent release was built |
| Output contract | Every label matches the contract template and passes the schema and pattern checks | Stage B: of labels pass, . Stage A: all VABench-P labels are valid eight-point XML, but no round-trip rate under one shared runtime parser was recorded across the remaining benchmarks. A byte-level prompt-match rate was not recorded for either stage |
| Coordinate protocol | No box labels, no non-canonical points, no conflicting protocols for the same question | Zero box outputs and zero non-canonical points on both stages |
| Reference integrity | Every record resolves to an existing media file | Zero media-schema errors on Stage A, and the media check ran and passed on Stage B; neither stage saved a separate count of missing references |
Table 15: Hard gates and the result recorded for each, on the two released stages. Both stages close as pass with documented residual risk. Where a quantity was not recorded separately, the cell says so rather than reporting a zero.
| Capability dimension | Physical | Share |
| Spatial & Relational | 144,902 | 31.34% |
| Scene & State | 150,806 | 32.61% |
| Temporal & Event | 11,905 | 2.57% |
| Point & Affordance | 110,594 | 23.92% |
| Long-Horizon Action Org. | 44,183 | 9.56% |
| Total | 462,390 | 100% |
Table 16: Realized composition of the Stage A joint training set by physical records, grouped by capability dimension. This release does not carry a regenerated token-proxy report, so no token-proxy share is stated for it; the M figure of the earlier R4 version does not transfer. Long-horizon action organization is the planning partition, comprising general planning records, Habitat-grounded and ALFRED-grounded plans.
| Component | Physical | Unique | Share |
| Task planning | 26,240 | 13,120 | 25.00% |
| VABench-P | 16,794 | 8,397 | 16.00% |
| OpenEQA | 12,000 | 6,000 | 11.43% |
| ERQA | 10,496 | 5,248 | 10.00% |
| Where2Place | 10,000 | 5,000 | 9.53% |
| RoboVQA | 8,000 | 4,000 | 7.62% |
Table 17: Realized composition of the Stage B Final Joint closure set by physical records. Each unique identity is materialized exactly twice. The release rebalances toward planning recovery under a question-answering guardrail, not toward point grounding alone.
| Stage A | Stage B | |
| Cross-Dimension Foundation | Final Joint Post-training | |
| Data | ||
| Dataset | Open-source joint pre-training set | Final Joint closure set |
| Physical training records | 462,390 | 104,960 |
| Unique identities | 294,762 | 52,480 |
| Materialization | on average, 1–4 per identity | Exactly |
Table 18: Where the two stages of the released model differ. Settings shared by both stages are listed in the note; per-stage schedules are omitted.
| Round | Diagnosis that triggered it | Data revision | Training change | Weight origin |
| Base | Uneven capability across the twelve embodied tasks, weakest on point grounding, video spatial reasoning, and closed-loop planning | None | None | Pretrained model |
| R1 | Validate joint planning and question-answering training and the evaluation pipeline end to end | records, M token proxy; planning to question answering about ; box labels removed and coordinates moved to | Short pilot run, sufficient only to exercise the pipeline | Restart from pretrained |
| R2 | The pilot budget was too small to support any conclusion about capability; the nine question-answering benchmarks need explicit balancing; point format, duplication, and leakage need cleaning | records, M token proxy; constrained sampling over nine benchmarks and fine-grained capabilities; duplicates and conflicting labels removed, test-overlapping records dropped, point data rewritten | Budget raised roughly threefold | Restart from pretrained |
| R3 | Perception improved but ALFRED and Habitat remained low; one point source held of the planning budget while grounded targets numbered records | records, M token proxy; grounded ALFRED and Habitat supervision added; over question answering, general planning, Habitat, ALFRED | Trained in two consecutive segments of one release | Restart from pretrained |
| R4 | R3 reported on EB-ALFRED from a summary covering of episodes, against on recomputation over all ; audit confirmed official test trajectories in the legacy planning pool | records, M token proxy; three high-risk trajectory sources removed in full ( records); R3 proportions and grounded improvements retained; zero box labels, broken media, and cross-task routing errors | Same recipe as the preceding round | Restart from pretrained |
| R5 | R4 improved planning only modestly and VABench-P remained low; targeted increase needed without sacrificing perception | records, M token; VABench , ALFRED , Habitat , question answering , general planning by token proxy; free-space prompt and label conflict rebuilt | Short post-training pass at a reduced learning rate | Continue from the preceding round |
Table 19: Per-round ledger. Weight origin makes explicit that the history is not a single continuation chain. R3 is a diagnostic round only; see Section 5.5 . The R4 scores in Table 5 were measured on a same-recipe continuation chain rather than on this round’s own checkpoint. Per-round hyperparameters are omitted here; those of the two released stages are given in Table 18 .
More in this topic
Embodied Reasoning
Embodied reasoning studies how agents use perception, memory, and physical knowledge to plan and act in interactive environments.
See all 78 papers