Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systems

Xue Qin; Simin Luan; Cong Yang; Zhijun Li

cs.ROApr 9, 2026

Evolving Skill Modules under a Fixed Planner: Versioning, Rollback, and Runtime Governance for Long-Lived Robot Systems

Authors: Xue Qin, Simin Luan, Cong Yang, Zhijun Li

Organizations: School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Future Science and Engineering, Soochow University, Suzhou, China

Abstract

Robots deployed for long periods keep improving their skills, and each update changes a released system. We treat this as a software-lifecycle problem: a fixed decision layer dispatches versioned skill modules and a runtime layer was built to screen each action. On six robosuite tasks we report three negative results and two measurements. First, peak task success is unstable across random seeds (within one method it spans 23.3 to 73.3%), so single-run peaks cannot rank these methods. Second, the system's four modules are whole-task policies with different labels, rotated on a clock, not the phase decomposition its documentation describes. At a matched budget one such policy holds the geometry at the final step in 0.734 of episodes reaching it, averaged over seeds, against 0.023 for the rotation, with no seed overlap at four seeds per arm (exact p=0.029). An intervention isolates why: restoring the termination condition the clock replaced raises retention on every seed. Third, our shield cut violations 98 to 100% on five single-arm tasks (34.9% on the sixth) by discarding whole actions, leaving success at zero: its acceptance criterion omitted completions, so a shield that stopped the robot scored perfectly. What survives is release machinery: a promotion gate kept all twelve injected regressions out, a rate its calibration nearly guarantees, at a 22.5% clean-candidate rejection cost; a dip detector caught nine of twelve, missing all three on one seed.

Figures & tables

Figure 1: Capability evolution loop for embodied agents. A fixed decision layer (blue, top) selects and sequences modules, while the capability modules (ECMs) themselves evolve over time through iterative execution, experience collection, and learning. A runtime layer (purple) screens each proposed action against configured predicates; § 4.5 reports the measured behaviour of this layer, including the failure mode that rules out a safety claim for it. The learning subsystem admits three modalities by design, but the imitation and code-synthesis paths (marked

\dagger

) never executed in any run reported in this paper, and reinforcement learning is the only modality behind every number below (§ 3.8 ).

Symbol	Meaning
$\mathcal{A}$	Persistent agent
$\pi_{\mathcal{A}}$	Decision policy (selects and sequences modules)
$\mathcal{M}_{\mathcal{A}}^{\text{id}}$	Identity memory (fixed beliefs, goals)
$\mathcal{M}_{\mathcal{A}}^{\text{ep}}$	Episodic memory (accumulative execution traces)
$c^{(k)}$	ECM at version $k$
$\theta^{(k)}$	Learnable parameters of ECM version $k$

Table 1: Summary of notation.

Task	Description	Steps	Environment
T1: Pick	Grasp object from table	2	PickPlace
T2: Place	Place at goal pose	3	PickPlace
T3: Stack	Stack A on B	4	Stack
T4: NutAssembly	Seat nut on peg	5	NutAssembly
T5: PickPlaceMulti	Place 3 objects into bins	6	PickPlace
T6: TwoArmPegInHole	Bimanual peg insertion	8	TwoArmPegInHole

Table 2: Manipulation tasks used in evaluation, ordered by composition depth. Success for every task is the underlying robosuite environment’s _check_success , read at the final step; there are no per-task criteria in the implementation. The environment column is the operative one: in the released code the task definitions forward no per-task construction arguments, so T1, T2 and T5 construct byte-identical default PickPlace environments and share one success criterion.

Method	Seed 7	Seed 123	Seed 2024	Mean $\pm$ std
Ours	73.3	23.3	70.0	$55.6{\,\pm\,}28.0$
Agent Modification	20.0	16.7	3.3	$13.3{\,\pm\,}8.8$
SkiMo	46.7	20.0	33.3	$33.3{\,\pm\,}13.3$
SPiRL	0.0	0.0	0.0	$0.0{\,\pm\,}0.0$
Static ECM	0.0	0.0	0.0	$0.0{\,\pm\,}0.0$

Table 3: T6 peak success rate (%) over 20 iterations for the five methods, per seed and as mean

\pm

std over the attested seeds

\{7,123,2024\}

. Every value comes from a run on the pinned cloud stack, and the per-seed result files are in the artifact ( analysis/method_comparison_n3.py regenerates this table). We attach no hypothesis test: an exact two-sided test at three seeds per arm cannot return

p

below

2/20{=}0.1

, and the spread within a single method (Ours spans

23.3

73.3

) is what this table should be read for. A fourth, baseline-only seed (42) exists in the artifact for the three recovered baselines and is excluded here because Ours and Agent Modification were not run on it; including it would raise no method above Ours. The budget-matched comparison this paper’s architectural claim rests on is in Section 4.8 .

	AM	SkiMo	AM+rotate	Ours
Four rotating whole-task policies	—	skill prior	✓	✓
Frozen agent ( $\Delta\pi{=}0$ )	—	—	—	✓
T6 peak (%)	$13.3{\,\pm\,}8.8$	$33.3{\,\pm\,}13.3$	$55.6{\,\pm\,}15.8$	$55.6{\,\pm\,}28.0$
Agent drift $\Delta\pi$	1.55	1.34	2.36	0

Table 4: The T6 comparison across the four configurations (peak success % over the attested seeds

\{7,123,2024\}

). The two right-hand columns spend

200{,}000

interaction steps per iteration against the monolithic column’s published default, so the table confounds structure with compute, and the budget-matched control of Section 4.8 reverses the ordering. The structure itself is four whole-task policies taking turns, not a split of the task ( Section 3.1 ). The frozen and unfrozen variants have equal mean peak success; they differ in drift, which is the factor under test.

Figure 2: Agent-policy drift

\Delta\pi

over the 20 T6 evolution iterations, thin lines per attested seed

\{7,123,2024\}

and thick lines seed means, rendered by figures/render_fig_identity_drift.py from the released per-seed files. Ours, SPiRL, and Static ECM hold

\Delta\pi{=}0

by construction; Agent Modification ends at a mean of

1.55

, SkiMo at

1.34

, and the mutable-dispatcher arm AM+rotate at

2.36

Property	SPiRL	SkiMo	Voyager	AM	Ours
Skill versioning	×	×	partial	—	✓
Continuous evolution	×	✓	✓	✓	✓
Rollback support	×	×	×	×	✓
Agent identity fixed	✓	×	×	×	✓
Runtime safety layer	×	×	heuristic	×	✓
Composition tested	×	✓	×	×	✓

Table 5: Skill-management properties across frameworks. The composition row refers to scheduling more than one module within a single episode, which T4–T6 exercise through the fixed time-slice rotation of Section 3.1 ; the type-compatibility checking of Section 3.5 is implemented but not independently evaluated. The runtime-safety row records that an enforcement layer is present, not that ours is usable: Section 4.5 shows it holds task success at zero. Voyager rewrites a skill in place when it is regenerated and keeps the earlier code as numbered files, with no mechanism for choosing or restoring a version, hence partial versioning and no rollback.

Arm	Regressions	False	Dip	Recovery	Rollbacks
	rejected	rejections	depth	(iters)	(false)
Gate-on	$100\pm 0$ %	$22.5\pm 26.3$ %	$-1.0\pm 4.3$ %	$0.0\pm 0.0$	n/a
Gate-off	n/a	n/a	$14.5\pm 8.1$ %	$0.9\pm 0.3$ †	n/a
Rollback	n/a	n/a	$17.6\pm 7.2$ %	$0.9\pm 0.3$	$2.2\pm 1.5$ (0)

Table 6: Lifecycle mechanisms on T6 under injected regressions, over four seeds and three injections per run. Dip depth is the drop at the injected iteration itself, measured against the mean of the three preceding iterations; the rollback trigger separately uses the median of the last three ( Section 4.6 ). The released aggregate file also records a windowed variant that takes the deepest drop anywhere in the interval following an injection. The two conventions agree on all twelve gate-off episodes and on eleven of twelve rollback episodes, but diverge on seven of twelve gate-on episodes, where the windowed statistic reads

0.7\pm 3.9

% against the at-injection

-1.0\pm 4.3

% printed here: with nothing to intercept, the deepest drop in a window is noise, whereas the injected iteration itself sits marginally above its own baseline. Dip and recovery statistics pool the twelve injection episodes, which cluster three-per-seed and are not independent; per-seed mean dips are

0.9,-0.5,-1.2,-3.4

% under the gate (seed-level

-1.0\pm 1.8

16.8,18.9,7.7,14.4

% without it (seed-level

14.5\pm 4.9

) and

19.3,17.2,21.9,12.2

% under rollback (seed-level

17.6\pm 4.1

), so the seed-level and pooled means agree on all three arms; the dispersions differ because one aggregates twelve episodes and the other four seeds. All dispersions in this table and caption are sample standard deviations. Clean-candidate rejection is computed per seed. Gate-on additionally spends two held-out evaluations of 30 episodes per iteration.

Figure 3: Deployed mean reward under injected regressions on T6, per deployment policy. Thin lines are the four seeds, thick lines their mean; dashed verticals mark the injection iterations

\{5,8,11\}

. Top: gated promotion intercepts every regressed candidate and the deployed curve is flat through the injections. Middle: without the gate the same regressions reach deployment and cost

14.5\pm 8.1

% of reward. Bottom: the dip detector reverses deployed regressions where it fires (triangles); seed 2024 fired zero times and carries no markers. Rendered by figures/render_fig_lifecycle.py in the artifact from the released per-seed result files.

$\sigma$	Mean drop % across seeds (range)	Triggers of 3, seeds $7/42/123/2024$
$0$	$-10.3$ to $-0.5$	0 / 0 / 0 / 0
$0.05$	$-7.7$ to $0.9$	0 / 0 / 0 / 0
$0.1$	$-3.0$ to $4.2$	0 / 0 / 0 / 0
$0.15$	$1.7$ to $7.1$	0 / 0 / 0 / 0
$0.2$	$1.1$ to $12.7$	0 / 0 / 0 / 0
$0.35$	$10.9$ to $21.7$	3 / 0 / 2 / 0

Table 7: Detector severity sweep on frozen final incumbents. Each cell: three perturbation replicates, 30 episodes each; drop is relative to that run’s own baseline (median of its last three clean deployed rewards); trigger applies the deployed rule (drop

>15\%

). The range column spans the four per-seed means, not individual replicates, whose spread is wider (the deepest single replicate at

\sigma{=}0.2

reaches

13.6\%

, still below the trigger). The

\sigma{=}0

row is clean re-evaluation and estimates the false-trigger rate. Produced by severity_runner.py ; the driver, the per-cell outputs, and the module weights it evaluates are all in the artifact.

Task	Policy	Viol. (off)	Viol. (on)	Reduction	Succ. off/on	Discarded
T1: Pick	random	7806	0	100.0% (100.0%)	0.0 / 0.0%	100.0%
T2: Place	random	9165	70	99.2% (99.3%)	0.0 / 0.0%	97.9%
T3: Stack	trained	10072	0	100.0% (100.0%)	0.0 / 0.0%	99.8%
T4: NutAssembly	random	8469	152	98.2% (98.2%)	0.0 / 0.0%	98.0%
T5: PickPlaceMulti	random	8847	0	100.0% (100.0%)	0.0 / 0.0%	100.0%
T6: TwoArmPegInHole	trained	6099	3973	34.9% (34.0%)	2.0 / 0.0%	75.1%

Table 8: Environment-verified violations, task success, and keep-out interventions under adversarial probing, 50 episodes per task per condition. Reduction excluding the velocity family is given in parentheses. The policy column records whether the episodes were driven by trained module checkpoints or by randomly initialised modules; only T3 and T6 had checkpoints available, and only T6 has a nonzero success rate to lose. “Discarded” is the fraction of the

25{,}000

actions per task on which the keep-out branch replaced the whole action vector with zero.

Figure 4: What the shield buys and what it costs, per task. Grey bars: environment-verified violation reduction. Orange bars: fraction of actions whose entire vector the keep-out branch discards. Flat markers: task success under governance,

0.0\%

on every task. The reduction is bought by the discard, and T6’s

34.9\%

marks where predicates that watch one arm of two stop covering the probes. Rendered by figures/render_fig_govstress.py from the released result files.

Arm	Modules	Steps/iter	Peak success	Past align
AM+single	1 whole-task	$1\times 200$ k	$95.0\pm 6.4$ %	0.740
Ours	4 whole-task, rotated	$4\times 50$ k	$3.3\pm 0.0$ %	0.589
AM+rotate	4 whole-task, rotated	$4\times 50$ k	$14.2\pm 19.5$ %	0.488
Agent Modification	1 monolithic	$1\times 20$ k	$0.0$ %	0.040
SkiMo	skill prior	$1\times 20$ k	$0.0$ %	0.052

Table 9: Budget-matched comparison on T6 over the first eight iterations, so that every arm has consumed the same cumulative interaction budget. The first three rows train whole-task policies and differ in how many; the first spends its whole budget on one, the next two spread it over four that take turns on a schedule. The last two spend

20{,}000

steps per iteration, their published default, and are shown for continuity. “Past align” is the fraction of evaluation episodes reaching the alignment step of the ladder of Section 4.9 or beyond, measured over these same first eight iterations. These cells pool episodes over seeds and conceal seed spread, so we give the per-seed values (seed order 7, 42, 123, 2024): AM+single

0.871,0.908,0.342,0.838

; Ours

0.683,0.683,0.558,0.433

; AM+rotate

0.458,0.392,0.637,0.463

; Agent Modification

0.058,0.021

; SkiMo

0.067,0.037

. AM+rotate’s peak-success mean conceals one escaping seed, per-seed

3.3,6.7,3.3,43.3

%. No inference is drawn from these fractions.

Figure 5: Per-iteration success over the budget-matched window. Thin lines are seeds, thick lines seed means. The single whole-task policy (AM+single, four seeds) climbs to peaks of

86.7

–

100\%

within eight iterations; of the eight rotating-arm curves (four seeds per arm), seven stay at or below

6.7\%

throughout, and the one exception is the learned dispatcher’s escaping seed at

43.3\%

( Section 4.8 ). Rendered by figures/render_fig_nophase_curves.py from the released per-seed result files.

	Clock			Hold
Seed	Reached	Retention	Success	Reached	Retention	Success
$7$	$14$	$0.071$	$3.3$ %	$9$	$0.556$	$16.7$ %
$42$	$8$	$0.375$	$10.0$ %	$13$	$0.462$	$20.0$ %
$123$	$9$	$0.000$	$0.0$ %	$9$	$0.667$	$20.0$ %
$2024$	$27$	$0.000$	$0.0$ %	$25$	$0.600$	$50.0$ %
Seed mean	—	$0.112$	$3.3$ %	—	$0.571$	$26.7$ %

Table 10: State-conditioned handoff intervention on the fixed-schedule arm, evaluation only, 30 episodes per arm per seed on that seed’s final checkpoint. Clock is the released schedule; hold freezes control in whichever module is acting when the success geometry is first reached. Retention is

P_{4}/(P_{3}{+}P_{4})

with the same ladder as Table 12 , but over these 30 final-checkpoint episodes rather than that table’s 240 pooled window episodes, so the clock column is a second estimate of that arm’s retention and not the same number; “reached” gives each cell’s denominator. Retention and success share the numerator

P_{4}

. Produced by handoff_runner.py ; the driver, the per-seed outputs, and the module weights it evaluates are all in the artifact, so this table can be regenerated from a clean checkout.

Method	Reached align or beyond	Decision drift	Action drift
Ours (4 rotating, frozen)	0.744	0.000	0.65
AM+rotate (4 rotating)	0.547	0.766	0.73
Agent Modification	0.146	n/a	0.31
SkiMo	0.050	n/a	0.32

Table 11: Fraction of evaluation episodes reaching the align step or beyond on T6, and mean per-iteration drift of the decision layer and of the emitted actions on the shared probe set. Two seeds, 20 iterations, 30 episodes per iteration.

Arm	Reached, per seed	Retention, per seed	Mean	Range
Budget-matched: first eight iterations, $1.6$ M training transitions per seed
AM+single	203, 208, 66, 183	0.813, 0.817, 0.788, 0.519	$0.734$	0.519–0.817
AM+rotate	46, 45, 100, 43	0.043, 0.067, 0.040, 0.326	0.119	0.040–0.326
Ours	107, 59, 48, 55	0.019, 0.034, 0.021, 0.018	0.023	0.018–0.034
Agent Modification	2, 1	0.000, 0.000	not estimable
SkiMo	6, 2	0.000, 0.000	not estimable

Table 12: Retention on T6 with the seed as the unit of analysis. Retention is

P_{4}/(P_{3}{+}P_{4})

, the fraction of evaluation episodes that reach the full success geometry and still hold it at the final step. AM+single trains one whole-task policy; AM+rotate and Ours rotate four of them, under a learned dispatcher and under a fixed schedule respectively. “Reached” is the

P_{3}{+}P_{4}

episode count that forms that seed’s denominator, and seeds appear in the order

7

42

123

2024

. Each seed contributes 30 episodes per iteration, so 240 episodes over the matched window; over the full run, 300 for AM+single, 600 for the rotating arms’ original seeds (7, 42), and 240 for their completion seeds (123, 2024), which ran the matched window only, on the attested cloud stack. The upper block cuts every arm to its first eight iterations, where cumulative training interaction is

1.6

M SAC environment transitions per seed for the three arms trained at

200{,}000

steps per iteration; Agent Modification and SkiMo run at their published

20{,}000

default over those same eight iterations and are never budget-matched to the other three. The lower block uses each arm’s full run (ten iterations for AM+single, twenty for the rotating arms’ original seeds, eight for their completion seeds), which is not budget-matched and is shown so the matched-window reading can be checked against the arms’ full histories. Episodes are clustered inside seeds, so the summary is the seed mean and the seed range and no interval here is computed from pooled episodes. At matched budget the two published baselines reach the geometry in three and eight episodes in total, so their retention is not estimable there. Produced by analysis/retention_seed_level.py in the artifact, which regenerates this table from the released per-seed files.

Figure 6: Seed-level retention over the budget-matched window. Circles are individual seeds (labelled), diamonds are seed means; the shaded band is the empty interval between the strongest rotating seed (

0.326

) and the weakest single-policy seed (

0.519

). The separation is complete at four seeds per arm, and each per-arm exact test returns its design floor of

p{=}0.029

. Rendered by figures/render_fig_retention.py from retention_seed_level.json .

Robotic Policy Evaluation

Robotic policy evaluation measures whether learned controllers succeed reliably, generalize across conditions, recover from failures, and behave safely.

See all 125 papers

Oct 8, 2026

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments

A generalist robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse. Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this experience around the task structure that gives it meaning, so that each repair is attributed to the responsible capability, supported by execution evidence, and validated before it is reused. We introduce RoboRSI, a robot self-improvement system built on Top-Down Skill Refinement (TSR). TSR decomposes tasks into compound, atomic, and base skills with scoped responsibilities and explicit input--output contracts, attributes each execution outcome to the responsible branch, and confines revision to that branch. Building upon this structure, a Manager, Planner, Engineer, and Reviewer coordinate planning, execution, diagnosis, and the validated release of new skills, while people steer the process through objectives and corrections; stable skill sequences are further consolidated into reusable compound skills. On a mobile manipulator, RoboRSI develops multi-object household cleanup over 104 rounds. In simulation, it achieves the highest success rate on LIBERO, LIBERO-PRO, LIBERO-Plus, and RoboTwin, exceeding the strongest baseline by 2.7 to 11.0 percentage points.

Zimo Wen, Yijin Chen, Yuxuan Cao +8

Robot Skill Learning Continual Robot Learning Robotic Manipulation

Ask PDF

Sep 29, 2026

Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents

Vision-language-action and world-action models have demonstrated impressive capabilities in robotics, yet generalization to unseen tasks remains challenging. More recently, general-purpose multimodal agents have shown great potential for zero-shot robotic task solving. However, they often incur high execution costs by reasoning and exploring the physical world from scratch. To reduce these costs, we introduce RoboSkill, a framework that connects skill acquisition and reuse through an Explore, Execute, Evolve loop. Within this loop, the agent explores to gather task-relevant information, executes tasks while adapting to feedback, and evolves its skill library based on execution records. It then reuses these skills to guide exploration and execution in the next cycle, closing the loop. To improve loop efficiency, we complement vision with tactile feedback to reduce uncertainty during physical interaction. We further augment textual guidance with reusable code to reduce reasoning overhead during skill reuse. On LIBERO-10, RoboSkill improves first-episode success rates by 12.5--25.0 percentage points and reduces average runtime by 7.6--72.4% across four agents. On real robots, it improves success rates by 8.3 percentage points and reduces average runtime for successful trials by at least 14.4%.

Sicheng Xie, Yitong Chen, Haidong Cao +3

Institute of Trustworthy Embodied AI, Fudan University · Shanghai Innovation Institute · NeoteAI.

Robot Skill Learning Agent Skill Learning

Ask PDF

Sep 30, 2026

RoboCoach: World Models as Active Coaches for Compositional Robot Skills

Long-horizon robot manipulation reuses skills across many task compositions, but improving these compositions with additional end-to-end demonstrations is costly. A practical self-improving system must decide both what to teach next and where to apply that supervision. We present ROBOCOACH, a world-model-guided coaching framework that uses imagined failures to guide demonstration requests and expert updates. Its Route-Imagine-Diagnose-Improve (RIDI) loop executes reusable skill experts inside COACHWORLD, our shared action-conditioned world model, and uses a progress judge to record the first subtask that fails to complete. Aggregated records select which subtask demonstrations to acquire and which expert adapters to update. Across two simulation suites and two real-robot platforms, imagined and deployed success correlate over 22 task-policy pairs (rho = 0.840). Controlled comparisons show that our coaching method outperforms matched baselines under matched data budgets and update schedules. With only 150 additional subtask demonstrations, success rises from 13.3% to 75.0% on Franka and from 40.0% to 83.8% on AgileX. The coached experts also transfer to four held-out compositions, achieving an average success of 35.0%, compared with 0% for a shared-policy baseline updated with uniformly acquired demonstrations. Together, these results show that world models can serve as active coaches, turning imagined failures into targeted supervision for modular policy improvement. Project Page: https://robocoach-ai.github.io/

Jiajun Liu, Yifan Chen, Yichao Liu +7

Renmin University of China · Tsinghua University · Shanghai Qizhi Institute +2

Robot Policy Learning Robot Policy Adaptation Long-Horizon Robotic Manipulation Robot Skill Learning World Models for Robotics Imitation Learning

Ask PDF

Abstract

Robotic Policy Evaluation

Explore similar work

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments

Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents

RoboCoach: World Models as Active Coaches for Compositional Robot Skills