SEAR: Self-Evolving Agentic Robotics

Xinyi Wang; Bangzheng Li; Hanyu Wang; Lin Zhang; Andrea Yaoyun Cui; Zhenggang Tang; Shyamal Buch; Dejia Xu

cs.ROOct 13, 2026

SEAR: Self-Evolving Agentic Robotics

Authors: Xinyi Wang, Bangzheng Li, Hanyu Wang, Lin Zhang, Andrea Yaoyun Cui, Zhenggang Tang, Shyamal Buch, Dejia Xu

Organizations: Luma AI

Abstract

Frontier language-model agents can now improve robots on their own: they write control code, test it, and keep what works as skills and memories for the next task. We introduce SEAR (Self-Evolving Agentic Robotics), a framework and benchmark built around two questions. Is the agent really learning, or just trying more often? And does what it learns still help on new tasks, in a new simulator, and on a real robot? The framework runs an agent in three modes: it writes code that becomes the policy (code-as-policy), acts live in the control loop (agent-as-policy), or works as a researcher that fine-tunes a vision-language-action model (auto-research). The benchmark adds controls that separate retained experience from repeated attempts, familiar tasks, privileged simulator state, larger budgets, and luck. Across seven frontier language models, 330 simulated and five physical tasks, and over four years of agent runtime, we highlight three findings. First, the learning curve of a robotic agent depends both on the overall time budget and the task difficulty relative to model capability. Harder tasks make longer sessions worthwhile; on easier tasks, re-rolling short attempts does just as well. Second, control code and shared routines transfer across tasks and simulators, while task-specific solutions do not. On a real robot, we observe a performance gap between agent-designed practice and novel human-designed scenes. Third, different models evolve more effectively in different ways. Some do best acting live, others writing code first. And a text-only model keeps pace with a comparable multimodal one: perception helps with error assessment, but agentic reasoning and tool use can matter more. We release the benchmark and codebase to support future research.

Figures & tables

Figure 2: The self-evolving loop, and what it looks like on a real robot. Top: within a time budget, an LM agent with its library updates a policy, by code-as-policy (CaP), agent-as-policy (AaP), or auto-research (AR); the policy executes in simulation or on hardware, the agent’s own checks or an external judge return feedback, and the library carries what worked into the next task ( Section 2 ). Bottom: Cup Upright and Spell “SEAR” attempts by Fable 5.1, GPT-6-astra, and Opus 5, labeled with elapsed hours and outcomes ( Section 4.4 ), then a whiteboard-writing demo ( website ) and a dexterous-hand task ( Section 4.4 ), both by Fable 5.1.

System	CaP	AaP	AR	Sim / real	tasks	Controls	Judge	Generalization measured
	Policy form				Library across
CaP-X ( Fu et al., 2026 )	$\CIRCLE$	$\LEFTcircle$	$\Circle$	both	$\LEFTcircle$	$\LEFTcircle$	–	perturbations, sim-to-real
ENPIRE ( Xiao et al., 2026 )	$\LEFTcircle$	$\Circle$	$\CIRCLE$	both	$\LEFTcircle$	$\Circle$	$\LEFTcircle$	qualitative
ASPIRE ( Lu et al., 2026 )	$\CIRCLE$	$\LEFTcircle$	$\Circle$	both	$\CIRCLE$	$\LEFTcircle$	$\LEFTcircle$	perturbations, new tasks
RATs ( Zhang et al., 2026a )	$\CIRCLE$	$\LEFTcircle$	$\Circle$	both	$\CIRCLE$	$\LEFTcircle$	$\LEFTcircle$	perturbations, new tasks, new sim.
Waddle ( Waddle Labs, 2026 )	$\CIRCLE$	$\CIRCLE$	$\LEFTcircle$	real	$\CIRCLE$	$\Circle$	$\Circle$	–

Table 1: Existing self-evolving robot agents as instances of SEAR .

\CIRCLE

yes,

\LEFTcircle

partial,

\Circle

no, – not reported, as stated in each system’s paper or announcement ( Section A.10 ); Table 11 adds seven systems released since mid-September 2026. Controls : budget-matched restarts and a no-library run. Judge : final policy graded on states and logic the agent cannot access.

Looks like learning	Control in SEAR	Results
More attempts	One long session against the best of three restarts at a third of the budget each ( best-of-3 ); runs with and without a library ( Lineage vs. Vanilla )	Sections 3.1 and 3.2
Familiar tasks	Scores on TEST initial states never shown to the agent; five tiers of distance from practice	Section 4
Privileged information	A sealed judge grades the frozen policy with state and logic the agent cannot reach	Sections 2.4 and A.9
Luck	Three independent repetitions of the whole run, library included	Section 3.3
A larger budget	Development time matched in every comparison; tokens, environment steps, and GPU time logged beside it	Appendix E

Table 2: Five ways a self-evolving agent can look better than it is, and the control for each.

Figure 3: The testbed. Five simulated task families (top) and five physical tasks (bottom): Banana into Bowl, Push-T, Cup Upright, Block Stacking, and Spell “SEAR”.

Figure 4: Five models over development time. Validation success, averaged over the five simulated families with equal weight ( Table 15 gives each family), one panel per model: with an inherited library (Lineage, solid), without (Vanilla, dashed), and best-of-3, three attempts at a third of the budget each (dotted). RoboLab’s 120-minute budget is rescaled onto the 60-minute axis.

Figure 5: One long session wins when a short attempt would end before the first success. Each model’s gain from one full-budget session (60 min) over the best of three short attempts (20 min) with the same total budget (Vanilla

-

best-of-3, VAL success at the end of the budget), against its median time to first validation success without a library, both averaged over the five task families. Shading ends at 20 minutes, the length of one short attempt. RoboLab’s budget is twice the others’ (120 minutes), so its times are halved here; its actual times are twice those plotted. Per-family values are in Table 15 .

Figure 6: The library helps most in the middle. One point per model and family: TEST success without a library (Vanilla) against the library’s gain (Lineage

-

Vanilla), from Table 6 . Labels give the mean gain below 30%, between 30 and 80%, and above 80%. Curve: least-squares quadratic fit over the 25 pairs, peaking at 43% (gain

+14.7

); shading: 95% bootstrap interval of the fit.

Figure 7: What an inherited skill changes. Kimi-k3 on the RoboLab task “put the Rubik’s cube and the banana in the bowl”, same scene. Without a library (left), the policy copies the gripper’s yaw from the object pose, the fingers land on the banana, and all ten TEST states fail. With the library (right), it calls an inherited skill, written on an earlier banana task, that finds the banana’s ridge and long axis and pinches across it; all ten TEST states succeed.

Tier	Graded on	Code-as-policy	VLA (AR)
T1	New initial states of the practiced tasks	87.8	73.2
T2	The same tasks under changed scenes or hardware faults	59.7	55.3
T3	New tasks in the same simulator	33.3	43.1
T4	New tasks in a new simulator and action API	28.7	10.2
T5	New initial states of tasks practiced on the physical robot	35.6	—

Table 3: Success falls with distance from practice. Mean TEST success (%) of Claude Opus 5 policies per tier, over the practice–test family pairs where both the code policy and the VLA have results ( Table 13 and Figure 15 give every pair). T3 and T4 policies are written after self-exploration on the new tasks; T1 and T2 policies are frozen. T5 is not a transfer result like T1–T4: the agent starts from an empty library and practices each of the five hardware tasks on the physical robot, so the T5 entry is its score on new initial states of those same tasks ( Table 4 ), with no VLA run on hardware.

Figure 8: One task-general program serves an explored and a never-explored scene. A self-exploration lineage, never told its tasks, ends with one program that parses each instruction into a goal and calls shared routines. “Put the bigger pumpkin in the bin”, in a scene the lineage never explored, calls the same grasp and placement as an explored banana-into-bowl task, and rises from 0 to 7 of 10 as those routines improve elsewhere.

Figure 9: Code and weights, side by side. (a) The TEST success of the VLA (the student) against the code policy (its teacher), for the VLA Lineage sessions of an Opus auto-research agent, pooled over the 305 tasks of LIBERO, LIBERO-PRO, and RoboLab; marker area counts tasks. The student beats its teacher on 15 tasks (orange), trails it on 126 (blue), and ties on 164 (grey). (b) LIBERO-practiced Opus policies re-graded on LIBERO-RIG, averaged over the fault axes in each group (per-axis values in Table 9 ).

Figure 10: Code against weights over accumulated session time. Code (blue) against weights (orange) over accumulated session time, with (solid) and without (dashed) inherited state: best-so-far per task validation success averaged over a family’s tasks, sessions laid end to end in curriculum order. Session time includes GPU training for auto-research VLA.

	CaP: frozen code policy			AaP: agent acts live
Task	Fable 5.1	GPT-6-astra	Opus 5	Fable 5.1	GPT-6-astra
Banana into Bowl	70.0	50.0	80.0	100.0	80.0
Push-T	100.0	60.0	40.0	100.0	100.0
Cup Upright	70.0	50.0	40.0	20.0	100.0
Block Stacking	71.0	17.0	18.0	52.0	60.0
Spell SEAR	40.0	30.0	0.0	60.0	100.0

Table 4: Held-out scores on the five physical tasks (%). Mean outcome (success 1, partial 0.5, failure 0) over ten operator-staged scenes per code-as-policy (CaP) task and five per agent-as-policy (AaP) task; Block Stacking reports the mean highest tower, capped at ten cubes, as a percentage. Each entry is one practice run per agent. Practice counts and costs are in Tables 18 and 19 .

Figure 11: What hardware exposes that simulation misses. (a) A grasp that squeezes a real banana. (b) Gripper pads worn through during practice. (c) How often a task starts in a hard state: in practice, where the agent resets the scene itself, and in the test, where an operator sets it up (counts from the recorded start states).

Figure 12: Dexterous-hand numpad typing. (a) The Sharpa Wave hand on a Flexiv Enlight arm at a physical numpad. (b) Accuracy (blue, left axis) and speed (purple, right axis) over 2.5 hours from first launch, including a restart. Dots are scored evaluations, hollow squares the speeds of unscored attempts; curves are Gaussian-smoothed with a 12-minute bandwidth. Shading marks phases annotated from the agent’s journals; the dashed line is the 5 keys/s speed gate. Completing the task also requires at least 90% accuracy over a sliding window of ten evaluations.

Figure 13: Model scores across task families, without a library. (a) TEST success, one line per model, families from easiest to hardest. (b) The same scores against the Artificial Analysis Intelligence Index ( Artificial Analysis, 2026 ) for robosuite, LIBERO-PRO, and RoboLab; colors as in (a). Scores with a library are in Table 6 , and the library’s effect is in Figure 6 .

Figure 14: The proposed co-evolving loop. Experience from each interaction round can update four components that return to the agent. Solid grey: what SEAR does today (the library carries over between tasks; a VLA is trained but stays with its task suite). Dashed blue: proposed (harness revision and training the LM on its own traces).

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Model	(median min)	Best-of-3	Vanilla	Vanilla $-$ BO3	Lineage $-$ Vanilla
		First success			Session term	Library term
RoboLab	Claude Opus 5	58	57.1	61.9	$+4.8$	$+6.1$
	GPT-5.6-sol	21	48.5	46.9	$-1.6$	$-4.5$
	Kimi-k3	88	6.4	30.1	$+23.7$	$+21.8$
	Gemini-3.6-flash	36	21.1	18.3	$-2.8$	$+2.0$
	DeepSeek-v4-flash	59	14.8	23.1	$+8.3$	$+4.1^{\dagger}$

Appendix

Table 5: Session and library terms of Equation 1 , the values behind Figure 6 ; TEST success (%); BO3 is best-of-3. RoboLab: one 120-minute session or three 40-minute attempts; LIBERO-RIG: 60 minutes or three of 20. First success: median time of the first validation success over the Vanilla runs that ever succeed. LIBERO-RIG best-of-3 is read over tasks with a graded winner (all 18; 9 for Gemini-3.6-flash). † Over the first 58 RoboLab tasks, where DeepSeek-v4-flash’s runs stopped when the model was retired (Lineage 30.7, Vanilla 26.6 on those tasks). § Its successor, DeepSeek-v4.1-flash.

	robosuite		LIBERO		LIBERO-PRO		LIBERO-RIG		RoboLab
Model	Lineage	Vanilla	Lineage	Vanilla	Lineage	Vanilla	Lineage	Vanilla	Lineage	Vanilla
Claude Opus 5	$97.1\pm 1.4$	$93.8\pm 7.0$	$94.4\pm 2.5$	$95.2\pm 1.1$	$91.8\pm 0.5$	$93.0\pm 1.8$	$97.0\pm 2.7$	$96.7\pm 2.2$	$68.0\pm 3.5$	$61.9\pm 1.0$
GPT-5.6-sol	$84.8\pm 6.4$	$89.0\pm 2.2$	$88.3\pm 3.9$	$87.1\pm 2.3$	$88.4\pm 2.8$	$83.1\pm 3.3$	$91.5\pm 5.7$	$88.0\pm 7.1$	$42.4\pm 0.6$	$46.9\pm 1.8$
Gemini-3.6-flash	$77.6\pm 6.6$	$70.5\pm 0.8$	$82.4\pm 3.9$	$71.4\pm 4.1$	$76.2\pm 1.2^{\lx@paragraphsign}$	$63.5\pm 3.8$	$83.3\pm 18.9^{\lx@paragraphsign}$	$36.7\pm 4.0$	$20.3\pm 2.4$	$18.3\pm 3.6$
Kimi-k3	$88.1\pm 2.2$	$76.2\pm 5.9$	$77.8\pm 1.3$	$77.8\pm 1.8$	$80.5\pm 3.5$	$78.3\pm 3.5$	$94.4\pm 5.3$	$72.0\pm 5.8$	$51.9\pm 4.9$	$30.1\pm 2.9$
DeepSeek-v4-flash	$77.0\pm 13.5$	$70.0\pm 1.4$	$65.9\pm 9.1$	$59.3\pm 3.6$	$75.5\pm 4.5$	$57.5\pm 1.2$	$88.6\pm 0.4^{\lx@sectionsign}$	$77.8\pm 1.5^{\lx@sectionsign}$	$30.7\pm 5.1^{\dagger}$	$23.1\pm 2.1$

Appendix

Table 6: Model scores across task families, with and without a library , the values behind Figures 13 and 6 . TEST success (%), mean

\pm

SD over three runs (two where marked ¶ ). Bold : highest in each column. † First 58 RoboLab tasks only, where DeepSeek-v4-flash’s runs stopped when the model was retired; its Vanilla score on the same tasks is

26.6\pm 2.2

. § DeepSeek-v4.1-flash, its successor.

Pairs	Outcome	Lineage rep.	Third Vanilla rep.	Margin
276 unsolved (first two Vanilla at 0)	above 0	32.5	22.5	$+10.0$
	at least 50%	23.1	13.0	$+10.1$
1,234 with some Vanilla success	above the reference	33.7	26.1	$+7.6$
	below the reference	23.1	31.9	$-8.8$

Appendix

Table 7: The library against a rerun’s noise. Share of comparisons (%) in which one repetition scores above (or below) the reference, the mean of the first two Vanilla repetitions of the same model–task pair; 1,510 pairs over robosuite, LIBERO, LIBERO-PRO, and RoboLab. A Lineage repetition is compared with what a third Vanilla repetition does against the same reference.

Figure 15: Where a gain comes from and how far it reaches, for Opus. Tiered TEST success (%) of code-as-policy (CaP) and auto-research (AR) agents driven by Claude Opus 5. Rows are the family an agent practiced in, columns the family its policy was graded on, both ordered easiest to hardest. The diagonal is the practiced family (T1); cells to its right are graded at increasing distance from practice (T2–T4). Table 13 gives the numbers. The two VLA LIBERO-RIG cells average each checkpoint’s own-suite cells over the ten fault axes. The real-robot cell (T5, 40.0) is Opus’s frozen physical code policy on the four non-stacking tasks; that run starts from an empty library ( Section 4.4 ), so it is not a transfer from simulation.

Figure 16: A task-blind lineage graded on scenes it has not yet reached. One Opus self-exploration lineage on RoboLab, its task-general program frozen at library versions v2, v13, v25, and v37 (after exploring 1, 11, 23, and 35 of its 47 scenes) and graded ten times on the same 44 probe tasks, one per scene, including scenes it had not yet reached. Nothing is rewritten per task. (A) Success on the same 44 tasks and on the 12 scenes the lineage never explored; grey is the 95% bootstrap interval over tasks. (B) Where graded episodes end. This is one lineage graded on one fixed layout per task, so the intervals cover tasks, not lineages. Of 33 tasks graded before and after their scene was explored, 26 did not change.

		Mean TEST success			All ten TEST states solved
Family	Tasks	CaP	VLA	VLA $\geq$ CaP	CaP	VLA
robosuite	7	97.1	61.0	14%	86%	29%
LIBERO	40	94.4	94.0	80%	85%	75%
LIBERO-PRO	160	91.8	90.6	79%	81%	70%
RoboLab	105	68.0	47.2	48%	46%	14%

Appendix

Table 8: Code against weights, family by family. Opus code (CaP) and weights (VLA) lineages. VLA

\geq

CaP compares the first trial of each paradigm task by task. All ten TEST states solved is the share of graded (trial, task) cells with perfect TEST success: three CaP trials per family, three VLA trials on robosuite, one elsewhere.

Fault	nominal	cam. orientation	cam. position	cam. field of view	wrist-cam mount	gripper pads	frame rotation	actuation gain	base pose	table height	proprio. noise
CaP	96.4	7.0	8.7	11.3	95.3	94.2	95.6	93.4	92.1	88.2	73.7
VLA	86.2	56.7	60.0	90.0	55.0	66.7	83.3	81.7	81.7	83.3	98.3

Appendix

Table 9: Code and weights under each LIBERO-RIG fault axis. TEST success (%) of LIBERO-practiced Opus policies on LIBERO-RIG at the moderate level. The control-frequency axis is excluded from every LIBERO-RIG result: the code-as-policy harness runs the rig at its nominal rate and cannot apply it. The VLA row is the LIBERO lineage checkpoints graded on the cells of their own suite, three to six checkpoint-cells per axis.

Share of memory entries touching…	RoboLab	LIBERO	RoboLab	LIBERO/PRO
	CaP		VLA
grasp / contact geometry	69%	64%	11%	12%
perception / grounding	58%	42%	8%	10%
frame / calibration / kinematics	46%	38%	7%	15%
motion / controller	40%	39%	8%	8%
training recipe / checkpoint	6%	6%	36%	39%

Appendix

Table 10: Code agents write about the world; VLA agents write about the pipeline. Share of memory entries touching each topic. CaP: the 2,368 memory entries of the three corrected-frame RoboLab lineages’ latest libraries and the 80 of the robosuite-and-LIBERO library. VLA: the 3,924 headings and bullets in the memory files of the 20 VLA lineages. An entry can touch several topics; topics are keyword classes.

System	CaP	AaP	AR	Sim / real	tasks	Controls	Judge	Generalization measured
	Policy form				Library across
Kuroki & Tang (2026)	$\CIRCLE$	$\Circle$	$\Circle$	sim	$\CIRCLE$	$\LEFTcircle$	$\LEFTcircle$	new tasks
Harness evolution ( Wei et al., 2026 )	$\Circle$	$\CIRCLE$	$\Circle$	sim	$\LEFTcircle$	$\Circle$	$\LEFTcircle$	new tasks, new benchmark
SimEX ( Hu et al., 2026a )	$\CIRCLE$	$\Circle$	$\Circle$	both	$\LEFTcircle$	$\Circle$	$\LEFTcircle$	new simulator, sim-to-real
RPG ( Wang et al., 2026 )	$\LEFTcircle$	$\CIRCLE$	$\Circle$	both	$\CIRCLE$	$\Circle$	$\LEFTcircle$	perturbations, sim-to-real
Robo-COP ( Zhang et al., 2026b )	$\LEFTcircle$	$\CIRCLE$	$\CIRCLE$	both	$\Circle$	$\Circle$	$\LEFTcircle$	new initial states

Appendix

Table 11: Self-evolving robot agents released since mid-September 2026 , in the columns of Table 1 .

\CIRCLE

yes,

\LEFTcircle

partial,

\Circle

no, – not reported, from each paper’s full text. Controls : budget-matched restarts and a no-library run. Judge : final policy graded on states and logic the agent cannot access.

	Library lineage				No-library best-of-3
Axis	s3	s4	s5	s3 TEST	s3	s4	s5
frame rotation	95.9	94.9	89.7	95.6	73.8	71.2	68.0
actuation gain	95.3	85.7	67.4	93.4	70.2	60.8	51.5
proprioception noise	73.4	37.1	4.0	73.7	62.0	40.8	6.2
camera orientation	6.8	7.1	7.5	7.0	2.8	2.5	3.0
camera position	8.7	11.4	10.2	8.7	2.8	4.8	6.2

Appendix

Table 12: Figure 17 as numbers: the LIBERO-practiced Opus policies re-graded on LIBERO-RIG, VAL success (%) averaged over tasks per perturbation axis and severity, the lineage as the mean of three runs over all 40 LIBERO tasks and the best-of-3 winners over the same 40 tasks. Columns s3 to s5 are severity levels 3 (moderate) to 5 (most severe). The lineage’s s3 TEST column gives the values behind Figure 15 ; its mean over the ten axes, 65.9 pooled and

66.0\pm 0.2

over the three runs, is that figure’s LIBERO

\to

LIBERO-RIG code-as-policy cell (T2). LIBERO-RIG’s control-frequency axis is omitted: the code-as-policy harness builds the rig at the nominal 20 Hz and cannot apply it, so it is excluded from every LIBERO-RIG result in this paper. Complete grades only.

Practiced on	Policy	robosuite	LIBERO	LIBERO-PRO	LIBERO-RIG	RoboLab
robosuite	CaP	$97.1\pm 1.4$	$41.7\pm 2.4$	$24.9\pm 12.6$	$35.7\pm 7.2$	$30.1\pm 6.9$
	AR	$61.0\pm 8.7$	$42.6\pm 7.7$	$43.6\pm 1.6$	—	$12.8\pm 5.7$
LIBERO	CaP		$94.4\pm 2.5$	$47.7\pm 2.9$	$66.0\pm 0.2$	$23.6\pm 3.8$
	AR		$94.0\pm 2.9$	$44.8\pm 4.9$	$75.7\pm 10.3$	$7.3\pm 1.6$
LIBERO-PRO	CaP			$91.8\pm 0.5$	$65.3\pm 1.9$	$32.3\pm 1.8$
	AR			$90.6\pm 3.1$	$45.3\pm 10.8$	$10.4\pm 2.1$

Appendix

Table 13: The cells of Figure 15 as numbers: TEST success (%) of the Claude Opus 5 code-as-policy (CaP) and auto-research (AR) policies, rows the family practiced in and columns the family graded on, mean

\pm

sample standard deviation over three replicates where replicated; the AR diagonal cells of LIBERO, LIBERO-PRO, and RoboLab are one trial each, with

\pm

over their four or twelve suites. The diagonal is T1, the off-diagonal cells T2 to T4 as Figure 15 marks them. The T5 cell, 40.0, is the physical lineage of Table 18 and is not repeated here. For the T3/T4 CaP cells the frozen exploration policy, graded with the instruction only at run time, scores (robosuite

\to

LIBERO)

26.9\pm 7.6

(robosuite

\to

LIBERO-PRO)

14.6\pm 9.5

(robosuite

\to

LIBERO-RIG)

24.3\pm 11.4

(robosuite

\to

RoboLab)

25.8\pm 3.7

(LIBERO

\to

RoboLab)

19.1\pm 5.8

(LIBERO-PRO

\to

RoboLab)

26.7\pm 2.5

(LIBERO-RIG

\to

RoboLab)

28.6\pm 4.8

. § Two of the three replicates found a wrong-axis gripper offset in the simulated tool frame (a rig defect) on their own; the third was rerun from the start with the offset corrected (authored 34.5, 44.3, and 31.8; the last over the 115 of its 120 exploration tasks that were graded).

Donor $\to$ target	Policy	Claude Opus 5	GPT-5.6-sol	Kimi-k3	Gemini-3.6-flash
robosuite $\to$ LIBERO	authored	$41.7\pm 2.4$	$34.7\pm 13.6$	$43.7\pm 4.4$	$10.2\pm 2.9$
	frozen	$26.9\pm 7.6$	$20.9\pm 6.1$	$38.0\pm 4.5$	$8.3\pm 7.2$
robosuite $\to$ LIBERO-PRO	authored	$24.9\pm 12.6$	$21.5\pm 12.1$	$26.7\pm 7.2$	$6.6\pm 6.2$
	frozen	$14.6\pm 9.5$	$11.3\pm 9.0$	$20.8\pm 8.9$	$4.2\pm 4.4$
robosuite $\to$ LIBERO-RIG	authored	$35.7\pm 7.2$	$23.3\pm 20.5$	$27.7\pm 5.2$	$1.2\pm 2.0$
	frozen	$24.3\pm 11.4$	$8.2\pm 7.1$	$8.9\pm 8.2$	$2.0\pm 2.5$

Appendix

Table 14: Self-exploration for every model that ran it: TEST success (%) of the policy authored once the instruction is disclosed (authored) and of the task-general exploration policy graded with the instruction only as a run-time input (frozen), for each donor library and target family of Figure 15 , mean

\pm

sample standard deviation over three replicate lineages. The Opus column is that figure’s T3/T4 code-as-policy cells and their frozen-policy ablation; DeepSeek-v4-flash did not run this campaign. The RoboLab rows are under the corrected tool frame: lineages that never discovered the wrong-axis gripper offset were rerun from session zero (one Opus, three GPT-5.6-sol, seven Kimi-k3, and all twelve Gemini-3.6-flash lineages, each through its 48 sessions except Gemini’s LIBERO-RIG-donor third replicate at 45), and the lineages that had found the offset themselves keep their full-course reading. A fully errored cell scores zero.

Model	Condition	robosuite	LIBERO	LIBERO-PRO	LIBERO-RIG	RoboLab	Mean
Claude Opus 5	with library	97.6	95.2	92.5	96.1	71.1	90.5
	no library	93.8	95.7	93.2	96.8	62.8	88.5
	best-of-3	87.1	62.5	49.3	31.1	52.8	56.6
	first success, min	13.7	27.9	27.7	28	58	25.3
GPT-5.6-sol	with library	85.7	90.5	90.5	92.4	47.8	81.4
	no library	91.9	90.5	86.1	90.2	51.3	82.0

Appendix

Table 15: The conditions of Figure 4 at the end of the budget, per family: best-so-far VAL success (%) averaged over each family’s cells at 60 minutes of self-evolve time (120 on RoboLab), and the equal-weight mean that the figure plots. First success: median minutes to the first validation success over the no-library runs that ever succeed; its Mean halves RoboLab’s time to put it on the 60-minute scale of the other families. Figure 5 plots, per model, the Mean of no library minus best-of-3 against the Mean first success. The LIBERO-RIG with-library value is the mean of the seed-matched runs and the no-library value the mean of three 60-minute Vanilla repetitions, neither told the fault, on the 18-cell course; the best-of-3 control is read over the cells with a graded winner. RoboLab is the corrected-frame campaign over all 105 cells of its curriculum, all three conditions on the same cells; † DeepSeek-v4-flash’s three RoboLab conditions are read on the first 58 cells, where its runs stopped when the model was retired.

	Code-as-policy			Auto-research VLA
Family	Lineage	Vanilla	Best-of-3	Vanilla	Lineage
robosuite	$97.1\pm 1.4$	$93.8\pm 7.0$	90.0	$61.4\pm 10.0$	$61.0\pm 8.7$
LIBERO	$94.4\pm 2.5$	$95.2\pm 1.1$	73.8 ∗	$94.0\pm 3.6$	$94.0\pm 2.9$
LIBERO-PRO	$91.8\pm 0.5$	$93.0\pm 1.8$	67.2	$92.4\pm 4.9$	$90.6\pm 3.1$
LIBERO-RIG	$97.0\pm 2.7$	$96.7\pm 2.2$	30.6	—	—
RoboLab	$68.0\pm 3.5$	$61.9\pm 1.0$	57.1	$43.1\pm 9.3$	$47.2\pm 3.7$

Appendix

Table 16: Opus on the practiced task under all three conditions. Claude Opus 5 on held-out seeds of the practiced task (T1), TEST success (%), mean

\pm

SD over three repeated runs where repeated; the auto-research cells on LIBERO, LIBERO-PRO, and RoboLab are one trial each, with

\pm

over their four or twelve suites. Lineage carries the library forward; Vanilla is the matched no-library session; Best-of-3 spends the same budget as three independent attempts and keeps the best by validation score. ∗ End-of-run TEST of the validation-selected winners on the 47 robosuite and LIBERO tasks. LIBERO-RIG: the 18 cells of the course at the moderate level, the lineage over three seed-matched runs, Vanilla over three 60-minute runs per cell that are not told the fault. ¶ Mean over the four non-stacking physical tasks, ten operator-staged scenes each ( Table 4 ).

Figure 17: Opus policies under LIBERO-RIG faults, with and without a library. LIBERO-practiced Opus policies re-graded on LIBERO-RIG by fault axis and severity, validation success averaged over tasks: the library lineage (three lineages, all 40 tasks) against the no-library best-of-3 winners on the same tasks. Table 12 lists the values.

Term	Meaning	Where
Policy forms
Code-as-policy (CaP)	The agent delivers a program; no language model runs at execution time.	Section 2
Auto-research (AR)	The agent runs training experiments that fine-tune a VLA from rollouts of its own code policies.	Section 2
Agent-as-policy (AaP)	The language model stays in the control loop and chooses each action from live observations.	Section 2
What persists
Library	The code skills and textual memories that persist between tasks; each task inherits and extends it.	Section 2

Appendix

Table 17: Glossary. Terms used in a specific sense in this paper, and where each is introduced.

	Fable 5.1		Opus 5		GPT-6-astra
Task, in lineage order	practice	score	practice	score	practice	score
Banana into Bowl	1h46 10/10	7.00	2h50 13/14	8.00	12h 27/45	5.00
Push-T	6h34 26/29	10.00	12h 8/58	4.00	12h 20/23	6.00
Cup Upright	4h18 14/19	7.00	9h33 68/105	4.00	12h 27/46	5.00
Block Stacking	12h 88/106	7.10	12h 15/210	1.80	12h 3/19	1.70
Spell SEAR	4h37 11/18	4.00	12h 1/45	0.00	12h 9/15	3.00

Appendix

Table 18: CaP practice and evaluation details. Scores are out of ten. Practice entries give time to the ten-success streak (bold) or the 12-hour budget, then scored forward successes/attempts. Bold mean score marks the best model. Counting conventions are defined in the accompanying text.

	Fable 5.1		GPT-6-astra
Task, in lineage order	practice	score	practice	score
Banana into Bowl	2h15 10/10	10.00	4h23 10/10	8.00
Push-T	7h03 17/19	10.00	9h56 10/10	10.00
Cup Upright	2h43 11/13	2.00	8h40 10/10	10.00
Block Stacking	12h 33/62	5.20	12h 14/23	6.00
Spell SEAR	3h36 12/15	6.00	12h 11/13	10.00

Appendix

Table 19: AaP practice and evaluation details. Scores are out of ten. Practice entries give time to the ten-success streak (bold) or the 12-hour budget, then successful/recorded forward evaluations. Bold mean score marks the best model. No Opus 5 AaP evaluation is available.

Task	Fable 5.1	Opus 5	GPT-6-astra	Fable 5.1	GPT-6-astra	AaP / CaP
	CaP, frozen policy			AaP, model in the loop
Banana into Bowl	1.3 (12)	1.4 (10)	1.3 (10)	7.8 (5)	12.8 (5)	6.7
Push-T	5.0 (12)	4.1 (10)	8.3 (10)	12.8 (5)	38.1 (6)	3.7
Cup Upright	1.6 (10)	1.8 (10)	1.6 (10)	10.5 (5)	8.9 (5)	5.6
Block Stacking	9.7 (11)	6.9 (10)	5.5 (10)	15.2 (5)	28.7 (5)	3.4
Spell SEAR	5.2 (10)	3.8 (10)	8.1 (10)	7.4 (6)	23.5 (5)	2.7

Appendix

Table 20: A model in the loop makes every episode several times longer. Rig time per TEST evaluation episode under the two execution modes: median minutes from the evaluation request to the verdict (number of episodes in parentheses). CaP is the frozen policy bundle of the offline evaluation; AaP keeps the model in the loop during evaluation. The last column is the ratio of the two medians over all episodes of the task.

Agent	Mode	Units	Harness refusal	Own limit	Other
Fable 5.1	CaP and AaP	34	3	29	2
Opus 5	CaP	35	3	32	0
GPT-6-astra	CaP	47	20	26	1

Appendix

Table 21: What stopped each failed or partial evaluation unit. A unit is an episode, except for GPT-6-astra’s Block Stacking, where each block-placement call counts. Harness refusal : all three of Fable’s arise from poses its own code selected when a working alternative existed; Opus’s are one refusal it steered into and two grasps that missed on the left arm’s provisional calibration; most of GPT-6-astra’s refuse paths its own rules chose, and 42 of its 47 units trace to its own choices.

Figure 18: Every physical practice attempt against time. Every forward attempt of the physical CaP campaign against hours since the task’s launch: tall ticks are scored attempts (green success, red failure), short grey ticks unscored attempts, and the black bar the tenth consecutive scored success, where the run stops; Block Stacking has no early stop. The right margin gives each task’s held-out score out of ten.

Figure 19: Self-evolution on hardware, every agent on every task (continued in Figures 20 and 21 ). Each block is one task, with one row per agent and four attempts per row, numbered in session order: the overview camera at the end of the attempt, with the elapsed session time, the attempt kind, and the judge’s verdict (green success, red failure). The axis under each block places the four numbered attempts of each agent at their session times, so the rows read as time lapses of 12-hour sessions. For Opus 5 on Banana into Bowl and Fable 5.1 on the other tasks the attempts are chosen to show how the session developed; the other rows generally show the first recorded attempt, the first scored success, a later scored failure, and the last recorded scored attempt. The displayed Opus and GPT-6-astra Push-T failures, GPT-6-astra Spell SEAR failure, and Fable Cup Upright examples are selected for visually unambiguous outcomes rather than proximity to a fixed session time. Each lineage starts from an empty library on Banana into Bowl and inherits its own earlier tasks’ workspace thereafter.

Figure 20: Self-evolution on hardware, continued from Figure 19 : Cup Upright and Block Stacking. Opus’s Cup Upright attempts after hour 2.9 practice a two-cup scene, because one cup left the table. On Block Stacking, Fable’s last tile is the build-and-topple cycle it settled into after its 310 mm record; Opus’s last stacking success comes at hour 5.3 and it has only single cubes after hour 6.3, and GPT-6-astra’s tower sits at 156 mm.

Figure 21: Self-evolution on hardware, continued from Figure 20 : Spell SEAR. Each tile states the instructed reading direction for SEAR in the displayed camera view. Left-to-right spells SEAR; right-to-left appears as RAES from camera left to right. Fable tiles 2–4 request RAES, while its first tile and all displayed Opus and GPT-6-astra tiles request SEAR; RAES is a valid forward spelling target, not a failed SEAR arrangement. Opus’s one success comes at hour 2.9; by hour 4.3 its S has left the table, and its last recorded scored attempt, at hour 5.4, reads a table with no S. GPT-6-astra’s first success comes at hour 6.4, and its last seven scored spellings all succeed.

Axis	Type	What physically changes	Levels 1 to 5
camera orientation	O	the scene camera’s mount is twisted; its position does not move	2 to 25 degrees
camera position	O	the scene camera is remounted elsewhere around the workspace, still aimed at it	5 to 50 degrees of arc
camera field of view	O	a different lens: the vertical field of view changes from the nominal 45 degrees	$\pm$ 2 to $\pm$ 18 degrees
wrist-camera mount	O	the wrist camera’s bracket is twisted and slid along its mounting arm	1 to 15 degrees, 2 to 30 mm
action-frame rotation	L	camera-to-arm miscalibration: every command executes in a frame rotated about vertical	2 to 25 degrees
actuation gain error	L	commands execute too strong or too weak by a scale factor	$\pm$ 5 to $\pm$ 50 percent

Appendix

Table 22: The ten LIBERO-RIG perturbation axes used in this paper. LIBERO-RIG also defines a control-frequency axis, the control loop running at a rate other than the trained 20 Hz; the code-as-policy harness runs the rig at the nominal rate and cannot apply that fault, so the axis is excluded from every result here. Type: O observable (at least one camera image changes at a matched configuration), L latent (no camera image changes), W weakly observable (visible only in the wrist view at close range). The last column gives the range from the mildest to the severest level.

Spatial	Object	Goal	LIBERO-10
00 action-frame rotation	00 camera position	00 gripper-pad geometry	00 camera orientation
02 actuation gain	02 camera field of view	02 action-frame rotation	02 camera position
	04 wrist-camera mount	04 actuation gain	04 camera field of view
06 proprioception noise	06 robot-base pose		06 wrist-camera mount
08 camera orientation	08 proprioception noise	08 table height	08 robot-base pose

Appendix

Table 23: The 18-cell LIBERO-RIG development course, all at the moderate level. Task indices follow LIBERO’s suite order.

Model	Level	Pairs	Repaired policy	Pre-course policy	Gain
Claude Opus 5	moderate	149	73.6	71.4	$+2.2$
	severe (4)	169	71.3	63.7	$+7.6$
	severe (5)	169	61.5	54.4	$+7.1$
GPT-5.6-sol	moderate	142	66.9	71.0	$-4.1$
Kimi-k3	moderate	118	56.3	66.3	$-10.0$
DeepSeek-v4-flash	moderate	142	51.1	41.0	$+10.1$

Appendix

Table 24: A policy repaired for one LIBERO-RIG fault, run on the same task under the other faults, against the pre-course LIBERO-PRO policy on the identical cells: validation success (%), macro over the ten axes, and the number of paired cells.

	All tasks		Tasks with teacher $\geq 90\%$
Successful demonstrations per task	Tasks	VLA success	Tasks	VLA success
none	9	0.0	—	—
1 to 49	12	20.8	4	40.0
50 to 99	20	41.0	10	60.0
100 to 299	59	58.0	46	63.0
300 or more	5	94.0	5	94.0

Appendix

Table 25: More demonstrations, higher success. The 105 RoboLab tasks grouped by the number of successful demonstrations recorded by their VLA lineage session, with the VLA’s mean TEST success (%) per bin; the right half keeps only tasks whose teacher passes at least 90% of TEST states.

What changed	Sessions	Median	Sessions	Median	Sessions	Median
	LIBERO		LIBERO-PRO		RoboLab
Collect more data	2	0.5	2	8.0	11	8.9
Improve inference settings	0	—	0	—	4	4.4
Average multiple model weights	0	—	2	4.2	0	—
Re-train on the same data	4	2.5	4	7.4	6	2.3
Re-grade same weights	2	0.0	0	—	3	1.1

Appendix

Table 26: What additional development time changes in AR. For each session with submissions before and after its first 5 hours on LIBERO or 10 hours on LIBERO-PRO and RoboLab, the best later submission is compared with the best earlier one. Entries give session counts and median in-session VAL gains in percentage points, grouped by the changes recorded in the 882 submissions. On LIBERO-PRO, retraining on the same data yields a median 7.4-point gain, close to the 8.0 points associated with collecting more data; returns are smallest on LIBERO, where scores are near the ceiling.

Agentic AI for Robotics

See all 40 papers

Oct 7, 2026

EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

Robot foundation models provide strong visuomotor control, yet their performance can degrade when object positions or task instructions change. Further improvements often require post-training on substantial robot data, which can be costly to collect through methods such as teleoperation. Agentic harnesses can adapt around the model, but current self-evolving harnesses use robot trials inefficiently when deciding which code and skill changes to pursue. We introduce EmbodiedRSI, a self-evolving agentic harness that autonomously decides where to explore next and turns the resulting physical interaction into improved code and skills. EmbodiedRSI realizes this through a Fast-Slow Dual-System Architecture, in which competing code and skill hypotheses are maintained in a Hypothesis Graph. Value-of-Information Experiment Selection chooses physical experiments that can distinguish these hypotheses. Their outcomes guide Code-Skill Co-Evolution. The Slow System builds Hierarchical Memory, and Reward-Grounded Memory Learning selects effective memory according to their value for later Fast-System improvement. On RoboCasa365, EmbodiedRSI reaches 77.0% overall success and 71.3% on Composite-Unseen, compared with 40.1% for the best baseline. EmbodiedRSI also reaches 86.8% overall success on LIBERO-Pro. Beyond benchmark performance, EmbodiedRSI transfers zero-shot to real-world robot, achieving 71.3% overall success across multiple challenging tasks.

Python Song, Zhixuan Liang, Kelsey Fu +3

Columbia University · Princeton University

Continual Robot Learning Robot Skill Learning Self-Evolving Agents Active Learning Robot Foundation Models

Ask PDF

Sep 30, 2026

Make Code as Policy Great Again: Frontier Agents Write, Call, and Evolve Robot Tools

Frontier models can control robots, but reasoning through every reach, grasp, and retreat makes manipulation slow and token-intensive. We revisit code as policy with a different division of labor: models build executable tools, code handles multi-phase motions, and models decide what to do next. We introduce URAI (Universal Robot-Agent Interface), which couples a programming agent that constructs robot tools with an execution agent that uses them in a feedback loop. The programming agent writes reusable and task-specific tools from task intent and refines them through execution feedback and human guidance. The execution agent selects and parameterizes these tools from current observations; each call runs a complete motion locally before returning control to the agent. Unlike delegating subsequent decisions to a generated program, this design retains model-level decision-making between tool executions. Validated tool revisions persist across episodes without updating foundation-model weights, and a shared GUI and API make the same tools available to humans and agents. Across five RoboDojo tasks and four frozen execution agents, URAI raises aggregate success from 18.0% to 53.0% relative to direct fingertip control, with the largest gain on Swap Blocks; with the same tools, a program written in advance reaches only 24% against 56% for two agents deciding after each call. Three of the four agents also finish episodes 1.3-1.5 times faster with 1.5-1.7 times fewer execution-agent output tokens; DeepSeek-V4-Flash's cost barely changes. We further evaluate URAI on seven real-world AgileX dual-arm tasks, spanning object manipulation, cloth folding, and human-interactive tic-tac-toe. URAI connects the coding and decision-making capabilities of frontier agents, organizing robot control around reusable tools that agents can both invoke and revise.

Shijia Ge, Alex Zhou, Jianshu Zeng +12

Hexafuture Inc. · Peking University · Beijing Institute of Technology +1

Robotic Tool Use Tool-Using Agents Robotic Manipulation Agentic AI for Robotics Code Generation for Robotics

Ask PDF

Apr 10, 2026

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment

Current LLM-based agents demonstrate strong performance in episodic task execution but remain constrained by static toolsets and episodic amnesia, failing to accumulate experience across task boundaries. This paper formalizes the Self-Evolving Agent (SEA) from the perspective of digital embodiment and continuous cross-task evolution, introduces the Evolutionary Flywheel as its minimal sufficient architecture, and presents SEA-Eval -- the first benchmark designed specifically for evaluating SEAs. Grounded in Flywheel theory, SEA-Eval establishes SR and T as primary metrics and, through sequential task stream design, is designed to quantify evolutionary gain, evolutionary stability, and implicit alignment convergence. Empirical evaluation reveals that, under comparable success rates, token consumption differs by up to 31.2 times between frameworks on individual tasks, with divergent evolutionary trajectories emerging under sequential analysis -- demonstrating that success rate alone creates a capability illusion and that the sequential convergence of

T

is the key criterion for distinguishing genuine evolution from pseudo-evolution.

Sihang Jiang, Lipeng Ma, Zhonghua Hong +9

Fudan University

Self-Evolving Agents AI Agent Benchmarks Long-Horizon Agent Evaluation

Ask PDF

Abstract

Agentic AI for Robotics

Explore similar work

EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

Make Code as Policy Great Again: Frontier Agents Write, Call, and Evolve Robot Tools

SEA-Eval: A Benchmark for Evaluating Self-Evolving Agents Beyond Episodic Assessment