DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents

Haoyuan Deng; Jiebin Liu; Tengxiao Zhang; Langning Yan; Hongye Cao; Ziwei Wang

cs.ROSep 30, 2026

DynaHarness: A Dynamic Physical Harness for Self-Evolving Robot Agents

Authors: Haoyuan Deng, Jiebin Liu, Tengxiao Zhang, Langning Yan, Hongye Cao, Ziwei Wang

Organizations: Nanyang Technological University · Nanjing University

Abstract

Pretrained robot policies provide useful action priors, but long-horizon manipulation still requires coordination between semantic reasoning and physical execution. Semantic reasoning operates at a coarser timescale than physical interaction, while episode-level failures provide limited guidance on which system component should be revised. We propose DynaHarness, a dynamic physical harness that couples semantic reasoning with physical governance through a shared execution contract and turns failure evidence into validated capability revisions. To be more specific, the slow brain proposes capabilities and symbolic arguments, while the fast brain grounds and monitors commands, refuses unresolved actions, substitutes capabilities, and requests replans when needed. The physical execution contract bounds each accepted command and records execution evidence across analytic skills, recovery skills, and the frozen VLA. Failure attribution localizes faults in these records and directs targeted revisions of reusable capabilities or execution mechanisms. Paired regression checks govern admission or rejection, closing the self-evolution loop. On LIBERO-Pro, DynaHarness achieves 75.2% on 800 newly sampled initial states, compared with 17.5% for the frozen policy. With the same capability library, full dynamic execution reaches 74.0% versus 63.9% under nominal one-step replanning. This demonstrates the value of DynaHarness as a dynamic physical harness that governs how existing capabilities are grounded, monitored, and coordinated during execution. Our project page is at https://denghaoyuan123.github.io/Dynaharness_page/.

Figures & tables

Figure 1: DynaHarness at a glance. (a) Robot-agent paradigms and DynaHarness . (b) LIBERO-Pro success against upper-model scale (Table 1 ). (c) Median scheduler and Qwen call times (Table 21 ); Sonnet/Claude Code slow-brain pilot (Table 23 ).

Figure 2: Overview of DynaHarness . The slow brain plans; the fast brain governs execution; the offline loop evaluates updates from recorded evidence.

Panel of Figure 3 — Figure 3: The fast brain within one episode ; the full episode is in Fig. 17 .

Figure 6: Execution and transfer (top), physical harness (bottom). a Budget used at episode end. b Steps after the last progress. c Frozen champion versus policy: new states (

800

) and untouched official states (

261

). d Without and with evolution (A2ctrl on development seeds), each against the frozen policy on its own seeds. e Evolved capabilities; the frozen policy solves none of these cells. f Steps by executor: one filmed success per cell (top) and all

20

seeds of four hard cells (bottom).

Figure 7: Development history. a Full evaluation of each round on seeds

21

–

40

. b Change from round

30

with the rejected candidate, on the hob and microwave cells. c Drawer update.

Appendix figures & tables40 assets

Supplementary material from the paper’s appendix.

Appendix

Command	Steps	Note
Approach	30
Guarded descent	about 30	41 before the speed work
Close the jaw	10	$0.5$ s gripper command; 20 at $1.0$ s
Lift	23
Carry over the corridor	60–80	depends on the corridor height
Lower	15

Appendix

Table 4: Steps per command stage , from the retained event stores.

Figure 9: The evolution loop and its registry. Registry entries track capability additions following fault probing, targeted revision and paired validation.

System	Upper model	Low-level policy	Evaluation setting
DynaHarness	Qwen3-VL-4B	frozen $\pi_{0.5}$	four cells, $200$ episodes each
$\pi_{0.5}$ (our run)	none	frozen $\pi_{0.5}$	four cells, $200$ episodes each
PhyAgentOS	Qwen3-VL-4B; GPT-4o-mini	frozen $\pi_{0.5}$	four cells, $200$ episodes each
Harness VLA	Qwen3-VL-4B	frozen $\pi_{0.5}$	four cells, $200$ episodes each
ENPIRE	Qwen3-VL-4B	$\pi_{0.5}$ via generated code	four cells, $200$ episodes each
Zetta	Qwen3-VL-4B	$\pi_{0.5}$ via tool interface	four cells, $200$ episodes each

Appendix

Table 5: Models and evaluation settings of our LIBERO-Pro runs.

Row	Source	Location and note
OpenVLA, $\pi_{0.5}$	Zhou et al. (2025)	Tables 2 and 4, Average rows (Task and Pos columns); 50 episodes per task
$\pi_{0.5}$ (our run)	ours	frozen $\pi_{0.5}$ LIBERO checkpoint, seeds $21$ – $40$ , all episode rows retained
$\pi_{0.5}$ -SFT	Zhang et al. (2026b)	Table 3, row $\pi_{\mathrm{RLinf}}$ (RLinf pi05_libero130_fullshot )
Fast-WAM	authors; Yuan et al. (2026)	LIBERO-Pro cells supplied by the authors; LIBERO from Table 2 of the publication
CaP-Agent0	Fu et al. (2026) ; Lu et al. (2026)	Goal cells: CaP-X Table 7 Average ( $0.168$ , $0.256$ ); 10-T and 10-S: ASPIRE Table 5 ( $0.024$ , $0.052$ )
RHO	Elmaaroufi et al. (2026)	Table 7 Average, RHO columns ( $0.556$ , $0.506$ )

Appendix

Table 6: Source of every row of Tables 1 and 8 , checked against the cited table.

Reasoner	Goal-T	Goal-S	Reasoner	Goal-T	Goal-S
GPT-5.5 (low)	28	42	Gemini 3.1 Pro	22	44
GPT-5.5 (medium)	24	44	Claude Haiku 4.5	20	38
GPT-5.5 (high)	26	38	Claude Sonnet 4.6	30	42
Gemini Rob-ER 1.6	34	44	Claude Opus 4.7	22	44
Gemini 3.5 Flash	28	48

Appendix

Table 7: Pigey with nine reasoners over the same frozen

\pi_{0.5}

(Table 15 of Galanti et al. (2026) ), 10 trials per task.

Figure 10: Real-world execution sequences for ring placement, cup stacking, bread placement, and ring placement in a drawer (top to bottom). Each row shows the initial scene followed by four execution stages, with the corresponding language instruction below.

Method	Upper VLM	Spatial	Object	Goal	Long	Avg.
Backbone policies
OpenVLA	none	$98.0$	$99.0$	$98.0$	$93.0$	$97.00$
$\pi_{0.5}$	none	$98.0$	$98.0$	$97.0$	$93.0$	$96.50$
$\pi_{0.5}$ (our run)	none	99.0	$99.0$	98.6	$95.4$	98.00
$\pi_{0.5}$ -SFT	none	99.0	$96.0$	$97.0$	$89.0$	$95.25$
$\pi_{0.5}$ , task-adapted	none	99.0	$98.6$	98.4	$93.6$	$97.40$

Appendix

Table 8: Success rate (%) on standard LIBERO.

Figure 11: Outcome composition with and without the rejected candidate. Round

31

ran

780

episodes; percentages are of each round’s episodes.

Outcome	Episodes
Success (benchmark verdict true)	541
Policy owned the episode	166
Infrastructure (socket dropped)	88
Plan refused	4
Other harness failure	1
Total, the reported denominator	800

Appendix

Table 9: Outcome of every episode in the analyzed round.

Zetta round	Goal-T	Goal-S	Mean
$\pi_{0.5}$	31.0	38.0	34.5
1	67.5	39.5	53.5
2	89.5	70.5	80.0
3	92.0	83.0	87.5
4	92.5	89.0	90.8
DynaHarness round	Episodes		Success (%)

Appendix

Table 10: Round histories of Zetta (top; Ding et al., 2026 ) and DynaHarness on seeds

21

–

40

Method	Spatial	Object	Goal	Long	Avg.
$\pi_{0.5}$ $\star$	99.0	99.0	98.6	95.4	98.00
LIBERO stdlive8c	99.0	99.2	96.4	93.6	97.05
LIBERO formal1	98.4	99.6	97.4	94.8	97.55
LIBERO formal2	98.8	99.6	97.0	96.8	98.05
Frozen champion ( 088ef2ea )	86.2	30.4	69.8	0.4	46.70

Appendix

Table 11: Configurations on unperturbed LIBERO (

2{,}000

episodes each). LIBERO-adapted builds and the frozen LIBERO-Pro champion use different settings; the champion includes

1064

tick-limit failures.

Method	Seeds	Goal-T	Goal-S	10-T	10-S	Avg.
$\pi_{0.5}$ , frozen	$1$ – $20$	20.5	20.5	21.0	6.5	17.1
DynaHarness without evolution	$1$ – $20$	10.5	20.0	20.0	5.0	13.9
$\pi_{0.5}$ , frozen	$21$ – $40$	19.5	17.5	20.0	8.0	16.2
DynaHarness , analyzed round	$21$ – $40$	75.0	69.5	73.0	53.0	67.6
DynaHarness , stat39	$21$ – $40$	74.5	76.0	76.0	64.5	72.8
DynaHarness , archived final	$21$ – $40$	75.0	81.0	76.0	65.0	74.25

Appendix

Table 12: Arms on both seed blocks. Blocks are separate experiments. The final development rates correspond to

74.25\%

overall.

Perturbation category	Successes	Success (%)
Camera viewpoints	1063/1599	66.5 [64.1, 68.8]
Robot initial states	1180/1550	76.1 [73.9, 78.2]
Objects layout	1322/1525	86.7 [84.9, 88.3]
Sensor noise	1391/1601	86.9 [85.1, 88.4]
Language instructions	1360/1537	88.5 [86.8, 90.0]
Light conditions	1098/1142	96.1 [94.9, 97.1]

Appendix

Table 13: DynaHarness on LIBERO-Plus , one trial per task, with Wilson

95\%

intervals.

Suite	C: DynaHarness	C: $\pi_{0.5}$	B: DynaHarness	B: $\pi_{0.5}$
Goal-T	79.0	26.5	78.1	26.0
Goal-S	78.0	19.5	81.8	25.8
10-T	77.5	17.5	75.3	7.8
10-S	66.5	6.5	71.1	2.2
Episodes	800	800	261	261
Success (%)	75.2	17.5	77.0	16.5

Appendix

Table 14: Post-selection success rates (%) by suite. C:

200

newly sampled states per suite. B: untouched official states, with

73

66

77

and

45

episodes in the listed suites, respectively.

Block	Comparison	$n$	W/L	$\Delta$ (pp)	$95\%$ CI	$p$
C	Champion vs. $\pi_{0.5}$	800	473/11	57.75	[45.75, 70.00]	$3.13\times 10^{-124}$
B	Champion vs. $\pi_{0.5}$	261	161/3	60.54	[44.44, 75.27]	$6.29\times 10^{-44}$
C	Champion vs. stat23	800	137/22	14.38	[6.62, 23.62]	$1.70\times 10^{-21}$
C	Champion vs. stat28	800	93/18	9.38	[3.88, 16.00]	$2.26\times 10^{-13}$
C	Champion vs. stat39	800	12/4	1.00	[0.00, 2.62]	0.0768
C	stat28 vs. stat23	800	65/25	5.00	[0.12, 11.50]	$2.97\times 10^{-5}$

Appendix

Table 15: Paired post-selection comparisons. W/L are exclusive wins of the first/second system. Intervals are cell-bootstrap

95\%

intervals;

p

is a two-sided exact test of discordant pairs.

Figure 12: Supporting panels. a Replayed stove requests. b PhyAgentOS retries and verifier. c Paired gate against full round for candidate C2.

Figure 13: Paired outcomes with the frozen

\pi_{0.5}

on the same

800

episodes (second measurement,

74.1\%

); each bar: won by both, won by this arm only, lost.

Arm	Goal-T	Goal-S	10-T	10-S	Total	Flags
Full DynaHarness	150	161	152	129	592	7
Nominal replanning (A2static)	130	150	117	114	511	—
Frozen sequence (A2seq)	130	159	115	106	510	0
No analytic contact skills	54	32	31	16	133	8
No recovery/intervention	149	161	145	130	585	7
Neither capability group	47	31	36	12	126	4

Appendix

Table 16: Capability and executor ablations by suite. All arms have

800

episodes,

200

per suite. Flags denote tick-limit or transport contamination, counted as failures; a dash denotes an unreported count. The bare-policy row is the archived same-block reference.

Comparison (Full minus treatment unless named)	W/L	$\Delta$ (pp)	$95\%$ CI	Exact $p$
Nominal replanning (A2static)	89/8	10.125	[3.375, 18.250]	$2.00\times 10^{-18}$
Frozen sequence (A2seq)	93/11	10.25	[3.50, 18.25]	$2.49\times 10^{-17}$
A2static minus A2seq	11/10	0.125	[ $-3.000$ , 2.875]	1.00
No analytic contact skills	469/10	57.38	[45.62, 69.00]	$2.09\times 10^{-124}$
No recovery/intervention	15/8	0.88	[ $-0.38$ , 2.62]	0.21
Neither capability group	473/7	58.25	[46.62, 69.62]	$7.25\times 10^{-130}$

Appendix

Table 17: Paired executor and capability comparisons. W/L are wins exclusive to the first/second arm; differences are first minus second. A uses cell-bootstrap

95\%

intervals. Full/A2seq bootstrap estimates use

20{,}000

whole-cell resamples, seed

20260926

. All

p

values are exact two-sided sign tests.

A. Prior routing archive
Arm/block	Event stores	Zero VLA (%)	VLA calls/ep.	Planner calls/ep.
A2 full control	770	74	2.39	3.68
No analytic contact skills	800	0	18.34	4.12
No recovery/intervention	750	75	2.11	4.09
Neither capability group	800	0	18.52	4.06
VLA unavailable	705	$\approx 100$	0.00	17.01

Appendix

Table 18: Routing, coverage and execution diagnostics. A: prior routing archive. B: result and trace coverage for the executor comparisons. C: common

783

Full/A2static pairs. D: all

800

A2static/A2seq pairs. Event counts are descriptive, may overlap, and do not identify individual causal contributions.

A. Whole run: $800$ episodes per arm
Metric	Full	A2seq
Planner calls	2,875	800
Environment steps	244,523	188,038
Environment-budget exhaustion	201	17
Tick-budget exhaustion	7	0
B. Common event traces: $783$ pairs

Appendix

Table 19: Full versus A2seq: execution diagnostics. A: directly recorded whole-run totals. B: events or steps on the

783

pairs with both trajectories. Rows may overlap and are not independent causal effects.

First divergence	Pairs	F	S	Both succeed	Both fail
Identical dispatches	468	0	0	468	0
Spent-capability substitution	109	16	3	12	78
Planned recovery insertion	84	18	1	0	65
Initial planner choice differs	61	22	6	19	14
Scheduler recovery insertion	27	2	0	0	25
Online planner selection	19	10	1	0	8

Appendix

Table 20: First dispatch divergence on

783

trace-matched pairs. F/S: Full-only/A2seq-only success; both columns denote shared outcomes. Categories are descriptive associations with the eventual result.

Figure 14: Execution schedule and decision latency. a Nominal execution periods. b Median scheduler decision and planner-call latencies in the

800

-episode run (Table 21 ).

Component	Calls	p50	p90	p99	max
Slow brain (Qwen3-VL-4B)	3,090	913 ms	1,277 ms	1,865 ms	5,581 ms
Fast brain	47,199	0.046 ms	0.056 ms	0.071 ms	5.58 ms
Fast brain, learned artifact	–	0.49 ms	–	–	–
Episode wall clock	Mean	p50	p90	Success	Failure
Goal-T	19.2 s	15.5 s	36.0 s	13.7 s	35.4 s
Goal-S	22.1 s	18.5 s	35.0 s	18.8 s	36.6 s

Appendix

Table 21: Latency and wall clock of the reported agent ,

22

channels running concurrently.

Variant	Success (%)	Net episodes	Paired $p$
Latching control	73.1	—	—
Without latched verdict	69.6	$-28$	$6.2\times 10^{-5}$
Five-capability control	73.6	—	—
Without five evolved capabilities	64.8	$-71$	$<10^{-13}$

Appendix

Table 22: Completion latching and five evolved capabilities. Separate paired campaigns on development seeds

21

–

40

800

episodes per arm.

	Qwen3-VL-4B	Claude Sonnet 5	Ratio
Latency per call, median (ms)	1,124	6,683	5.9
Latency per call, maximum (ms)	1,485	10,348	7.0
Prompt tokens per call	4,036	74,600	18
Prompt tokens per episode	21,253	444,102	20.9
Slow-brain calls per episode	5.35	5.35	1.0
Calls over the $20$ s timeout	0	0	–

Appendix

Table 23: Latency and token counts in the slow-brain pilot. Sonnet 5 via Claude Code,

20

episodes and

107

calls; Fig. 1 c uses its median.

Figure 15: goal_swap[5] , seed 22, both arms , cut at the same environment steps.

Capability	Inv.	Compl.	Cells	Conversion
push_object	1	0 %	1	goal_swap[5] $0\to 8$
slide_drawer_object	21	0 %	2	10_task[3] $0\to 0$
turn_knob_object	33	21 %	1	10_task[2] $0\to 19$
keyframe_recovery	35	100 %	2	cross-cell
swing_door_object	1	0 %	1	10_swap[9] $0\to 7\to 10^{\dagger}$
† later rounds, for which only aggregates were retained.

Appendix

Table 24: Evolved capabilities as measured objects. Inv. : dispatches; Compl. : completed fraction; Conversion : source-cell successes before and after.

Figure 16: Successes of 20 per cell, four arms ; the early seeds

1

–

20

block (top) and the development block (bottom) are compared only within a block.

Cell	Agent	First change	Both	Paired outcome
goal_swap[0] (reachable)	11	18	20	2 pairs won, none lost
goal_swap[3] (unreachable)	2	3	3	2 against 2
goal_task[7] (unreachable)	18	17	19	within repeat noise
17 further cells (unreachable)	270	270	270	no discordant pair
Goal-T and Goal-S ( $/400$ )	301	308	312

Appendix

Table 25: The two drawer changes ,

400

episodes per arm on seeds

21

–

40

, paired by task and seed; unreachable cells are not scored.

Cell	Instruction	Prom.	Rej.	$\Delta$
goal_task[1]	plate on stove	20	0	$-20$
10_task[2]	stove on + pan	19	1	$-18$
goal_swap[1]	bowl on stove	17	0	$-17$
10_task[8]	moka pot on stove	17	0	$-17$
10_swap[2]	stove on + moka	7	0	$-7$
10_swap[9]	mug in microwave	0	5	$+5$

Appendix

Table 26: The rejected candidate over 39 shared cells , successes of

20

Panel of Figure 17 — Figure 17: Refusal and substitution of an infeasible capability by the fast brain.

Panel of Figure 18 — Figure 18: An evolved capability at work , beside the frozen policy on the same seed.

Panel of Figure 19 — Figure 19: Two long-horizon episodes whose frozen-policy counterparts fail on the same seeds.

Cell	Method	0	1	2	3	4	5	6	7	8	9	Avg.
Goal-T	$\pi_{0.5}$ $\dagger$ ( Ding et al., 2026 )	0	95	10	0	100	0	20	80	5	0	31.0
	Zetta $\dagger$ ( Ding et al., 2026 )	80	100	95	80	100	100	95	95	80	100	92.5
	$\pi_{0.5}$ $\star$ seeds 1–20	0	15	30	0	25	0	20	80	30	5	20.5
	$\pi_{0.5}$ $\star$ seeds 21–40	0	5	25	0	10	0	15	85	55	0	19.5
	DynaHarness $\star$ 21–40	0	100	100	0	90	95	95	90	95	80	74.5
Goal-S	$\pi_{0.5}$ $\dagger$ ( Ding et al., 2026 )	0	60	0	45	0	0	0	100	100	75	38.0

Appendix

Table 27: Per-task success (%) on LIBERO-Pro.

\dagger

Seeds

1

–

20

, Table 3 of Ding et al. (2026) ;

\star

ours, DynaHarness at round 39.

	GPT-4o-mini			Qwen3-VL-4B
Suite or cell	F	Fi	V	F	Fi	V
Spatial	100	100	100	98	98	99
Object	99	100	100	99	99	99
Goal	98	98	98	98	98	99
Long	80	92	95	81	96	97
LIBERO ( $/400$ )	377	390	393	376	391	394

Appendix

Table 28: PhyAgentOS run by us , successes of

100

per suite or cell.

	LIBERO		LIBERO-Pro
	GPT	Qwen	GPT	Qwen
Attempts per episode, mean	1.07	1.04	1.30	1.16
Episodes retried	21	16	67	46
Verifier calls	39	25	200	200
Environment steps per episode, mean	168	158	501	442
Policy calls per episode, mean	34.0	32.0	100.2	88.4

Appendix

Table 29: Execution statistics of our PhyAgentOS runs , from their episode records.

Stage	Recorded outcome
Online recovery agent	3 invocations; image read requested as text, 0 tool calls; failed closed with no environment write
Offline diagnosis	3 visual-evidence claims citing images that were not delivered; diagnosis rejected by the validator
Recovery execution	proposal accepted at confidence 0.95; reported completed after 78 steps; joint target unmet; episode failed
Candidate proposal	8 of 9 proposals failed validation
Same-seed gate	3 gates, 0 successes in either arm; none passed
GPT-5.6-sol candidate	13/20 against the bare policy’s 6/20; discordant pairs 8 and 1, exact sign test $p=0.039$

Appendix

Table 30: Zetta’s released code on drawer tasks : where each Qwen3-VL-4B stage stopped (top) and a GPT-5.6-sol candidate against the bare policy (bottom).

Task, seed	Outcome	Turns	Tools	Policy	Tokens	Time (s)
1, 1	success	31	62	3	3.32	348
1, 3	failure	37	70	4	4.51	505
0, 5	failure	33	81	4	3.27	448

Appendix

Table 31: Harness VLA agent, three Spatial-T episodes. Tokens in millions.

Paired mechanism benchmark	Incumbent	C1	C2
goal_task[9]	16	16	18
10_task[0]	17	17	19
goal_swap[1]	16	18	18
10_swap[8]	8	9	9
goal_swap[5]	10	10	10
10_swap[3]	0	0	0

Appendix

Table 32: Two-stage admission of candidate C2 : paired mechanism benchmark, full round and matched Goal-S comparison, top to bottom.

Explore similar work

Sep 24, 2026cs.RO

HarnessPAI: An Evolving Harness for Physical AI

Physical AI aims to build embodied agents that perceive the world, understand and reason about it, and decide how to act. Yet the field has focused primarily on the last component: the action model that maps observations to low-level controls. The prevailing training recipe can erode the perceptual and reasoning capabilities needed for robust behavior, leaving even strong action models vulnerable to scene perturbations and long-horizon tasks. We introduce HarnessPAI, a model- and embodiment-agnostic Harness framework for Physical AI that treats code as the executable and evolvable interface that organizes the underlying action primitive. The framework separates two timescales: within a rollout, it executes open-loop at the program level, with a fixed program guiding and checking execution; across rollouts, it evolves closed-loop, using execution feedback to revise the program and distill failures into reusable skills. Across desktop robot arms, household robots, a robot vacuum, and a legged walking agent, HarnessPAI improves on both pure action models and code-as-policy baselines without retraining the underlying model: a 61.6-point gain over

π_{0.5}

on LIBERO-PRO and a 27.2-point gain over WorldDreamer on RoboCasa atomic tasks. Once a program is selected, rollout execution requires no online high-level LLM deliberation. Beyond execution, the converged program is also a cheap and reliable expert-data collector, and fine-tuning

π_{0.5}

on collected expert data lifts success rate on LIBERO-PRO by 38.8 points. Our results suggest that the frontier of Physical AI depends not only on stronger action models, but also on executable harnesses that integrate perception, task understanding and reasoning, and action execution into a unified, verifiable, and feedback-driven system. Website: https://darwin-agent.github.io/HarnessPAI

Xin Wang, Wenhao Wu, Menghao Zhang +20

Embodied Artificial Intelligence Agent Harness Embodied Agents Perception Pipeline Artificial Intelligence Systems Evolving Harness+1

Ask PDF

Jul 20, 2026cs.RO

RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning

Long-horizon robotic tasks require a breadth of capabilities beyond what any single existing robot control policy can reliably provide. Combining heterogeneous policies with complementary strengths offers a promising solution, but introduces two key challenges: uncertain capability boundaries and distribution mismatches during policy handoffs. These challenges remain largely unaddressed by existing planning methods, which typically assume homogeneous, predefined skills with fixed applicability. We propose RoboHarness, a unified framework that encapsulates independently developed heterogeneous policies, including vision-language-action models (VLAs), world-action models (WAMs), reinforcement learning (RL) policies, and task and motion planners (TAMP), as reusable agentic skills. RoboHarness integrates understanding, memory, and evolution skills to reason about policy capabilities and support capability-aware task decomposition and policy routing. To mitigate distribution mismatches during policy handoffs, we introduce Memory Bridge, a plug-in policy-chaining mechanism that enables reliable transitions between heterogeneous policies without joint retraining. Extensive experiments across five public benchmarks, 500 customized tasks across 10 classes, and 135 real-robot trials demonstrate substantial gains in long-horizon and memory-dependent tasks, as well as robustness to out-of-distribution conditions.

Jinbang Huang, Zhiyuan Li, Yuanzhao Hu +6

Huawei Noah’s Ark Lab · University of Toronto · University of British Columbia +2

Robot Policies Long-Horizon Planning Robocasa Robot Systems Orchestration Reusable Agent Skills

Ask PDF

Aug 1, 2026cs.RO

From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning

Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring agents to adapt their plans under non-stationary conditions. We study this challenge through long-horizon embodied planning under environment deviations and execution uncertainty. Existing embodied-task benchmarks can expose such failures, but these failures are usually treated as evaluation outcomes instead of learnable signals for training agents to recover. In this work, we introduce DynamicEnvPlan, a closed-loop framework for high-level planning in dynamic environments. It extends embodied task execution with humanoid agents, high-level primitive skills, structured semantic memory, and controllable perturbations. Our data synthesis design consists of planning, perturbation, and guarded correction modules that turn dynamic execution states into recovery-oriented traces. The resulting traces are used for staged supervised fine-tuning, enabling the planner to learn from both nominal execution and perturbed recovery trajectories. Using 104 task-scene combinations spanning i.i.d., compositional generalization, and out-of-distribution settings for fine-tuning and evaluation, DynamicEnvPlan boosts success rate from 33.3% for the base planner to 76.2%, while improving across all seven evaluation metrics critical to physical-world interaction, including safety and affordance compliance.

Hao Yuan, Yuxin Wang, Lei Ji +1

University of Chinese Academy of Sciences · †Independent researchers. · 3Beijing Academy of Artificial Intelligence (BAAI)

Embodied Agents Dynamic Environments Planning

Ask PDF