MetaAct: Self-Evolving Executable Action Constraints for Embodied Language Agents
Authors: Jiaxing Zhao, Hanwen Wang, Di Liu, Shaoteng Liu, Jing-Ran Su, Yanheng Li, Qian Zha, Jiaxi Li, Hongbin Xie, Haoran Zhang, Xuan Song, Haoxuan Che
Organizations: Jilin University · Southern University of Science and Technology · Northwestern Polytechnical University · The Hong Kong Polytechnic University · City University of Hong Kong · Peking University · XGEN Labs
Self-evolving language agents have emerged as a promising paradigm for continuously improving agent behavior through experience. However, most existing approaches evolve textual artifacts, which can influence action preferences but cannot reliably enforce action admissibility in embodied environments with discrete and often irreversible actions. We propose MetaAct, which shifts self-evolution from textual guidance to executable action constraints by synthesizing GEAR (Grounded Executable Action Rules) from execution feedback. Each rule is represented as a typed specification, compiled into a deterministic guard, and validated offline through trajectory replay without additional model calls or environment interactions. Across four benchmarks, MetaAct improves task performance; on ScienceWorld and WebShop, violations drop from 70% to 0% and 40% to 1%, with score gains of +10.1 and +8.0. Our code is available at https://github.com/BaiCubed/MetaAct.
Figures & tables
Figure 1: From textual lessons to executable constraints. (1) In ScienceWorld, a single irreversible failure reveals an ordering rule: the thermometer must be focused before the target object. (2) Textual self-evolution distills the failure into a correct lesson, but the violating action remains in the candidate set and is still selectable. (3) MetaAct compiles the same rule into a GEAR guard that removes the violating action from the admissible set at execution time: text changes preference, whereas the constraint changes admissibility.
Figure 2: Overview of MetaAct. (1) Failed and successful trajectories provide contrastive execution evidence. (2) The LLM synthesizes typed GEAR specifications, which are compiled into deterministic guards and verified offline by trajectory replay. (3) Accepted guards constrain action execution at deployment time and can be continually expanded from new interaction feedback.
ALFWorld (134)
SciWorld (60)
TextCraft (60)
WebShop (100)
SR ↑
SR ↑
score ↑
SR ↑
score ↑
SR ↑
score ↑
Single-agent methods
Act-only
0.799
0.167
44.3
0.617
80.6
0.180
57.6
ReAct
0.851
0.050
27.5
0.667
72.3
0.120
21.7
SC-ReAct ( k=3 )
0.851
0.150
45.7
0.850
91.2
0.090
18.0
Reflexion ( ≤3 )
0.955
0.217
53.5
0.750
82.7
0.210
36.2
Table 1: Main benchmark comparison. Performance of published baselines and MetaAct on four embodied-language benchmarks using the DeepSeek-V4-flash backbone. MetaAct-synth denotes automatically synthesized executable constraints and MetaAct-oracle the hand-written reference. Best values are bold; the second-best distinct value in each metric column is underlined.
Figure 3: Matched-evidence comparison. Each row shows one variant. Left: task-score change relative to the unconstrained baseline, with 95% confidence intervals. Right: episode-level win/loss/tie counts against the same baseline. Filled markers indicate confidence intervals that do not include zero.
Figure 4: Effect of action masking and rule messages at execution time. Results on SciWorld with GPT-4.1-mini over 60 episodes per setting. Left: task score under different enforcement strategies ( ∗ : the 95% confidence interval for the score difference is entirely above zero). Right: fraction of episodes with at least one rule violation. Action masking removes violations, while combining masking with rule messages gives the largest score improvement.
Figure 5: Offline replay predicts harmful constraint behavior before deployment. SciWorld candidate specifications. (a) Held-out harmful blocking as a function of the offline gate score. (b) ROC curves for the gate criteria. (c) Harmful and catastrophic blocking rates for accepted and rejected candidates on held-out replay.
Offline
Online
Variant
Coverage ↑
Fidelity F1↑
Harmful ↓
Violation ↓
Score ↑
A0
Full MetaAct
0.211
0.135
0.000
0.167
50.77
A1
− Contrastive Evidence
0.189
0.411
0.000
0.229
47.62
A2
− Counterexample Repair
0.211
0.164
0.000
0.229
50.66
A3
− Constraint Union
0.189
0.164
0.000
0.292
47.37
A4
− Replay Verification
0.433
0.135
0.069
0.229
48.36
Table 2: Ablation study on SciWorld. Offline metrics are computed by replay; online metrics are averaged over three evaluation seeds with 60 episodes per seed.
Figure 6: Executable constraints can be acquired and deployed during an interaction stream. WebShop with GPT-4.1-mini over 100 consecutive episodes. The panels show 20-episode rolling task score, rolling violation rate, and the cumulative number of accepted constraints.
Backbone
No constr.
Prompt rule
MetaAct
Oracle
Δ vs. prompt [95% CI]
DeepSeek-V4-flash
44.45
44.30
54.60
55.30
+10.30[+1.15,+19.83]∗
GPT-4.1-mini
45.72
40.60
54.35
50.83
+13.75[+6.43,+21.92]∗
Qwen3-8B
34.77
30.90
36.73
32.97
+5.83[−0.77,+12.93]
Table 3: Executable-constraint gains persist across language-model backbones. SciWorld task score (60 paired episodes per backbone) with no added constraint, the textual prompt rule, MetaAct, and the hand-written reference.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
SciWorld test60
SR
score
steps
tokens/ep
ReAct
0.050
27.5
24.5
43.9k
ReflAct
0.050
31.9
17.9
48.7k
AutoGen (strongest baseline)
0.267
54.6
15.0
27.7k
constraint variant (ours)
0.300
55.3
33.2
139.9k
Appendix
Table 4: ReflAct compared with ReAct under matched settings.
Goal perturbation
Consistency ↑
Jaccard ↑
Δ blocks
paraphrase
1.0000
1.000
+0
clause reordering
1.0000
1.000
+0
synonym substitution
1.0000
1.000
+0
distractor clause
1.0000
1.000
+0
meaning-changing control
0.9692
0.774
+22
Appendix
Table 5: Guard stability under goal rewriting. SciWorld: 39 episodes, 715 actions, and 63 guard firings under the original goals. The same compiled guards are replayed after each rewrite.
Gate variant
Pass rate
Coverage
False block ↓
Harmful ↓
Catastrophic ↓
Full gate
0.643
16/23
0.0248
0.188
0.000
− cov
0.829
20/23
0.0198
0.150
0.000
− eff
0.700
19/23
0.0208
0.316
0.000
− ok
0.643
16/23
0.0248
0.188
0.000
− eff & ok
0.814
20/23
0.0231
0.300
0.000
no gate
1.000
23/23
0.0843
0.261
0.087
Appendix
Table 6: Contribution of the replay-gate criteria on SciWorld. The candidate pool is fixed; only the acceptance rule changes.
Figure 7: Replay validation remains selective under corrupted evidence. SciWorld candidate constraints under label noise and trajectory truncation. (a) Gate acceptance rate. (b) Harmful rate among the accepted constraints, evaluated on clean held-out trajectories.
Figure 8: Constraint accumulation preserves compiler composability while overlapping scopes increase behavioral restriction. SciWorld. (a) Action block rate as the number of compiled rules grows. (b) Fraction of episodes in which more than half of the actions are blocked.
Variant
complied within budget
budget exhausted
compliance
SciWorld, synthesized
60
111
35.1%
SciWorld, hand-written
116
234
33.1%
WebShop, synthesized
52
7
88.1%
WebShop, hand-written
64
11
85.3%
Appendix
Table 7: Compliance when the exact rule is supplied at the time of violation with two retries.
w/o
synth ↑
oracle
R↑ (95% CI)
Tokens/ep ↓
SciWorld test60
41.8 ± 3.5
55.3 ± 5.9
54.6 ± 0.9
0.95 [0.26,3.78]
101.3k / 137.6k ( −26% )
WebShop test100
55.8 ± 1.5
60.4 ± 1.5
61.4 ± 2.0
0.78 [−2.83,4.75]
19.4k / 21.0k ( −8% )
Appendix
Table 8: Three-seed comparison with the hand-written reference. Results use seeds 42/43/44 and are paired by episode ID. R=(synth−w/o)/(oracle−w/o) measures recovery of the hand-written headroom.
Stratum
types
per seed
synth − oracle ↑
95% CI (seeds)
Constraint emitted
10
−0.8/−5.8/−9.2
−5.27
[−15.8,+5.2]
Abstained
20
+10.6/+0.8/−0.4
+3.67
[−11.3,+18.7]
Pooled
30
+6.8/−1.4/−3.4
+0.67
[−12.8,+14.1]
Appendix
Table 9: SciWorld performance stratified by whether synthesis emits a verified constraint. Strata are determined from training evidence; intervals are computed over the three evaluation seeds.
Method
ALFWorld
SciWorld
TextCraft
WebShop
Act-only
32.9k
22.2k
31.9k
11.2k
ReAct
40.2k
43.9k
49.8k
32.6k
SC-ReAct ( k=3 )
116.0k
113.7k
106.9k
89.6k
Reflexion ( ≤3 )
58.3k
138.4k
91.5k
97.3k
AdaPlanner
4.8k
4.1k
4.1k
39.8k
AutoGen
34.2k
27.7k
26.7k
41.4k
Appendix
Table 10: Per-episode model-token use for the methods in Table 1 .
Figure 9: Composition of verified specifications. Marker area indicates the number of constraints using each primitive; × indicates that a primitive is absent. Column totals report aggregate use.
Benchmark
Variant
Episodes
Online additions/ep
Median uses
SciWorld
w/o
60
0.78
1
prompt (seeded)
60
0.18
0
WebShop
w/o
100
0.73
4
prompt-evolve
100
0.76
6
prompt-frozen
100
0.00
100
Appendix
Table 11: Online lesson evolution under the evaluation protocols.
Figure 10: Episode-level rule-violation rates. Diagonal hatching denotes rules provided in the prompt, and grid hatching denotes compiled constraints.
Figure 11: Cumulative distribution of the first violation’s relative position within an episode. Curves above the diagonal indicate violations concentrated earlier in the episode.
Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions. Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use. These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect. Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used. Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response. We introduce EnactOpt, which uses limited execution feedback to jointly evolve reusable textual skills and context-construction code while keeping the underlying models and runtime fixed. It diagnoses adjacent decisions by relating action feedback to the actual model input and subsequent action. Structured episode summaries guide joint revisions through prompt-level edit operators, and candidate harnesses undergo compatibility checks and validation-based retention. Across VLABench, ALFWorld, and Embodied Agent Interface (EAI), EnactOpt improves task success by approximately 10-27 percentage points over the initial harnesses and outperforms the evaluated GEPA, TextGrad, and Trace2Skill baselines. These gains extend to task conditions not used for harness evolution or candidate selection. The results support a complementary route to embodied adaptation: turning limited execution experience into reusable behavioral improvements by jointly refining how evidence is presented and acted upon.
ZhiYuan Feng, Zhuoshi Pan, Zhaolu Kang +5
Tsinghua University · Peking University · Kling AI +1
Coding agents can construct executable programs and revise them through feedback, offering a foundation for embodied control without target-task fine-tuning. Through a programmatic robot interface, the agent observes the scene, executes control programs, and adapts them to the resulting physical state, serving as the closed-loop policy. However, this within-episode adaptation does not automatically preserve experience across scenes. Our trajectory analysis shows that agents spend interaction and inference budgets establishing effective operations and identifying their preconditions, while fresh agents cannot reuse these discoveries. This motivates EmbodiedEvo, a self-evolving coding agent that distills physical feedback into reusable code Skills and diagnostic Lessons. Experience is acquired on one Playground scene per task, then frozen for independent agents in unseen scenes, without updating model weights. On RoboDojo-Sim, EmbodiedEvo achieves an average score of 44.48 and a success rate of 37.82%, surpassing all officially verified results. Paired tests show the largest gains on memory-dependent tasks and successful reuse of coordinated procedures, while fine contact control remains difficult. Experience from stronger models also improves weaker executors: Astra's experience raises Sol and Luna's success rates by 8.86 and 5.50 percentage points over no experience. Compared with raw interaction history, Skills and Lessons achieve comparable task performance at 53.0% lower execution cost.
Embodied agents can benefit from skills that guide object search, action execution, and state changes across diverse environments. Since embodied environments vary across layouts, object states, and other execution factors, these skills must self-evolve from trajectories generated during task execution. However, existing skill self-evolution methods are mainly developed in digital environments and often convert trajectories into coarse skill updates. Directly applying this paradigm to embodied settings is problematic, because a failed task execution may reflect not only incorrect skill content, but also an execution lapse in which the agent fails to follow valid guidance. We propose EmbodiSkill, a training-free framework for embodied skill self-evolution through skill-aware reflection and targeted revision. EmbodiSkill interprets each trajectory with respect to the current skill, uses skill-changing evidence to update the skill body, and uses execution-lapse evidence to preserve and emphasize valid guidance. Experiments on ALFWorld and EmbodiedBench show that EmbodiSkill consistently improves embodied task success. On ALFWorld, EmbodiSkill enables a frozen Qwen3.5-27B executor to reach 93.28% task success, outperforming GPT-5.2 used as a direct agent without skills by 31.58%. These results show that skill-aware self-evolution helps embodied agents accumulate reusable procedural knowledge from their own trajectories.
Ruofei Ju, Xinrui Wang, Xin Ding +12
Nanjing University · Institute for AI Industry Research (AIR), Tsinghua University · Huazhong University of Science and Technology +2