MetaAct: Self-Evolving Executable Action Constraints for Embodied Language Agents
Authors: Jiaxing Zhao, Hanwen Wang, Di Liu, Shaoteng Liu, Jing-Ran Su, Yanheng Li, Qian Zha, Jiaxi Li, Hongbin Xie, Haoran Zhang, Xuan Song, Haoxuan Che
Organizations: Jilin University · Southern University of Science and Technology · Northwestern Polytechnical University · The Hong Kong Polytechnic University · City University of Hong Kong · Peking University · XGEN Labs
Self-evolving language agents have emerged as a promising paradigm for continuously improving agent behavior through experience. However, most existing approaches evolve textual artifacts, which can influence action preferences but cannot reliably enforce action admissibility in embodied environments with discrete and often irreversible actions. We propose MetaAct, which shifts self-evolution from textual guidance to executable action constraints by synthesizing GEAR (Grounded Executable Action Rules) from execution feedback. Each rule is represented as a typed specification, compiled into a deterministic guard, and validated offline through trajectory replay without additional model calls or environment interactions. Across four benchmarks, MetaAct improves task performance; on ScienceWorld and WebShop, violations drop from 70% to 0% and 40% to 1%, with score gains of +10.1 and +8.0. Our code is available at https://github.com/BaiCubed/MetaAct.
Figures & tables
Figure 1: From textual lessons to executable constraints. (1) In ScienceWorld, a single irreversible failure reveals an ordering rule: the thermometer must be focused before the target object. (2) Textual self-evolution distills the failure into a correct lesson, but the violating action remains in the candidate set and is still selectable. (3) MetaAct compiles the same rule into a GEAR guard that removes the violating action from the admissible set at execution time: text changes preference, whereas the constraint changes admissibility.
Figure 2: Overview of MetaAct. (1) Failed and successful trajectories provide contrastive execution evidence. (2) The LLM synthesizes typed GEAR specifications, which are compiled into deterministic guards and verified offline by trajectory replay. (3) Accepted guards constrain action execution at deployment time and can be continually expanded from new interaction feedback.
ALFWorld (134)
SciWorld (60)
TextCraft (60)
WebShop (100)
SR ↑
SR ↑
score ↑
SR ↑
score ↑
SR ↑
score ↑
Single-agent methods
Act-only
0.799
0.167
44.3
0.617
80.6
0.180
57.6
ReAct
0.851
0.050
27.5
0.667
72.3
0.120
21.7
SC-ReAct ( k=3 )
0.851
0.150
45.7
0.850
91.2
0.090
18.0
Reflexion ( ≤3 )
0.955
0.217
53.5
0.750
82.7
0.210
36.2
Table 1: Main benchmark comparison. Performance of published baselines and MetaAct on four embodied-language benchmarks using the DeepSeek-V4-flash backbone. MetaAct-synth denotes automatically synthesized executable constraints and MetaAct-oracle the hand-written reference. Best values are bold; the second-best distinct value in each metric column is underlined.
Figure 3: Matched-evidence comparison. Each row shows one variant. Left: task-score change relative to the unconstrained baseline, with 95% confidence intervals. Right: episode-level win/loss/tie counts against the same baseline. Filled markers indicate confidence intervals that do not include zero.
Figure 4: Effect of action masking and rule messages at execution time. Results on SciWorld with GPT-4.1-mini over 60 episodes per setting. Left: task score under different enforcement strategies ( ∗ : the 95% confidence interval for the score difference is entirely above zero). Right: fraction of episodes with at least one rule violation. Action masking removes violations, while combining masking with rule messages gives the largest score improvement.
Figure 5: Offline replay predicts harmful constraint behavior before deployment. SciWorld candidate specifications. (a) Held-out harmful blocking as a function of the offline gate score. (b) ROC curves for the gate criteria. (c) Harmful and catastrophic blocking rates for accepted and rejected candidates on held-out replay.
Offline
Online
Variant
Coverage ↑
Fidelity F1↑
Harmful ↓
Violation ↓
Score ↑
A0
Full MetaAct
0.211
0.135
0.000
0.167
50.77
A1
− Contrastive Evidence
0.189
0.411
0.000
0.229
47.62
A2
− Counterexample Repair
0.211
0.164
0.000
0.229
50.66
A3
− Constraint Union
0.189
0.164
0.000
0.292
47.37
A4
− Replay Verification
0.433
0.135
0.069
0.229
48.36
Table 2: Ablation study on SciWorld. Offline metrics are computed by replay; online metrics are averaged over three evaluation seeds with 60 episodes per seed.
Figure 6: Executable constraints can be acquired and deployed during an interaction stream. WebShop with GPT-4.1-mini over 100 consecutive episodes. The panels show 20-episode rolling task score, rolling violation rate, and the cumulative number of accepted constraints.
Backbone
No constr.
Prompt rule
MetaAct
Oracle
Δ vs. prompt [95% CI]
DeepSeek-V4-flash
44.45
44.30
54.60
55.30
+10.30[+1.15,+19.83]∗
GPT-4.1-mini
45.72
40.60
54.35
50.83
+13.75[+6.43,+21.92]∗
Qwen3-8B
34.77
30.90
36.73
32.97
+5.83[−0.77,+12.93]
Table 3: Executable-constraint gains persist across language-model backbones. SciWorld task score (60 paired episodes per backbone) with no added constraint, the textual prompt rule, MetaAct, and the hand-written reference.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
SciWorld test60
SR
score
steps
tokens/ep
ReAct
0.050
27.5
24.5
43.9k
ReflAct
0.050
31.9
17.9
48.7k
AutoGen (strongest baseline)
0.267
54.6
15.0
27.7k
constraint variant (ours)
0.300
55.3
33.2
139.9k
Appendix
Table 4: ReflAct compared with ReAct under matched settings.
Goal perturbation
Consistency ↑
Jaccard ↑
Δ blocks
paraphrase
1.0000
1.000
+0
clause reordering
1.0000
1.000
+0
synonym substitution
1.0000
1.000
+0
distractor clause
1.0000
1.000
+0
meaning-changing control
0.9692
0.774
+22
Appendix
Table 5: Guard stability under goal rewriting. SciWorld: 39 episodes, 715 actions, and 63 guard firings under the original goals. The same compiled guards are replayed after each rewrite.
Gate variant
Pass rate
Coverage
False block ↓
Harmful ↓
Catastrophic ↓
Full gate
0.643
16/23
0.0248
0.188
0.000
− cov
0.829
20/23
0.0198
0.150
0.000
− eff
0.700
19/23
0.0208
0.316
0.000
− ok
0.643
16/23
0.0248
0.188
0.000
− eff & ok
0.814
20/23
0.0231
0.300
0.000
no gate
1.000
23/23
0.0843
0.261
0.087
Appendix
Table 6: Contribution of the replay-gate criteria on SciWorld. The candidate pool is fixed; only the acceptance rule changes.
Figure 7: Replay validation remains selective under corrupted evidence. SciWorld candidate constraints under label noise and trajectory truncation. (a) Gate acceptance rate. (b) Harmful rate among the accepted constraints, evaluated on clean held-out trajectories.
Figure 8: Constraint accumulation preserves compiler composability while overlapping scopes increase behavioral restriction. SciWorld. (a) Action block rate as the number of compiled rules grows. (b) Fraction of episodes in which more than half of the actions are blocked.
Variant
complied within budget
budget exhausted
compliance
SciWorld, synthesized
60
111
35.1%
SciWorld, hand-written
116
234
33.1%
WebShop, synthesized
52
7
88.1%
WebShop, hand-written
64
11
85.3%
Appendix
Table 7: Compliance when the exact rule is supplied at the time of violation with two retries.
w/o
synth ↑
oracle
R↑ (95% CI)
Tokens/ep ↓
SciWorld test60
41.8 ± 3.5
55.3 ± 5.9
54.6 ± 0.9
0.95 [0.26,3.78]
101.3k / 137.6k ( −26% )
WebShop test100
55.8 ± 1.5
60.4 ± 1.5
61.4 ± 2.0
0.78 [−2.83,4.75]
19.4k / 21.0k ( −8% )
Appendix
Table 8: Three-seed comparison with the hand-written reference. Results use seeds 42/43/44 and are paired by episode ID. R=(synth−w/o)/(oracle−w/o) measures recovery of the hand-written headroom.
Stratum
types
per seed
synth − oracle ↑
95% CI (seeds)
Constraint emitted
10
−0.8/−5.8/−9.2
−5.27
[−15.8,+5.2]
Abstained
20
+10.6/+0.8/−0.4
+3.67
[−11.3,+18.7]
Pooled
30
+6.8/−1.4/−3.4
+0.67
[−12.8,+14.1]
Appendix
Table 9: SciWorld performance stratified by whether synthesis emits a verified constraint. Strata are determined from training evidence; intervals are computed over the three evaluation seeds.
Method
ALFWorld
SciWorld
TextCraft
WebShop
Act-only
32.9k
22.2k
31.9k
11.2k
ReAct
40.2k
43.9k
49.8k
32.6k
SC-ReAct ( k=3 )
116.0k
113.7k
106.9k
89.6k
Reflexion ( ≤3 )
58.3k
138.4k
91.5k
97.3k
AdaPlanner
4.8k
4.1k
4.1k
39.8k
AutoGen
34.2k
27.7k
26.7k
41.4k
Appendix
Table 10: Per-episode model-token use for the methods in Table 1 .
Figure 9: Composition of verified specifications. Marker area indicates the number of constraints using each primitive; × indicates that a primitive is absent. Column totals report aggregate use.
Benchmark
Variant
Episodes
Online additions/ep
Median uses
SciWorld
w/o
60
0.78
1
prompt (seeded)
60
0.18
0
WebShop
w/o
100
0.73
4
prompt-evolve
100
0.76
6
prompt-frozen
100
0.00
100
Appendix
Table 11: Online lesson evolution under the evaluation protocols.
Figure 10: Episode-level rule-violation rates. Diagonal hatching denotes rules provided in the prompt, and grid hatching denotes compiled constraints.
Figure 11: Cumulative distribution of the first violation’s relative position within an episode. Curves above the diagonal indicate violations concentrated earlier in the episode.