Organizations: The Hong Kong Polytechnic University · ACE Robotics · Nanyang Technological University · Tsinghua University · Zhejiang University · Fudan University
In this work, we aim to develop an embodied MLLM agent that understands and solves embodied tasks by learning, abstracting, and reusing transferrable and generalizable behavioral patterns. To this end, we propose StateSkill, a new skill representation for embodied agents, which introduces the concept of "finite-state machines" into skill modeling for effective and efficient skill construction and learning. The core idea of StateSkill is to model embodied interaction experiences as a finite-state machine, where a skill is organized around a set of task-progress states, state transitions, and state-conditioned guidance, enabling embodied agents to dynamically adjust their skills as the task progresses while keeping skill contexts compact and stage-relevant. We further develop StateSkill-Evolve, an experience-driven evolution mechanism that iteratively refines StateSkill through retrospective analysis of new successful and failed state-guided trajectories. Upon this, we introduce StateGRPO, which integrates StateSkill evolution and model optimization via state-conditioned GRPO into a unified iterative learning loop, enabling state-conditioned skills and model capabilities to co-evolve and reinforce each other. Extensive experiments demonstrate the effectiveness of our proposed methods and trained model (i.e., StateSkill-Agent) across various embodied tasks.
Figures & tables
Figure 1: Motivation. (a) Imitating from task-specific demonstrations offers limited transferability and generalization. (b) Textual LLM skills face two key challenges for embodied agents: Limited Skill Representation in representing complex behavioral experiences in interactive physical environments; and Limited Skill Efficiency due to loading entire skill descriptions that introduce redundant and irrelevant context. (c) StateSkill organizes shared behavioral patterns into finite-state machines as skills and dynamically provides state-conditioned guidance as tasks progress, while new interaction experiences enable StateSkill and the model to co-evolve and mutually reinforce each other.
Figure 2: Overview. (a) StateSkill guides embodied execution through explicit state transitions and progressively loaded state-conditioned guidance, adapting to task progress and environment states. (b) StateSkill abstracts shared behavioral patterns from embodied trajectories into a finite-state machine, equipping each task-progress state with language guidance, a transition gate, and trajectory-derived action priors. (c) StateSkill-Evolve iteratively refines StateSkill by reviewing new successful and failed state-guided trajectories and validating candidate revisions. (d) StateGRPO integrates StateSkill evolution and state-conditioned embodied agent optimization into a unified iterative loop, enabling StateSkill and model capabilities to co-evolve.
Method
Spat-T
Spat-S
Obj-T
Obj-S
Goal-T
Goal-S
L10-T
L10-S
Overall
Vision-Language-Action Models
OpenVLA ( Kim et al., 2024 )
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
π0 ( Black et al., 2025b )
0.0
0.0
0.0
2.0
0.0
0.0
0.0
0.0
0.3
π0.5 ( Black et al., 2025a )
1.0
20.0
1.0
17.0
2.0
38.0
1.0
8.0
11.0
MolmoAct ( Lee et al., 2025 )
0.0
0.0
0.0
6.0
0.0
0.0
6.0
0.0
1.5
NORA ( Hung et al., 2025 )
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Table 1: Results on LIBERO-PRO. Success rates (%) under task (T) and swap (S) perturbations. Gains (pp) are relative to the corresponding baseline; “–” means unavailable.
Method
Atomic-Seen
Composite-Seen
Composite-Unseen
Overall
PRTS ( Zhang et al., 2026c )
66.3
30.3
18.8
39.6
RLDX-1 ( Kim et al., 2026 )
67.6
27.9
8.5
36.0
WorldDreamer ( Wang et al., 2024b )
66.3
26.7
9.0
35.3
GR00T N1.5 ( Bjorck et al., 2025 )
50.7
14.8
2.7
23.9
π0.5 ( Black et al., 2025a )
39.6
7.1
1.2
16.9
π0 ( Black et al., 2025b )
34.6
6.1
1.1
14.8
Table 2: Results on RoboCasa365 (Target50). Success rates (%) on atomic and composite tasks. Gains (pp) are relative to the corresponding baseline.
Method
Block Stacking
Flower Insertion
Object Sorting
Pen Placement
Cup Nesting
Overall
GPT-5.5 (Baseline)
80.0
55.0
100.0
55.0
85.0
75.0
GPT-5.5 + StateSkill
90.0
70.0
100.0
75.0
95.0
86.0 ↑11.0
Qwen3.5-9B (Baseline)
70.0
30.0
70.0
60.0
70.0
60.0
Qwen3.5-9B + StateSkill
80.0
50.0
95.0
75.0
80.0
76.0 ↑16.0
Table 3: Real-world manipulation results. Success rates (%) on five real-world tasks. GPT-5.5 and Qwen3.5-9B are used as baselines.
StateSkill
StateSkill-Evolve
StateGRPO
Overall
–
–
–
15.3
✓
–
–
63.5
✓
✓
–
68.4
✓
✓
✓
70.9
Table 4: Ablation studies on LIBERO-PRO. Success rates (%) with Qwen3.5-9B. Rows progressively add the indicated components.
Metric
Harness VLA
XSkill
SkillGen
Trace2Skill
StateSkill-Agent
StateSkill-Agent w/ FT
Success Rate (%) ↑
25.8
23.6
28.9
31.5
68.4
70.9 ↑45.1
Skill Length (K) ↓
169.4
186.7
104.3
231.5
11.0
11.0 ↓93.5%
Interaction Turns ↓
74.96
76.80
69.50
66.10
23.10
21.91 ↓70.8%
Token Usage (M) ↓
3.62
3.74
3.05
3.22
0.47
0.44 ↓87.8%
Table 5: Comparison with LLM skills on LIBERO-PRO. All methods use Qwen3.5-9B as the baseline. Superscripts in the shaded column indicate improvements over Harness VLA: percentage-point gains in success rate and relative reductions (%) in other metrics.
Figure 3: Qualitative results of StateSkill. Without StateSkill, the agent incorrectly assumes a successful grasp and continues with empty transport and release, causing repeated errors and eventual failure in a pick-and-place task. With StateSkill, the missed grasp is detected, enabling the agent to regrasp, verify the updated task state, and successfully complete the task.