cs.AIOct 9, 2026

EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents

Authors: Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, +1 more

Organizations: School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · University of Hong Kong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Continual learning requires models to accumulate past experience and transfer it to new tasks. Despite substantial research on continual learning, existing work remains largely confined to non-embodied settings, without extending to embodied environments. To address this gap, we introduce EmbodiedCLBench, the first benchmark in embodied simulation for evaluating the continual learning capability of self-evolving agents. With compositional generalization as the core evaluation principle, our benchmark evaluates whether agents can first learn from basic tasks and then recombine the acquired skills to solve complex, unseen advanced tasks. Based on EmbodiedCLBench, we conduct extensive experiments across multiple agent harnesses and learning methods, revealing consistent limitations and distinctive behaviors. Our results show that while advanced tasks can be hardly solved under zero-shot setting, the experience from basic tasks enables consistent improvement. Such benefit from the experience remains stable even if the task complexity increases. However, this improvement does not scale accordingly with the number of learning samples, and current agents cannot extract meaningful information from additional experience. Collectively, our findings highlight continual learning as a fundamental yet underexplored capability, and our benchmark offers valuable resources for designing effective self-evolving agents in embodied environments. Our code and dataset are available at https://pku-value-lab.github.io/EmbodiedCLBench-homepage.

Figures & tables

Figure 1: Compositional generalization is leveraged as a principled way to measure continual learning capability. An agent that has experience from basic tasks should recombine previously acquired skills to solve novel, unseen advanced tasks, rather than learning each new task from scratch.
Figure 2: Overview of the construction process of EmbodiedCLBench . During task generation, basic tasks connect a few atomic actions into simple household operations, and advanced tasks combine elements from multiple basic tasks into compositional tasks. For each task, a state-transition graph is built, with each graph node as an intermediate state and paired with action options and a first-person-view image. During evaluation, the agent observes the current state and selects from candidate actions. After each selection, the agent transitions to the next node on the graph.
Figure 3: Benchmark statistics of EmbodiedCLBench : environment configuration (a), task composition across six categories (b), and task complexity at two levels (c).
Figure 4: Effect of compositional complexity on continual learning. The xx -axis varies the number of atomic actions per advanced task from 3 to 6. (a) reports the raw success rate achieved by each setting. (b) reports the gain over zero shot, defined as the success rate of ICL or Skill Learning minus that of zero shot, isolating the improvement attributed to the learning phase.
Figure 5: Success rate versus the number of learning samples. Each learning sample is a basic task that the agent completes before being tested on advanced tasks.
Figure 6: Error analysis of decision behavior. Each group of bars shows the change in selection rate relative to the zero-shot setting for three action types: optimal actions follow the shortest path, suboptimal actions are executable but deviate from the shortest path, and invalid actions cannot be executed in the current state.
Figure 7: Step budget sensitivity. The xx -axis shows the number of extra steps allowed beyond the shortest path. By default, agents are allowed 3 extra steps in the testing phase.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

CategoryCore Question
Object ManipulationCan the agent locate and manipulate everyday objects into a usable or manageable state?
Spatial PlacementCan the agent understand spatial relationships between objects and surfaces or containers?
Accessibility ControlCan the agent change the state of doors, drawers, and cabinet doors to enable or restrict access?
Appliance OperationCan the agent toggle household appliances and environmental devices?
Activity-Driven Scene ConfigurationCan the agent set up multiple objects and devices to support activities such as reading, dining, or leisure?
Household State RestorationCan the agent restore a room to a tidy, energy-efficient, and reusable state?
Appendix
Table 2: Six capability categories of basic tasks in EmbodiedCLBench .
Figure 8: Visual states along the shortest path for a basic-task example.
Task: I’m about to watch a movie in the living room. Turn off the overhead light so the screen isn’t washed out, and open up the media cabinet so everything’s within reach.
StateOptionText shown to the agentHidden role
State 0AHead over to the ceiling light area to get ready to turn it off.Invalid in current state
State 0BSwitch on the overhead light to brighten the living room before the movie.Invalid in current state
State 0CShut off the ceiling light right away to set the mood for the movie.Invalid in current state
State 0DThe overhead light is still on — that’s the first thing to deal with before the movie.Correct shortest-path action
State 1ACheck the ceiling light to see if it’s already been switched off.Invalid in current state
Appendix
Table 3: Basic-task example: action options and hidden graph roles at each state.
Figure 9: Visual states along the shortest path for a basic-task example.
Task: Get the living room ready for a relaxed evening — turn on some background entertainment and open up the storage cabinet so everything’s within reach.
StateOptionText shown to the agentHidden role
State 0AStart by checking the TV area — that’s where the background entertainment will come from.Correct shortest-path action
State 0BShut off the TV screen since it’s not needed for the living room ambiance.Invalid in current state
State 0CHead over to the TV wall to get closer to the entertainment controls.Invalid in current state
State 0DPull open the storage cabinet to make everything accessible for the evening.Invalid in current state
State 1AShut the storage cabinet to keep the living room looking tidy.Invalid in current state
Appendix
Table 4: Basic-task example: action options and hidden graph roles at each state.
Figure 10: Visual states along the shortest path for an advanced-task example, including the final state.
Task: I’m done for the night and heading to bed — dim the bedroom lamp, open up the study desk so it’s ready for tomorrow, shut the bedside drawer, and toss the old remote into the bin.
StateOptionText shown to the agentHidden role
State 0ACheck the trash can area to see what needs to be tossed before bed.Executable detour
State 0BSwitch on the table lamp to brighten the bedside area for cleanup.Invalid in current state
State 0CThe lamp’s still glowing — that’s the first thing to sort out before the room can rest.Correct shortest-path action
State 0DHead over to the table lamp to adjust the lighting for bedtime.Invalid in current state
State 1ASwitch off the table lamp to dim the room for bedtime.Invalid in current state
Appendix
Table 5: Advanced-task example: action options and hidden graph roles.

Explore similar work

CardsList

  1. ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?

    Oct 6, 2026Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2Continual Learning for LLM AgentsLLM Agent Evaluation

    Carnegie Mellon University · Amazon

  2. Learning New Tasks via Reusable Skills: Skill-Compositional Experts for Embodied Continual Learning

    Jun 14, 2026Shuaike Zhang, Shaokun Wang, Haoyu Tang +2Continual Robot LearningRobot Skill Learning

    Harbin Institute of Technology, Shenzhen · Shandong University · Shenzhen Loop Area Institute

  3. Self-Evolving Embodied Agents via Skill-Harness Evolution

    Aug 11, 2026Peidong Wang, Zhiming Ma, Ying Chang +7Test-Time AdaptationAgent Skill Learning

    Northeastern University · Microsoft Research