Self-Evolving AI for Humanoids: Mechanisms, Safety, and Evaluation of Post-Deployment Self-Improvement

Loc X. Nguyen; Avi Deb Raha; Huy Q. Le; Eui-Nam Huh; Dusit Niyato; Choong Seon Hong

cs.ROSep 2, 2026

Self-Evolving AI for Humanoids: Mechanisms, Safety, and Evaluation of Post-Deployment Self-Improvement

Authors: Loc X. Nguyen, Avi Deb Raha, Huy Q. Le, Eui-Nam Huh, Dusit Niyato, Choong Seon Hong

Organizations: School of Computing, Kyung Hee University, Yongin-si, 17104, Republic of Korea · G-LAMP NEXUS Institute, Kyung Hee University, Yongin-si, 17104, Republic of Korea · College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract

Humanoid robots are becoming an important part of embodied artificial intelligence, driven by advances in reinforcement learning for locomotion, world models for prediction, and vision-language-action models for general control. However, most of these systems remain static after deployment. A policy is trained offline for a fixed objective and then frozen, even though the tasks, environments, and robot bodies keep drifting over time. An emerging paradigm of self-evolving agents aims to address this problem by allowing systems to improve from their own post-deployment experience. Since most existing studies focus on disembodied software agents, this survey examines how self-evolution changes when an agent has a physical body. We first define self-evolution for humanoids and represent a deployed robot using a state tuple that includes its policy, perception, memory, workflow, and body. This state is updated by an evolution operator in a slow outer loop with a lifelong objective. We then organize the literature into four complementary mechanisms of self-evolution, presented in increasing order of autonomy: self-learning, self-adaptation, self-optimization, and self-generation. Since changes to a humanoid can introduce physical hazards, we treat safety and uncertainty as key design dimensions of the evolution operator, and further formulate admissible evolution as a constraint enforced by a world-model verification gate within a human-oversight envelope. Finally, we present that evaluation should track the robot's evolving trajectory rather than a fixed checkpoint, and we identify the lack of a benchmark designed specifically for self-evolving humanoids. Moreover, we outline open challenges spanning AI algorithms, on-board systems, and governance.

Figures & tables

Figure 1: From frozen deployment to self-evolution. (a) The conventional pipeline trains a policy offline, freezes it, and deploys it for a long period (months/years) while the world, tasks, and robot body change. As these changes occur, the system’s utility can gradually degrade. (b) A self-evolving humanoid uses its deployment experience to update itself through an evolution process. Examples include TTT-Parkour [ 173 ] , DreMa [ 6 ] , LOTUS [ 137 ] , SkillsCrafter [ 142 ] , Egocentric model [ 49 ] , and SplitAdapter [ 62 ] .

Driver	Why self-evolution is needed
Changing World	Dynamic environments, new objects, evolving tasks, and shifting user needs create continual distribution shift.
Changing Robot	Hardware degradation, sensor drift, wear, maintenance, and morphology changes require ongoing adaptation.
Changing Knowledge	Foundation models, semantic understanding, and planning capabilities must remain current as the world evolves.
Changing Intelligence	Long-term autonomy demands continual skill acquisition, reflection on experience, self-correction, and open-ended improvement rather than fixed capabilities.

Table 1: Four drivers that keep changing after deployment and create the distribution shift.

Survey classification	Embodied	Post-deployment self-improvement	Body/hardware evolution	Real-time safety constraint	Lifelong / trajectory evaluation
Humanoid system reviews [ 128 , 15 ]	✓	✗	✗	$\circ$	✗
Humanoid perception [ 8 ]	✓	✗	✗	✗	✗
Motion planning [ 24 ]	✓	✗	✗	$\circ$	✗
VLA models [ 99 ]	✓	✗	✗	✗	✗
Self-evolving agents [ 34 , 31 ]	✗	✓	✗	$\circ$	✓
Agent evaluation [ 157 ]	✗	$\circ$	✗	✗	✓

Table 2: Positioning against related surveys. ✓ = treated as a central concern;

\circ

= partial or incidental; ✗ = not addressed.

Module (state component)	Representative model approach	Primary task	Improves from
Perception $\Phi$	ViT/CNN encoders; self-supervised representations [ 43 , 16 ]	State estimation and semantic perception	Unlabeled sensor streams (self-supervised learning)
Policy/control $\pi$	RL policies; VLA action heads, autoregressive [ 66 ] and flow-based [ 10 ]	Locomotion, whole-body control, and manipulation	Rewards; demonstrations
Memory $\mathcal{M}$	Skill libraries, replay buffers, and learned world models [ 137 , 41 ]	Skill reuse; future-state prediction	Replay; consolidation
Workflow $\mathcal{W}$	Classical motion planning/TAMP [ 24 ] ; VLM/LLM planners [ 53 ]	Task decomposition; long-horizon planning	Reflection; execution feedback
Body $\mathcal{B}$	Calibration; kinematic/dynamic self-models [ 49 , 50 ]	Tracking morphology; dynamics	Proprioception; self-modeling

Table 3: Coverage map: each humanoid module realizes a component of the state tuple

s_{\tau}=(\pi_{\tau},\Phi_{\tau},\mathcal{M}_{\tau},\mathcal{W}_{\tau},\mathcal{B}_{\tau})

, runs a different approach of learning machinery, and improves from a different signal.

Figure 2: The deployed humanoid is the state tuple $s_{\tau}=(\pi,\Phi,\mathcal{M},\mathcal{W},\mathcal{B})$ . On the fast inner loop (milliseconds to seconds) the tuple is fixed and the robot acts; on the slow outer loop (hours to months) accumulated experience $e_{\tau}$ drives the evolution operator $\mathcal{E}$ , whose candidate $s_{\tau+1}$ must pass the admissible region $\mathcal{C}(s_{\tau})$ before being committed, optimizing the lifelong objective $V$ under the plasticity-stability trade-off.

Work	Sub-op.	Trig.	Platform / Timing	Work	Sub-op.	Trig.	Platform / Timing
Self-learning (§ 3 )				Self-optimization (§ 5 )
LEGION [ 106 ]	$\mathcal{E}_{\mathcal{M}},\mathcal{E}_{\pi}$	P	Manipulator, Epis.	H2O / OmniH2O / ExBody2 [ 46 , 45 , 57 ]	$\mathcal{E}_{\pi}$	P	Humanoid, Off.
LOTUS [ 137 ]	$\mathcal{E}_{\mathcal{M}}$	P	Manipulator, Epis.	BFM-Zero [ 78 ]	$\mathcal{E}_{\pi}$	A	Humanoid, Off.
LifeLong-RFT [ 88 ]	$\mathcal{E}_{\pi}$	P	VLA manip., Epis.	DreamVLA [ 167 ]	$\mathcal{E}_{\mathcal{M}},\mathcal{E}_{\pi}$	P	VLA manip., Off.
DayDreamer [ 149 ]	$\mathcal{E}_{\pi},\mathcal{E}_{\mathcal{M}}$	A	Quadruped/arms, Online	WMPO [ 171 ]	$\mathcal{E}_{\pi}$	A	VLA manip., Epis.
ResFiT [ 4 ]	$\mathcal{E}_{\pi}$	P	Humanoid, Online	REFLECT [ 90 ]	$\mathcal{E}_{\mathcal{W}}$	A	Manipulator, Epis.

Table 4: Classification of representative methods by sub-operator(s), trigger (A: active, i.e., self-initiated; P: passive, i.e., externally supplied), demonstrated platform, and update timing.

Figure 3: Three architectural address to catastrophic forgetting, distinguished by where accumulated knowledge lives: (a) A generative non-parametric memory [ 106 ] , (b) An explicit, ever-growing skill library with a routing meta-controller [ 137 ] , (c) A single foundation policy with a revised update rule, in which chunk-level reinforcement fine-tuning with multidimensional process rewards replaces forgetting-prone supervised fine-tuning [ 88 ] .

Figure 4: Three loci of online improvement ordered by decreasing physical risk and increasing reliance on learned models: directly on hardware with a constrained update ResFiT [ 4 ] , on hardware inside a safety envelope Robot-Trains-Robot [ 47 ] , and inside a just-in-time reconstruction of the current world TTT-Parkour [ 173 ] . DayDreamer’s world model provides the key mechanism by turning each real interaction into many imagined ones.

Figure 5: Conceptual framework for constrained humanoid self-adaptation across domain, morphological, and personalization shifts. Domain adaptation combines pre-deployment distribution absorption [ 70 ] with deployment-time mismatch detection and online adjustment [ 133 , 111 ] . Morphological adaptation similarly couples embodiment generalization and self-modeling [ 49 , 132 ] with constrained post-deployment corrections [ 62 , 145 ] , while personalization learns or exposes user preferences as conditioning variables or explicit interaction constraints [ 13 , 71 ] . These complementary mechanisms are synthesized into a unified protect-and-adapt lifecycle in which a validated base policy is preserved, only admitted bounded updates are executed, and deployment outcomes feed subsequent monitoring, adaptation, or retraining.

Figure 6: World-model-centered architecture for safe self-optimization in humanoid robots. RL, planning, and self-reflection generate candidate policy, action, and corrective updates, while the world model predicts their consequences before real-world execution [ 149 , 42 ] . Predictive world modeling couples to the self-optimization stack through three complementary pathways: prediction-to-perception [ 167 ] , prediction-to-policy optimization [ 171 ] , and prediction-to-planning [ 85 ] , enabling anticipatory control, imagination-based learning, and consequence-aware planning.

Figure 7: Self-reflection-driven self-optimization pipeline for humanoid robots. Failures are detected or anticipated during execution through progress- and sentinel-based monitoring [ 97 , 77 ] , and then explained from multimodal execution traces [ 90 ] . The resulting explanation is translated into motor-level corrections [ 150 ] or learning signals such as dense rewards [ 72 ] , allowing the corrected experience to improve subsequent behavior through deployment-time learning [ 5 , 84 ] . The world-model verifier evaluates its consistency, feasibility, and safety.

Figure 8: From the data flywheel to the self-generation autonomy ladder. Moving from left to right requires progressively less human effort: HOMIE’s cockpit [ 7 ] , DART’s cloud AR [ 117 ] , DexFlywheel automated per-cycle data generation [ 172 ] , and finally the GR00T N1 foundation models [ 9 ] .

Figure 9: A candidate update $\Delta s$ proposed by the evolution operator must pass five gates in sequence: a trust region bounding the step, the world-model verification gate [ 56 ] , a monotonicity check on protected safety-critical skills, an alignment check against operator intent, and only then commit. The whole track runs inside the human envelope, with protective falling SafeFall [ 108 ] as the physical floor beneath it.

Resource	Type	Sim / real	Targets lifelong/drift
HumanoidBench [ 126 ]	Whole-body benchmark	Sim	No
LIBERO [ 83 ]	Manipulation benchmark	Sim	Lifelong task suites
MimickingBench [ 89 ]	Scene-interaction benchmark	Sim	No
Humanoid Everyday [ 170 ]	Manipulation dataset	Real	No
Genie Sim [ 158 ]	Simulation platform	Sim	No
Isaac Gym / Isaac Lab [ 104 , 110 ]	Simulator	Sim	No

Table 5: Representative benchmarks, simulators, and datasets for humanoid learning.

Self-Evolving Agents

Self-evolving agents improve their skills, memory, tools, policies, or environments by learning from interaction and evaluation feedback.

See all 416 papers

Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems
Jun 4, 2026Dianxing Shi, Bowen Wang, Junqi He +2Human-in-the-Loop Evaluation LLM Safety Alignment
The University of Osaka · Independent researcher
Safety in Self-Evolving Agents: A Survey
Sep 8, 2026Jiahao Chen, Zhou Feng, Oubo Ma +28AI Agent Safety LLM Agent Safety
Zhejiang University
Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation
Sep 25, 2026Chang Gong, Jingping Bi, Di Yao +3Recursive Self-Improvement AI Safety Evaluation
Institute of Computing Technology, Chinese Academy of Sciences, China

Jun 4, 2026J/K move · Enter open · S save

Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems

Dianxing Shi, Bowen Wang, Junqi He, Junhao Chen+1

The University of Osaka · Independent researcher

Human-in-the-Loop Evaluation LLM Safety Alignment LLM Agent Safety Self-Evolving Agents

Open paper AskPDF arXiv

Self-evolving agents improve through continual self-play and self-generated learning signals, but autonomous evolution can also cause capability degradation and safety drift. Although human feedback has proven effective for static and post-trained agents, its role in self-evolving systems remains underexplored. We introduce Agent Norm Correction through Human-like Oversight and Review (ANCHOR), an LLM-based framework that simulates human supervision and delivers feedback at various phases of self-evolution. With ANCHOR, we evaluate two representative open-source self-evolving agent systems across coding, mathematical reasoning, and safety. Our results show that even limited supervision substantially mitigates safety degradation while preserving stable performance on core evolutionary objectives. Further analysis shows that supervision over the output verification phase is the most effective for intervention, whereas increasing supervision frequency yields diminishing returns. These findings provide empirical evidence and practical guidance for designing more stable, controllable, and human-aligned self-evolving agent systems.

Abstract

Self-Evolving Agents

Explore similar work

Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems

Safety in Self-Evolving Agents: A Survey

Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation