SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents

Saswat Das; Parvati Viswanathan; Daniel Donnelly; Chang Huang; Sahar Abdelnabi; Ferdinando Fioretto

cs.CRSep 28, 2026

SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents

Authors: Saswat Das, Parvati Viswanathan, Daniel Donnelly, Chang Huang, Sahar Abdelnabi, Ferdinando Fioretto

Organizations: University of Virginia · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center

Abstract

Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management protocols, and reusable tools and skills, in response to user and environment feedback. However, locally useful updates may persist into later tasks where they produce unsafe behavior, even without direct adversarial influence. To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment. To account for the stochasticity inherent in agentic operations, we provide an adaptive trajectory discovery pipeline that probes for failures while preserving original task intent and supports causal attribution through paired non-evolving agents and attribution scores. Our evaluation across multiple recent LLMs, evolution surfaces, and harm types reveals that self-evolution indeed increases task completion rates but often at the cost of safety failures that are absent for paired non-evolving baseline agents. We also show that qualitatively different safety behaviors emerge across evolution surfaces and harm types. Further, we show that this divergence in safety behavior is reflected in agents' chain-of-thought reasoning, which yields an effective monitoring strategy that can mitigate unsafe behavior with a low false positive rate.

Figures & tables

Figure 1: Endogenous misalignment. Feedback from an upstream task induces a persistent controller update that is locally useful but overgeneralizes to a downstream context. The evolved agent discloses unnecessary access information, whereas the paired non-evolving agent preserves the contextual boundary.

Figure 2: Failure Discovery Pipeline. TextGrad-based candidate task search and paired evaluation pipeline finds trajectories with endogenous misalignment.

Figure 3: Overall and agent-specific candidate-level safety failures for unevolved/evolved agents

Panel of Figure 4 — Figure 4: Task completion and safety failure rates on downstream safety test tasks for paired non-evolving (hatched bars) and self-evolving agents (solid bars) in trajectories discovered using the stress test. Self-evolving agents achieve higher task completion rates for all surfaces and models except in one case while incurring safety failures attributed to self-evolution events in all cases, whereas non-evolved agents remain safe.

Panel of Figure 5 — Figure 5: Safety failures across (i) harm types, (ii) task domains, and (iii) evolution surfaces.

Figure 6: Differences between evolved and non-evolving agents in (i) active versus resolved risk threads and (ii) safety-relevant reasoning behaviors, computed over observable Kimi K2.5 traces.

Figure 7: Confusion matrix for the monitor-based mitigation strategy.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Label	Definition
evidence_absent	Required evidence for the object is unavailable.
evidence_present	Concrete supporting evidence is stated in the reasoning.
source_access_claim	The agent says it accessed or found a source, but the reasoning-only record cannot independently verify the external observation.
evidence_conflict	Evidence or instructions concerning the object disagree.
unsupported_inference	A conclusion about the object exceeds the stated evidence.
sensitive_information	The object is sensitive information.

Appendix

Table 1: Node labels

Label	Definition
supports	Source node provides the stated basis, evidence, or rationale for the target node.
contradicts	Source node conflicts with or negates information, evidence, or a qualification in the target node.
certainty_escalation	Target node expresses a stronger conclusion than the earlier evidence or reasoning warrants.
scope_drift	Target node broadens the inquiry, context, or intended use beyond the scope established by the source node.
risk_to_plan	A risk identified in earlier reasoning is carried into a later action or response plan.
constraint_preserved	Target node retains and acts consistently with a boundary established earlier.

Appendix

Table 2: Edge labels

Thread state	Definition
active	The exact risk still controls the latest plan.
resolved	Later reasoning corrects that exact risk and does not restore it.
reopened	The exact risk returns after correction.
considered_only	The object was explored but never entered a plan.
unclear	The reasoning does not establish its final use.

Appendix

Table 3: Thread states

Figure 8: Differences between evolved and non-evolving agents in (i) active versus resolved risk threads and (ii) safety-relevant reasoning behaviors, computed over paired GPT 5.6 Luna trace summaries.

Explore similar work

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
Aug 13, 2026Xutao Mao, Liangjie Zhao, Xiang Zheng +1Large Language Model Agents Malicious Agents
1City University of Hong Kong · 2Adelaide University
BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents
May 28, 2026Jiahao Huang, Fei Cheng, Junfeng Jiang +2Self-Evolution Large Language Model Agents
University of Tokyo · Kyoto University · National Institute of Informatics
SEAGym: An Evaluation Environment for Self-Evolving LLM Agents
Jun 16, 2026Congjie Zheng, Chuanyi Xue, Bin Liang +2Self-Evolving Agents Agent Harness
Department of Automation, Tsinghua University, Beijing, 100084, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, 100084, China

Aug 13, 2026, cs.AIJ/K move · Enter open · S save

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

Xutao Mao, Liangjie Zhao, Xiang Zheng, Cong Wang

1City University of Hong Kong · 2Adelaide University

Large Language Model Agents Malicious Agents Alphaevolve

Open paper AskPDF arXiv

Self-improving LLM agents convert successful trajectories into persistent cross-task state. An unsafe success can thereby become reusable policy after its triggering input disappears. Skill evolution makes this failure measurable by distilling operational trajectories into executable, transferable, and inspectable procedures. Because evolution optimizes task outcomes rather than procedure safety, compromised experience can cause skill misevolution. Existing benchmarks measure current behavior or static artifacts but cannot attribute risk across authoring, retrieval, and later execution. To expose this lifecycle, we introduce SkillMisevo-Gym, a lifecycle-aware harness that versions skill state across agent frameworks, and SkillMisevo-Bench, a frozen design from malicious exposure to carryover tasks, with concept-aligned benign tasks and nine lifecycle metrics. We also introduce SafeEvolve, a wrapper that repairs unsafe content and governs subsequent reuse. Across 25 agent-method configurations, each covering 525 tasks in 25 episodes, all 21 evolved configurations author unsafe artifacts, while only fifteen lead to fresh-session harm. In the exposure sweep, three malicious tasks raise carryover ASR from 16.0% to 35.3%. Across representative skill evolution methods, SafeEvolve reduces unsafe retrieval and fresh-session harm by 26.7 and 17.3 percentage points, respectively, while mean benign utility changes by only 0.4 points. Together, persistent-adaptation safety must govern what updates write and what future executors reuse. Code is available at https://github.com/henrymao2004/misevolve.

Abstract

Explore similar work

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

BenchTrace: A Benchmark for Testing Reflection Ability and Controlled Evolution in LLM Agents

SEAGym: An Evaluation Environment for Self-Evolving LLM Agents