Evolutionary Computation for Trustworthy AI: From Attacks and Defenses to Self-Evolving Era
Organizations: Nanyang Technological University, Singapore · Google · Sichuan University, China · Lingnan University, Hong Kong, China ·
Abstract
As Artificial Intelligence (AI) has evolved from task-specific models to foundation models and agents, the scope of trustworthy AI has expanded from model-level robustness to the reliability and safety of broader AI systems. This evolution has also expanded the attack surface from individual models to broader system-level interactions, including tool use, context, and interaction trajectories with dynamic environments. As a result, maintaining reliable and safe behavior under changing or deliberately manipulated conditions has become increasingly challenging. The search for effective attacks and defenses often relies on black-box feedback to navigate discrete choices among words, actions, system components, or their combinations. Multiple objectives and expensive candidate evaluations further limit what can be explored. Evolutionary Computation (EC), with its population-based, gradient-free search and flexible variation and selection mechanisms, is well suited to these settings. This survey reviews how EC has been applied to trustworthy AI across three directions: evolutionary attacks, evolutionary defenses, and trustworthy self-evolving AI systems. Unlike prior reviews that treat trustworthy AI, EC, and self-evolving systems largely separately, we connect these lines through a common evolutionary perspective. For self-evolving AI, we examine how trustworthiness governs the generation and retention of updates that shape subsequent adaptation. We further synthesize evaluation methods and benchmark resources from both trustworthiness and evolutionary-search perspectives. Finally, we discuss key challenges and future research directions toward more effective and reliable use of EC in trustworthy AI.
Figures & tables
| Survey | Object | Trustworthy AI | EC-Centric | Trustworthy Self-Evolving AI |
| Trustworthy AI Surveys | ||||
| Huang et al. (2020) [ 42 ] | Predictive | ✓ | ✗ | ✗ |
| Zhang et al. (2022) [ 43 ] | Predictive | ✓ | ✗ | ✗ |
| Goyal et al. (2023) [ 44 ] | Predictive | ✓ | ✗ | ✗ |
| Tao et al. (2024) [ 41 ] | Generative Agentic | – | ✗ | – |
| Costa et al. (2024) [ 45 ] | Predictive | ✓ | ✗ | ✗ |
| Benchmark Type | Evaluation Scope | Representative Resources | Evaluation Target | Main Metrics |
| Predictive robustness | RobustBench [ 85 ] ; NAS-RobBench-201 [ 86 ] | Adversarial robustness of models and architectures | Clean accuracy; robust accuracy | |
| LLM/VLM safety | JailbreakBench [ 87 ] ; HarmBench [ 88 ] ; StrongREJECT [ 89 ] ; MM-SafetyBench [ 90 ] | Jailbreak and harmful-response behavior | ASR; harmfulness; refusal quality | |
| Ethical judgment and bias | ETHICS [ 91 ] ; BBQ [ 92 ] | Moral judgments in text; social bias in question answering | Judgment accuracy; bias score | |
| Trustworthiness [-0.1pt] Benchmarks | Agent safety | Agent-SafetyBench [ 93 ] ; R-Judge [ 94 ] ; OS-Harm [ 95 ] | Risk and unsafe behavior during execution | Safety violations; risk recognition; task success |
| Single-objective optimization | COCO/BBOB [ 79 ] ; IOHexperimenter [ 82 ] ; PINN neuroevolution benchmarks [ 96 ] | Efficiency and reliability under limited or noisy evaluations | Fitness evaluations; target attainment; convergence; run statistics | |
| Multi-objective optimization | DTLZ [ 97 ] ; WFG [ 98 ] ; mDTLZ1–4 [ 99 ] ; DtA-PF generator [ 100 ] | Pareto-front convergence and coverage | Hypervolume; inverted generational distance |
| Base EC | Method | Search Design | Evaluation Design | Evolved Object | Trustworthy AI Problem |
| GenAttack [ 8 ] | Org.: elitism, adaptive mutation Aux.: dimensionality reduction | Crit.: target-class log odds Signal: class probabilities | Input perturbations | Black-box adversarial attacks | |
| AutoDAN [ 10 ] | Org.: hierarchical GA Aux.: LLM initialization, mutation | Crit.: target-response likelihood Signal: target-token probabilities | Jailbreak prompts | Jailbreak generation | |
| GA | DIGA [ 11 ] | Org.: importance-guided GA Aux.: TF–IDF token guidance | Crit.: query-centroid similarity Signal: retriever embeddings | Adversarial passages | RAG corpus poisoning |
| CMA-ES Attack [ 107 ] | Org.: covariance adaptation Aux.: perturbation compression | Crit.: success, perturbation Signal: model scores | Image perturbations | High-dimensional black-box attacks | |
| ES | EvoIBA [ 64 ] | Org.: boundary-guided Org.: Sep-CMA-ES Aux.: fractal initialization, jumps | Crit.: perturbation under success Signal: top-1 class labels | Input perturbations | Hard-label black-box attacks |
| DE | One Pixel Attack [ 20 ] | Org.: crossover-free DE Aux.: — | Crit.: class-probability objective Signal: class probabilities | Pixel coordinates and values | Sparse adversarial input generation |
| Base EC | Method | Search Design | Evaluation Design | Evolved Object | Trustworthy AI Problem |
| EMO-GAT [ 130 ] | Org.: bi-objective NSGA-II Aux.: adversarial training loop | Crit.: two attack-specific accuracies with Crit.: clean-accuracy penalty Signal: clean and adversarial accuracy | Adversarial-example selection masks | Multi-attack adversarial training | |
| MORAS-SH [ 24 ] | Org.: three-objective NSGA-II Aux.: weight sharing, surrogate | Crit.: clean error, adversarial error, Crit.: surrogate score Signal: low-fidelity errors, Signal: surrogate predictions | Neural architectures | Robust architecture search | |
| ER-APT [ 131 ] | Org.: genetic example refinement Aux.: gradient initialization, Aux.: dynamic weighting | Crit.: adversarial loss Signal: prompt–model loss | Adversarial examples | Adversarial prompt tuning | |
| GA | TRG-ASO [ 132 ] | Org.: phase-adaptive GA Aux.: fitness-record early stopping | Crit.: robust and clean improvement Signal: adversarial and clean losses | PGD attack configurations | Robust training strategy search |
| SNES [ 133 ] | Org.: confidence-weighted ES Aux.: Bayesian model checking | Crit.: return, constraint cost Signal: Bayesian satisfaction confidence | Neural-policy parameters | Safe policy synthesis | |
| ES | SMC-ES [ 134 ] | Org.: specification-constrained ES Aux.: statistical model checking | Crit.: worst-case reward, constraint violation Signal: scenario rewards, violation costs | Neural-policy parameters | Verified policy synthesis |
| Trustworthiness Pattern | Representative Methods | Evolving State | Persistent State |
| MermaidFlow [ 32 ] | Typed agent workflows | Admissible workflow variants | |
| Search-Space-Constrained [-0.1pt] Exploration | SEVerA [ 158 ] | Candidate programs | Verified programs used in later updates |
| AgentBreeder [ 33 ] | Multi-agent scaffolds | Selected scaffold population | |
| HarnessBank [ 34 ] | Prompts, knowledge, tools, and runtime control | Verified harness archive | |
| SHE; JIT Memory [ 159 , 160 ] | Safety harnesses and mitigation skills | Validated harness updates and verified Meta-Skills | |
| Selection-Guided Refinement | TAME; SkillHarness; Membrane [ 161 , 162 , 163 ] | Memory, reusable skills, and safety knowledge | Governed memory and skill state |
Trust in AI
Trust in artificial intelligence concerns when people and institutions should rely on AI, and how systems can make that reliance warranted and safe.
See all 175 papers