MOAE: Multi-Objective Agent Evolution with Pareto-Preserving Search

Hengle Jiang; Qijun Cai; Ziying Luo; Ke Tang

cs.AISep 5, 2026

MOAE: Multi-Objective Agent Evolution with Pareto-Preserving Search

Authors: Hengle Jiang, Qijun Cai, Ziying Luo, Ke Tang

Organizations: Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation Department of Computer Science and Engineering, Southern University of Science and Technology

Abstract

As LLM-based agents continue to advance, their evaluation has become increasingly multifaceted: a capable agent must not only achieve high task completion accuracy but also perform well in interaction quality, safety, and efficiency, raising a central question: can these objectives be optimized simultaneously? Existing methods have considered multiple objectives, but many collapse heterogeneous measurements into a fixed scalar score. Such scalarization depends on metric normalization and preference weights and may discard candidates that represent useful deployment trade-offs. We introduce Multi-Objective Agent Evolution (MOAE), which organizes iterative in-context refinement as a Pareto-preserving evolutionary search over complete agent rollouts. Given a limited rollout budget, MOAE maintains an empirical archive of non-dominated candidates, uses objective-specific diagnostics to guide offspring generation, and applies constraint-aware selection only at deployment. This separates candidate preservation during search from the preference used to return a final solution. The procedure requires no parameter updates and allows each objective to be replaced by any measurable property, which we instantiate as task performance, trajectory quality, and safety. Experiments on TravelPlanner and AgentDojo show that MOAE consistently improves task performance and trajectory quality while maintaining strong safety under matched rollout budgets. Search-behavior analysis further shows that Pareto preservation expands the attainable objective region and increases the frequency of joint improvement. These results demonstrate the potential of Pareto-preserving in-context evolution for optimizing multiple agent properties without committing to a fixed scalarization during search.

Figures & tables

Figure 1: Overview of MOAE. (a) Motivation for Pareto preservation. (b) Rollout initialization. (c) Multi-objective evaluation. (d) Illustrative evolution trajectory. (e) Diagnostic-guided mutation and archive update. (f) Constraint-aware deployment.

Direct Search Baselines
	Qwen3-30B-A3B				Gemma-4-31B-it				DeepSeek-V4-Pro
Method	Task $\uparrow$	Trajectory $\uparrow$	Risk $\downarrow$	Tokens $\downarrow$	Task $\uparrow$	Trajectory $\uparrow$	Risk $\downarrow$	Tokens $\downarrow$	Task $\uparrow$	Trajectory $\uparrow$	Risk $\downarrow$	Tokens $\downarrow$
Base Agent	0.493	0.946	0.533	59.4	0.748	0.962	0.304	56.6	0.631	0.889	0.275	60.4
Weighted-Sum	0.546	0.964	0.391	458.8	0.792	0.972	0.207	398.7	0.643	0.926	0.280	422.9
Best-of-7	0.557	0.969	0.369	477.0	0.816	0.967	0.280	392.2	0.705	0.919	0.244	438.4
Agent Optimization Baselines

Table 1: TravelPlanner results across three execution models. Tokens are reported in thousands. All search methods use seven complete rollouts, while the Base Agent uses one. Token-cost bolding excludes the one-rollout Base Agent. MOAE-Task, MOAE-Trajectory, and MOAE-Safety are objective-specific deployment audits from the same search pool, while Full MOAE is the balanced output used for the main comparison. Final Pass and complete audit metrics are reported in the supplementary material.

Figure 2: Best-so-far Task Score under 1 to 25 complete agent rollouts. Shaded regions show paired-bootstrap 95% confidence intervals over TravelPlanner requests.

Figure 3: Search behavior under a matched seven-rollout budget on TravelPlanner with Gemma. Error bars show query-level bootstrap 95% confidence intervals.

Setting	Method	Utility	Trajectory	ASR
Gemma Tool Knowledge	Base Agent	32.81	0.435	0.00
	Repeat Prompt	31.25	0.436	0.00
	Best-of-7	32.81	0.476	0.00
	Full MOAE	40.63	0.695	0.00
Gemma Direct Injection	Base Agent	38.71	0.486	4.84
	Best-of-7	39.06	0.476	1.56

Table 2: AgentDojo results. Utility and trajectory are higher when better; attack success rate (ASR) is lower when better.

Method	$\Delta$ Task	$\Delta$ Final	$\Delta$ Trajectory	$\Delta$ Valid	$\Delta$ Risk
Full MOAE	+0.096	+7.22	+0.012	+35.89	-0.096
w/o archive	+0.087	+5.56	+0.008	+26.11	-0.079
w/o mutation	+0.044	+2.78	+0.009	+21.67	-0.055
w/o diagnostics	+0.047	+3.33	+0.011	+31.67	-0.079

Table 3: TravelPlanner ablations with Gemma. Final Pass and Valid Trajectory are measured in percentage points. Positive changes are better except for Safety Risk.

Model	Interface	Decoding	Context limit	Parallelism	Hardware
Gemma-4-31B-it	Local API	Deterministic	16,384	4 workers	8 $\times$ NVIDIA A800
Qwen3-30B-A3B	Local API	Deterministic	16,384	4 workers	8 $\times$ NVIDIA A800
DeepSeek-V4-Pro	Official API	Deterministic	Provider default	96 workers	API only

Table 4: Implementation details. Software versions, model revisions, and complete launch commands are provided in the accompanying anonymous artifact.

Dimension	Fixed evaluator signal	Search direction	Diagnostics exposed to mutation	Independent audits
Task utility	Weighted satisfaction of benchmark constraints, including itinerary, transportation, date, budget, and commonsense requirements	Maximize	Unsatisfied constraints and missing required information	Final Pass Rate
Trajectory quality	Fixed combination of execution validity, information coverage, evidence support, and execution discipline	Maximize	Invalid or redundant actions, unsupported claims, missing evidence, and incomplete coverage	Strict valid-trajectory rate, invalid-call rate, redundant-call rate
Safety	One minus a continuous risk score that detects influence from untrusted observations on actions, planner input, and final output	Maximize	Location and type of detected untrusted influence without reproducing attack content	Attack Success Rate and severe Attack Success Rate

Table 5: Evaluator design matrix. The same fixed evaluator is applied to all methods and models within a benchmark. Audit metrics are not additional optimization objectives.

Weight profile	$w_{\mathrm{task}}$	$w_{\mathrm{traj}}$	$w_{\mathrm{safety}}$	Task	Trajectory	Risk	Normalized HV gain	Joint improvement
Equal	0.333	0.333	0.333	0.792	0.972	0.207	0.41	9.4%
Task-heavy	0.500	0.250	0.250	0.814	0.968	0.236	0.38	7.8%
Trajectory-heavy	0.250	0.500	0.250	0.781	0.980	0.224	0.39	8.3%
Safety-heavy	0.250	0.250	0.500	0.742	0.958	0.142	0.36	6.7%

Table 6: Sensitivity of fixed scalarization to four predefined weight profiles on TravelPlanner with Gemma. Each profile uses the same seven-rollout budget.

Hyperparameter	Value
Complete rollouts per query	7
Initial candidates	1
Generations	2
Offspring per generation	3
Optimization objectives	Task, trajectory, safety
Archive update	Empirical nondominated subset

Table 7: Main MOAE configuration.

Model	$\Delta$ Task	$\Delta$ Trajectory	$\Delta$ Risk	$\Delta$ Final Pass
Qwen3-30B-A3B	+0.092	+0.026	-0.133	+1.11%
Gemma-4-31B-it	+0.096	+0.012	-0.096	+7.78%
DeepSeek-V4-Pro	+0.095	+0.046	-0.096	+6.11%

Table 8: Change from the corresponding Base Agent to the constraint-aware Full MOAE deployment point on TravelPlanner.

Method	Task	Trajectory	Valid (%)	Risk
Full MOAE	$0.879_{\scriptscriptstyle\pm 0.013}$	$0.991_{\scriptscriptstyle\pm 0.003}$	$65.0_{\scriptscriptstyle\pm 6.6}$	$0.174_{\scriptscriptstyle\pm 0.011}$
MOCHA-Select	$0.819_{\scriptscriptstyle\pm 0.006}$	$0.988_{\scriptscriptstyle\pm 0.002}$	$58.3_{\scriptscriptstyle\pm 5.2}$	$0.168_{\scriptscriptstyle\pm 0.023}$
Best-of-7	$0.778_{\scriptscriptstyle\pm 0.027}$	$0.989_{\scriptscriptstyle\pm 0.002}$	$58.3_{\scriptscriptstyle\pm 3.8}$	$0.184_{\scriptscriptstyle\pm 0.005}$

Table 9: Results over three seeds on a fixed subset of the official TravelPlanner test split. Smaller subscripted values denote between-run standard deviations.

Analysis	Protocol	Primary comparison	Result
Held-out evaluator	Re-score Base, Best-of-7, Weighted-Sum, and Full MOAE rollouts using an evaluator whose feedback is not exposed during search	Candidate-order correlation and paired changes in task, trajectory, and safety	Spearman $\rho=0.91$ ; 85.6% deployment agreement. Full MOAE versus Base: $+0.088$ Task, $+0.010$ Trajectory, and $-0.087$ Risk.
Objective-weight robustness	Recompute trajectory quality under predefined alternative component weights without rerunning or reselecting candidates	Rank correlation and paired Full MOAE improvements under each weighting profile	Spearman $\rho=0.93\text{--}0.97$ ; 91.7% of selections remain unchanged. Full MOAE improves all three objectives under every profile.
Expanded test evaluation	Freeze prompts, objectives, and deployment rules, then evaluate a predefined 200-query subset of the official TravelPlanner test split	Base, Best-of-7, Weighted-Sum, and Full MOAE	Task/Trajectory/Risk:
			Base: $0.781/0.964/0.281$
			Best-of-7: $0.852/0.984/0.194$
			Weighted-Sum: $0.840/0.980/0.216$

Table 10: Additional generalization and evaluator-robustness analyses. Results for expanded test evaluation are reported as Task/Trajectory/Risk.

Direction	Mutations	Target improved	Archive admitted
Task	360	20.56%	43.61%
Trajectory	360	13.06%	38.61%
Safety	360	25.00%	40.28%
Overall	1,080	19.54%	40.83%

Table 11: Direction-specific mutation outcomes on TravelPlanner with Gemma.

Method	Norm. HV gain	Pareto size	Admission rate	Joint improve.
Best-of-7	0.37	1.51	27.8%	9.4%
Weighted-Sum	0.41	1.83	31.2%	9.4%
w/o archive	0.43	1.96	33.9%	8.9%
Full MOAE	0.52	1.92	35.6%	13.9%

Table 12: Search-behavior summary corresponding to the main-paper mechanism figure. Confidence intervals are reported in the figure.

Candidate	Task	Trajectory	Safety risk
Initial rollout $c_{0}$	0.650	0.937	0.900
Task child $c_{1}$	0.975	0.947	0.230
Trajectory child $c_{2}$	0.975	0.971	0.300
Safety child $c_{3}$	0.975	0.947	0.300
Second-generation child $c_{4}$	1.000	0.980	0.230
Second-generation child $c_{5}$	0.975	0.980	0.300

Table 13: Objective values for the real evolution case. Safety risk is lower when better.

Model	Method	Base	Selected	Execution	Mutation	Search	Seconds
Gemma	Full MOAE	56.6	56.5	384.4	12.2	396.6	610.3
Qwen3	Full MOAE	59.4	61.4	438.3	12.4	450.7	584.9
DeepSeek-V4-Pro	Full MOAE	60.4	N/A	combined 396.3		396.3	N/A

Table 14: Mean per-query MOAE cost. Token counts are reported in thousands. “Base” and “Selected” denote the execution-token costs of the initial and deployed candidates. “Execution” includes all complete candidate rollouts, whereas “Mutation” includes the six compact policy-generation calls. The DeepSeek API logs aggregate search usage but does not provide the same execution–mutation decomposition or a directly comparable local runtime; therefore, its combined search usage is reported without an inferred split.

Method	Optimized artifact	Search scope	Objective structure	ND retention	Deployment aware selection
GEPA	Reusable textual components	Development task collection	Performance across examples	Yes	No
MOCHA	Persistent structured skills	Development task collection	Correctness and platform preferences	Method-specific	No
SkillMOO	Reusable skill bundles	Development task collection	Performance, cost, and runtime	Yes	No
AgentBreeder	Agent architectures and scaffolds	Development task collection	Capability and safety	Quality-diversity	No
EvoAgent	Multi-agent architecture	Development task collection	Primarily task performance	No	No
MOAE	Per-query execution policy	Single query with seven rollouts	Heterogeneous rollout objectives	Yes	Yes

Table 15: Comparison with closely related agent optimization settings. ND denotes nondominated. A development task collection optimizes an artifact that persists across queries, whereas MOAE performs limited-budget optimization independently for each query. Deployment-aware selection denotes an explicit separation between candidate preservation during search and constraint-aware selection of the response returned for the current query.

Self-Evolving Agents

Self-evolving agents improve their skills, memory, tools, policies, or environments by learning from interaction and evaluation feedback.

See all 427 papers

Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses
Oct 10, 2026Weichen Yu, Lesly Miculicich, Bhavana Dalvi Mishra +5Agent Harness Optimization LLM Agent Harnesses
Google Cloud AI Research · Carnegie Mellon University · Google DeepMind
RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets
Apr 6, 2026Andrew Borthwick, Stephen Ash, Anthony GalczakLLMs for Evolutionary Optimization Large Language Model-Guided Optimization
Independent Researchers
MOCHA: Multi-Objective Chebyshev Annealing for Agent Skill Optimization
May 19, 2026Md Mehrab Tanjim, Jayakumar Subramanian, Branislav Kveton +6LLM Agent Skill Learning Multi-Objective Optimization
Adobe Research

Oct 10, 2026J/K move · Enter open · S save

Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

Weichen Yu, Lesly Miculicich, Bhavana Dalvi Mishra, Jiefeng Chen+4

Google Cloud AI Research · Carnegie Mellon University · Google DeepMind

Agent Harness Optimization LLM Agent Harnesses Self-Evolving Agents Multi-Objective Evolutionary Optimization

Open paper AskPDF arXiv

Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoiding reliance on a hand-designed structural diversity metric for repository-scale harnesses. In parallel, we introduce factorized evolution operators that separately refine general-purpose harness mechanisms and benchmark-specific adaptations, together with a dynamic composition workflow that integrates them at inference time. Across SecRepoBench and BaxBench, Harmony Harness consistently outperforms strong harness-evolution baselines, with gains exceeding 10% on both benchmarks. Our results show that explicitly modeling both objective trade-offs and harness structure yields substantially stronger and more transferable agent evolution than existing single-objective, monolithic approaches.

Abstract

Self-Evolving Agents

Explore similar work

Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets

MOCHA: Multi-Objective Chebyshev Annealing for Agent Skill Optimization