A self-evolving agent rewrites its own harness and must decide which rewrites to keep, so its improvement signal is the check a rewrite has to pass. Verifying a rewrite by agent rollout is expensive and subjective, while executable checks such as the repository's own tests are too sparse to catch behavior-changing rewrites, rejecting only a quarter of them. We instead take test feedback as the executable signal, and search for test suites that are both small and strong with a quality-diversity evolutionary algorithm, so that the signal is dense and informative. TeSearch treats suite size as a behavior descriptor, so that quality is pursued at every size and a small, strong suite is reachable, and learns a transition kernel during the search that denoises the LLM as an actuator on that descriptor. Across 129 real-world modules and three backbones, TeSearch delivers the highest-quality suites of any method compared, LLM-driven evolutionary search included, and the smallest of any LLM-based one. Used as the improvement signal of a self-evolving agent, it rejects 59.9% of behavior-changing rewrites against 25.3% for the repository's tests, at a 4.0% false-alarm rate and 3.5 s per decision. The code and data are available at https://github.com/ccsnow127/TeSearch.
Figures & tables
Figure 1: Overview of tesearch . The archive (bottom) keeps a Pareto front on (coverage, mutation score) per size niche. (1) Size-UCB picks a target niche through the transition kernel; (2) the nearest front suite on each side of the target size becomes a parent; (3) parents, module, execution feedback and target size form the context; (4) the LLM writes a suite, possibly off the requested size; (5) the suite is filed where it lands, and the size error updates the kernel.
Method
cov
mut
hv
size
SBST
Pynguin ∗
0.464
0.157
0.090
20.4
GPT-5.4
Claude Opus 4.8
Gemini 3.1 Pro
cov
mut
hv
size
cov
mut
hv
size
cov
mut
hv
size
Inference-time scaling
Iter. refinement
0.798
0.647
0.516
50.8
0.842
0.849
0.716
72.5
0.694
0.764
0.539
56.8
Best-of- N
0.908
0.758
0.693
47.6
0.953
0.861
0.824
61.5
0.837
0.823
0.691
51.4
Beam search
0.859
0.905
0.775
44.5
0.971
0.910
0.886
68.8
0.890
0.934
0.838
52.6
Table 1: Delivered suite on the 129 TestGenEval modules, means over modules, same call budget for every method. Higher is better except size , ranked separately; the smallest per backbone is in blue . Claude Code runs only on its own backbone. ∗ Pynguin is matched in wall clock and scored on the 40 modules it supports.
Repository tests
tesearch
Module
Role in the harness
div/eq
DDR
FAR
s
DDR
FAR
s
agents/default
agent loop: model calls, actions, limits
17/41
0.529
0.000
15.7
0.588
0.049
3.0
environments/local
runs bash commands, captures output
25/31
0.720
0.000
4.9
0.920
0.065
3.3
actions_toolcall
parses tool calls; renders observations
29/34
0.276
0.000
3.0
0.552
0.029
3.1
actions_text
parses text actions; renders observations
25/35
0.040
0.000
7.7
0.560
0.029
2.9
cache_control
places prompt-cache breakpoints
29/26
0.172
0.000
2.9
0.414
0.038
3.0
Table 2: Improvement signals for self-revisions of mini-swe-agent’s harness. div/eq: divergent / equivalent revisions in the module’s pool. DDR: divergent revisions rejected ( ↑ ); FAR: equivalent revisions rejected ( ↓ ); s: mean wall clock per decision.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Condition on (prev, curr)
Emitted note
Pareto dominance
no lower hypervolume and smaller size
perfect improved version of prev (hypervolume grow 0.852->0.916, size decline 24->13)
lexicographic dominance
higher hypervolume and larger size
improved version of prev (hypervolume grow …, size grow …)
Appendix
Table 3: The comparison note attached to the second parent. Together with the target, which always comes last, the ordered parents form a gradient along the order on (hv,size) , so the strongest example sits nearest the output position, where the model’s recency bias weighs it most [ 70 , 33 , 60 ] . Because the parents lie on opposite sides of the target size and are ordered first, the later of the two is never the weaker, so the relation reduces to two cases, and the note reports both changes with their numbers.
Case (order)
T against its neighbor
Hypervolume of T
Emitted note
compress ( S,L,T )
smaller than L
hv(T)≥hv(L)
perfect improved version of prev (hypervolume grow or hold from <hv(L)>, size decline <size(L)>-><n>)
expand ( L,S,T )
larger than S
hv(T)>hv(S)
improved version of prev (hypervolume grow from <hv(S)>, size grow <size(S)>-><n>)
Appendix
Table 4: What the target T must achieve against the parent placed next to it. S and L are the parents below and above the target size. The requirement is asymmetric in hypervolume because the step toward T changes size in opposite directions.
Family
Intent
Class
n
Performance
cache a function whose result depends only on its arguments
preserving
16
cache a function that reads changing state (clock, counter, file)
changing
16
list to set, where order and duplicates are unused
preserving
16
list to set, where order or duplicates are relied on
changing
16
batch independent calls, keeping results and their order
preserving
16
batch calls, changing which error surfaces and when
changing
16
Appendix
Table 5: Revision intents. Class is what the intent asks for, not what the revision turns out to do; a revision enters Table 2 only if human verification confirms that it realizes its intent ( Section 5.3 ). Intents come in pairs: each behavior-changing intent is the failure mode of the behavior-preserving intent above it, with the same goal and the same shape of edit. n counts revisions generated under each intent, pooled over the eight modules.
Module
Intent
What the revision did
Label
agents/default
narrow except Exception , missing a type the block raises
narrows except Exception around the model call to ValueError and TypeError ; a model raising KeyError or OSError now escapes the loop
div
environments/local
return early, skipping a side effect (log, counter, cleanup)
checks for the submission sentinel only when the command failed, so a successful submit no longer ends the episode
div
extract a helper, keeping captured variables and defaults
moves the decoding of a failed command’s partial output into a module-level helper, called at the same point on the same exception
eq
actions_toolcall
list to set, where order and duplicates are unused
tests for the required command argument with a one-element set’s issubset instead of in
eq
actions_text
cache a function that reads changing state (clock, counter, file)
reads the clock once at import, so every observation carries the same stale timestamp
div
cache_control
stop emitting a field read only on a rare path (e.g., errors)
deletes both places that attach the ephemeral cache_control marker, so no breakpoint is ever set
div
Appendix
Table 6: Example divergent and equivalent revisions for six of the eight harness modules, with the revision intent and content. eq and div abbreviate equivalent and divergent.