Knowledge in the Harness, Reliability in the Weights: Teacher-Free Co-Evolution of an LLM Agent
Organizations: Brightendata
Abstract
Small open models suit private customer-service deployments but trail frontier models in two ways: they miss what the written policy requires in a given situation, and they act inconsistently on what they know. We study whether such a model can close both gaps from its own deployment logs and their reference outcomes, without a stronger teacher model. Our starting point is a Gemma 4 12B agent that reads its domain policy from on-demand skills rather than from its system prompt. In each round, the deployed system serves a fresh batch of tasks; the base model diagnoses failed conversations and writes rules into the skills, and a rule survives only if it passes code checks, counterexample self-review, a paired retest, and a noise-calibrated validation gate. Reinforcement learning (GRPO) then trains a LoRA adapter on complete multi-turn, tool-using conversations. On the retail and airline domains of -bench, without training or selecting on official tasks, the agent after two rounds improves on this starting point by 12.5 pass points in retail and 19.5 in airline, and by 18.4 and 22.0 pass points. It also exceeds the benchmark's reference agent, which runs the same model with the whole policy in its system prompt, by 11.8 and 8.0 pass points and by 22.8 and 16.0 pass points, closing most of the gap to three frontier models under the same protocol. A ablation shows that the rules rescue tasks the agent never solved and carry policy knowledge that RL with the rules in context does not internalize, whereas the trained weights make solved tasks reliable.
Figures & tables
| System | Editor | Weights | Dialogue | Held out |
|---|---|---|---|---|
| HarnessX | Claude Opus | GRPO | no | no |
| SIA | Claude Sonnet | LoRA RL | no | no |
| HarnessForge | GPT-5.5 | LoRA SFT | no | yes |
| CoTrace | frontier | SFT, RL | no | yes |
| Co-Harness | critic | SFT | no | – |
| Yu et al. | Gemini | LoRA SFT | no | yes |
| Retail | Airline | |||
| System | pass 1 | pass 4 | pass 1 | pass 4 |
| Gemma 4 12B | ||||
| Reference agent | 73.2 | 43.9 | 67.5 | 44.0 |
| H0-M0 (start) | 72.6 | 48.2 | 56.0 | 38.0 |
| H1-M0 | 73.2 | 48.2 | 67.0 | 46.0 |
| H1-M1 | 77.4 | 53.5 | 71.5 | 54.0 |
| Bin | Ref. | H0-M0 | H2-M0 | H0-M2 | H2-M2 | ||
|---|---|---|---|---|---|---|---|
| Retail | 0/4 | 8 | 18.8 | 0.0 | 43.8 | 25.0 | 68.8 |
| 1–3/4 | 51 | 64.2 | 54.4 | 72.1 | 68.6 | 77.5 | |
| 4/4 | 55 | 89.5 | 100.0 | 87.3 | 95.0 | 94.5 | |
| Airline | 0/4 | 15 | 31.7 | 0.0 | 26.7 | 5.0 | 43.3 |
| 1–3/4 | 16 | 68.8 | 56.2 | 78.1 | 73.4 | 82.8 | |
| 4/4 | 19 | 94.7 | 100.0 | 93.4 | 98.7 | 94.7 |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Type | Domain | Rule (verbatim; … marks omissions) | Edit | Retest |
|---|---|---|---|---|
| Verification | Airline | Before offering a compensation certificate, call get_flight_status and verify that the status confirms the flight was actually cancelled (for a cancellation complaint) or delayed (for a delay complaint). | rewrite | 12/24 23/24 |
| Verification | Airline | Before calling cancel_reservation , if the user claims a flight was cancelled by the airline, call get_flight_status … to verify that the status is ‘cancelled’. | add | 0/6 6/6 |
| Verification | Airline | Before calling cancel_reservation , verify that the reservation was made within the last 24 hours, the flight was cancelled by the airline, it is a business flight, or the user has travel insurance and the reason for cancellation is a health or weather reason … | rewrite | 3/12 9/12 |
| Computation | Airline | Before calling update_reservation_baggages , calculate the value for nonfree_baggages by subtracting the number of free bags from the total_baggages , and use 0 if the result is negative … | rewrite | 4/6 6/6 |
| Computation | Airline | When listing flight options, the agent must sum the prices of all flight segments in each option to calculate the total price per passenger. | add | 3/6 5/6 |
| Retrieval | Airline | When the user asks for information about other upcoming reservations, call get_user_details to retrieve all reservation IDs, and then call get_reservation_details for each reservation … | add | 5/12 8/12 |
| Write | update_reservation_baggages with nonfree_baggages 2; reference 0 (total 4) |
|---|---|
| DB diff | an extra payment of $100 on the customer’s card |
| Loads | the entry document and airline:modify ; never airline:book |
| Rerun | 0/6 successes |
| Diagnosis | “The agent failed to calculate the correct number of non-free baggages because it did not load the ‘Book flight’ section of the policy, which contains the rules for baggage allowances based on membership level and cabin class.” Root cause: not loaded . |
| Edit | ADD to the entry document: “When the user wants to modify baggage, load both the airline:modify and airline:book skills.” |
| Retest | step kept, with 7 more successes than the current version over its 31 tasks |
| Root cause | Prescribed edit | Share |
|---|---|---|
| Missing | add a rule | 43.8% |
| Not loaded | add a loading rule to the entry document | 3.9% |
| Unclear or wrong | rewrite or delete the rule; for a policy sentence, add a rule | 8.7% |
| Not followed | keep it and add a more specific rule | 43.6% |
| Following the rule, the agent would… | Blocked |
|---|---|
| do for every user what one user asked for or claimed | 2 |
| choose something the user must choose or provide | 4 |
| change the database without listing the details and getting an explicit yes | 0 |
| refuse, block, or transfer where the policy allows the action | 21 |
| compute or state an amount or count against the policy or a tool | 3 |
| transfer where the policy says to refuse, or the reverse | 3 |
| True change | Harness gate | Model gate | |
|---|---|---|---|
| in | one attempt | with retry | (best of 3) |
| 9.8% | 18.6% | 21.5% | |
| 50.0% | 75.0% | 74.9% | |
| 66.7% | 88.9% | 87.8% | |
| 80.6% | 96.2% | 95.1% | |
| 95.8% | 99.8% | 99.6% | |
| Agent and editor ( Gemma 4 12B, base or LoRA) | |
| Sampling | , top- , top- |
| Output tokens | at most 8,192 (agent), 32,768 (editor) |
| Skill loads | at most 4 before one reply |
| Editor input | at most 30k tokens per failure |
| Conversation | at most 100 turns (pipeline), 200 (official) |
| Harness evolution | |
| Retail | Airline | |||
| System | pass 2 | pass 3 | pass 2 | pass 3 |
| Reference agent | 59.4 | 50.7 | 55.0 | 49.0 |
| H0-M0 | 60.1 | 52.9 | 47.3 | 42.0 |
| H1-M0 | 61.0 | 53.5 | 56.0 | 50.0 |
| H1-M1 | 65.2 | 58.1 | 61.0 | 56.5 |
| H2-M1 | 70.5 | 64.3 | 63.7 | 59.0 |
| Effect | pass 1 | pass 4 |
|---|---|---|
| Retail | ||
| H2-M0 H0-M0 | [ , 10.3] | 0.0 [ , 10.5] |
| H0-M2 H0-M0 | [1.1, 10.1] | [ , 14.9] |
| H2-M2 H2-M0 | [2.6, 12.9] | [7.9, 28.9] |
| H2-M2 H0-M2 | [2.0, 11.8] | [4.4, 21.1] |
| Interaction | [ , 8.8] | [ , 25.4] |
| Never solved | Lifted from never | |||
|---|---|---|---|---|
| Run | collection | profile | tasks | success |
| Retail 1 | 16 | 22 | 6 | 35% |
| Airline 1 | 71 | 42 | 41 | 58% |
| Retail 2 | 20 | 20 | 10 | 38% |
| Airline 2 | 25 | 28 | 11 | 34% |
| Loads | Success | ||
| System | book | loaded | not loaded |
| Reference agent | – | 18/40 (policy in prompt) | |
| H0-M0 | 30% | 6/12 | 0/28 |
| H0-M2 | 30% | 7/12 | 0/28 |
| H1-M0 | 40% | 12/16 | 6/24 |
| H2-M0 | 40% | 11/16 | 6/24 |
| V | Official | ||||
| Version | on V | pass 1 | pass 4 | pass 1 | pass 4 |
| Retail | |||||
| H0-M0 | 72.44 | 76.5 | 56.2 | 72.6 | 48.2 |
| H1-M0 | 72.53 | 76.7 | 55.9 | 73.2 | 48.2 |
| H1-M1 | 76.71 | 80.6 | 61.3 | 77.4 | 53.5 |
| H2-M1 | 78.21 | 81.9 | 63.3 | 80.5 | 59.6 |
| Airline | Ref. | H0 M0 | H2 M0 | H0 M2 | H2 M2 | |
|---|---|---|---|---|---|---|
| Find a reservation | 27 | 64.8 | 52.8 | 60.2 | 60.2 | 70.4 |
| Fallback condition | 19 | 55.3 | 35.5 | 56.6 | 43.4 | 64.5 |
| Multiple requests | 17 | 39.7 | 17.6 | 38.2 | 25.0 | 50.0 |
| Refused cancel | 15 | 76.7 | 60.0 | 78.3 | 70.0 | 81.7 |
| Persistent user | 14 | 80.4 | 66.1 | 83.9 | 75.0 | 89.3 |
| Change cabin | 13 | 42.3 | 25.0 | 42.3 | 28.8 | 59.6 |
LLM Agent Self-Improvement
Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.
See all 296 papers