Small open models suit private customer-service deployments but trail frontier models in two ways: they miss what the written policy requires in a given situation, and they act inconsistently on what they know. We study whether such a model can close both gaps from its own deployment logs and their reference outcomes, without a stronger teacher model. Our starting point is a Gemma 4 12B agent that reads its domain policy from on-demand skills rather than from its system prompt. In each round, the deployed system serves a fresh batch of tasks; the base model diagnoses failed conversations and writes rules into the skills, and a rule survives only if it passes code checks, counterexample self-review, a paired retest, and a noise-calibrated validation gate. Reinforcement learning (GRPO) then trains a LoRA adapter on complete multi-turn, tool-using conversations. On the retail and airline domains of τ2-bench, without training or selecting on official tasks, the agent after two rounds improves on this starting point by 12.5 pass1 points in retail and 19.5 in airline, and by 18.4 and 22.0 pass4 points. It also exceeds the benchmark's reference agent, which runs the same model with the whole policy in its system prompt, by 11.8 and 8.0 pass1 points and by 22.8 and 16.0 pass4 points, closing most of the gap to three frontier models under the same protocol. A 2×2 ablation shows that the rules rescue tasks the agent never solved and carry policy knowledge that RL with the rules in context does not internalize, whereas the trained weights make solved tasks reliable.
Figures & tables
Figure 1: One round of co-evolution. The deployed system serves a fresh batch Bk ; the base model, acting as editor, turns failed conversations into rules; GRPO then trains the agent’s LoRA adapter under the new rules. Each layer is adopted only if it passes its gate on the validation set V, and a rejected harness candidate is evolved once more from the current version. Official tasks are run only for reporting.
System
Editor
Weights
Dialogue
Held out
HarnessX
Claude Opus
GRPO
no
no
SIA
Claude Sonnet
LoRA RL
no
no
HarnessForge
GPT-5.5
LoRA SFT
no
yes
CoTrace
frontier
SFT, RL
no
yes
Co-Harness
critic
SFT
no
–
Yu et al.
Gemini
LoRA SFT
no
yes
Table 1: Systems that update both the harness and the weights. Editor: the model that writes harness edits (for Yu et al., the model that corrects the student’s turns). Dialogue: co-evolution on multi-turn tasks with a simulated user. Held out: results on tasks used neither for evolution nor for selection. –: not stated.
Retail
Airline
System
pass 1
pass 4
pass 1
pass 4
Gemma 4 12B
Reference agent
73.2
43.9
67.5
44.0
H0-M0 (start)
72.6
48.2
56.0
38.0
H1-M0
73.2
48.2
67.0
46.0
H1-M1
77.4
53.5
71.5
54.0
Table 2: Official τ2 -bench tasks, 4 trials per task. The reference agent places the whole policy in its system prompt; all other rows use the skill-based harness. H2-M0 combines the final rules with the base weights, and H0-M2 the trained weights with the original skills. Differences in the text are computed from unrounded scores; Table 10 adds pass 2 and pass 3 .
Figure 2: Official scores over two rounds. Dashed lines: the reference agent; gray bands: the range of the three frontier models with the H0 harness. Each update was adopted on V alone.
Figure 3: The 2×2 ablation on the official tasks: the evolved rules alone ( H2-M0 ), the trained weights alone ( H0-M2 ), and both ( H2-M2 ). (a) Gains over H0-M0 . (b) Success on the tasks that H0-M0 solved in 0, 1–3, or 4 of its four trials; the three systems are independent runs.
Bin
n
Ref.
H0-M0
H2-M0
H0-M2
H2-M2
Retail
0/4
8
18.8
0.0
43.8
25.0
68.8
1–3/4
51
64.2
54.4
72.1
68.6
77.5
4/4
55
89.5
100.0
87.3
95.0
94.5
Airline
0/4
15
31.7
0.0
26.7
5.0
43.3
1–3/4
16
68.8
56.2
78.1
73.4
82.8
4/4
19
94.7
100.0
93.4
98.7
94.7
Table 3: Mean success (%) on the official tasks, grouped by how many of each task’s four H0-M0 trials succeeded (Bin). The other columns are independent runs of four trials per task, so the binning does not bias their comparison. Ref.: reference agent.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Domain
Rule (verbatim; … marks omissions)
Edit
Retest
Verification
Airline
Before offering a compensation certificate, call get_flight_status and verify that the status confirms the flight was actually cancelled (for a cancellation complaint) or delayed (for a delay complaint).
rewrite
12/24 → 23/24
Verification
Airline
Before calling cancel_reservation , if the user claims a flight was cancelled by the airline, call get_flight_status … to verify that the status is ‘cancelled’.
add
0/6 → 6/6
Verification
Airline
Before calling cancel_reservation , verify that the reservation was made within the last 24 hours, the flight was cancelled by the airline, it is a business flight, or the user has travel insurance and the reason for cancellation is a health or weather reason …
rewrite
3/12 → 9/12
Computation
Airline
Before calling update_reservation_baggages , calculate the value for nonfree_baggages by subtracting the number of free bags from the total_baggages , and use 0 if the result is negative …
rewrite
4/6 → 6/6
Computation
Airline
When listing flight options, the agent must sum the prices of all flight segments in each option to calculate the total price per passenger.
add
3/6 → 5/6
Retrieval
Airline
When the user asks for information about other upcoming reservations, call get_user_details to retrieve all reservation IDs, and then call get_reservation_details for each reservation …
add
5/12 → 8/12
Appendix
Table 4: Representative rules of the final harnesses and one deletion, all written by the 12B editor. Type: what the rule adds (a verification, a computation, a retrieval, a policy boundary, or a correction of an earlier rule). Retest: successes of the supporting tasks in the paired retest, current → candidate.
Write
update_reservation_baggages with nonfree_baggages 2; reference 0 (total 4)
DB diff
an extra payment of $100 on the customer’s card
Loads
the entry document and airline:modify ; never airline:book
Rerun
0/6 successes
Diagnosis
“The agent failed to calculate the correct number of non-free baggages because it did not load the ‘Book flight’ section of the policy, which contains the rules for baggage allowances based on membership level and cabin class.” Root cause: not loaded .
Edit
ADD to the entry document: “When the user wants to modify baggage, load both the airline:modify and airline:book skills.”
Retest
step kept, with 7 more successes than the current version over its 31 tasks
Appendix
Table 5: From an evidence pack to a rule (airline, round 1, step 1).
Root cause
Prescribed edit
Share
Missing
add a rule
43.8%
Not loaded
add a loading rule to the entry document
3.9%
Unclear or wrong
rewrite or delete the rule; for a policy sentence, add a rule
8.7%
Not followed
keep it and add a more specific rule
43.6%
Appendix
Table 6: Root causes the editor assigns to a failure, the edit each permits, and their shares among the 715 of 761 diagnoses with a valid reply (572 proposed additions, 30 rewrites or deletions, and 113 no edit). The policy itself is never edited.
Following the rule, the agent would…
Blocked
do for every user what one user asked for or claimed
2
choose something the user must choose or provide
4
change the database without listing the details and getting an explicit yes
0
refuse, block, or transfer where the policy allows the action
21
compute or state an amount or count against the policy or a tool
3
transfer where the policy says to refuse, or the reverse
3
Appendix
Table 7: Violation types in the counterexample self-review, and the number of the 30 blocked edits for which at least one of the two reviews reported each type.
True change
Harness gate
Model gate
in S
one attempt
with retry
(best of 3)
−1.5
9.8%
18.6%
21.5%
0.0
50.0%
75.0%
74.9%
+0.5
66.7%
88.9%
87.8%
+1.0
80.6%
96.2%
95.1%
+2.0
95.8%
99.8%
99.6%
Appendix
Table 8: Probability of adoption under the noise model, with σS=1.16 , the mean of the two domains. With retry: two attempts with the same true effect. Model gate: the best of three checkpoints against a shared control; because that control is the adopted run of the harness gate, whose selection biases it upward, the real model gate is slightly stricter than simulated.
Agent and editor ( Gemma 4 12B, base or LoRA)
Sampling
T=1.0 , top- p=0.95 , top- k=64
Output tokens
at most 8,192 (agent), 32,768 (editor)
Skill loads
at most 4 before one reply
Editor input
at most 30k tokens per failure
Conversation
at most 100 turns (pipeline), 200 (official)
Harness evolution
Appendix
Table 9: Hyperparameters, identical in both rounds and both domains.
Retail
Airline
System
pass 2
pass 3
pass 2
pass 3
Reference agent
59.4
50.7
55.0
49.0
H0-M0
60.1
52.9
47.3
42.0
H1-M0
61.0
53.5
56.0
50.0
H1-M1
65.2
58.1
61.0
56.5
H2-M1
70.5
64.3
63.7
59.0
Appendix
Table 10: pass 2 and pass 3 on the official tasks, complementing Table 2 .
Effect
pass 1
pass 4
Retail
H2-M0 − H0-M0
+4.8 [ −0.7 , 10.3]
0.0 [ −10.5 , 10.5]
H0-M2 − H0-M0
+5.7 [1.1, 10.1]
+6.1 [ −2.6 , 14.9]
H2-M2 − H2-M0
+7.7 [2.6, 12.9]
+18.4 [7.9, 28.9]
H2-M2 − H0-M2
+6.8 [2.0, 11.8]
+12.3 [4.4, 21.1]
Interaction
+2.0 [ −4.4 , 8.8]
+12.3 [ −0.9 , 25.4]
Appendix
Table 11: Effects in the 2×2 ablation with 95% paired bootstrap intervals over tasks (10,000 resamples). The interaction is H2-M2 − H2-M0 − H0-M2 + H0-M0 .
Figure 4: Gain in pass 4 against gain in pass 1 on the official tasks for sixteen layer contrasts: the eight accepted updates and eight ablation contrasts, over both domains. All eight weight contrasts lie above the diagonal, and all four rule contrasts on the base weights lie on or below it; three of the four rule contrasts on trained weights lie above it, as rules written in round 2 from the failures of M1 act on tasks the trained weights nearly solve.
Never solved
Lifted from never
Run
collection
profile
tasks
success
Retail 1
16
22
6
35%
Airline 1
71
42
41
58%
Retail 2
20
20
10
38%
Airline 2
25
28
11
34%
Appendix
Table 12: Batch tasks never solved in four collection trials and in four profiling trials under the accepted harness, and tasks lifted from never to at least one success, with their success in GRPO’s first-epoch rollouts.
Loads
Success
System
book
loaded
not loaded
Reference agent
–
18/40 (policy in prompt)
H0-M0
30%
6/12
0/28
H0-M2
30%
7/12
0/28
H1-M0
40%
12/16
6/24
H2-M0
40%
11/16
6/24
Appendix
Table 13: The ten airline baggage-allowance tasks (40 conversations per system): share of conversations that load the booking skill, and successes with and without it.
V
Official
Version
S on V
pass 1
pass 4
pass 1
pass 4
Retail
H0-M0
72.44
76.5
56.2
72.6
48.2
H1-M0
72.53
76.7
55.9
73.2
48.2
H1-M1
76.71
80.6
61.3
77.4
53.5
H2-M1
78.21
81.9
63.3
80.5
59.6
Appendix
Table 14: Validation score S and pass 1 and pass 4 on V (8 trials per task) and on the official tasks (4 trials per task) for each accepted version; a version gated twice shows its first gate’s score. In two cases the first harness candidate fell short (retail round 1, 72.18 vs. 72.44; airline round 2, 74.48 vs. 74.74), and the candidate evolved on retry passed against a rerun of the current version (72.53 vs. 71.94; 75.44 vs. 73.07); averaging the two runs of the current version gives the same decisions (72.53 vs. 72.19; 75.44 vs. 73.90).
Figure 5: pass 1 gain of each accepted update on V and on the official tasks. The dotted line marks equal gains.
Airline
n
Ref.
H0 M0
H2 M0
H0 M2
H2 M2
Find a reservation
27
64.8
52.8
60.2
60.2
70.4
Fallback condition
19
55.3
35.5
56.6
43.4
64.5
Multiple requests
17
39.7
17.6
38.2
25.0
50.0
Refused cancel
15
76.7
60.0
78.3
70.0
81.7
Persistent user
14
80.4
66.1
83.9
75.0
89.3
Change cabin
13
42.3
25.0
42.3
28.8
59.6
Appendix
Table 15: pass 1 by annotated knowledge point: every knowledge point with at least 10 official tasks (outcome types omitted), plus price computation and adding bags, discussed in § 5.4 . A task carries several knowledge points. Ref.: reference agent; H i over M j : H i -M j .
Post-training is becoming a service (PTaaS): a customer hands an operator data and a goal, and a forward-deployed engineer (FDE) returns a fine-tuned, evaluated, and deployed model under a budget, a human-approval gate, and reproducibility requirements. Seating an LLM agent in the FDE seat raises a question existing benchmarks cannot answer: not whether an agent can raise a metric, but whether it can be trusted to deliver. We answer it on a governed delivery plane, where an agent drives ten stages and an oracle scores each stage from platform-recorded facts. The central silent failure is the run that trains but does not learn (TBDL): loss falls, every signal stays green, and the delivered model is no better than the base. An operator-run acceptance gate catches every such run before payment, and a detector calibrated on known-corrupted runs flags severe corruption mid-run. We ran four frontier agents (Claude Opus 5, GPT-5.6-luna, Gemini 3.7 Flash, DeepSeek V4-Pro) end to end on metered L40S, A100, and H200 GPUs across 8B to 70B open bases, certifying every scenario before scoring. We also ran a human FDE arm under the same oracle and compare every agent against it.
Weihang Ding, Junfei Zhan
Department of Industrial Engineering and Operations Research University of California, Berkeley · Department of Computing Imperial College London, United Kingdom
Deploying LLM agents typically requires a compact test-time student, even if a stronger teacher is available during training. We study knowledge brewing: distilling a teacher's interactive experience into a persistent external memory for the student. Crucially, this requires no weight updates, expert demonstrations, ground-truth labels, or test-time teacher access. This setting poses two challenges: environments provide only sparse, binary feedback, and teacher-authored notes must be inherently tailored to be concretely executable by a substantially weaker student. To address these hurdles, we propose AgentBrew, comprising two coupled components. First, a failure-triggered teacher--Ralph Loop mitigates sparse feedback by transforming student failures into environment-validated notes. Second, student-aware synthesis calibrates teacher knowledge to the weak executor's operational granularity, yielding model-specific, actionable guidance. Extensive evaluations and comprehensive ablations across coding, math, and tool-use tasks demonstrate that this asymmetric, training-free brewing paradigm produces highly capable yet deployable LLM agents.
Effective teaching requires reasoning about the learner as well as the task, a capacity that Theory of Mind describes as inferring another agent's latent state from its behavior. We ask whether a stronger LLM teaches a fixed student better when it applies this principle. The student is an LLM agent acting through a harness that fixes its tools, context, and action budget, so a procedure that helps one model can exhaust the budget of another. We introduce Student-Aware Teaching (SAT). The teacher builds a learner model that pairs an empirical profile with hypotheses about the student's limitations, tests these hypotheses with diagnostic probes, and writes reusable skills for the diagnosed failures. A skill enters the library only after it raises success on matched development tasks. With Claude-Sonnet-5 teaching DeepSeek-V4-Flash, SAT reaches 86.46% task success across four primary runs, leading both a profile-free teacher with the same framework (81.04%) and the untaught student (75.00%) in each run. Removing the analytical learner model lowers both prediction and success, and under misleading learner profiles none of the 48 candidate skills passes admission. When DeepSeek-V4-Flash and Gemini-3.8-Flash exchange libraries, each performs better with its own. Saved libraries keep average gains of up to 11.1 points after the teacher leaves. The benefit depends on a student that can follow instructions and a deployment route that delivers them.