Does persistent memory accumulated from previous attempts help an LLM agent solve new problems, and does memory design affect that benefit? We study programmatic Chinese-character drawing across five agent conditions attempting the same 668 items. Accumulated memory produces modest aggregate gains despite substantial growth in the memory stores: free-form notes improve first-attempt success by 4.6 percentage points, while a design combining free-form memory with reusable code improves it by 2.8 points. The two designs do not differ significantly overall, but the code bank shows a suggestive advantage on later, harder items. A separate post-hoc comparison finds a much larger improvement when the code-bank agent receives privileged, target-specific structural supervision. The agent can therefore exploit highly informative structural guidance, although the privileged per-item information does not establish what could have been learned from experience alone. These results suggest that memory design may affect where accumulated experience becomes useful, even when its average benefit is modest.
Figures & tables
Figure 1: The verdict tiers, real attempts (bottom row) with their references (top row), drawn from the four-tier window (B9–B13). Perfect (two examples): correct and calligraphically modulated. Pass: correct but crude. Close: nearly right (here the left radical collapses into a blob). Fail: components present but misassembled.
Figure 2: First-attempt success by batch; “boot” contains 18 items. Distinct markers and line patterns identify the five conditions. Shading marks B9–B13, the common four-tier evaluation window.
Success, all 668
Perfect, B9–B13 ( n=250 )
Condition
n
Rate
n
Rate
G1 no memory
219
32.8%
6
2.4%
G2 notes
250
37.4%
5
2.0%
G3 code bank
238
35.6%
1
0.4%
G4 grid + supervision
332
49.7%
51
20.4%
G5 code bank + supervision
359
53.7%
38
15.2%
Table 1: First-attempt outcomes. Five of G1’s six Perfect outputs were rendered with a system font rather than drawn; the original image verdicts are retained, with provenance documented in Appendix C .
Curriculum position
G2 lines
G3 primitives
G4 primitives
0
0
0
0
≈150
1,011
67
78
≈400
3,707
184
140
668
5,111
251
130
Table 2: Observed memory checkpoints. G2 counts curator-authored operational memory, excluding automatically generated indexes and process logs; G3/G4 count bank primitives. These are architecture-specific units, not comparable information quantities. G4’s final decrease reflects pruning.
Figure 3: Batch-matched advantage over the memoryless control. Points show observed success-rate differences; lines show linear fits with pointwise 95% confidence bands. Circles denote G2 − G1 and triangles G3 − G1. Bootstrap contains 18 items; subsequent batches contain 50.
Group
Failed
Retried
Recovered
Perfect
Frozen
Never retried
G2 notes
418
45 (10.8%)
12 (26.7%)
0
33 (7.9%)
89.2%
G3 code bank
430
56 (13.0%)
6 (10.7%)
0
50 (11.6%)
87.0%
G4 grid + sup.
336
87 (25.9%)
33 (37.9%)
4
54 (16.1%)
74.1%
G5 code + sup.
309
107 (34.6%)
27 (25.2%)
3
80 (25.9%)
65.4%
Table 3: Retry outcomes. Retried is a share of failed items; Recovered is a share of retried items; Perfect counts failed items recovered to the top tier; Frozen and Never retried are shares of failed items. The control (449 failures) has no curator and hence no retries. G4 additionally re-attempted five already-passing items hunting Perfect upgrades (two succeeded); these are excluded from the failure partition. Diagnostic reruns under altered prompts are excluded from this accounting.
Figure 4: Retry behavior. Left: proportion of initially failed items selected for retry. Center: proportion of selected items recovered, with 95% confidence intervals. Right: items remaining unsuccessful after the curriculum.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Position
Change
Reason
150
Curators may freely restructure their own memory files within format constraints
Fixed organization was bottlenecking G2/G3
270
Retry cool-down and terminal freeze introduced
Immediate re-attempts were wasting budget
350
Obligation to reuse bank functions lifted; imports still available, fresh inline rendering equally allowed
Drawers deferred to imperfect bank entries
419
Four-tier rubric (Perfect/Close added) from batch B9
Binary rubric could not distinguish crude success from mastery
500
Bank-deviation channel with curator note
Let drawers bypass bad primitives without silent divergence
post-B11
G5 added (G3 format + supervision); G4 left unchanged
G4’s curator misread its setup instructions; injection found reaching G4 only (§ 4.2 )
Appendix
Table 8
Figure 5: The four items where the memoryless control alone exceeded Close. Top: reference. Bottom: the control’s render—typeset glyphs via system font, not drawn strokes.
Condition
Perfect
Pass
Close
Fail
G1
6
33
25
186
G2
5
39
34
172
G3
1
56
34
159
G4
51
57
41
101
G5
38
80
33
99
Appendix
Table 4: Four-tier first-attempt outcomes on the same 250 items from B9–B13 for every condition.