Coding Agents for Coding Theory
Organizations: Stanford University
Abstract
We spent five weeks using an LLM coding agent on open problems in coding theory: finding large sets of four-letter words, such as DNA barcodes, that stay far apart in edit distance. The agent wrote the verifiers and search code; a human chose the problem and set the verification protocol. Restricting the search to codes with a prescribed symmetry, a classical technique, shrank the problem about fourfold and raised the best known code of length 6 and minimum edit distance 3 from 114 to 120 words (). The same pipeline improved twelve further lower bounds at lengths 6 to 9 and distances 3 to 6. We give the failures equal space. Our own search stopped at 116 and recorded the last symmetry class as topping out at 112; a second agent session, running the same search with a better operator, found the 120. A later verdict that the method did not carry over to length 7 was wrong for the same reason, and an earlier instance cost three weeks. Each time, an intermediate result was written down, never rechecked, and treated as a fact that ruled out further search. Checking final outputs, as our protocol required, does not catch such errors.
Figures & tables
| the three runs | operator vs. budget | |||||
|---|---|---|---|---|---|---|
| run | core-min | best | calib. | budget | calib. | |
| 1 | original operator | 0 6 | 100 | 0 99 | 00 6 (broken op.) | 0 99 |
| 2 | repaired operator | 60 | 112 | 110 | 00 6 | 109 |
| 3 | later sweep, same op. | 20 | 116 | 109 | 0 15 | 109 |
| 0 20 | 109 | |||||
| 0 60 | 110 | |||||
| cell | published | new lower bound | class | orbits | seeds |
| 114–176 | 120 | (free) | 1021 | 8/96 at 1 min | |
| 356–614 | 364 | 2218 | 2/26 at 2 min | ||
| 28–32 | 30 | diagonal, order 4, optimum | 518 | 3/3 at 26 s | |
| 65–128 | 72 | diagonal, order 8 | 1277 | 11/11 at 1–2 min | |
| 1132–2340 | 1148 | (free) | 16381 | 1/20 at 10 min | |
| 3451–9360 | 3760 | diagonal (free) | 65531 | ReduMIS, 2/7 at 15–30 min |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Decision or component | Source |
|---|---|
| Problem: open cells of the quaternary edit-metric table | Human |
| Protocol: verifier-first rules and escalation order | Human |
| Redirects between sessions | Human |
| Prescribed-symmetry search, a classical method [ 4 ] | Agent, within the protocol’s escalation order, which as shipped lists it second; we cannot date that entry against the agent’s first use |
| Verifiers, search code, orbit-graph builders, and C implementations of the ARW operator and NuMVC | Agent |
| Recorded conclusions, including the ceiling and “did not transfer” | Agent |