Compile-Gated Lean Proof Refactoring with Enterprise Agent Practices: A Model per Step under a US$3 Cap, Followed by Programmatic Rewrite Rules
Organizations: Vexorium
Abstract
Our closed-track entry (Arena username Vexorium-Agent) scored 86.31 points on the final board (L = 77.07, H = 81.86, Z = 100) over the 60 final problems, against 33.33 for the reference proofs, 8th at the deadline (top: 92.57), for US$166.33 at list prices with every call counted (per problem: mean US$2.772, at most US$2.874; US$3 cap). Three practices carry the entry. First, each paid stage runs the model that measured best per dollar for it among the 9 we tested in our own harness: GPT-6.1 Sol the early sessions, GPT-6.1 Terra the later ones and the tail at 70% of Sol's cost per turn, and Gemini 4 Argon the review and repair; Sol also runs the style rewrite. Second, what our development and research runs learned ships as compiler-checked programmatic rewrite rules rather than prompt text (+0.93 points from rules fixed before the release; +0.26 and +0.30, in sample, from passes built or switched on after the release). Third, every call is recorded and reviewed before each change (all 9 board scores came within -0.063 to +0.015 points of our meter's prediction). The harness, gate and many of these rules also serve our open-track entry. We report the full bill and the spread of three runs.
Figures & tables
| Role | OpenRouter id | Output cap | List price |
|---|---|---|---|
| Sessions to $1.25; style rewrite | openai/gpt-6.1-sol | 2,001–12,000; 23,139 | 2.00 / 10.00 |
| Sessions to $2.75; tail | openai/gpt-6.1-terra | 2,001–12,000 | 1.00 / 5.00 |
| Review; repair | google/gemini-4-argon | 16,000; 8,000 | 2.00 / 10.00 |
| Component | Model | Sessions | Calls | US$ |
| Sessions to $1.25 | GPT-6.1 Sol | 300 | 7,847 | 64.74 |
| Sessions 2.75 | GPT-6.1 Terra | 461 | 15,510 | 89.53 |
| Tail | GPT-6.1 Terra | 17 | 363 | 2.20 |
| Review | Gemini 4 Argon | 138 | 4.78 | |
| Repairs | Gemini 4 Argon | 279 | 2.86 | |
| Style rewrite | GPT-6.1 Sol | 73 | 2.21 |
| Stage | Paid by | Run 1 | Run 2 | Run 3 |
|---|---|---|---|---|
| Reference proof (score) | – | 33.33 | 33.33 | 33.33 |
| CPU auto-brief + sessions to $1.25, cleanup | Sol | 48.28, 0.38 | 48.13, 0.43 | 47.81, 0.44 |
| Sessions to $1.90, cleanup | Terra | 1.81, 0.10 | 1.70, 0.10 | 2.19, 0.08 |
| Sessions to $2.75, cleanup | Terra | 1.22, 0.08 | 1.26, 0.07 | 1.31, 0.08 |
| Review, repair, stack | Argon | 0.15 | 0.18 | 0.15 |
| Tail | Terra | 0.01 | 0.00 | 0.01 |
| Decision | Evidence (alternatives tried in italics) | Choice |
|---|---|---|
| Model per step | Terra vs Sol throughout on later steps from the same 15 states (US22.61): [ 0.113, +0.654]; end to end, Sol then Terra vs Sol throughout [ 0.12, +0.46]. Terra first 2.06 (4 problems), Opus 5.5 second 1.62 (5; no interval). Sonnet 5.5 first : 72.75 vs Sol 83.19, at 2.6 Sol’s cost/turn. Argon as main model 84.29 at about US2.57 (5 hard problems). | Sol to $1.25, then Terra |
| One budget ladder | A fresh Sol plan at \Delta=-0.42-2.30 vs US--+0.116--$ 0.972, +0.091] held out. | one ladder, no stop rule |
| Review | Multi-round summed over 15 problems (US+0.100.43); net of the end stage [+0.001, +0.069] (45 problem-runs); repair + stack vs neither [+0.00, +0.13], 3 up, 0 down, 86 % of it on one problem. Other reviewers (Kimi K3, Astra): Appendix B . | Argon, 3 rounds, repair + stack |
| Late extras | Against none of them : tail [+0.001, +0.056] (US+0.7500.56–0.63 spent, but on the warm-up with the budget left in the final runs. Approach pick [+0.001, +0.024] out of sample (64 samples, 0 worse), below its pre-registered bar. | tail, style rewrite and, by the author’s decision, approach pick |
| End passes (post-release) | Final, 12 problems, paid bets (Terra at a stall, a planned restart, a profiler fix) vs control: [ 1.37, +1.64], so the bets were dropped. Final, +0.333+0.285$ on the problems the rules came from. | the three $0 end passes |
| Upload | Runs 85.709 / 85.785 / 86.009 ( run 1 or 2 ); a per-problem union of runs is not reproducible by the code. With debug.skipKernelTC on 4 problems the board read 87.13 vs 86.24 without it (same run, an earlier post-step); the option turns off Lean’s kernel check. | run 3 + post-step, no skip: 86.31 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Problem | Sol | Terra | Argon | Total | At cap price | Score | ||
|---|---|---|---|---|---|---|---|---|
| PutnamBench | ||||||||
| putnam_1964_a4 † | 1.239 | 1.516 | 0.097 | 2.852 | 2.949 | 97.43 | 94.5 | 97.8 |
| putnam_1995_a3 † | 1.225 | 1.530 | 0.094 | 2.848 | 2.942 | 97.05 | 97.6 | 93.6 |
| putnam_1988_a2 | 1.066 | 1.695 | 0.070 | 2.831 | 2.901 | 96.85 | 94.8 | 95.7 |
| putnam_2001_b1 | 1.184 | 1.525 | 0.117 | 2.827 | 2.944 | 96.60 | 94.1 | 95.7 |
| putnam_1964_b2 † | 1.153 | 1.670 | 0.051 | 2.874 | 2.925 | 99.29 | 98.6 | 99.3 |
| Start | Data | Tested | Outcome |
|---|---|---|---|
| Before the release: engine and first capped pipeline | |||
| 10-06 | warm-up | vendor agent CLIs or our own harness for hosted models | our own harness (one tool set, every call metered) |
| 10-09 | warm-up | Claude Code with Opus 5.5 and Codex with GPT-6 Astra, uncapped, resumed passes | both plateaued; their general practices fed our harness; their proofs are in no package |
| 10-10 | warm-up | free lexer-level rewrites | adopted as a $0 normalizer inside cleanups and the end stage |
| 10-10 | warm-up | why GPT-6.1 Sol beats our best proofs on some problems | sessions fenced off from other proofs; a check that each proof starts from the reference; briefs with facts, no proofs |
| 10-11 | warm-up | reasoning effort and tool-output size against cost | Sol at medium effort with full tool output |
| Item | Pin |
| Code repository | https://github.com/bobbercheng/lean-refactor-closed |
| Code archive there | final/ at tag closed-final-sub41-20261104 |
| Code archive (sha256) | abc1b0e29dcd29d37ddd8889f35265d57f5233f9310cb3eca8882d672fec877a |
| Run 3 code (the archive) | commit 402c44d2 , 2026-11-02 15:37 UTC |
| Run 2 / run 1 code | 0ea568e2 (2026-11-02 12:23 UTC) / b5f036ab (2026-10-31 18:42 UTC) |
| Post-step code | both passes ran commit 715cd39d ; the archive carries 93959dcb , the same ops plus one that our passes did not run |
| Op | Version | What it proposes | Steps | Problems | Gain |
|---|---|---|---|---|---|
| implied | 1 | a decide -family closer decides the hypothesis of a library implication an environment query finds | 1 | 1 | 0.102 |
| closerprep | 1 | conv => reduce before a decide -family closer, with a minimal drop of the simp list | 0 | 0 | 0.000 |
| inlineeq | 1 | an equation proved inline in a simp list, rewritten from its own hint | 0 | 0 | 0.000 |
| hintgrind | 2 | a tactic span replaced by the closer Lean prints for its goal ( grind? , simp? ) | 1 | 1 | 0.002 |
| projpath | 2 | a destructuring pattern becomes one name and projection paths ( t.2.1 ) | 15 | 12 | 0.038 |
| lexsurf | 2 | lexer-cheap surface rules (leading-dot constructors, inferable ascriptions dropped) | 15 | 15 | 0.038 |
| Sent (UTC) | What | Ours | Board | ||
|---|---|---|---|---|---|
| 2026-11-02 00:44 | run 1’s output (a probe) | 85.72 | 85.66 | 75.62 | 81.38 |
| 2026-11-02 03:00 | run 1’s state early in the run, uploaded on purpose as a lower placeholder score (a probe) | 70.33 | 70.30 | 51.89 | 59.01 |
| 2026-11-02 18:11 | run 2’s state while it ran (a probe) | 80.89 | 80.89 | 67.76 | 74.91 |
| 2026-11-03 05:56 | run 3’s state while it ran (a probe) | 84.84 | 84.85 | 73.85 | 80.71 |
| 2026-11-03 07:21 | run 2’s output | 85.78 | 85.80 | 75.52 | 81.87 |
| 2026-11-03 17:23 | run 1’s output after a first post-step with the kernel-check skip on some problems | 86.90 | 86.85 | – | – |