cs.MAAug 27, 2026

GVS5H: Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance

Authors: Victor Gao, Vida Khosrowshahi, Ali Khosrowshahi, Xihao Sun, Juhyun Lee, Simon, Lee

Organizations: Sang Won · Persis Capital Inc.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Frontier coding performance is typically attained with large, costly proprietary models. We introduce ledger-based zero-shot self-orchestration (GVS5H), a training-free method in which fresh instances of one model decompose problems and coordinate through a shared file system. Across eleven open and closed-weight models on the 100 latest hard LiveCodeBench problems, the method yields as much as 25.6 points improvement, boosting several cheaper models to frontier-level performance. Orchestrated Qwen3.8 Flash Next scores 93.0% against Fable 5's 90.4% at 9% of the cost, while the smaller Qwen3.8-27B reaches 92.4%. Gains are not universal: some models are unchanged or worse. Transcript analysis attributes the gain to decomposition and persistent context. Inference-time organization can reach or exceed frontier coding accuracy at a fraction of the cost on self-hostable weights.

Figures & tables

ModelSingle callWith managerΔ\Delta vs singleppΔ\Delta vs Fable 595% boundpp
Claude Fable 590.4±1.490.4\pm 1.4———reference——
GPT-5.6-Terra80.8±1.080.8\pm 1.088.0±1.288.0\pm 1.2+7.2+7.28.0×10−58.0\times 10^{-5}−2.4-2.4−5.8-5.80.880.88
GPT-5.6-Luna70.4±4.670.4\pm 4.681.2±2.881.2\pm 2.8+10.8+10.8<2.5×10−5<2.5\times 10^{-5}−9.2-9.2−13.4-13.40.00130.0013
Qwen3.8-27B66.8±5.266.8\pm 5.292.4±2.392.4\pm 2.3+25.6+25.6<2.5×10−5<2.5\times 10^{-5}+2.0+2.0−0.8-0.80.880.88
Qwen3.8-Flash-Next84.2±1.884.2\pm 1.893.0±1.293.0\pm 1.2+8.8+8.8<2.5×10−5<2.5\times 10^{-5}+2.6+2.6−0.1-0.10.560.56
DeepSeek-V4.1-Flash88.4±2.688.4\pm 2.692.0±1.892.0\pm 1.8+3.6+3.60.0020.002+1.6+1.6−1.3-1.30.880.88
Table 1: Manager vs. single call, five passes. pass@1 on LCB-100 at a 128k cap, reasoning on; ±\pm is a 95% tt interval across the five passes (df = 4). Δ\Delta is in percentage points, against the model’s own single call and against Fable 5’s single call. Each pp is a paired sign-flip permutation test, unit = problem ( n=100n=100 ), Holm-corrected within its family of five; << marks the permutation floor. 95% bound is the one-sided lower bound on Δ\Delta vs Fable 5 over the same per-problem differences (t, df = 99): the largest deficit the data leave open.
ArmRate $/MTok in / outIn (MTok)Out (MTok)$/pass$/solvedpp vs singlepp vs Fable 5
Qwen3.8-27B single0.21/0.21 / 2.550.07537.4247$18.95$0.28——
Qwen3.8-27B manager0.21/0.21 / 2.551.505318.6277$47.82$0.521.5×10−41.5\times 10^{-4}6.1×10−46.1\times 10^{-4}
Qwen3.8-Flash-Next single0.15/0.15 / 0.470.07535.7751$2.73$0.032——
Qwen3.8-Flash-Next manager0.15/0.15 / 0.471.293011.8380$5.76$0.0623.5×10−43.5\times 10^{-4}3.6×10−73.6\times 10^{-7}
DeepSeek-V4.1-Flash single0.30/0.30 / 1.200.06802.8626$3.46$0.039——
DeepSeek-V4.1-Flash manager0.30/0.30 / 1.201.499312.4873$15.43$0.171.6×10−41.6\times 10^{-4}2.2×10−92.2\times 10^{-9}
Table 2: What one pass cost each arm. List rate ×\times the tokens it consumed. The two pp columns test each manager arm against its own single call and against Fable 5’s single call: Welch over the five passes, Holm-corrected across 11 comparisons. Qwen3.8-27B’s single arm is the 128k cap-matched one; at 250k it costs 26.95/pass.Retriedanddiscardedattemptsarecounted:theyweregeneratedandwouldbebilled.ThecheapestarmisGPT−5.6−Lunasingleat26.95/pass. Retried and discarded attempts are counted: they were generated and would be billed. The cheapest arm is GPT-5.6-Luna single at 0.41 a pass and the most accurate is Qwen3.8-Flash-Next manager at 5.76—a145.76 — a 14 \times$ spread in price for +22.6 points.
ModelCap hits singleCap hits managerNo code s / mof which refusalspass@1 s / mEmitted only s / mCap
GPT-5.6-Terra0 / 5000 / 3,3920 / 0080.8 / 88.080.8 / 88.0128k
GPT-5.6-Luna0 / 5000 / 3,4980 / 0070.4 / 81.270.4 / 81.2128k
Qwen3.8-27B150 / 500108 / 3,23535 / 0066.8 / 92.471.8 / 92.4128k a
Qwen3.8-27B (250k)124 / 500—21 / —070.0 / —73.1 / —250k a
Qwen3.8-Flash-Next51 / 50092 / 3,14120 / 0084.2 / 93.087.7 / 93.0128k
DeepSeek-V4.1-Flash11 / 50049 / 3,52510 / 1088.4 / 92.090.2 / 92.2128k
Table 3: Cap hits and empty solutions per arm. A cap hit is a generation that stopped at the token limit ( finish_reason=length ). No code counts any run containing no code, due to a cap hit or other reason.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

 Single callWith manager
ModelAs gradedΔ\Delta harnessΔ\Delta keysRe- portedAs gradedΔ\Delta harnessΔ\Delta keysRe- ported
Pinned backends, five passes
Claude Fable 587.4+0.0+0.0+3.0+3.090.4————
GPT-5.6-Terra69.8+8.4+8.4+2.6+2.680.872.6+12.6+12.6+2.8+2.888.0
GPT-5.6-Luna60.4+8.0+8.0+2.0+2.070.470.4+8.4+8.4+2.4+2.481.2
Qwen3.8-27B62.4+2.6+2.6+1.8+1.866.884.4+5.4+5.4+2.6+2.692.4
Appendix
Table 4: Every reported arm as the stock evaluator graded it, and after each correction (Appendix B ). As graded is LiveCodeBench’s own verdict on the stored generations. Harness is the move contributed by running each submission as a subprocess, which is what repairs the sys.stdin mock and sys.stdout.buffer ; keys is the move contributed by judging the four problems that accept more than one answer by the contest’s rule. Reported is the number used everywhere else in this paper. Upper block: five-pass means at a 128k cap, reasoning on (Table 1 ). Lower block: the 128k reasoning-on, one-pass OpenRouter arms (Table 6 ); Qwen3.5-9B is unusable in this condition and is omitted. Across the 48 arm-passes of the two remaining OpenRouter conditions the harness correction moves six by one point and the key correction moves each arm by 0 to 3 points. Qwen3.8-27B’s single call is the one row where as graded is not the stock evaluator on the code being reported: that arm is cap-matched by replaying its 250k generations at 128k (Appendix A ) and the truncated solutions were never put through the stock evaluator, so the column is the 250k grading and the two Δ\Delta columns carry the cap match as well as the corrections. Against that baseline the row gains 35 problem-passes and loses 13, eight of them generations whose 128k prefix contains no code at all.
ModelManager onlySingle call onlyProblem-passesExact McNemar pp
GPT-5.6-Terra47115002.0×10−62.0\times 10^{-6}
GPT-5.6-Luna72185008.1×10−98.1\times 10^{-9}
Qwen3.8-27B13465001.4×10−321.4\times 10^{-32}
Qwen3.8-Flash-Next4955003.9×10−103.9\times 10^{-10}
DeepSeek-V4.1-Flash2245005.3×10−45.3\times 10^{-4}
Appendix
Table 5: Problem-passes won by one arm alone. Of the 500 problem-passes per model (100 problems ×\times 5 passes), the number the manager arm solved and the single call did not, against the number the single call solved and the manager did not. The gains are not compensating wins and losses. The test is exact McNemar on those discordant pairs (Section 2.2 ).
ModelParams128k ⋅\cdot ON (1 pass)128k ⋅\cdot OFF (1 pass)16k ⋅\cdot OFF ( ×\times 5)
Opus-5n/a88 →\to 95 ( +7 )––
Kimi-K3∼2.8{\sim}2.8 T86 →\to 86 (+0)33 →\to 76 ( +43 )33.4 →\to 65.0 ( +31.6 )
MiniMax-M3428B62 →\to 69 ( +7 )26 →\to 40 ( +14 )22.2 →\to 33.2 ( +11.0 )
Qwen3.6-35B35B26 →\to 44 ( +18 )35 →\to 27 ( −- 8)28.2 →\to 27.4 ( −- 0.8)
Qwen3.5-9B9Bunusable18 →\to 21 (+3)15.0 →\to 22.6 ( +7.6 )
Appendix
Table 6: LCB-100 pass@1 (%), single →\to manager. For the OpenRouter-served models on the original scaffold. “–” = not run; Qwen3.5-9B reasoning-on returns reasoning-only replies and is unusable. Scores are from the corrected grader (Appendix B ): the four LCB-100 problems whose reference answer is not unique are judged by the original contest rule, and submissions run as real subprocesses so sys.stdout.buffer behaves as it does on the contest judge. A bolded delta clears the ∼4.5{\sim}4.5 point pass-to-pass band. The eleven pinned-backend, five-pass arms are reported separately in Section 3 and are not pooled here, because both the serving path and the scaffold version differ (Section 2.2 ).
Model (condition)ProblemHow the single call failedWhat the scaffold added
Qwen3.5-9B (128k, off)abc385_dTimed out: a naïve per-segment scan of a path simulation.The brainstorm flagged in writing that the check is O(N⋅M)≈4×1010O(N\cdot M)\approx 4\times 10^{10} and prescribed a range-query structure before any code - the up-front planning a single pass skips (Section 4 ).
Qwen3.6-35B (on)abc394_fCut off mid-reasoning at 32,768 tokens; returned nothing.Solved in one worker cycle: the brainstorm fixed the structural insight (an “alkane” subtree needs a degree-4 center and ≥5\geq 5 vertices); the worker used an iterative DFS to avoid a recursion-depth failure. The same 32k clamp was hit 117 times across this model’s manager calls, but with several attempts per problem one clamped call no longer costs the problem.
MiniMax-M3 (on)3701Printed nothing on cdcd : the lexicographic-reconstruction branch was broken.Split the two coupled objectives across worker cycles - one worker on the forward cost DP over capped run-lengths, a separate worker on a suffix DP used purely for the lexicographic reconstruction.
Kimi-K3 (128k, off)3687Over-counted the longest unique-value path (9 and 3 where the answers were 6 and 2), never shrinking its window on a repeated value.The brainstorm wrote the reduction into notes.md first - longest substring without repeating characters along each root-to-leaf path, window start via last-seen depths, O(n)O(n) DFS. Only then did a worker implement it.
GPT-5.6-Terra (on)36884,615 characters implementing a Segment-Tree-Beats-shaped structure indexed by candidate value, and wrong.The plan named the difficulty in advance - the state must be compressed rather than maintaining all distinct values per index. Nine calls later, the manager arm returned 1,860 characters over one standard segment tree. The contribution is subtraction . Manager-only win in four of five passes.
GPT-5.6-Luna (on)abc397_eCarried a set of unfinished path lengths per subtree across 4,092 characters, a general state it never closed.The brainstorm established the bounding lemma (at most two unfinished paths at any vertex), after which a worker implemented the postorder scan directly in 2,166 characters over five calls. Manager-only win in two of five passes.
Appendix
Table 7: One transcript per model where the single call failed and the manager arm solved the problem (Section 4 ). LCB problem IDs as in release_v6; call counts are manager-arm model calls. Character counts are of the emitted program.

Explore similar work

CardsList

  1. Embarrassingly Simple Self-Distillation Improves Code Generation

    Apr 1, 2026Ruixiang Zhang, Richard He Bai, Huangjie Zheng +3Code GenerationLarge Language Model Generation

    Apple

  2. FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale

    May 14, 2026Runyuan He, Qiuyang Mang, Shang Zhou +14Large Language Model GenerationFrontiers

    UC Berkeley · UC San Diego · University of Washington +4

  3. Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models

    Sep 24, 2026Ehsan Barkhordar, Surendrabikram ThapaCode QualityMle-Bench Lite

    Koç University · Virginia Tech