Retaining Repair Hypotheses for Cost-Aware Execution

Test-time compute has emerged as an effective paradigm for improving large language model capability at inference time. Existing allocation strategies primarily prioritize tasks according to difficulty, uncertainty, or expected performance gain, implicitly treating prediction errors as equally costly. This assumption is often misaligned with real deployment, where failures can differ substantially in their downstream tasks. To address this limitation, this paper introduces consequence-aware test-time compute allocation by formulating a cost-weighted scheduling problem where the priority of a task is its failure consequence with the marginal gain of additional compute. In practice, however, marginal gain is difficult to predict before execution, so we propose a deployable scheduler that uses consequence as the routing signal. The scheduler predicts task consequence from pre-solution inputs and allocates the available premium compute to the corresponding top-ranked tasks. Experiments on the SWE bench Lite show that consequence provides information beyond task difficulty and can be predicted before solving. Under a fixed compute budget, consequence-aware routing achieves the best high-consequence task success, while overall accuracy remains competitive. A controlled within-model experiment further confirms the same advantage when only inference attempts are reallocated.

Language-model agents can repair analog layouts when sign-off tools check every edit, but which model should plan a repair, which should edit it, and at what cost remain open questions. In our tool-grounded repair agent, models only choose bounded layout operators; sign-off tools rerun each task's contract checks after every edit, so failing layouts are never accepted. On layout-versus-schematic (LVS), design-rule checking (DRC), and post-layout repairs of three designs (a production high-voltage digital-to-analog converter (DAC), a 5-V oscillator, and a SKY130 operational transconductance amplifier (OTA)) with six tier-matched models from two vendors, 182 blind runs show that cheap models execute plans but rarely succeed alone (the cheapest model: 1 of 11 runs alone, 16 of 17 under a flagship planner; p < 0.001), cheap planners fail more often, and no fixed assignment is cheapest on every task. We propose a learning controller that selects models for each role and the escalation order from the first tool evaluation and updates Bayesian estimates after every attempt. With rules and initial estimates frozen before testing, the controller fixed 7 of 8 episodes on four sealed held-out tasks, matching the OpenAI flagship alone, at 15-40% of the flagship's spend per fix on three tasks but 8% higher spend overall. The cheapest mid-tier model alone fixed 6 of 8 at the lowest overall spend per fix but missed the hardest task. All 32 extracted DAC repairs match the human-guided LVS repair's integral nonlinearity (INL) within 0.005 least significant bits (LSB) at every code.

A repair action proposed by a large language model (LLM) specifies which records to change and how to change their values. A failed action may still contain a useful choice of records or a useful update, depending on how the two are paired. We introduce a counterfactual evaluation framework that separates selected records (targets) from the applied change (an update). Four executions combine proposed and reference targets and updates to measure the effects of correcting or retaining each component and their interaction. We evaluate the framework on two tasks: traffic assignment, where affected demand must be identified from road counts, and database repair, where affected rows must be identified from aggregate reports. An injected fault supplies a reference action that reverses it, and the recovery rate measures an action's loss reduction relative to that reference. For sparse-target actions in the traffic experiments, retaining the proposed update on reference targets achieves 79.4% of the reference loss reduction, compared with 43.7% from tuning update strength on the proposed targets. In the database experiments, update replacement worsens 17 of 108 actions, all selecting unaffected records, with sharply different frequencies across models. The value of a proposed update depends on the targets it is paired with. Missed targets hide useful updates, while excess targets can make a milder update better than the reference correction. In a component-reuse study, testing new target-update pairings from two proposals raises mean held-out recovery rate to 0.616, compared with 0.580 when selecting between the original proposals.