cs.AIOct 11, 2026

Visible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loops

Authors: Jianlin Chen, Wenhui Chen, Chi Man Vong

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

LLM optimization loops often show the proposer the candidates it has already tried and rejected, but leave it to the proposer to avoid them. We ask whether avoidance measured in a static probe predicts avoidance in the states an optimizer's own trajectory produces. In catalog-based loops where three open LLMs improve a GUI agent by naming one of 40 edits, a re-proposal is identified exactly by the edit's name. Clean base-state probes underpredict re-proposal in the proposers' own loops (macro-averaged loop-minus-probe rate 0.32 [0.27, 0.37]), and proposals of already evaluated harness-edit pairs consume 28-63% of valid proposer iterations. The two settings reject different edits: a probe's rejected edits are mostly ones the proposer did not choose, whereas a loop's are its own earlier choices, which remain preferred when the history is removed. Ordinary history lowers re-proposal for two proposers but raises it for the third, and the effects of failure evidence are model-specific. Stating the rejection status in one sentence lowers re-proposal in every model (-0.16 [-0.22, -0.09] relative to a neutral list of the same IDs). For catalog-based loops with exact edit identities, rejection avoidance should be measured on the rejected sets the loop produces.

Figures & tables

Figure 1: State-dependent re-proposal and the interventions used to localize it. H: no history; B, R, J, K, S: failure-evidence edits; N, F, P: neutral ID list, status sentence, status plus prohibition. Without history, XsX_{s} is strongly enriched, with no detectable excess over the proposer’s other earlier choices (post hoc; § 5.1 ). Failure-evidence edits have model-specific effects (§ 5.2 ). Stating rejection status lowers re-proposal in every model; a prohibition gives a further negative rate estimate, not significant after Holm correction ( pp 0.054; § 5.3 ). α=a−λ\alpha=a-\lambda is a conceptual accounting; channel-specific readings require Assumption 2 .
  Own loopsClean probe 
ProposerCat.nB/nn_{B}/nα^\hat{\alpha} [95% CI]nB/nn_{B}/nα^\hat{\alpha}Δ\Delta [95% CI]
Gemma-4-26B-A4BA22/130++ 0.43 [ −- 0.56, 1.21]0/30≤\leq −- 1.460.17 [0.08, 0.26]
Gemma-4-26B-A4BB34/156++ 0.30 [ −- 0.27, 0.84]0/30≤\leq −- 1.460.22 [0.13, 0.31]
Mistral-Small-3.2-24BA121/167++ 3.52 [3.27, 3.69]2/30−- 1.840.66 [0.52, 0.75]
Mistral-Small-3.2-24BB94/158++ 2.69 [1.98, 3.38]3/30−- 1.400.49 [0.31, 0.66]
Qwen3-VL-32BA16/161−- 0.89 [ −- 1.74, −- 0.30]0/30≤\leq −- 1.460.10 [0.05, 0.16]
Table 1: Probe and loop. nB/nn_{B}/n : re-proposals over eligible valid loop steps (6 runs per cell) or valid probe cells. α^\hat{\alpha} : rejection-enrichment log-odds with run-cluster CI; ≤\leq : exact one-sided 95% upper bound for a probe with no re-proposal. Δ\Delta : registered primary endpoint, loop rate minus probe rate, two-level cluster bootstrap CI.
  Original 120 statesFresh R4 intervention subset (120 states)
RoleConditionRateX−AX-A [95% CI]log ORRateX−AX-A [95% CI]log OR
ReferenceA: unchanged state0.34––0.30––
Baseline preferenceH: no history (matched)   0.51++ 0.21 [0.11, 0.31]++ 0.68
LocalizationM: same-harness probe0.30−- 0.05 [ −- 0.11, 0.01]−- 0.350.27Δ360\Delta_{360} −- 0.04 [ −- 0.07, −- 0.01] †−- 0.36 †
Failure evidenceB: failure examples removed0.15−- 0.19 [ −- 0.25, −- 0.13]−- 1.920.14−- 0.17 [ −- 0.23, −- 0.10]−- 1.49
 R: neutral text, approx. length-matched   0.24−- 0.07 [ −- 0.13, −- 0.01]−- 0.56
Table 2: State edits, by role. Rate: mean of state-level rates (3 calls per state). X−AX-A : state-level difference with stratified bootstrap CI ( † : 90% CI over 360 states for the equivalence test). log OR: Mantel–Haenszel, stratified by state. S: share of choices that follow a shuffled failure pair. For M, the displayed rate is from the 120-state subset; the daggered equivalence contrast and log OR ( Δ360\Delta_{360} ) use all 360 fresh states, on which the unchanged and reconstructed rates are 0.34 and 0.30. Subset-defined contrasts (T, on the nn states where a strict swap exists; J −- K in App. B ) are paired against A on the same subset; the overall A rate cannot be used to reconstruct them. Registered tests and hierarchical CIs: App. B .
Figure 2: Rejection-enrichment log-odds against a uniform choice (dashed); arrows: exact 95% upper bounds. H–P: conditions on the 120 fresh states (Table 2 ); state-cluster CIs.
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Registration fileRegistered (UTC)First paid call (UTC)USD
VRR_CONFIRM.json2026-10-04 06:49:052026-10-04 06:49:430.393
VRR_INTERVENE.json2026-10-04 06:49:052026-10-04 07:15:520.375
VRR_REASONING.json2026-10-04 06:49:052026-10-04 07:17:111.208
VRR_REASONING_AMEND1.json2026-10-04 08:34:012026-10-04 08:34:250.144
VRR_QUAL.json2026-10-04 06:49:052026-10-04 07:20:410.014
VRR_R2_REASONING.json2026-10-04 10:10:242026-10-04 10:12:040.282
Appendix
Table 3: Registration vs. first paid call (UTC) and spend per experiment.
ClaimWhereEvidenceFalsifierStatus
Loop re-proposal exceeds clean-probe re-proposal§ 5.1H1: Δˉ=\bar{\Delta}= 0.32 [0.27, 0.37]; H1b: 6/6 cellsCI includes 0; fewer than 4 cellsregistered, supported
M and A satisfy the preregistered ±0.10\pm 0.10 equivalence criterion§ 5.1H15: M −- A −- 0.04 [ −- 0.07, −- 0.01] (90%), 360 fresh statesCI outside ±0.10\pm 0.10registered, passes ±0.10\pm 0.10 , not ±0.05\pm 0.05
Ordinary history changes re-proposal relative to matched no-history§ 5.1H14a: A −- H −- 0.21 [ −- 0.30, −- 0.11]CI includes 0registered, supported pooled (decrease); reversed for Mistral
Failure deletion lowers re-proposal§ 5.2H2a −- 0.19; H13a −- 0.17 [ −- 0.23, −- 0.10]CI includes 0registered, supported (both rounds)
Failure content beyond approximate text-bulk control§ 5.2H13b: R −- A −- 0.07 [ −- 0.13, −- 0.01]Holm p≥0.05p\geq 0.05registered, not supported (Holm pp 0.071)
Choice follows shuffled failure attachment§ 5.2H13c: J −- K ++ 0.07; H13d: follow 0.03 →\to 0.05no differenceregistered, not supported
Appendix
Table 4: Claims, evidence and falsifiers. Registered tests are differences in rates (sign-flip tests, Holm within their family). H13–H16 were registered after the framework of § 3 .
TestContrastnnRate difference [95% CI]Hierarchical CIlog OR [95% CI]pHolmp_{\mathrm{Holm}}
H13aB −- A <0<0120−- 0.17 [ −- 0.23, −- 0.10][ −- 0.25, −- 0.08]−- 1.49 [ −- 2.21, −- 0.94]<< 0.001
H13bR −- A <0<0120−- 0.07 [ −- 0.13, −- 0.01][ −- 0.15, 0.02]−- 0.56 [ −- 1.16, −- 0.04]0.071
H13cJ −- K <0<065++ 0.07 [ −- 0.02, 0.16][ −- 0.03, 0.18]++ 0.53 [ −- 0.10, 1.24]0.928
H13dfollow S >> A102++ 0.02 [ −- 0.01, 0.05]––0.425
H13eP −- F <0<0120−- 0.05 [ −- 0.10, −- 0.01][ −- 0.11, 0.00]−- 1.15 [ −- 2.40, −- 0.31]0.054
H13fE −- A <0<0120−- 0.27 [ −- 0.32, −- 0.22][ −- 0.34, −- 0.20]−- 3.61 [ −- 5.05, −- 2.89]<< 0.001
Appendix
Table 5: Round-4 registered tests on fresh states. One-sided tests H13a–f (Holm over six), two-sided H14a–c (Holm over three); stratified and hierarchical (run, then state) bootstrap CIs; Mantel–Haenszel log odds ratios. H15: equivalence of M −- A within ±0.10\pm 0.10 (90% CI over 360 states). H16: provider-pinned regime comparison.
TestContrastnnEffect [95% CI]pHolmp_{\mathrm{Holm}}
H4athinking −- instruct (both 4000 tok.)40 states−- 0.40 [ −- 0.55, −- 0.25]<< 0.001
H4binstruct 4000 −- 200 tok.36 states++ 0.06 [ −- 0.06, 0.17]0.689
H5same-harness probe −- loop state117 states−- 0.05 [ −- 0.11, 0.01]0.177
H6aDO-NOT −- neutral list120 states−- 0.23 [ −- 0.29, −- 0.18]<< 0.001
H6bneutral list −- unchanged117 states−- 0.06 [ −- 0.13, 0.01]0.177
Appendix
Table 6: Round-2 registered tests. Paired differences in re-proposal rate over states; sign-flip tests, Holm within each registration (H4a/b; H5, H6a/b).
ModelCallsProvidersLargest shares
Gemma-4-26B-A4B329513NextBit 34%, Cloudflare 22%, DekaLLM 11%, …
Mistral-Small-3.2-24B29623Venice 57%, DeepInfra 34%, Parasail 9%
Qwen3-VL-32B30211Alibaba 100%
Qwen3-30B-A3B-Instruct3476Alibaba 39%, StreamLake 32%, SiliconFlow 10%, …
Qwen3-30B-A3B-Thinking2561Alibaba 100%
Qwen3-VL-8B (agent)26403PrimeIntellect 44%, Alibaba 43%, Parasail 13%
Appendix
Table 7: Serving providers per model over all paid calls except the pilot (from the per-call ledger).
ModelProviderF/N callsPaired statesF −- N [95% CI]
Gemma-4Cloudflare26/269−- 0.15 [ −- 0.33, 0.00]
Gemma-4Io Net14/135++ 0.07 [0.00, 0.20]
Gemma-4Makora18/187−- 0.33 [ −- 0.67, −- 0.05]
Gemma-4NextBit47/4516−- 0.11 [ −- 0.28, 0.01]
MistralDeepInfra18/119−- 0.11 [ −- 0.39, 0.11]
MistralParasail15/147−- 0.21 [ −- 0.79, 0.43]
Appendix
Table 8: Provider sensitivity of F −- N (post hoc). Valid F/N calls are counted within paired states, which have at least one valid call under both conditions from the same provider. Differences average state-level rates within each provider, with catalog-stratified 95% bootstrap CIs over paired states. Only groups with at least five paired states are shown.
ExperimentUSD
VRR-P0-10.3381
VRR-P0-1-probe0.0545
VRR-P0-20.3748
VRR-P11.1997
VRR-P1-cross0.1440
VRR-P1-probe0.0085
Appendix
Table 9: Spend per experiment (USD, from the per-call ledger).
Figure 3: Instruct vs. thinking (Qwen3-30B-A3B-2507). (a) Re-proposal rate in each model’s own loops. (b) The same 40 fresh states answered by the instruct model with a 200- or 4000-token cap and by the thinking model with a 4000-token cap: unpinned routing, one call per state (left), and both endpoints pinned to one provider with three seeds (hatched).
Figure 4: Raw re-proposal rates against the uniform-menu reference. Filled: own loops, run-cluster 95% CIs; open: clean probe. Red ticks: each loop’s reference rate ∣Xt∣/(40−∣ht∣)|X_{t}|/(40-|h_{t}|) ; dotted line: the probe’s reference, L/40L/40 .
Figure 5: State edits as differences in rate , pooled and per model (state counts in the legend), 95% bootstrap CIs over states.
State subsetnnM −- A [90% CI]Run-hierarchical [90% CI]
A and M identical117++ 0.000 [0.000, 0.000]–
A and M differ243−- 0.057 [ −- 0.097, −- 0.016][ −- 0.113, 0.003]
Gemma-4-26B-A4B98++ 0.005 [ −- 0.060, 0.068]–
Mistral-Small-3.2-24B71−- 0.099 [ −- 0.178, −- 0.019]–
Qwen3-VL-32B74−- 0.099 [ −- 0.164, −- 0.036]–
Appendix
Table 10: Same-harness reconstruction M −- A by whether the prompt changes (post hoc). The A/M equality was checked by rerendering all 360 prompts. Differences average state-level rates from valid replies; 90% catalog-by-model stratified bootstrap CIs, with a run-then-state hierarchical sensitivity for the changed subset. Per-model rows subdivide that subset. The registered H15 equivalence test remains the all-state contrast in Table 5 .
ContrastGemma-4-26B-A4BMistral-Small-3.2-24BQwen3-VL-32B
B −- A++ 0.03 [ −- 0.05, 0.12]−- 0.31 [ −- 0.45, −- 0.17]−- 0.22 [ −- 0.30, −- 0.13]
R −- A++ 0.05 [ −- 0.03, 0.14]−- 0.04 [ −- 0.18, 0.10]−- 0.21 [ −- 0.30, −- 0.12]
J −- K++ 0.17 [0.05, 0.30]++ 0.04 [ −- 0.17, 0.24]−- 0.02 [ −- 0.16, 0.12]
A −- H−- 0.57 [ −- 0.71, −- 0.42]++ 0.31 [0.12, 0.49]−- 0.36 [ −- 0.53, −- 0.19]
F −- N−- 0.14 [ −- 0.25, −- 0.04]−- 0.22 [ −- 0.37, −- 0.05]−- 0.11 [ −- 0.18, −- 0.05]
P −- F++ 0.00 [ −- 0.07, 0.08]−- 0.15 [ −- 0.26, −- 0.05]−- 0.01 [ −- 0.04, 0.00]
Appendix
Table 11: Round-4 state-level rate differences per model (fresh states; registered as descriptive, no multiplicity correction), with bootstrap CIs over states. The models differ in kind: deletion and content effects are small for Gemma-4, deletion acts mostly through non-semantic text bulk for Mistral, and the history raises re-proposal for Mistral only; the status sentence (F −- N) is negative for all three.
Figure 6: Stated reasons for re-proposals and avoiding choices: annotator 1 and the two blinded LLM annotators (30 replies per group; blinded pair κ=\kappa= 0.62). “Failure-motivated rationale” abbreviates the preregistered code failure-target attraction (App. G ), which describes the stated rationale, not a causal mechanism.
Proposer (states)B−AB-AC−AC-AD−AD-AE−AE-A
Gemma-4-26B-A4B (37)−- 0.11 [ −- 0.19, −- 0.05] (0.054)−- 0.01 [ −- 0.12, 0.09] (1.000)−- 0.19 [ −- 0.31, −- 0.09] (0.018)−- 0.16 [ −- 0.27, −- 0.05] (0.063)
Mistral-Small-3.2-24B (40)−- 0.31 [ −- 0.45, −- 0.17] (0.002)++ 0.03 [ −- 0.09, 0.15] (1.000)−- 0.11 [ −- 0.23, 0.01] (0.546)−- 0.57 [ −- 0.69, −- 0.43] ( << 0.001)
Qwen3-VL-32B (40)−- 0.14 [ −- 0.22, −- 0.06] (0.021)++ 0.10 [0.00, 0.20] (0.546)−- 0.05 [ −- 0.15, 0.06] (1.000)−- 0.16 [ −- 0.23, −- 0.10] ( << 0.001)
Appendix
Table 12: Per-model state-level rate differences on the original 120-state intervention round (registered secondary family; Holm correction over 12 contrasts; pHolmp_{\mathrm{Holm}} in parentheses). States with no valid reply under A are dropped from that model’s contrasts.
ProposerCat.ViewSeedElig.ValidBBKeptFinal dev
Gemma-4-26B-A4BAFULL202621530.850
Gemma-4-26B-A4BAFULL212717130.867
Gemma-4-26B-A4BAFULL222523830.850
Gemma-4-26B-A4BAW8202522230.850
Gemma-4-26B-A4BAW8212624130.850
Gemma-4-26B-A4BAW8222523530.867
Appendix
Table 13: Every replication loop. Elig.: steps with Xt≠∅X_{t}\neq\varnothing ; Valid: those with a valid reply; BB : re-proposals of a visible rejection; Kept: edits in the final harness; Final dev: final development accuracy.
ABKindCatalog-A description
E01RF1promptBriefly describe the screen before choosing an action.
E02ZQ7promptDouble-check your answer before replying.
E03NK3promptAccuracy is very important for this task.
E04WV1decodeLower the output token limit from 48 to 16 to save cost.
E05VG3promptIf you are unsure, choose the most prominent element on the…
E06YV2promptPut the final reply on the last line, prefixed with FINAL:.
Appendix
Table 14: The 40 edits. A: catalog-A ID; B: catalog-B code; kind; catalog-A description (truncated at 62 characters).
Line of workWhat is establishedAssumption we testHow we test it
LLM optimizers with shown history ( Yang et al., 2024 ; Agrawal et al., 2026 ; Lee et al., 2026 ; Lu et al., 2026 ; Yang et al., 2026 )showing tried candidates and failures helps searcha shown rejection is avoidedα\alpha in reached states; no-history baseline (§ 5.1 )
Static instruction following ( Zhou et al., 2023a ; Jiang et al., 2024 )compliance in one responseavoidance is a trait measured onceclean probe vs. own loop vs. reconstructed state (§ 5.1 )
Negative and omission constraints ( Castricato et al., 2024 ; Rana, 2026 ; Gamage, 2026 )prohibitions are fragile; prior pressure drives violationspressure is a fixed priorno-history baseline at the reached harness; status vs. prohibition (§ 5.3 )
Failure records and repetition ( Gumaan, 2026 ; Yang, 2026 ; Xu et al., 2022 )a visible failed action is repeated; “do not repeat” does not help (135M–1.7B)the failure record drives repetitionapproximately length-matched neutral text, attachment shuffle, status sentence (§ 5.2 – 5.3 )
Knowing vs. acting ( Kruthof, 2026 ; Dong et al., 2026 )models restate what they violatestated reasons explain choicesstated reasons vs. content controls (§ 5.4 )
Reasoning vs. compliance ( Fu et al., 2025 ; Li et al., 2025 )reasoning lowers static-instruction compliancethis generalises to all constraintsinstruct/thinking pair, provider-pinned (§ 5.4 )
Appendix
Table 15: Positioning matrix. Each line of work, what it establishes, and the assumption that a history-implied exclusion constraint puts under test.
ProposerStatesReplay (A)No failures (B)DO-NOT (E)
Gemma-4-26B-A4B271022
Mistral-Small-3.2-24B4839426
Qwen3-VL-32B441800
Appendix
Table 16: Earlier replay on violation-selected states (one call per state): states with a visible re-proposal.
More in this topic

Large Language Model-Guided Optimization

Large Language Model-Guided Optimization uses language models to propose, steer, and refine solutions to hard search and design problems.

See all 613 papers

Explore similar work

CardsList

  1. Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders

    Oct 13, 2026Xin Li, Hao Jiang, Jinghao Guo +2RL for RecommendationLarge Language Model-Based Recommendation

    Nanyang Technological University, Singapore

  2. Reasoning with Evidence, Not Merely Rationales: Verifiable Preference Proofs for LLM-Based Recommendation

    Oct 2, 2026Yu Hou, Nathaniel Kang, Pengkai Wang +1Large Language Model-Based RecommendationExplainable Recommendation

    School of Mathematics and Computing (Computational Science and Engineering), Yonsei University Seoul, Republic of Korea · School of Computer Science and Engineering, Kyungpook National University Daegu, Republic of Korea · Department of Electrical and Computer Engineering, Sungkyunkwan University Suwon, Republic of Korea +1

  3. Inheritance as an Admission Test for LLM Prompt Merge Operators

    Oct 21, 2026Zan-Kai Chong, Hiroyuki Ohsaki, Bryan NgLanguage Model MergingPrompt Engineering

    School of Science and Technology, Kwansei Gakuin University, Sanda, Japan · School of Engineering & Computer Science, Victoria University of Wellington, Wellington, New Zealand