Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation

Roberto I. Ono Filho

cs.CLAug 28, 2026

Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation

Authors: Roberto I. Ono Filho

Organizations: Independent researcher

Abstract

What does a generation loop gain from learning on its own verified successes? In cycles of generate, verify, select and LoRA-consolidate on online bin packing, training on value-filtered candidates shifts what the model writes on held-out variants toward value (-1.7 points of excess, p=0.008; -3.1 against a random-consolidation control, p=0.004) while the best observed candidate converges to the classic heuristic's level and no further. A confirmation battery replicates the whole procedure three times, with fresh seeds and a never-consulted held-out set read exactly once: the mean was nearly identical in all three lineages (-2.0, -1.8, -1.9), and after aggregating within held-out variant all seven evaluable variants favored consolidation (p=0.008). The best observed candidate moved to the classic heuristic's level, exactly (0.021028, the lowest variant's classic level, in all three lineages and both arms); attract never went beyond its variant's classic level, while the random control did in 7 of 24 lineage-variant cells. A matched SFT-only control shows the supervised anchor, not repulsion from bad candidates, does the concentrating (96% of candidates land exactly at the classic heuristic's level). On far families the tails cut both ways: consolidation lowers the per-candidate rate of better-than-classic candidates (10% to 3.9%) while its larger production yields more such candidates absolutely (5 against 1, on few events). As motivation we report the inference-time ledger that led here: a model-written schematic recap buys judged document integration and nothing buys development; a verifier written into the stream is imitated, 16.4 fabricated verdict lines per notebook. In this arrangement (an 8-bit model in MLX, five cycles), mean quality among valid candidates can be bought and replicated; under attraction the observed best goes to the classic and, so far, never beyond it.

Figures & tables

arm (Qwen3-30B-A3B-Base, period 300)	integration	development	coherence	surprise
habituation, no interruption	1.40	1.10	1.50	1.40
verbatim context (clock300)	0.80	0.40	1.50	0.50
reset + new subject	2.00	1.60	2.20	2.50
schema: reset + recap + new subject	2.70	1.80	2.30	2.70
question: preserved + open question	1.50	1.10	1.90	1.00
agenda: reset + recap + open question	2.50	2.00	1.90	2.60

Table 1: Document-level reading (whole 4,500-token stream, injected text removed; Opus 5,

k=3

; cell means over ten premises, 0–10). The schematic recap gives the highest integration of the program; no arm passes 2.0 on development.

arm	cycle	candidates	mean excess	rel. to min(BF,FF)	best	distinct	copies
base	0	65	0.0633	+0.0322	0.0408	0.99	–
base	1	53	0.0562	+0.0251	0.0327	1.00	–
attract	1	235	0.0559	+0.0248	0.0319	0.98	10%
random	1	276	0.0708	+0.0397	0.0332	1.00	5%
base	2	42	0.0583	+0.0272	0.0317	0.97	–
attract	2	198	0.0430	+0.0119	0.0312	0.99	6%

Table 2: Battery C on the eight held-out variants (never in any training set; test instances): candidates per arm and cycle, mean test excess of valid candidates (cell means over variants), the same relative to the better of best fit and first fit, the best candidate, the distinct/valid ratio and the share of literal copies of the arm’s training set. The value-filtered adapter lowers the mean; its best reaches the classic level and stays there.

Figure 1: The title of this paper, as data (held-out variants, never in any training set; cycle 0 is the shared pre-consolidation base). Left: mean excess over the lower bound falls under attraction and nowhere else. Right: from the second cycle the trained arms’ best stays within 0.0005 of the 0.0311 classic level, while the base’s best fluctuates above it (0.0317–0.0414). Consolidation moves the mean; attraction reaches but does not exceed the classic level, while the random control dips below it at cycles 2 and 4.

	held-out variants (8)				far families (6)
adapter (cycle-5 lineage)	cand.	mean exc.	at classic	levels	mean exc.	$>$ classics
base (none)	47	0.0556	20%	0.81	0.0838	10.0%
random	287	0.0695	16%	–	0.1004	0.0%
repel-early (DPO, 20 steps)	71	0.0721	34%	–	0.0994	2.4%
attract (SFT on value)	226	0.0386	72%	0.14	0.0803	3.9%
repel-anch (DPO + anchor)	144	0.0311	100%	0.07	0.0758	0.0%

Table 3: The ladder of adapters. Held-out: mean test excess of valid candidates, share scoring exactly at the level of the better classic heuristic, distinct score levels per valid candidate; far families: mean excess and share of candidates better than the classics. Means and shares are averaged over variants (held-out) or families (far), as in Table 2 ; candidate counts are totals. The mass walks onto the known attractor; attraction lowers the far-family tail rate per valid candidate while raising absolute tail yield through production. Rows below attract are single-lineage mechanistic follow-ups.

Figure 2: What consolidation costs. Left: the share of candidates better than the stronger classic heuristic on the far families, the tail where discovery lives, shrinks from 10% (base) to 3.9% (attraction) to at most 0.6% under every tail-preserving objective we built. Right: on held-out variants the consolidated arms concentrate exactly at the classic’s level (attraction 72%, anchored repulsion 100%, a functional collapse), against 20% for the base. The bar heights are per-family average rates (the registered estimand); the counts above them are pooled events over all candidates, a different aggregation (base: 10.0% macro but 1/18 = 5.6% pooled; attraction: 3.9% macro, 5/156 = 3.2% pooled). Both rest on 0–5 events per arm and are preliminary. Per notebook, attraction’s absolute tail yield is higher than the base’s (0.28 vs 0.06). Competence per candidate is bought with idiosyncrasy; total yield depends on production.

lineage	arm	candidates	valid	mean excess	best	$\Delta$ vs base
1	base	71	63	0.0612	0.0247
1	attract	201	184	0.0416	0.0210	$-0.0196$
1	random	219	191	0.0619	0.0210	$+0.0007$
2	base	79	51	0.0605	0.0243
2	attract	162	140	0.0430	0.0210	$-0.0176$
2	random	232	205	0.0548	0.0210	$-0.0057$

Table 4: The replication battery on the never-consulted held-out set (means averaged over the seven common variants;

\Delta

= per-lineage variant-averaged difference to base; best = minimum over all held-out variants). Three independent runs of the whole procedure produce three nearly identical effects on the mean and the same movement of the best: to the classic level, in every arm that trains, and never beyond it under attract.

Figure 3: Independent replication, variant by variant: each line pairs one held-out variant’s mean under base and under attract, one panel per lineage; the right panel shows the best candidate. The slope repeats across lineages; the best moves to the classic level in all three lineages and stops there.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

battery	registered	data seen	primary	status
S (selection)	2026-08-19	paper-1 batteries	held-out gain	null
M/M+ (memory, tension)	2026-08-20	paper-1, S	schema vs verbatim	supported
V (verifier)	2026-08-20	M, S	held-out gain	null; fabr. verdicts
C cycles 1–3	2026-08-21	none	attract vs base (c3)	not supported ( $p=0.11$ )
C cycles 4–5	2026-08-21	c3 result	attract vs base (c5)	sequential, $p=0.008$
C-repel, N (tails)	2026-08-21/22	C	keep the tails	degenerate / not kept

Appendix

Table 5: Order of registrations and looks (dates from the commit history; M+ was registered after the M result). The cycle-5 result is a prospective sequential extension, not an independent confirmation; the independent lineages with never-consulted held-outs are the confirmation battery.

Inference-Time Optimization

Inference-time optimization adapts model computation, outputs, or runtime settings to improve quality under deployment constraints.

See all 97 papers

May 23, 2026

Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction

Major LLM platforms deploy models in an inference-only configuration: the model serves requests but never updates per-user weights. Users must repeatedly re-teach preferences, corrections, and project context, and context-based workarounds consume context-window space and degrade under cascading compaction. We evaluate an alternative: nightly consolidation of interaction knowledge into model weights via reflection, synthesis, and Low-Rank Adaptation (LoRA) fine-tuning on a single consumer GPU. Across ten realistic software development conversations (n = 10, 1,146 test questions across three memory types), three cycles of cascading compaction retain 36.8 +/- 3.0% of knowledge (between an 11.8% no-context floor and a 90.1% full-context ceiling), while consolidation retains 80.4 +/- 1.3% -- a 43.6 pp gain (paired t(9) = 14.8, p < 0.001) that more than doubles what compaction preserves, with the largest gains on procedural corrections (36.3% -> 74.6%) and episodic project facts (31.5% -> 78.2%). As a methodological aside, mean per-token validation cross-entropy is negatively correlated with LLM-judged accuracy (r = -0.51) while median per-token validation cross-entropy tracks accuracy almost exactly (r = +0.99): under evaluators that tolerate surface-form variation, the mean is misleading and a heavy-tail-robust statistic is the faithful signal. Persistent personalization requires moving beyond inference-only deployment toward architectures that consolidate knowledge into weights.

Simon Dennis, Kevin Shabahang, Hao Guo +1

i14 / University of Melbourne · i14

Continual Learning for LLMs LLM Personalization LLM Inference

Ask PDF

Oct 1, 2026

ReSolve: Reusing Candidate Reasoning through Selective Generative Moderation

Sampling multiple solutions spends computation on intermediate deductions and unfinished arguments as well as final answers. We introduce ReSolve, a training-free inference procedure that reuses this candidate reasoning through selective generative moderation. An answer-distribution controller invokes a model to examine existing derivations when candidates disagree or lack a parseable answer, then incorporates the generated solution into a bounded loop. Under Hybrid scoring on 130 competition-mathematics problems evaluated with two independently sampled candidate pools, ReSolve obtains 100 and 99 correct answers, compared with 91 and 92 for voting over the same four candidates, with no correct-to-incorrect changes relative to that vote in either pool. Eight-sample self-consistency obtains 94 and 96 correct answers while consuming substantially more tokens; ReSolve uses 46.3% and 47.2% fewer tokens in the two evaluations. A controlled ablation removes visible derivations while retaining answer keys, vote counts, and the per-state output-cap rule, reducing accuracy from 100 to 93 correct despite increasing computation. Selective and always-on Uniform moderation both solve 97 problems, while selectivity reduces moderation tokens by approximately 54% and total pipeline tokens by 6.2%. These results support candidate reasoning as reusable inference computation. They do not establish an accuracy advantage over additional sampling or a distinct benefit from specialized route instructions.

Bangji Yang, Jiajun Fan, Hongbo Ma +6

University of Illinois at Urbana-Champaign · Rutgers University · University of Illinois Chicago +2

Efficient Language Model Reasoning Inference-Time Compute Allocation Efficient Language Model Inference LLM Mathematical Reasoning

Ask PDF

Jun 2, 2026

Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics

Best-of-

N

inference scaling (drawing

N

candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by an amount that varies across models, but predicting that amount in advance currently requires running the procedure end-to-end. Prior work links cheap statistics of a model's sampled outputs and validation-set correctness (how often samples agree, how diverse they are, how confident the model is, and where correct samples appear) to model behavior, but does not isolate which of these form a stable, compact predictor of best-of-

N

gain. We fit ridge predictors on features computed from a single labeled validation-set sampling pass, use bootstrap-Lasso as a stability analysis of the candidate feature set, and give a concentration analysis with an explicit linear-approximation residual. Across three base-model families, six post-training methods, and math and reasoning task domains, the stability analysis identifies a strict three-feature core spanning prompt-level agreement spread, label-assisted first-correct-sample position, and completion-length variance; a compact ridge predictor built from this core plus an entropy add-on reaches Spearman

ρ= 0.90

with actual best-of-

N

gain under a reward-model verifier. The intended use is labeled validation-set screening of candidate configurations before paying the full reward-model scoring cost.

Luyang Zhang, Jingyan Li

Carnegie Mellon University · Johns Hopkins University

LLM Evaluation Language Model Performance Prediction Inference-Time Scaling Best-of-N Sampling

Ask PDF

Abstract

Inference-Time Optimization

Explore similar work

Beyond Inference-Only Deployment: Comparing Weight-Based Consolidation Against Cascading Compaction

ReSolve: Reusing Candidate Reasoning through Selective Generative Moderation

Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics