Does reflective LLM feedback survive a wet-lab-scale label budget?

Haoxuan Zeng; Xin Luo

cs.AINov 1, 2026

Does reflective LLM feedback survive a wet-lab-scale label budget?

Authors: Haoxuan Zeng, Xin Luo

Organizations: University of Michigan, Ann Arbor, Michigan, USA

Abstract

Can reflective language-model feedback improve a biological predictor when each cycle buys only one plate of labels? We separate three possible sources of benefit: proposing model edits, choosing measurements, and revising decisions through repeated feedback. We replay $36$ ProteinGym assays, starting with $96$ labels and acquiring $96$ per round for six rounds, with a sealed test set and a shared edit schema. The pre-specified comparison of full LLM control against random edits detects no advantage in test rank correlation ( $Δz=-0.008$ , $95\%$ CI $[-0.034,+0.018]$ ); equivalence is also unestablished. Exploratory ablations nevertheless identify useful model editing. With random acquisition fixed, LLM edits improve Fisher-transformed test Spearman by $0.043$ over no edits; with acquisition fixed to upper confidence bound, they improve normalized top-variant recovery by $0.023$ . LLM acquisition alone does not detectably outperform fixed adaptive rules, and repeated feedback does not detectably outperform one initial recommendation under full control. Thus, the observed gains support constrained specialist editing, while the incremental value of recursive reflection remains unestablished at this label budget.

Figures & tables

Figure 1: How model editing, acquisition, and feedback affect the two objectives. (a,b) Improvement over no edits as labels accumulate; shading gives pointwise

95\%

intervals across

36

assays, with seeds

0

–

2

averaged within assay. The gray window marks rounds 3–6 used for endpoint analysis. (c,d) Paired endpoint differences and

95\%

t

intervals; positive values favor the first arm. Only the marked ranking comparison is confirmatory. LLM + random/UCB fixes acquisition while allowing model edits.

Controller	Acquisition	Mean $z$	$R_{96}$
No edits	Random	0.603	0.349
Random edits	Editable	0.597	0.359
Local search ( $5\times$ )	Editable	0.623	0.354
Full LLM	Editable	0.590	0.391
LLM, one call	Chosen once	0.600	0.390
LLM model edits	Random	0.647	0.358

Table 1: Random split,

108

runs per arm, averaged over rounds 3–6. Acquisition is prescribed unless marked editable or chosen once.

R_{96}

is normalized top-variant recovery; Appendix C reports all

20

complete arms.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Information flow in the budgeted loop. Purchased-label CV informs model edits and the controller. Non-acquisition edits pass an acceptance gate; acquisition follows a separate path and reveals the next

96

labels. The sealed test set is used only for external scoring.

Controller	Mean $z$	$\Delta z$ vs. no edits	$R_{96}$
Fixed div.-greedy	0.597	$-0.007$	0.384
Fixed greedy	0.566	$-0.038$	0.380
Fixed UCB	0.589	$-0.014$	0.383
No edits	0.603	$+0.000$	0.349
LLM acq. only	0.591	$-0.012$	0.377
LLM full	0.590	$-0.014$	0.391

Appendix

Table 2: Random split: all

20

complete arms,

108

runs per arm. Unqualified LLM arms use the configured Opus 5 controller.

Controller	Mean $z$	$\Delta z$ vs. no edits	$R_{96}$
Fixed div.-greedy	0.402	$+0.000$	0.285
Fixed greedy	0.404	$+0.003$	0.288
Fixed UCB	0.402	$+0.001$	0.285
No edits	0.401	$+0.000$	0.276
LLM full	0.461	$+0.060$	0.319
LLM no menu	0.453	$+0.052$	0.293

Appendix

Table 3: Position-disjoint split: all

11

complete arms,

108

runs per arm.

Controller	Mean $z$	$\Delta z$ vs. no edits	$R_{96}$
Fixed div.-greedy	0.591	$-0.011$	0.383
Fixed greedy	0.564	$-0.038$	0.382
Fixed UCB	0.584	$-0.018$	0.385
No edits	0.602	$+0.000$	0.354
LLM full	0.587	$-0.015$	0.392
LLM + fixed UCB	0.573	$-0.029$	0.405

Appendix

Table 4: All

11

complete random-split arms with seeds

0

–

4

180

runs per arm, averaged over rounds 3–6. Eligibility follows Appendix A .

Field	Allowed values	Default
Curation
drop_outliers.method	none, iqr, zscore	none
drop_outliers.k	[1.0, 4.0] (real)	3.0
dedup_by_position	False, True	False
reweight	none, inverse_zs_confidence, fitness_decile_balanced	none
clip_targets	none, winsorize_1, winsorize_5	none

Appendix

Table 5: Allowed edits and default values. Numeric ranges are inclusive.

Assay	Length	Variants	Readout	Organism
A0A247D711_LISMN_Stadelmann_2021	87	1,653	Activity	Listeria monocytogenes
A4GRB6_PSEAI_Chen_2020	266	5,004	Fitness	Pseudomonas aeruginosa
AACC1_PSEAI_Dandage_2018	177	1,801	Fitness	Pseudomonas aeruginosa
BLAT_ECOLX_Firnberg_2014	286	4,783	Fitness	Escherichia coli
CALM1_HUMAN_Weile_2017	149	1,813	Fitness	Homo sapiens
CASP3_HUMAN_Roychowdhury_2020	258	1,567	Activity	Homo sapiens

Appendix

Table 6: Assay metadata. Length counts residues; variants are single substitutions.

LLM Self-Refinement

Large language model self-refinement studies how models can improve outputs through repeated critique, verification, revision, and feedback.

See all 117 papers

Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds
Jul 31, 2026Yefan Tao, Gerald Friedland, Madhusudhanan Chandrasekaran +1LLM Self-Correction LLM Self-Refinement
Amazon
Language Models Learn to Improve Language Models
Oct 16, 2026Minhua Lin, Zijun Wang, Zewen Liu +7Language Model Self-Improvement Large Language Model-Guided Optimization
The Pennsylvania State University · UC Santa Cruz · Emory University +2
In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration
Oct 1, 2025Youngbin Choi, Minjong Lee, Saemi Moon +4LLM Reliability Multi-Turn Language Model Reasoning
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH

Jul 31, 2026J/K move · Enter open · S save

Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds

Yefan Tao, Gerald Friedland, Madhusudhanan Chandrasekaran, Luyang Kong

Amazon

LLM Self-Correction LLM Self-Refinement Language Model Self-Assessment LLM Reliability Metacognition in Language Models Self-Reflection in Language Model Agents

Open paper AskPDF arXiv

Reflection, the ability to revisit and revise prior reasoning, is central to how humans improve their answers. Large language models (LLMs) are increasingly prompted to "reflect," yet whether this resembles human revision remains unclear. We introduce the Human-LLM Reflection Framework (HRF), a controlled two-pass protocol comparing human and LLM revision under identical conditions across self-, peer-, and cross-agent settings. Using an information-theoretic analysis based on per-iteration cross-entropy reduction, we find two failure modes of LLM reflection. On objective tasks with finite answer spaces, reflection yields near-zero information gain (Delta I approx 0), behaving as neutral re-generation indistinguishable from re-sampling. On subjective tasks, it yields significant negative gain (Delta I < 0), moving predictions away from the target. Human revision, by contrast, yields positive gain in both settings. Cross-agent experiments localize the failure to the revision step, not input quality: LLMs degrade even high-quality human responses. Diagnostic analyses (revision conditioned on first-pass correctness, and oracle-guided revision against a random-reshuffle baseline) show that which sub-step dominates varies by task and by model rather than reducing to a single mechanism: self-error detection is present on objective multiple-choice tasks but weak on subjective ones, and recovery under an oracle error signal exceeds the baseline for some models and falls below it for others. The unifying account is structural: without external information, self-conditioned revision cannot reduce uncertainty about the target, so LLM reflection is better understood as conditioned re-generation than as genuine error-driven revision.

Abstract

LLM Self-Refinement

Explore similar work

Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds

Language Models Learn to Improve Language Models

In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration