PID-Gated Continual Prompt Adaptation Under Noisy Feedback

Zhentao Xu

cs.LGOct 12, 2026

PID-Gated Continual Prompt Adaptation Under Noisy Feedback

Authors: Zhentao Xu

Organizations: Independent Researcher, Sunnyvale, CA, USA

Abstract

Deploying LLM agents in non-stationary environments requires continually revising their instructions as the task drifts, yet rewriting after every reported mistake can turn noisy feedback into persistent errors. This raises an often-overlooked scheduling question in prompt adaptation: when should feedback trigger a lasting instruction update, and at what editor-call cost? Taking inspiration from proportional-integral-derivative (PID) control, we introduce a family of lightweight gates that react to the current error, accumulate evidence over time, and roll back deterioration after an edit. We benchmark these gates against both simple baselines and advanced prompt optimizers (TextGrad and GEPA), over a diverse suite of drifting tasks, LLM parameter sizes, feedback-noise levels, and random seeds. PID-gated continual prompt adaptation yields a more favorable Pareto trade-off between task performance (high accuracy, low error) and editor-call budget than the baselines considered, pointing to control-theoretic scheduling as a simple, general tool for robust prompt adaptation under unreliable feedback.

Figures & tables

Panel of Figure 1 — Figure 1: The regulated textual feedback loop. Signals enter from outside: target $y_{t}$ from the left, input $x_{t}$ from above. The Sensor flips the label with probability $\eta$ (labels are either categorical, as in Parity, or bucketed categorical, as in Guess number / moving target). The comparator produces the observed error $e_{t}$ (binary for Parity, bucketed distance for numeric tasks); when the controller fires, editor $g$ rewrites the prompt $\theta$ .

Figure 2: Cost-quality snapshot at

\eta=0.1

on Qwen3-14B (one marker per method, mean over 5 seeds). Higher accuracy and lower MAE/RMSE are better; fewer calls are cheaper. Points near the top-left (or bottom-left, for MAE/RMSE) indicate a better cost-quality trade-off.

	Parity		Guess number		Track moving target
Policy	Acc.(%) $\uparrow$	Calls $\downarrow$	MAE $\downarrow$	Calls $\downarrow$	RMSE $\downarrow$	Calls $\downarrow$
No Edit	$47.8\pm 5.8$	$0.0\pm 0.0$	$17.61\pm 3.05$	$0.0\pm 0.0$	$19.42\pm 2.05$	$0.0\pm 0.0$
Fixed Cadence	$68.5\pm 10.1$	$7.0\pm 0.6$	$12.17\pm 3.54$	$8.0\pm 0.0$	$19.70\pm 4.10$	$8.0\pm 0.0$
TextGrad	$88.0\pm 4.4$	$34.8\pm 13.1$	$5.20\pm 1.47$	$105.0\pm 21.1$	$8.33\pm 1.57$	$165.0\pm 3.3$
GEPA	$79.7\pm 6.4$	$44.4\pm 7.6$	$9.18\pm 1.55$	$142.4\pm 18.3$	$10.12\pm 0.91$	$159.6\pm 9.2$
Proportional Gate (P)	$\mathbf{90.2\pm 3.6}$	$20.2\pm 5.1$	$4.70\pm 2.76$	$63.8\pm 14.3$	$7.14\pm 1.42$	$110.2\pm 5.5$

Table 1: Primary results on Qwen3-14B at

\eta=0.1

(

10\%

feedback corruption, 5 random seeds). Bold marks the best mean per column, excluding No Edit and Fixed Cadence.

	Parity		Guess number		Track moving target
Policy	Acc.(%) $\uparrow$	Calls $\downarrow$	MAE $\downarrow$	Calls $\downarrow$	RMSE $\downarrow$	Calls $\downarrow$
No Edit	$47.8\pm 5.8$	$0.0\pm 0.0$	$17.61\pm 3.05$	$0.0\pm 0.0$	$19.42\pm 2.05$	$0.0\pm 0.0$
Fixed Cadence	$61.0\pm 15.7$	$7.2\pm 0.7$	$14.59\pm 4.48$	$7.8\pm 0.4$	$20.03\pm 5.50$	$8.0\pm 0.0$
TextGrad	$87.3\pm 5.9$	$34.8\pm 13.9$	$\mathbf{3.92\pm 1.39}$	$99.6\pm 20.2$	$7.03\pm 1.22$	$160.8\pm 8.8$
GEPA	$79.7\pm 6.4$	$44.4\pm 7.6$	$8.27\pm 1.75$	$107.0\pm 10.6$	$10.31\pm 0.98$	$161.6\pm 3.5$
Proportional Gate (P)	$89.2\pm 3.7$	$22.6\pm 6.4$	$3.93\pm 1.34$	$67.6\pm 19.5$	$6.45\pm 1.20$	$108.0\pm 3.9$

Table 2: Scaling-up ablation on Qwen3.8-27B at

\eta=0.1

(

10\%

feedback corruption, 5 random seeds). Bold marks the best mean per column, excluding No Edit and Fixed Cadence.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Cost–quality tradeoff across all three feedback-noise levels on Qwen3-14B (mean over 5 seeds). Marker face shade encodes noise: hollow

\eta=0

, half-filled

\eta=0.1

, solid

\eta=0.2

. Lines connect the three noise points for each method. Points near the top-left (bottom-left for MAE/RMSE) indicate a better cost–quality trade-off. PID gates sit on a better frontier than TextGrad and GEPA at every noise level.

Symbol	Meaning
$t,\,T$	Step index and horizon; $T=120$ steps per trajectory.
$x_{t},\,y_{t}$	Input and hidden ground-truth target; $y_{t}$ never reaches the editor.
$\hat{y}_{t},\,\tilde{y}_{t}$	Predictor output $\hat{y}_{t}=f(\theta_{t},x_{t})$ and observed (possibly corrupted) feedback.
$e_{t}$	Observed error: $\mathbf{1}[\hat{y}_{t}\neq\tilde{y}_{t}]$ for categorical tasks, or a bucketed distance for numeric ones.
$\theta_{t}$	Current prompt (plant state); the editor’s target of change.
$f,\,g$	LLM predictor $\hat{y}_{t}=f(\theta_{t},x_{t})$ and LLM editor $\theta_{t+1}=g(\theta_{t},\mathcal{H})$ (invoked when the gate fires).

Appendix

Table 3: Recurring symbols.

PID Control

See all 9 papers

RECAP: Regression Evaluation for Continual Adaptation of Prompts
Jun 4, 2026Harsh Deshpande, Kushal Chawla, Sangwoo Cho +2Continual Learning for LLMs LLM Evaluation
Capital One
Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs
Oct 13, 2026Mikhail Aleksandrov, Nail Bashirov, Maksim Ivanov +6Prefix Tuning LLM Prompting
BRAIn Lab
Prior or Feedback? What an LLM Uses When Adapting Neural Operators
Oct 8, 2026Julian Chan, Javier Mora JimenezPartial Differential Equations LLM Decision-Making
Compare the Market

Jun 4, 2026J/K move · Enter open · S save

RECAP: Regression Evaluation for Continual Adaptation of Prompts

Harsh Deshpande, Kushal Chawla, Sangwoo Cho, William Campbell+1

Capital One

Continual Learning for LLMs LLM Evaluation Instruction Following

Open paper AskPDF arXiv

Production agentic systems routinely face evolving constraints and must comply from the very next interaction. Scenarios like a tool-call notification changing a compliance threshold or a policy update adding disclosure requirements fit this criteria, having close to no room for errors in production. This proactive adaptation setting is common in deployment, but absent from current benchmarks, which assume either static constraint sets or reactive protocols with evaluation feedback. We introduce RECAP, a benchmark that measures continual-learning phenomena (forgetting, regression, forward transfer) at the constraint level under a strictly proactive adapt-then-test protocol: prompt optimization methods receive only the constraint specification and must generalize before seeing any test data. Evaluating six methods across five LLMs and three schedules with evolving constraints, we find that these methods show no significant improvement in performance, even after incurring a higher latency. These methods, designed for offline or reactive settings, are inadequate for the proactive paradigm. Our work emphasizes the growing need for designing proactive prompt adaptation methods, where the models must remain robust to evolving needs in deployment.

Abstract

PID Control

Explore similar work

RECAP: Regression Evaluation for Continual Adaptation of Prompts

Not All Drift Is a Mechanism: Tracing Prompt Adaptation in LLMs

Prior or Feedback? What an LLM Uses When Adapting Neural Operators