cs.LGOct 9, 2026

History, Adaptation, and Hidden Gain: Two Prospective Tests of Learned Predictive Control in a Synthetic Neural Population

Authors: Mahesh Reddy Pagadala

Organizations: Aston University, Birmingham, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We study learned one-step predictive control of a synthetic population of 64 leaky integrate-and-fire neurons whose light sensitivity is scaled by a hidden factor s, using two prospective confirmations on plants simulated only after the analysis was frozen. In the first (50 plants, 11 conditions), removing the 200-ms input-output history substantially worsened a parameter-count-matched neural predictor at low and moderate gain (s = 0.6, 0.8; RMSE rose by 2.47 Hz at s = 0.6, 97.5% CI 2.43-2.51). In secondary comparisons, recursively adapted ARX had the lowest mean error of the deployable controllers in 10 of 11 conditions, tuned PI with anti-windup matched or beat the network for s >= 0.7, and training for 10,000 rather than 900 epochs lowered prediction error but worsened control. Development work suggested the network's learned action sensitivity did not track the hidden gain. A second protocol (50 new plants, 9 sensitivities, 2 noise levels) confirmed three pre-specified predictions at the primary noise level. Re-estimating a single multiplicative input gain recovered 87% (95% CI 86-88%) of the benefit of full adaptive ARX. Supplying that online estimate to the network as an extra input reduced high-gain tracking error by 1.84 Hz [1.81, 1.88]. Longer training worsened the plain network by 1.09 Hz but improved the gain-input network by 1.07 Hz, a differential of 2.16 Hz [2.13, 2.20]. Under high process noise (sigma = 2.0) the effects shrank to about 0.04 Hz or less and gain-only adaptation recovered only 24% of the adaptive-ARX benefit. In this synthetic plant, at the primary noise level, one-parameter online gain estimation accounts for most of the gap between a history-based network and full adaptive ARX, and for its prediction-control mismatch.

Figures & tables

ArmDefinitionProtocol
PI tunedpositional PI, Kp=0.02K_{p}=0.02 , Ki=0.20K_{i}=0.20 , integrator clip ±10\pm 10 (development grid of 400 gain pairs)v3, v4
PI tuned + AWas above with back-calculation anti-windup, Kp=0.02K_{p}=0.02 , Ki=0.25K_{i}=0.25 , kt=0.5k_{t}=0.5v3, v4
PI sens. ref.best PI of the family when ss is known; not deployable, reference onlyv3
ARX fixedleast-squares prior, no online updatev3, v4
ARX + RLSall ten coefficients adapted by RLS, P0=109 Σ0P_{0}=10^{9}\,\Sigma_{0} , λ=0.98\lambda=0.98v3, v4
ARX- ggone-parameter gain RLS (Section 3.4 ), P0=1000P_{0}=1000 , λ=0.98\lambda=0.98v4
Table 1: Controllers. Network arms have five initialisations each, averaged within plant. Every gain, epoch and checkpoint was fixed on training, validation or development plants before the corresponding confirmation.
ProtocolIDContrastLevel / decision
v3P1Mem-141 −- H4-10k at s=0.6,0.8s=0.6,0.897.5% each; both lower bounds >0>0
v3P2ARX fixed −- H4-10k at s=0.6s=0.695%; interval within ±0.25\pm 0.25 Hz
v3B1PI tuned (+AW) −- H4-10k, 11 conditions99.77% each; no direction
v3S1, S3–S5RLS effect, chronology shuffle, ARX+RLS −- H4-10k, H4-900 −- H4-10k97.5–99.55%; exploratory
v4Q1gain fraction, σ=0.5\sigma=0.595%; lower bound ≥0.70\geq 0.70
v4Q3H4-10k −- I2, mean over H\mathcal{H} , σ=0.5\sigma=0.595%; lower bound >0>0
Table 2: Pre-specified tests. Secondary results do not change primary decisions. Development values for v4 came from plants 5200–5249.
ConditionPIPI+AWARXARX+RLSH4-10kH4-900Mem-141PI ref. †
s=0.50s=0.504.3162.8442.8732.7692.7842.7945.5722.664
s=0.60s=0.601.8701.6701.6921.5671.6421.6274.1111.471
s=0.70s=0.701.5041.3111.3941.2731.4121.3593.1191.226
s=0.80s=0.801.2641.1281.2781.0801.3841.2182.4251.124
s=0.90s=0.901.1261.0691.3531.0021.5421.2061.9661.070
s=1.00s=1.001.0841.0751.7070.9711.8861.3051.7481.075
Table 3: Protocol v3: mean tracking RMSE (Hz) over 50 confirmation plants (7000–7049). Lowest of the deployable arms shown in bold. † Uses knowledge of ss ; excluded from bold. A legacy PI with phase-1 gains ( Ki=0.15K_{i}=0.15 , not shown) was 0.004 Hz lower than ARX+RLS at s=1.4s=1.4 and higher elsewhere.
Figure 1: Protocol v3: mean tracking RMSE against hidden sensitivity at σ=0.5\sigma=0.5 . The memoryless network (1.7–5.6 Hz) is omitted for scale; all values are in Table 3 .
ConditionPI tuned −- H4-10kPI + AW −- H4-10k
s=0.50s=0.50+1.532 [+1.485, +1.569]+0.060 [+0.045, +0.076]
s=0.60s=0.60+0.228 [+0.209, +0.248]+0.028 [+0.008, +0.046]
s=0.70s=0.70+0.092 [+0.070, +0.110]−- 0.101 [ −- 0.122, −- 0.082]
s=0.80s=0.80−- 0.119 [ −- 0.146, −- 0.097]−- 0.256 [ −- 0.285, −- 0.233]
s=0.90s=0.90−- 0.416 [ −- 0.456, −- 0.379]−- 0.473 [ −- 0.511, −- 0.431]
s=1.00s=1.00−- 0.801 [ −- 0.851, −- 0.759]−- 0.810 [ −- 0.860, −- 0.762]
Table 4: Protocol v3: PI minus H4-10k RMSE (Hz) with simultaneous 95% intervals (99.77% each). Negative values favour PI.
IDCriterionDevelopmentσ=0.5\sigma=0.5Decisionσ=2.0\sigma=2.0
Q1lower bound ≥0.70\geq 0.700.860.871 [0.863, 0.878]supported0.24 [0.17, 0.32]
Q3lower bound >0>0+1.84 Hz+1.84 [1.81, 1.88]supported+0.042 [0.033, 0.054]
Q4lower bound >0>0+1.0 to +3.8 Hz ∗+2.16 [2.13, 2.20]supported+0.018 [0.008, 0.027]
Table 5: Protocol v4 primary decisions at σ=0.5\sigma=0.5 (plants 8000–8049; 95% BCa intervals) and the same statistics at σ=2.0\sigma=2.0 (98.3% intervals; robustness, not decisions). ∗ Development values at s=1.0s=1.0 and 1.41.4 ; no high-gain-set mean was recorded.
σ=0.5\sigma=0.5 (primary)
ssPI+AWARXARX+RLSARX- ggH4-10kH4-900I1I3I2
0.502.8722.9012.8062.8572.8222.8252.8512.8082.792
0.601.6681.6961.6081.6911.6381.6271.6471.5951.571
0.701.3171.4001.2901.4211.4211.3621.3621.3211.269
0.801.1311.2611.0881.2711.3971.2251.2061.1791.095
0.901.0621.3670.9851.2061.5491.2101.1581.1601.037
Table 6: Protocol v4: mean tracking RMSE (Hz) over 50 plants. ARX- gg : one-parameter gain RLS; I1–I3: interventions. Lowest in each row in bold. H4 and I2 at fixed epochs are in Figure 4 .
Figure 2: Protocol v4: mean tracking RMSE against hidden sensitivity for six arms. Shading marks the pre-specified high-gain set. All arms are in Table 6 .
Figure 3: Gain estimate gg (mean of the last 60 steps of ARX- gg ) against ss for each of the 50 v4 plants (thin lines) and their mean, σ=0.5\sigma=0.5 .
ssH4-10k −- I2 (Hz)plants >0>0
1.00+0.89 [+0.85, +0.93]100%
1.10+1.36 [+1.32, +1.41]100%
1.20+1.90 [+1.84, +1.97]100%
1.40+3.21 [+3.15, +3.28]100%
Table 7: H4-10k −- I2 at each high-gain setting ( σ=0.5\sigma=0.5 ; 98.75% intervals) and the share of plants with a positive difference.
Figure 4: Matched-epoch comparison at σ=0.5\sigma=0.5 : plain DR-H4 and gain-input I2 at 900 (dashed) and 6000 (solid) epochs. Shading marks the high-gain set.
ssI2 −- ARX+RLSI2 −- PI+AWH4-10k −- I1H4-10k −- I3
0.50−- 0.015 [ −- 0.040, +0.007]−- 0.081 [ −- 0.096, −- 0.067]−- 0.030 [ −- 0.038, −- 0.021]+0.014 [+0.006, +0.022]
0.60−- 0.037 [ −- 0.080, −- 0.009]−- 0.097 [ −- 0.113, −- 0.078]−- 0.009 [ −- 0.023, +0.006]+0.043 [+0.031, +0.056]
0.70−- 0.021 [ −- 0.056, +0.004]−- 0.048 [ −- 0.065, −- 0.031]+0.058 [+0.044, +0.070]+0.099 [+0.085, +0.112]
0.80+0.008 [ −- 0.012, +0.028]−- 0.036 [ −- 0.051, −- 0.021]+0.191 [+0.171, +0.214]+0.218 [+0.202, +0.236]
0.90+0.052 [+0.025, +0.076]−- 0.025 [ −- 0.047, −- 0.003]+0.391 [+0.364, +0.431]+0.389 [+0.360, +0.423]
1.00+0.043 [+0.009, +0.066]−- 0.038 [ −- 0.055, −- 0.020]+0.712 [+0.675, +0.748]+0.657 [+0.622, +0.690]
Table 8: Protocol v4 secondary contrasts at σ=0.5\sigma=0.5 (Hz; 99.44% intervals). For I2 −- X, negative favours I2; for H4-10k −- X, positive favours X.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

StagePlantsOutcome
v0.1–v0.6 (exploratory)0–69, 1000–1049, 1400–1649learned controller brittle to gain shift; history helps
C0 confirmation3000–3049chronology effect +2.1 / +5.4 Hz
Experiment A4000–4049matched history ablation +2.76 Hz at s=0.6s=0.6
Experiment B6000–6049linear equivalence at s=0.6s=0.6 ; PI later found under-tuned, RLS inactive
Phase-2 development5000–5149PI search, adaptive RLS, 10 000-epoch retraining, rehearsal
v3 confirmation7000–7049Section 4
Appendix
Table 13
More in this topic

Neural Network Control

See all 29 papers

Explore similar work

CardsList
Oct 20, 2026

When Does Feedback Help? Auditing Learned Voltage-Clamp Experimental Design

A learned scientific controller can outperform a misspecified planner without benefiting from feedback. We audit two synthetic voltage-clamp tasks with a training-selected fixed protocol, current masking, a Bayesian reference, and an excluded target. Five recurrent policies repeat one fixed protocol in all 1,200 off-grid rollouts; masking current changes no decision. In a crossed four-hypothesis task, a fixed diagnostic pulse is followed by one learned choice. Policies then use the first current, but tie a correctly specified Bayesian controller, and a fitted threshold reproduces the learned actions without being that controller. The excluded-target prediction gain is 4.6991×10−64.6991\times10^{-6} nats per sample. Feedback use and protocol quality are separate tests. These cases establish neither biological discovery nor general RL superiority.

Rayan Nyaupane, Abhiram Bhupatiraju

The University of Texas at Austin

Active Experimental DesignFeedback ControlPolicy Learning
AskPDF
Oct 7, 2026

When does a network's training history predict its future learning better than its current state? Evidence from a response probe and a forecasting screen

Networks that behave alike now can still learn differently when training continues. Work on loss of plasticity and critical periods shows that the path to a state shapes what follows; it does not show whether the path carries information that a measurement of the state itself misses. We ask when the training history of a network predicts its future learning better than its current state. In a main study, small multilayer perceptrons were trained under three history regimes (42 histories), and future learning was measured at four checkpoints by a short probe: a copy of the network trained for 100 updates on a new task. Before the prediction result was read, the protocol checked the probe. It responded monotonically to a function-preserving rescaling of hidden units, repeated measurements agreed (intraclass correlation 0.940, [0.903, 0.997], in the least reliable class, mean of three repeats), and a re-initialisation of units was visible directly after it but not 100 to 200 updates later. A history state of at most four dimensions did not improve on a calibrated model of the current state (gain -21.4%, 90% interval [-91.9, 8.1]; required in advance: 10%). A companion screen on 1,560 synthetic regression runs asked the same question for a target further away, the final error of the run. There, history models forecast better than the current validation error after 12 of up to 240 epochs (compact state 30.3%, [15.8, 39.4], a contextual comparison) and were not distinguishable from it after 48. In both studies the history was informative only while the current state was not yet informative about the target; this reading was formed after the results.

Martin Hofmann, Patrick Mäder

Data-intensive Systems and Visualization Group (dAI.SY), Technische Universität Ilmenau, Max-Planck-Ring 14, 98693 Ilmenau, Thuringia, Germany · German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Deutscher Platz 5e, 04103 Leipzig, Saxony, Germany · Faculty of Biological Sciences, Friedrich Schiller University, Fürstengraben 1, 07745 Jena, Thuringia, Germany

Neural Network Training Dynamics
AskPDF
Apr 5, 2026

Adapting Neural Robot Dynamics on the Fly for Predictive Control

Accurate dynamics models are critical for the design of predictive controllers for autonomous mobile robots. Physics-based models are often too simple to capture relevant real-world effects, while data-driven models are data-intensive and slow to train. We introduce an approach for fast adaptation of neural robot dynamic models that combines offline training with efficient online updates. Our approach learns an incremental neural dynamics model offline and performs low-rank second-order parameter adaptation online, enabling rapid updates without full retraining. We demonstrate the approach on a real quadrotor robot, achieving robust predictive tracking control in novel operational conditions.

Abdullah Altawaitan, Nikolay Atanasov

Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA · Kuwait University

Robot Dynamics LearningRobot Dynamics IdentificationOnline System Identification
AskPDF