A learned scientific controller can outperform a misspecified planner without benefiting from feedback. We audit two synthetic voltage-clamp tasks with a training-selected fixed protocol, current masking, a Bayesian reference, and an excluded target. Five recurrent policies repeat one fixed protocol in all 1,200 off-grid rollouts; masking current changes no decision. In a crossed four-hypothesis task, a fixed diagnostic pulse is followed by one learned choice. Policies then use the first current, but tie a correctly specified Bayesian controller, and a fitted threshold reproduces the learned actions without being that controller. The excluded-target prediction gain is 4.6991×10−6 nats per sample. Feedback use and protocol quality are separate tests. These cases establish neither biological discovery nor general RL superiority.
Figures & tables
Figure 1: (a) Fixed sequences on the original reward, including post-steps. (b) Independent gains within twelve cells. Points are designed conditions. RL, masked RL, and the training-selected fixed protocol coincide.
Controller
400 pA ↓
800 pA ↓
Validation-selected fixed
0.2844
0.5366
Learned design (four seeds)
0.1992
0.4887–0.4912
Bayesian contingent design
0.1992
0.4887
Learned, current masked
0.2844
0.5366
Table 2: Crossed task on 200 simulated draws from four fixed hypotheses, not biological replicates. Entropy is nats and is lower when the experiment is informative. At 800 pA the learned values span four seeds.
Figure 2: (a) Mean first-pulse current on seeds 5000–5199 around the validation-fit threshold ( 537.7 pA). (b) Held-out variant entropy. The threshold reproduces all learned actions on this test block; its entropy is nearly identical to the Bayesian reference, which agrees on 199 of 200 actions.
Question
Original task
Crossed task
Does feedback change actions?
No. All 1,200 rollouts are LL. Masking changes none.
Yes at nominal noise: the second pulse follows the first current.
No. It ties Bayes. The threshold reproduces learned actions; Bayes agrees on 199 of 200 and is not that rule.
Table 3: Three questions, answered separately. Crossed-task entropy is the training objective. The independent outcome is the fresh excluded waveform. The simpler references are Bayes and the fitted threshold.
Position controllers have become the dominant interface for executing learned manipulation policies. Yet a critical design decision remains understudied: how should we choose controller gains for policy learning? The conventional wisdom is to select gains based on desired task compliance or stiffness. However, this logic breaks down when controllers are paired with state-conditioned policies: effective stiffness emerges from the interplay between learned reactions and control dynamics, not from gains alone. We argue that gain selection should instead be guided by learnability: how amenable different gain settings are to the learning algorithm in use. In this work, we systematically investigate how position controller gains affect three core components of modern robot learning pipelines: behavior cloning, reinforcement learning from scratch, and sim-to-real transfer. Through extensive experiments across multiple tasks and robot embodiments, we find that: (1) behavior cloning benefits from compliant and overdamped gain regimes, (2) reinforcement learning can succeed across all gain regimes given compatible hyperparameter tuning, and (3) sim-to-real transfer is harmed by stiff and overdamped gain regimes. These findings reveal that optimal gain selection depends not on the desired task behavior, but on the learning paradigm employed. Project website: https://younghyopark.me/tune-to-learn
We consider the stabilization of Vlasov--Poisson plasma dynamics, a central control problem in nuclear fusion. Our focus is the gap between what an ideal controller would use and what experiments can actually observe: while optimal policy may rely on the full phase-space state, practical feedback is typically limited to sparse macroscopic diagnostics. We therefore study imitation learning methods that distill a fully observed expert policy into controllers operating only on macroscopic measurements. We show the stability guarantees of the learned policy, where the error floor depends on the minimal behavior cloning loss achievable under the observation constraints. We further characterize this minimal loss in terms of a notion of entropy that quantifies the complexity of the initial distribution. Our results demonstrates the theoretical feasibility of learning stabilizing feedback policies for kinetic plasma dynamics from macroscopic observations, and exhibits the adaptivity of the learning approach to low-complexity structures. Through extensive numerical experiments, we validate our theory and show that the learned policies can stabilize the system using only macroscopic observations, within a significantly longer time horizon than non-adaptive baseline controllers.
Xiaofan Xia, Qin Li, Wenlong Mou
Department of Statistical Sciences, University of Toronto† · Department of Mathematics, University of Wisconsin–Madison⋄
We study learned one-step predictive control of a synthetic population of 64 leaky integrate-and-fire neurons whose light sensitivity is scaled by a hidden factor s, using two prospective confirmations on plants simulated only after the analysis was frozen. In the first (50 plants, 11 conditions), removing the 200-ms input-output history substantially worsened a parameter-count-matched neural predictor at low and moderate gain (s = 0.6, 0.8; RMSE rose by 2.47 Hz at s = 0.6, 97.5% CI 2.43-2.51). In secondary comparisons, recursively adapted ARX had the lowest mean error of the deployable controllers in 10 of 11 conditions, tuned PI with anti-windup matched or beat the network for s >= 0.7, and training for 10,000 rather than 900 epochs lowered prediction error but worsened control. Development work suggested the network's learned action sensitivity did not track the hidden gain. A second protocol (50 new plants, 9 sensitivities, 2 noise levels) confirmed three pre-specified predictions at the primary noise level. Re-estimating a single multiplicative input gain recovered 87% (95% CI 86-88%) of the benefit of full adaptive ARX. Supplying that online estimate to the network as an extra input reduced high-gain tracking error by 1.84 Hz [1.81, 1.88]. Longer training worsened the plain network by 1.09 Hz but improved the gain-input network by 1.07 Hz, a differential of 2.16 Hz [2.13, 2.20]. Under high process noise (sigma = 2.0) the effects shrank to about 0.04 Hz or less and gain-only adaptation recovered only 24% of the adaptive-ARX benefit. In this synthetic plant, at the primary noise level, one-parameter online gain estimation accounts for most of the gap between a history-based network and full adaptive ARX, and for its prediction-control mismatch.