When Does Feedback Help? Auditing Learned Voltage-Clamp Experimental Design

Rayan Nyaupane; Abhiram Bhupatiraju

cs.AIOct 20, 2026

When Does Feedback Help? Auditing Learned Voltage-Clamp Experimental Design

Authors: Rayan Nyaupane, Abhiram Bhupatiraju

Organizations: The University of Texas at Austin

Abstract

A learned scientific controller can outperform a misspecified planner without benefiting from feedback. We audit two synthetic voltage-clamp tasks with a training-selected fixed protocol, current masking, a Bayesian reference, and an excluded target. Five recurrent policies repeat one fixed protocol in all 1,200 off-grid rollouts; masking current changes no decision. In a crossed four-hypothesis task, a fixed diagnostic pulse is followed by one learned choice. Policies then use the first current, but tie a correctly specified Bayesian controller, and a fitted threshold reproduces the learned actions without being that controller. The excluded-target prediction gain is $4.6991\times10^{-6}$ nats per sample. Feedback use and protocol quality are separate tests. These cases establish neither biological discovery nor general RL superiority.

Figures & tables

Figure 1: (a) Fixed sequences on the original reward, including post-steps. (b) Independent gains within twelve cells. Points are designed conditions. RL, masked RL, and the training-selected fixed protocol coincide.

Controller	400 pA $\downarrow$	800 pA $\downarrow$
Validation-selected fixed	0.2844	0.5366
Learned design (four seeds)	0.1992	0.4887–0.4912
Bayesian contingent design	0.1992	0.4887
Learned, current masked	0.2844	0.5366

Table 2: Crossed task on 200 simulated draws from four fixed hypotheses, not biological replicates. Entropy is nats and is lower when the experiment is informative. At 800 pA the learned values span four seeds.

Figure 2: (a) Mean first-pulse current on seeds 5000–5199 around the validation-fit threshold (

537.7

pA). (b) Held-out variant entropy. The threshold reproduces all learned actions on this test block; its entropy is nearly identical to the Bayesian reference, which agrees on 199 of 200 actions.

Question	Original task	Crossed task
Does feedback change actions?	No. All 1,200 rollouts are LL. Masking changes none.	Yes at nominal noise: the second pulse follows the first current.
Does it improve an independent outcome?	No. Learned and fixed scores coincide.	Fresh excluded-waveform gain: $4.6991\times 10^{-6}$ nats/sample.
Does learning beat a simpler adaptive choice?	No. It copies the training-selected sequence.	No. It ties Bayes. The threshold reproduces learned actions; Bayes agrees on 199 of 200 and is not that rule.

Table 3: Three questions, answered separately. Crossed-task entropy is the training objective. The independent outcome is the fresh excluded waveform. The simpler references are Bayes and the fitted threshold.

Active Experimental Design

See all 17 papers

Tune to Learn: How Controller Gains Shape Robot Policy Learning
Apr 2, 2026Antonia Bronars, Younghyo Park, Pulkit AgrawalRobot Policy Learning Robotic Control
MIT · MIT Improbable AI Lab
Provable imitation learning for control of instability in partially-observed Vlasov--Poisson equations
May 6, 2026Xiaofan Xia, Qin Li, Wenlong MouImitation Learning
Department of Statistical Sciences, University of Toronto† · Department of Mathematics, University of Wisconsin–Madison⋄
History, Adaptation, and Hidden Gain: Two Prospective Tests of Learned Predictive Control in a Synthetic Neural Population
Oct 9, 2026Mahesh Reddy PagadalaNeural Network Control Adaptive Control
Aston University, Birmingham, UK

Apr 2, 2026J/K move · Enter open · S save

Tune to Learn: How Controller Gains Shape Robot Policy Learning

Antonia Bronars, Younghyo Park, Pulkit Agrawal

MIT · MIT Improbable AI Lab

Robot Policy Learning Robotic Control Robotic RL Sim-to-Real Transfer Robot Imitation Learning

Open paper AskPDF arXiv

Position controllers have become the dominant interface for executing learned manipulation policies. Yet a critical design decision remains understudied: how should we choose controller gains for policy learning? The conventional wisdom is to select gains based on desired task compliance or stiffness. However, this logic breaks down when controllers are paired with state-conditioned policies: effective stiffness emerges from the interplay between learned reactions and control dynamics, not from gains alone. We argue that gain selection should instead be guided by learnability: how amenable different gain settings are to the learning algorithm in use. In this work, we systematically investigate how position controller gains affect three core components of modern robot learning pipelines: behavior cloning, reinforcement learning from scratch, and sim-to-real transfer. Through extensive experiments across multiple tasks and robot embodiments, we find that: (1) behavior cloning benefits from compliant and overdamped gain regimes, (2) reinforcement learning can succeed across all gain regimes given compatible hyperparameter tuning, and (3) sim-to-real transfer is harmed by stiff and overdamped gain regimes. These findings reveal that optimal gain selection depends not on the desired task behavior, but on the learning paradigm employed. Project website: https://younghyopark.me/tune-to-learn

Abstract

Active Experimental Design

Explore similar work

Tune to Learn: How Controller Gains Shape Robot Policy Learning

Provable imitation learning for control of instability in partially-observed Vlasov--Poisson equations

History, Adaptation, and Hidden Gain: Two Prospective Tests of Learned Predictive Control in a Synthetic Neural Population