Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification

Junghoon Seo

cs.ROApr 15, 2026

Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification

Authors: Junghoon Seo

Organizations: PIT IN Corp., South Korea

Abstract

Behavior cloning (BC) on position-controlled robots is shaped by the PD loop that executes policy actions. We give a finite-horizon, nonasymptotic analysis of how controller gains affect BC failure. Independent sub-Gaussian action errors propagate through gain-dependent closed-loop dynamics into sub-Gaussian position errors. The resulting failure tail is controlled by controller amplification multiplied by validation loss and generalization slack, so validation loss alone can mis-rank gains. Under shape-preserving upper-bound assumptions, the analysis separates label difficulty, injection strength, and contraction, ranking compliant-overdamped gains as tightest and stiff-underdamped gains as loosest, with the mixed regimes system-dependent. In the canonical scalar second-order PD system, stationary position-error variance increases with stiffness and decreases with damping over the stable range, and exact zero-order-hold discretization inherits the ordering to leading order. This extends the error-attenuation explanation of bronars et al. (2026) to finite-horizon failure bounds.

Figures & tables

Fig. 1: Hasse diagram of the regime ordering established by Theorem 7 . Solid arrows indicate

\Psi(K_{1})\leq\Psi(K_{2})

. CO is the unique minimum and SU the unique maximum. The dashed line between SO and CU indicates that their ordering is system-dependent.

Fig. 2: Monte Carlo position-error envelopes (

N=50{,}000

rollouts) for the four gain regimes under exact ZOH discretization. Shaded bands show the 95th and 99th percentiles. The dashed line is the steady-state 95th-percentile bound from Theorem 1 . All panels share the same vertical scale. CO yields the tightest envelopes, confirming the predicted ordering.

Fig. 3: Stationary proxy

X_{\infty}(K)

(normalized by

X_{\infty}^{\mathrm{CO}}

, log scale) over the gain parameter space, computed via exact ZOH discretization at

\Delta t=0.02

s. Markers locate the four canonical regimes. The landscape is monotone increasing in

K_{p}

and decreasing in

K_{d}

over the entire stable region, consistent with Theorem 9 and the leading-order ZOH relation in Proposition 10 .

Fig. 4: Empirical failure rate

\hat{P}(\mathrm{Fail}_{T})

(solid) versus the Theorem 3 upper bound (dashed) as a function of the success-tube radius

r

for the four canonical regimes. The bound dominates the Monte Carlo curve in every regime, and the regime ordering CO

\prec

\approx

\prec

SU is preserved at all

r

	CO	SO	CU	SU
$K_{p}$	50	100	50	100
$K_{d}$	40	40	20	20
$\rho(A_{K})$	0.974	0.948	0.943	0.819
$X_{\infty}^{\mathrm{c}}=\sigma^{2}K_{p}/(2K_{d})$	0.625	1.250	1.250	2.500
$X_{\infty}^{\mathrm{d}}$ (ZOH discrete)	0.012	0.025	0.025	0.050
$X_{\infty}^{\mathrm{d}}/X_{\infty}^{\mathrm{d,CO}}$	1.0	2.0	2.0	4.0

TABLE I: System quantities and empirical failure rates for the scalar canonical system.

X_{\infty}^{\mathrm{c}}

from ( 24 ),

X_{\infty}^{\mathrm{d}}

from the ZOH discrete Lyapunov.

Regime	$k_{p}$	$\zeta$	$\widehat{L}_{\rm val}$	$C_{\rm rs}$	$Z_{K}$	Fail
CO	50	2.0	0.679	$1.5{\times}10^{5}$	2.75	0.000
SO	200	2.0	0.382	$1.7{\times}10^{6}$	3.12	0.013
CU	50	0.5	0.714	$2.4{\times}10^{6}$	3.98	0.213
SU	200	0.5	0.270	$1.4{\times}10^{8}$	6.00	0.893

TABLE II: Robosuite PickPlaceCan regimes and mean online trained-policy results over three seeds. Each row averages separately trained MLP deployments. Failure is measured with tube radius

r=4.6

and

C_{\rm rs}=(\widehat{S}_{T}^{\rm pulse})^{2}\widehat{L}_{\rm cal}

Fig. 5: Online robosuite PickPlaceCan deployments of trained MLP policies. Panel (a) shows post-training local amplification estimates, and panels (b)–(d) show seed-level peak joint error from neural-policy rollouts against expert validation loss, rollout-local calibration loss, and the product

C_{\rm rs}

. Validation loss alone favors the stiff-underdamped regime, whereas

C_{\rm rs}

separates the high-error SU regime from the compliant-overdamped regime.

PID Control

See all 9 papers

Tune to Learn: How Controller Gains Shape Robot Policy Learning
Apr 2, 2026Antonia Bronars, Younghyo Park, Pulkit AgrawalRobot Policy Learning Robotic Control
MIT · MIT Improbable AI Lab
Training-free Behavior Cloning
Sep 24, 2026Maximilian Adang, Timothy Chen, Lars Osterberg +2Robot Policy Learning Behavior Cloning
Stanford University, Stanford, CA 94404, USA.
Understanding Behavior Cloning with Action Quantization
Mar 20, 2026Haoqun Cao, Tengyang XieBehavior Cloning Imitation Learning
University of Wisconsin–Madison

Apr 2, 2026J/K move · Enter open · S save

Tune to Learn: How Controller Gains Shape Robot Policy Learning

Antonia Bronars, Younghyo Park, Pulkit Agrawal

MIT · MIT Improbable AI Lab

Robot Policy Learning Robotic Control Robotic RL Sim-to-Real Transfer Robot Imitation Learning

Open paper AskPDF arXiv

Position controllers have become the dominant interface for executing learned manipulation policies. Yet a critical design decision remains understudied: how should we choose controller gains for policy learning? The conventional wisdom is to select gains based on desired task compliance or stiffness. However, this logic breaks down when controllers are paired with state-conditioned policies: effective stiffness emerges from the interplay between learned reactions and control dynamics, not from gains alone. We argue that gain selection should instead be guided by learnability: how amenable different gain settings are to the learning algorithm in use. In this work, we systematically investigate how position controller gains affect three core components of modern robot learning pipelines: behavior cloning, reinforcement learning from scratch, and sim-to-real transfer. Through extensive experiments across multiple tasks and robot embodiments, we find that: (1) behavior cloning benefits from compliant and overdamped gain regimes, (2) reinforcement learning can succeed across all gain regimes given compatible hyperparameter tuning, and (3) sim-to-real transfer is harmed by stiff and overdamped gain regimes. These findings reveal that optimal gain selection depends not on the desired task behavior, but on the learning paradigm employed. Project website: https://younghyopark.me/tune-to-learn

Abstract

PID Control

Explore similar work

Tune to Learn: How Controller Gains Shape Robot Policy Learning

Training-free Behavior Cloning

Understanding Behavior Cloning with Action Quantization