Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset. Yet not all states are created equal: in a small number of critical states, even a slight deviation from the expert action can lead to a failure, while in many other states such errors are recoverable and inconsequential. We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state. We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy. SCRIM assigns each demonstration state a score reflecting how vulnerable the task outcome is to small, policy-relevant action errors. We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance. We further show that SCRIM enables data curation by selecting compact, high-value subsets of demonstrations that preserve performance while substantially reducing the amount of training data. We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success. Project website: https://sia-lab-git.github.io/SCRIM/
Figures & tables
Figure 3: Faucet models ordered by mean SCRIM over the demonstrations, from the two easiest (top) to the two hardest (bottom).
Figure 5: SCRIM distinguishes a brittle (left) from a robust (right) cube-stacking demonstration.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
StackCube
TurnFaucet
PegInsertion
Hammer
Disturbance bound ϵ (per axis)
0.005
0.005
0.002
0.20
SCRIM adversary
200k
100k
1M
500k
Appendix
Table 1: Disturbance bounds and interaction budgets. Environment interactions used to train the SCRIM adversary behind Table 1 baselines.
StackCube
TurnFaucet
PegInsertion
Hammer
ESS% target τ
44.7
40.6
69.7
50.0
Temperature T⋆
1.97
1.72
3.75
5.62
Appendix
Table 2: Sampling target. Top: selected ESS% target τ and resulting temperature T⋆ for SCRIM. Bottom: success rate (%) of SCRIM-weighted BC across ESS% targets.
ϵ
0
0.001
0.003
0.005
0.01
0.02
0.05
Success (%)
72.0
64.0
42.0
24.0
14.0
0.0
0.0
Appendix
Table 3: Random-residual calibration (StackCube). Success rate (%) of the frozen base policy under uniform random residuals of size ϵ .
ϵ
0.003
0.005
0.01
0.02
0.05
StackCube (uniform BC 68.0)
Success (%)
83.0
91.0
73.0
77.0
73.0
TurnFaucet (uniform BC 39.0)
Success (%)
42.0
47.0
37.0
42.0
42.0
Appendix
Table 4: Sensitivity to the perturbation bound. Success rate (%) of SCRIM-weighted BC for each ϵ at a fixed ESS% target.
StackCube
TurnFaucet
PegInsertion
Hammer
Gated fraction k
5%
5%
30%
10%
Clean success (%)
69.0
29.0
13.0
66.0
Adversary at high-SCRIM states
13.0
13.0
13.0
20.0
Adversary at random states
6.0
4.0
6.0
1.0
Uniform ϵ -noise at high-SCRIM states
1.0
0.0
0.0
3.0
Appendix
Table 5: Score-gated disturbance. Drop in success rate (percentage points) relative to clean rollouts when the disturbance is applied at a fraction k of steps.
StackCube
TurnFaucet
PegInsertion
Hammer
Noise scale c=0
13.0
13.0
13.0
20.0
c=0.003
16.0 (14.3)
12.0 (4.1)
13.0 (12.2)
20.0 (18.4)
c=0.01
14.0 (4.3)
9.0 (1.2)
8.0 (3.7)
19.0 (5.5)
c=1
3.0 (0.04)
−1.0 (0.01)
5.0 (0.04)
2.0 (0.06)
Halfway checkpoint
4.0
1.0
5.0
23.0
Random states (reference)
6.0
4.0
6.0
1.0
Appendix
Table 6: Degraded critics. Drop in success rate (percentage points) from the adversary at high-SCRIM states, with the score SNR in parentheses, at the gated fractions of Table 5 .
Parametric imitation learning via behavior cloning can suffer from poor generalization to out-of-distribution states due to compounding errors during deployment. We show that reusing the training data during inference via a semi-parametric retrieval-based imitation learning approach can alleviate this challenge. We present Difference-Aware Retrieval Policies for Imitation Learning (DARP), a semi-parametric retrieval-based imitation learning approach that addresses this limitation by reparameterizing the imitation learning problem in terms of local neighborhood structure rather than direct state-to-action mappings. Instead of learning a global policy, DARP trains a model to predict actions based on k-nearest neighbors from expert demonstrations, their corresponding actions, and the relative distance vectors between neighbor states and query states. DARP requires no additional assumptions beyond those made for standard behavior cloning -- it does not require additional data collection, online expert feedback, or task-specific knowledge. We demonstrate consistent performance improvements of 15-46% over standard behavior cloning across diverse domains, including continuous control and robotic manipulation, and across different representations, including high-dimensional visual features. Code and demos are available at https://weirdlabuw.github.io/darp-site/.
Quinn Pfeifer, Ethan Pronovost, Paarth Shah +3
1Paul G. Allen School of Computer Science & Engineering, University of Washington · 2Toyota Research Institute · 3Google DeepMind +1
Imitation-learning policies inherit the quality of the demonstrations they are trained on, and a growing set of curation metrics promise to score and filter low-quality demonstrations automatically. These metrics are each validated on different data with different protocols, so it is unclear which of them actually identify the demonstrations that harm a policy. We build a controlled testbed in which demonstration defects are injected with known type, and audit seven curation metrics along two axes: how well each separates defective from clean demonstrations, and whether training a behavior-cloning policy on each metric's curated subset improves task success. We study two defect regimes. Subtle perturbations (correlated action noise, tremor, truncation) are detectable by multivariate outlier scoring and, once removed, recover the full downstream gap. Structural errors, where the demonstration executes a wrong action at a key moment, are invisible to every action-only metric we test, and two of them are inverted: they score defective demonstrations as higher quality and, used for curation, tend to leave the policy at or below the uncurated baseline rather than above it. Only metrics that examine the state trajectory detect structural errors, and even the best of them recovers just a third of the downstream gap. High detection accuracy does not guarantee downstream improvement. We release the testbed and all curation implementations.
Aarav Bedi
University of California, Berkeley · Department of Mechanical Engineering University of California, Berkeley Berkeley, California, USA
Behavior cloning provides strong imitation learning guarantees when training and test environments share the same dynamics. However, in many deployment settings the test environment's transitions differ from training, and classical offline IL offers no recourse: the learner must commit to an action at every state, even when its demonstrations are uninformative and could lead to arbitrary degradation of performance. This motivates the study of selective imitation, where the learner may choose to stop when it cannot act reliably. We introduce a model for selective imitation under arbitrary dynamics shift: given labeled expert demonstrations from a training environment and unlabeled state trajectories from the same expert in a test environment, the learner outputs a selective policy that is complete (rarely stops in training) and sound (incurs low regret before stopping in test). Our algorithm, SeqRejectron, constructs a stopping rule using a small set of validator policies whose size is independent of the horizon or policy class. For deterministic policies, this yields horizon-free O~(log∣Π∣/ε2) sample complexity, assuming sparse costs. For stochastic policies, we obtain analogous horizon-free guarantees using a cumulative Hellinger stopping time. We extend the framework to misspecified experts and different expert policies across train and test and obtain results that gracefully degrade with the amount of misspecification.