cs.RONov 12, 2026

Inner Product Regularization for Imitation Learning

Authors: Benjamin A. Christie, Tianyu Qiu, Amy H. Choi, David Fridovich-Keil, Dylan P. Losey

Organizations: Virginia Tech, Department of Mechanical Engineering · The University of Texas at Austin, Department of Aerospace Engineering and Engineering Mechanics · Korea Advanced Institute of Science and Technology, Department of Industrial and Systems Engineering

Abstract

Robots that learn from demonstrations should extrapolate their policy despite state perturbations such as sensor noise, scene variation, and compounding errors. One approach to this robustness is regularization. Prior regularizers consider each state on its own and limit how sensitive the policy is to any perturbation at that state. By contrast, we explore how the policy's output shifts when given the same perturbation at different states. Our insight is that, locally, the robot should respond to state perturbations in a consistent manner: e.g., if changing an irrelevant object position does not impact the policy at one state, that same change should not shift the policy output at a nearby state. Applying this insight, we develop inner product regularization (InneR). We show that the inner product on action space induces an inner product on state space through the policy Jacobian (the Fisher information for stochastic policies), and that the null space of this inner product governs which state perturbations have a first-order effect on the policy. By clustering states whose inner products are aligned, averaging the inner products within each cluster, and minimizing the smallest eigenvalues of that average, InneR shapes the learned policy so that nearby states share a common null space. InneR enforces consistent responses to perturbations without limiting policy expressiveness, and can be appended to existing imitation learning pipelines for deterministic or stochastic policies. In simulated experiments, InneR makes learned policies more robust to irrelevant state dimensions and shifts between training and test distributions, whereas norm penalties often perform worse than behavior cloning. On a real robot arm performing three manipulation tasks, InneR raises the average success rate of a diffusion policy from 18% to 57% without additional data.

Figures & tables

Explore similar work

CardsList
  1. A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning

    Sep 11, 2026Amine Andam, Jamal Bentahar, Mustapha HedabouKL-Regularized RLAdversarial Robustness

  2. Difference-Aware Retrieval Policies for Imitation Learning

    Jun 8, 2026Quinn Pfeifer, Ethan Pronovost, Paarth Shah +3Imitation LearningInteractive Imitation Learning

  3. Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning

    Sep 28, 2026Hanbin Zhou, Shangzhe Li, Alexander Braverman +1Reinforcement LearningAdversarial Imitation Learning