cs.RONov 4, 2026

Online Adaptation to Latent Physical Properties for Learning from Demonstration

Authors: Kotaro Kajitsuka, Masaki Murooka, Fumio Kanehiro

Organizations: University of Tsukuba, Tsukuba, Japan. · CNRS-AIST JRL (Joint Robotics Laboratory), IRL3218, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan.

Abstract

Manipulating visually identical objects with different physical properties requires adapting actions based on information acquired through interaction. We address the problem of learning from demonstrations how to adapt manipulation actions, such as regrasping and changing contact locations, based on physical-property information acquired during interaction. To this end, we propose a learning-from-demonstration method that learns latent representations of physical properties needed to predict object responses and conditions a policy on online estimates of these representations. A dynamics model and object-specific latent representations are jointly learned from demonstrations that include behaviors adapting to the objects' physical properties, and the learned representations are then used to condition an imitation policy. During execution, the dynamics model and policy weights remain fixed, while the latent representation is updated based on the discrepancy between predicted and actual observations, allowing interaction with the object to inform action generation. Real-world and simulation experiments with objects whose centers of mass differ suggest that online adaptation through latent representations of physical properties can improve task success rates. However, in simulation, success rates on center-of-mass locations absent from the demonstrations remained around 30%, indicating limited adaptation to unseen physical properties. Project page: https://isri-aist.github.io/latent-dynamics-project-page/

Figures & tables

Explore similar work

CardsList
  1. Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback

    Sep 28, 2026Yishu Li, Liyuan Geng, Xinyi Mao +2Robot Policy LearningTest-Time Adaptation

  2. PEARS: Physical-Prior-Guided Efficient Adaptation via Failure Reasoning and Diffusion Steering for Tactile Manipulation

    Oct 6, 2026Kun Song, Yiming Wang, Yilin Chen +5Robot Policy AdaptationRobotic RL