DRAP: Dynamics-Risk Adaptive Plasticity for Continual Learning in Physical AI
Abstract
Continual learning in physical AI differs from conventional continual learning because small per-step policy errors can accumulate over a trajectory. Each deviation may move the agent outside the training distribution, increase subsequent prediction errors, and ultimately cause closed-loop task failure. This work introduces Dynamics-Risk Adaptive Plasticity (DRAP), a replay-free method that estimates the historical importance of adaptable model tensors from sample-level action gradients weighted by local trajectory-amplification risk. A ridge-regularized local linear model is fitted to expert state-action transitions, and the largest singular value of its state-transition matrix provides a policy-independent, Lyapunov-inspired estimate of one-step perturbation amplification. After each task, risk-weighted squared gradients are accumulated in a tensor-level importance memory. During subsequent tasks, DRAP uses normalized cumulative importance to scale the effective AdamW learning rate of each tensor, immediately reducing the plasticity of components that supported previous behavior at dynamically sensitive trajectory points. We instantiate DRAP on a fixed KappaTune-selected subnetwork of OpenVLA and evaluate replay-free sequential adaptation across the LIBERO Spatial, Object, and Goal suites. Compared with standard AdamW fine-tuning and EWC, the results show substantially improved retention of previously learned behaviors while preserving strong new-suite acquisition.
Figures & tables
| Dimension | CLASSP | EWC | Proposed DRAP |
|---|---|---|---|
| Historical signal | Powered gradient/update accumulation | Diagonal Fisher information estimated from previous-task gradients | Dynamics-risk-weighted sample-level gradients |
| Physical grounding | None | None | Local trajectory-amplification risk |
| Protection mechanism | Update denominator and gradient threshold | Fisher-weighted quadratic displacement penalty | Tensor-specific AdamW learning rate |
| Reference weights | Not required | Required, | Not required |
| Onset of protection | Immediate | Zero at the new-task initial point; increases with displacement from | Immediate |
| Optimizer | Custom CLASSP optimizer | AdamW plus EWC regularization loss | AdamW with per-tensor learning-rate groups |
| Method | Checkpoint | Spatial | Object | Goal |
|---|---|---|---|---|
| Standard fine-tuning | After Spatial | 0.77 | – | – |
| After Object | 0.11 | 0.83 | – | |
| After Goal | 0.00 | 0.07 | 0.71 | |
| KappaTune+EWC | After Spatial | 0.76 | – | – |
| After Object | 0.41 | 0.82 | – | |
| After Goal | 0.03 | 0.30 | 0.64 |
| After Object | After Goal | |||||
|---|---|---|---|---|---|---|
| Method | Avg. success | Spatial forget. | Avg. success | Spatial forget. | Object forget. | Avg. forget. |
| Standard fine-tuning | 0.47 | 0.66 | 0.26 | 0.77 | 0.76 | 0.77 |
| KappaTune+EWC | 0.62 | 0.35 | 0.32 | 0.73 | 0.52 | 0.63 |
| KappaTune+DRAP | 0.77 | 0.02 | 0.55 | 0.43 | 0.19 | 0.31 |
| KappaTune+Uniform-risk DRAP | 0.72 | 0.14 | 0.51 | 0.49 | 0.23 | 0.36 |
| DRAP without KappaTune | 0.77 | 0.07 | 0.28 | 0.67 | 0.74 | 0.71 |
Continual Robot Learning
Continual robot learning develops robots that acquire new skills and adapt to changing conditions while retaining previously learned behavior.
See all 100 papers