SPREAD: Subspace Representation Distillation for Lifelong Imitation Learning

Kaushik Roy; Giovanni D'urso; Nicholas Lawrance; Brendan Tidd; Peyman Moghadam

cs.LGMar 9, 2026

SPREAD: Subspace Representation Distillation for Lifelong Imitation Learning

Authors: Kaushik Roy, Giovanni D'urso, Nicholas Lawrance, Brendan Tidd, Peyman Moghadam

Organizations: CSIRO Robotics, DATA61, CSIRO, Australia. · Queensland University of Technology (QUT), Brisbane, Australia.

Abstract

A key challenge in lifelong imitation learning (LIL) is enabling agents to acquire new skills from expert demonstrations while retaining prior knowledge. This requires preserving the low-dimensional manifolds and geometric structures that underlie task representations across sequential learning. Existing distillation methods, which rely on L2-norm feature matching in raw feature space, are sensitive to noise and high-dimensional variability, often failing to preserve intrinsic task manifolds. To address this, we introduce SPREAD, a geometry-preserving framework that employs singular value decomposition (SVD) to align policy representations across tasks within low-rank subspaces. This alignment maintains the underlying geometry of multimodal features, facilitating stable transfer, robustness, and generalization. Additionally, we propose a confidence-guided distillation strategy that applies a Kullback-Leibler divergence loss restricted to the top-M most confident action samples, emphasizing reliable modes and improving optimization stability. Experiments on the LIBERO, lifelong imitation learning benchmark, show that SPREAD substantially improves knowledge transfer, mitigates catastrophic forgetting, and achieves state-of-the-art performance.

Figures & tables

Fig. 2 : Overview of our proposed $SPREAD$ method. Subspace Representation Distillation aligns the latent representations from different input modality encoders ( e.g. , Task, AgentView, HandEye, Joint, and Gripper information), while confidence-guided policy distillation maps the action distribution of the GMM policy between incremental steps $T^{k-1}$ and $T^{k}$ .

Method	LIBERO-OBJECT			LIBERO-GOAL			LIBERO-SPATIAL
Method	FWT ( $\uparrow$ )	NBT ( $\downarrow$ )	AUC ( $\uparrow$ )	FWT ( $\uparrow$ )	NBT ( $\downarrow$ )	AUC ( $\uparrow$ )	FWT ( $\uparrow$ )	NBT ( $\downarrow$ )	AUC ( $\uparrow$ )
Sequential	62.0 ( $\pm$ 1.0)	63.0 ( $\pm$ 2.0)	30.0 ( $\pm$ 1.0)	55.0 ( $\pm$ 1.0)	70.0 ( $\pm$ 1.0)	23.0 ( $\pm$ 1.0)	72.0 ( $\pm$ 1.0)	81.0 ( $\pm$ 1.0)	20.0 ( $\pm$ 1.0)
EWC [ 6 ]	56.0 ( $\pm$ 3.0)	69.0 ( $\pm$ 2.0)	16.0 ( $\pm$ 2.0)	32.0 ( $\pm$ 2.0)	48.0 ( $\pm$ 3.0)	6.0 ( $\pm$ 1.0)	23.0 ( $\pm$ 1.0)	33.0 ( $\pm$ 1.0)	6.0 ( $\pm$ 1.0)
ER [ 8 ]	56.0 ( $\pm$ 1.0)	24.0 ( $\pm$ 1.0)	49.0 ( $\pm$ 1.0)	53.0 ( $\pm$ 1.0)	36.0 ( $\pm$ 1.0)	47.0 ( $\pm$ 2.0)	65.0 ( $\pm$ 3.0)	27.0 ( $\pm$ 3.0)	56.0 ( $\pm$ 1.0)
BUDS [ 26 ]	52.0 ( $\pm$ 2.0)	21.0 ( $\pm$ 1.0)	47.0 ( $\pm$ 1.0)	50.0 ( $\pm$ 1.0)	39.0 ( $\pm$ 1.0)	42.0 ( $\pm$ 1.0)	-	-	-
LOTUS [ 9 ]	74.0 ( $\pm$ 3.0)	11.0 ( $\pm$ 1.0)	65.0 ( $\pm$ 3.0)	61.0 ( $\pm$ 3.0)	30.0 ( $\pm$ 1.0)	56.0 ( $\pm$ 1.0)	-	-	-

TABLE I : Experimental results across three different LIBERO task suites. The reported values are averages from three seeds, including the mean and standard error. The best values are highlighted in bold, and the second-best values are underlined. The dash (-) indicates a failure to reproduce results. All metrics are measured based on success rates (%).

Panel of Figure 3 — Fig. 3 : Average success rate across incremental tasks on (a) LIBERO-OBJECT and (b) LIBERO-GOAL (Higher numbers are better).

Panel of Figure 4 — Fig. 4 : Forward Transfer (FWT) across incremental tasks on (a) LIBERO-OBJECT and (b) LIBERO-GOAL (Higher numbers are better).

Fig. 5 : Negative Backward Transfer (NBT) across incremental tasks on LIBERO-GOAL (lower is better).

Fig. 6 : Drift analysis across incremental tasks for different modalities on LIBERO-OBJECT. The radial axes (1-9) indicate the incremental steps in LIL. Lower representation drift indicates better preservation of previously learned knowledge across tasks in lifelong imitation learning.

$\mathcal{L}_{\text{text}}$	$\mathcal{L}_{\text{image}}$	$\mathcal{L}_{\text{action}}$	$\mathcal{L}_{\text{extra}}$	LIBERO-GOAL
$\mathcal{L}_{\text{text}}$	$\mathcal{L}_{\text{image}}$	$\mathcal{L}_{\text{action}}$	$\mathcal{L}_{\text{extra}}$	FWT $\uparrow$	NBT $\downarrow$	AUC $\uparrow$
✓	✓	✓	✓	0.78	0.09	0.72
✗	✓	✓	✓	0.78	0.11	0.69
✓	✗	✓	✓	0.67	0.20	0.57
✓	✓	✗	✓	0.70	0.02	0.70
✓	✓	✓	✗	0.74	0.03	0.71

TABLE II : Ablation studies on the contribution of each component in our method. Experiments were performed on the LIBERO-GOAL task suite. Reported results are the mean across three different seed values.

Fig. 7 : Ablation study on policy distillation strategies using LIBERO-GOAL task suite. Bar represents mean across three random seeds while standard error is represented with darker color. Our top-

M

confidence-weighted approach outperforms baselines in FWT, NBT and AUC.

Rank of Subspace	FWT ( $\uparrow$ )	BWT ( $\downarrow$ )	AUC ( $\uparrow$ )
$SPREAD$ (Full-Rank)	76.0 ( $\pm$ 1.0)	16.0 ( $\pm$ 8.0)	66.0 ( $\pm$ 5.0)
$SPREAD$ (Rank $=48$ )	78.0 ( $\pm$ 1.0)	9.0 ( $\pm$ 5.0)	72.0 ( $\pm$ 1.0)
$SPREAD$ (Rank $=32$ )	74.0 ( $\pm$ 1.0)	6.0 ( $\pm$ 4.0)	71.0 ( $\pm$ 2.0)
$SPREAD$ (Rank $=16$ )	76.0 ( $\pm$ 3.0)	9.0 ( $\pm$ 4.0)	69.0 ( $\pm$ 2.0)

TABLE III : Ablation study on the dimensionality/rank of subspace representation distillation strategies using LIBERO-GOAL task suite. Our proposed

SPREAD

loss with

75\%

-rank performs best in FWT, NBT and AUC.

Explore similar work

Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment
Mar 11, 2026Fanqi Yu, Matteo Tiezzi, Tommaso Apicella +2Imitation Learning Robot Imitation Learning
AI for Good (AIGO), Istituto Italiano di Tecnologia, Genoa, Italy · DITEN, University of Genoa, Genoa, Italy · PAVIS, Istituto Italiano di Tecnologia, Genoa, Italy +1
Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning
Jun 5, 2026Fatema Siddika, Md Anwar Hossen, Tanwi Mallick +1Continual Learning for LLMs Mixture-of-Experts Language Models
Iowa State University, Ames, USA · Argonne National Laboratory, USA
Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery
Apr 15, 2026Noureddine KermicheModular Neural Networks
Western Digital Corporation, Irvine, CA, USA

Mar 11, 2026J/K move · Enter open · S save

Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment

Fanqi Yu, Matteo Tiezzi, Tommaso Apicella, Cigdem Beyan+1

AI for Good (AIGO), Istituto Italiano di Tecnologia, Genoa, Italy · DITEN, University of Genoa, Genoa, Italy · PAVIS, Istituto Italiano di Tecnologia, Genoa, Italy+1

Imitation Learning Robot Imitation Learning Replay-Based Continual Learning

Open paper AskPDF arXiv

We introduce a lifelong imitation learning framework that enables continual policy refinement across sequential tasks under realistic memory and data constraints. Our approach departs from conventional experience replay by operating entirely in a multimodal latent space, where compact representations of visual, linguistic, and robot's state information are stored and reused to support future learning. To further stabilize adaptation, we introduce an incremental feature adjustment mechanism that regularizes the evolution of task embeddings through an angular margin constraint, preserving inter-task distinctiveness. Our method establishes a new state of the art in the LIBERO benchmarks, achieving 10-17 point gains in AUC and up to 65% less forgetting compared to previous leading methods. Ablation studies confirm the effectiveness of each component, showing consistent gains over alternative strategies. The code is available at: https://github.com/yfqi/lifelong_mlr_ifa.

Abstract

Explore similar work

Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment

Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery