DiPLO: Self-distilled Diffusion Planning for Loco-manipulation

Shafeef Omar; Sebastian Sanokowski; Elham Daneshmand; Dian Yu; Daniel Cremers; Cordelia Schmid; Majid Khadiv

cs.ROOct 17, 2026

DiPLO: Self-distilled Diffusion Planning for Loco-manipulation

Authors: Shafeef Omar, Sebastian Sanokowski, Elham Daneshmand, Dian Yu, Daniel Cremers, Cordelia Schmid, Majid Khadiv

Organizations: Technical University of Munich (TUM), Munich, Germany. · Institute for Advanced Study (IAS), Technical University of Munich, Munich, Germany. · McGill University, Montreal, Canada. · INRIA, Paris, France.

Abstract

Humanoid loco-manipulation is particularly challenging because it requires coordinating locomotion, balance, and object interaction, making it difficult to learn from limited demonstrations or from scratch through Reinforcement Learning (RL). We introduce DiPLO (self-distilled Diffusion Planning for Loco-manipulation), a framework that combines a self-distilled diffusion planner with a low-level whole-body tracking policy. Our planner is pretrained via supervised learning on motions derived from human-to-humanoid retargeting, while the tracker is trained via RL to execute them. DiPLO centers around two complementary post-training stages. First, we introduce Self-Distillation Diffusion Policy Optimisation (SD:DPO) to remove the planner's dependence on strong task conditioning, which we identify as a crucial ingredient for effective pretraining, as downstream fine-tuning does not recover comparable performance when it is omitted.. We then fine-tune this distilled model with Diffusion-Augmented Proximal Policy Optimization (DA:PPO), a maximum-entropy RL method for diffusion models, to improve task success and align generated plans with the frozen tracker's execution capabilities. Together, these stages enable motion generation through state-history and goal conditioning alone, enabling 99% task success in pick-carry-place tasks, compared to just 45% for the pretrained model. The resulting planner also generates motions for unseen object goals, with further evaluations on kicking and pushing/pulling tasks. Finally, we deploy our pipeline on the hardware, using our DA:PPO planner-tracker on a Unitree G1, achieving repeated successful executions without any real-world fine-tuning.

Figures & tables

Fig. 2 : Overview of the pipeline. (a) SBTO turns each IK-retargeted demonstration $s^{IK}$ into a set of dynamically feasible motions $s^{D}$ , and symmetries further enlarge the data corpus. (b) The diffusion motion generator $G_{\phi}$ is pretrained to generate an $n$ -step feasible motion conditioned on the IK-retargeted motion prior, the state history and the goal, while the tracking policy $\pi_{\theta}$ is trained separately with RL to track $s^{D}$ . (c) Self-distillation with SD:DPO removes the need for the task prior: the student $G_{\phi}$ , with the prior token nulled, rolls out with the frozen tracker $\pi_{\theta}$ ; each plan it produces is noised to a random denoising step $k$ and denoised twice by the same network, with and without $s^{IK}$ , and the student’s reverse kernel $p_{\phi}$ is matched to the self-teacher’s stop-gradient kernel $\mathrm{sg}(p_{\phi})$ . (d) The distilled $G_{\phi}$ is then post-trained with DA:PPO, conditioned on the state history and the goal alone. In closed loop, $G_{\phi}$ plans $n$ steps, $\pi_{\theta}$ executes the first $m\leq n$ of them, and $G_{\phi}$ re-plans from the reached state. DA:PPO credits the rollout reward $R_{\mathrm{env}}$ to every denoising step of $G_{\phi}$ while $\pi_{\theta}$ stays fixed; the deployed pipeline runs the same loop without the reward.

Pretraining corpus	Reference source	Survival	Task success	MPKPE	Obj. pos.	Obj. ori.
		(%) $\uparrow$	(%) $\uparrow$	(cm) $\downarrow$	(cm) $\downarrow$	( ∘ ) $\downarrow$
SBTO Aug + Symmetry	Pretrained planner	96.5 $\pm$ 1.5	51.9 $\pm$ 4.5	8.5 $\pm$ 0.6	12.6 $\pm$ 2.4	14.0 $\pm$ 1.3
	$+$ DPPO-style ( $\mathcal{T}=0$ )	99.1 $\pm$ 0.1	94.0 $\pm$ 3.5	4.9 $\pm$ 0.3	5.2 $\pm$ 2.1	6.3 $\pm$ 1.2
	$+$ DA:PPO	100.0 $\pm$ 0.0	96.4 $\pm$ 3.4	4.3 $\pm$ 0.2	5.0 $\pm$ 2.0	5.8 $\pm$ 1.0
SBTO Aug	Pretrained planner	92.3 $\pm$ 4.0	47.5 $\pm$ 6.3	8.6 $\pm$ 0.7	14.7 $\pm$ 3.9	14.9 $\pm$ 1.5
SBTO Aug	$+$ DA:PPO	100.0 $\pm$ 0.0	92.5 $\pm$ 4.7	4.1 $\pm$ 0.1	4.9 $\pm$ 1.9	5.8 $\pm$ 1.1

TABLE I : Post-Training for Alignment on different pretraining corpora, evaluated in closed-loop using motions in the

Single

dataset. Each generator is finetuned for 500 iterations on its own pretraining corpus, with 2048 parallel environments. The last two rows involve no learned planner: the same tracker is tasked to track the IK-retargeted motion or its SBTO solution.

Fig. 3 : Planar robot trajectories aligned at the origin illustrating data augmentation for pick-carry-place task. Left: the original motions. Middle: dynamically feasible variations generated by SBTO from perturbed initial conditions. Right: the dataset after symmetry augmentation.

Fig. 4 : Closed-loop survival rates of the pretrained generator under different conditioning priors. Each pictogram marks in blue what the prior specifies; hollow markers indicate sparse frames.

Rotation	Survival	Task success	Obj. ori.
diffusion	(%) $\uparrow$	(%) $\uparrow$	( ∘ ) $\downarrow$
$\mathrm{SO}(3)$	96.5 $\pm$ 1.5	51.9 $\pm$ 4.5	14.0 $\pm$ 1.3
Euclidean	85.9 $\pm$ 5.0	46.4 $\pm$ 5.7	19.6 $\pm$ 2.5

TABLE II: Comparison of diffusing the rotational components of the generated motion (root orientation, object orientation) on

\mathrm{SO}(3)

vs. treating their

6

D coordinates as Euclidean. Evaluated closed-loop with the frozen tracker.

Model	Survival (%) $\uparrow$	Task success (%) $\uparrow$
State Diffusion + Tracker	96.5 $\pm$ 1.5	51.9 $\pm$ 4.5
Action Diffusion [ 13 ]	87.5 $\pm$ 3.7	45.3 $\pm$ 4.8
State-Action Diffusion [ 14 ]	58.2 $\pm$ 7.1	8.2 $\pm$ 3.6

TABLE III : Diffusion motion generator with tracking policy vs. diffusion action generators executing joint PD targets directly for loco-manipulation tasks. The state-action diffusion baseline is an unconditioned prior that is steered at test time.

		$+$ DA:PPO
Metric	Pretrained	$r_{\text{kin}}$	$r_{\text{goal}}$
Survival (%) $\uparrow$	96.5 $\pm$ 1.5	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0
Task success (%) $\uparrow$	51.9 $\pm$ 4.5	96.4 $\pm$ 3.4	92.9 $\pm$ 4.7
MPKPE (cm) $\downarrow$	8.5 $\pm$ 0.6	4.3 $\pm$ 0.2	8.0 $\pm$ 0.4
Obj. pos. (cm) $\downarrow$	12.6 $\pm$ 2.4	5.0 $\pm$ 2.0	7.2 $\pm$ 2.0
Obj. ori. ( ∘ ) $\downarrow$	14.0 $\pm$ 1.3	5.8 $\pm$ 1.0	7.5 $\pm$ 1.3

TABLE IV : DA:PPO finetuning with the IK-motion tracking reward

r_{\text{kin}}

vs. a goal-driven reward

r_{\text{goal}}

Task	Generator	Survival	Task success
		(%) $\uparrow$	(%) $\uparrow$
Pick-place	No-prior (pretrained)	46.9 $\pm$ 8.3	32.2 $\pm$ 6.8
	No-prior $+$ DA:PPO ( $r_{\text{kin}}$ )	74.1 $\pm$ 8.0	67.0 $\pm$ 8.6
	Self-Teacher (pretrained)	96.5 $\pm$ 1.5	51.9 $\pm$ 4.5
	Student (pretrained)	57.7 $\pm$ 9.2	45.4 $\pm$ 9.0
	Student $+$ DA:PPO	92.1 $\pm$ 4.8	92.1 $\pm$ 4.8

TABLE V : Diffusion motion generators post-trained using SD:DPO on different tasks. Self-Teacher and Student are the same pretrained generator, evaluated with the IK-retargeted task prior and with the null token

\varnothing

in its place, respectively. No-prior is a separate motion generator described in Section V-B that doesn’t receive any task prior upon pretraining.

Fig. 5 : Mean final position error (left) and orientation error (right) of the object for Student $+$ SD:DPO $+$ DA:PPO, with goals placed around the robot’s start. The robot starts at the center facing right; black arrowheads are the $29$ demonstrated goals, pointing along the goal yaw, and the dashed circle marks the farthest of them ( $1.9$ m). Each goal pose is evaluated using $100$ rollouts, with the objects’s initial pose randomised up to $5$ cm and $12^{\circ}$ .

Motion	Tracker	Survival	Task success
generator		(%) $\uparrow$	(%) $\uparrow$
Pretrained	Pretrained	96.8 $\pm$ 2.2	51.9 $\pm$ 4.5
	PPO ( $r_{\text{gen}}$ )	99.0 $\pm$ 1.0	87.9 $\pm$ 4.9
	PPO ( $r_{\text{gen}}{+}r_{\text{goal}}$ )	99.5 $\pm$ 0.3	89.3 $\pm$ 5.2
$+$ DA:PPO ( $r_{\text{kin}}$ )	Pretrained	100.0 $\pm$ 0.0	96.5 $\pm$ 1.5

TABLE VI : Comparison of fine-tuning the motion generator against fine-tuning the tracker. In the top block we use the pretrained diffusion model, and in the bottom block the diffusion model post-trained with DA:PPO. Tracker fine-tuning updates all parameters of the tracking policy with PPO for 1000 learning iterations, using the reward for tracking the generated reference (

r_{\text{gen}}

), with or without an additional goal term (

r_{\text{goal}}

); see Appendix -G .

Motions	Success	MPKPE	Final obj. pos.	Final obj. ori.
	$\uparrow$	(cm) $\downarrow$	(cm) $\downarrow$	( ∘ ) $\downarrow$
Repeated	10/10	6.5 $\pm$ 0.7	1.8 $\pm$ 0.8	3.4 $\pm$ 1.1
Distinct	5/5	6.8 $\pm$ 1.1	5.7 $\pm$ 5.0	12.8 $\pm$ 12.5

TABLE VII : Real-robot execution on the Unitree G1, for ten executions of a single goal ( repeated ) and one execution of distinct goals ( distinct ). Success is the number of successful over attempted executions; the remaining values are mean

\pm

sample standard deviation over executions.

Fig. 6 : Architecture of the denoiser network of the diffusion-based motion generator

G_{\phi}

, based on

L

-layers of Diffusion Transformer (DiT) blocks. At each denoising step

k

, along with the noised target tokens

s^{D,k}_{\tau}

(the motion chunk over the frames

\tau=t{+}1{:}t{+}n

at denoising step

k

) and the denoising step

k

via adaptive layer norm, the network is also conditioned on the goal token, the current state and history tokens, and the IK-retargeted motion token which serves as the prior to solve the task. The output is the denoised target

s^{D,k-1}_{\tau}

, and

G_{\phi}

iterates the network from

k=K

down to

k=1

, whose output

s^{D,0}_{\tau}

is the fully denoised motion.

Parameter	Value
Representation
Data rate	$50$ Hz
History / prediction horizon $n_{h}$ / $n$	$2$ / $64$ frames
Per-frame feature dimension	$48$
Goal dimension $g_{t}$	$10$
Feature normalization	per-channel mean/std

TABLE VIII : Pretraining configuration of the diffusion motion generator.

Stage	Quantity	Range
Reset	Object position $(x,y,z)$	$(\pm 0.05,\pm 0.05,\pm 0.0125)$ m
	Object orientation (r,p,y)	$(\pm 0.05,\pm 0.05,\pm 0.2)$ rad
	Robot root pose $(p,r)$	$(\pm 0.05,\pm 0.1)$ m, rad
	Robot root height / yaw	$\pm 0.01$ m / $\pm 0.2$ rad
	Robot joint positions	$\pm 0.1$ rad
Push	Robot linear / angular velocity	$\pm 0.3$ m/s / $\pm 0.45$ rad/s

TABLE IX : Domain randomization ranges used during finetuning of the diffusion motion generator. Values denote symmetric half-widths of uniform perturbations.

Reward Term	Equation	Weight
Motion Tracking
Root Position	$\exp\!\left(-5\\|\mathbf{p}_{t}-\mathbf{\hat{p}}_{t}^{\text{D}}\\|^{2}\right)$	0.5
Root Orientation	$\exp\!\left(-3\\|\mathbf{q}_{t}-\mathbf{\hat{q}}_{t}^{\text{D}}\\|^{2}\right)$	0.5
Body Position	$\exp\!\left(-5\\|\mathbf{p}_{\text{body},t}-\mathbf{\hat{p}}_{\text{body},t}^{\text{D}}\\|^{2}\right)$	1.0
Body Orientation	$\exp\!\left(-3\\|\boldsymbol{\theta}_{\text{body},t}-\boldsymbol{\hat{\theta}}_{\text{body},t}^{\text{D}}\\|^{2}\right)$	1.0
Body Linear Velocity	$\exp\!\left(-0.5\\|\mathbf{v}_{t}-\mathbf{\hat{v}}_{t}^{\text{D}}\\|^{2}\right)$	1.0

TABLE X : Reward terms used by the robot-object multi-motion tracking controller.

Reward Set	Equation	Weight
Kinematic Fidelity ( $r_{\text{kin}}$ )
Anchor Position	$\exp(-5\overline{\\|\mathbf{\hat{p}}_{t}-\mathbf{p}_{t}^{IK}\\|^{2}})$	0.25
Anchor Orientation	$\exp(-3\overline{\\|\mathbf{\hat{q}}_{t}-\mathbf{q}_{t}^{IK}\\|^{2}})$	0.25
Body Position	$\exp(-5\overline{\\|\mathbf{\hat{p}}_{\text{body},t}-\mathbf{p}_{\text{body},t}^{IK}\\|^{2}})$	0.50
Body Orientation	$\exp(-3\overline{\\|\mathbf{\hat{q}}_{\text{body},t}-\mathbf{q}_{\text{body},t}^{IK}\\|^{2}})$	0.50
Object Position	$\exp(-8\overline{\\|\mathbf{\hat{p}}_{\text{obj},t}-\mathbf{p}_{\text{obj},t}^{IK}\\|^{2}})$	0.25

TABLE XI : Post-training reward terms for the diffusion motion generator over a chunk of

T

steps.

(\hat{\cdot})

: realised motion;

(\cdot)^{IK}

: IK-retargeted motion;

(\cdot)^{G}

: goal object pose the generator is conditioned on. All squared-norm errors below are averaged over the chunk:

e\equiv\tfrac{1}{T}\sum_{t=1}^{T}e_{t}

. In the goal terms,

\theta_{t}

is the angle between the realised and goal object orientations, with

\sigma_{p}=7.8

cm and

\sigma_{\theta}=0.244

rad.

Parameter	Value
DA-MDP
Chunk horizon $n$ / executed frames $m$	$64$ / $56$
Denoising steps finetuned	all $K=10$
Temperature $\mathcal{T}$	$10^{-4}$
Discount $\gamma$ / GAE $\lambda$	$1.0$ / $0.95$
Step-variance multipliers	learned, initialized at $1$

TABLE XII : DA:PPO post-training configuration of the diffusion motion generator.

Parameter	Value
Rollouts
Executed frames per re-plan $m$	$56$
Parallel environments	$2048$
Iterations	$1000$
PPO update
Discount $\gamma$ / GAE $\lambda$	$0.99$ / $0.95$

TABLE XIII : Configuration of tracker fine-tuning with PPO.

Parameter	Value
Expert rollouts
Expert	Pretrained Tracking Policy
Randomization	Table IX
Noise on applied joint targets	Ornstein–Uhlenbeck, $\theta=0.8$
scale $\sigma$ (normalized action units)	$0.1$ and $0.25$
Stored action	tracker output without noise

TABLE XIV : Configuration of the action-level diffusion baselines. Where two values are separated by a bar, the first holds for the two conditioned models and the second for the unconditioned prior.

$\boldsymbol{n}$	$\boldsymbol{m}$	Survival	Task success	MPKPE	Obj. pos.	Obj. ori.
		(%) $\uparrow$	(%) $\uparrow$	(cm) $\downarrow$	(cm) $\downarrow$	( ∘ ) $\downarrow$
Without randomization
$20$	$12$	97.6 $\pm$ 2.3	89.7 $\pm$ 5.1	5.9 $\pm$ 0.4	4.6 $\pm$ 1.3	9.0 $\pm$ 2.7
$32$	$24$	100.0 $\pm$ 0.0	92.9 $\pm$ 4.7	5.6 $\pm$ 0.3	3.6 $\pm$ 0.4	6.6 $\pm$ 1.7
$48$	$40$	97.4 $\pm$ 2.5	87.7 $\pm$ 5.5	5.8 $\pm$ 0.5	5.2 $\pm$ 1.8	8.9 $\pm$ 3.0
$64$	$56$	100.0 $\pm$ 0.0	93.1 $\pm$ 4.7	5.0 $\pm$ 0.2	3.4 $\pm$ 0.4	7.0 $\pm$ 1.8

TABLE XV : Closed-loop performance of pretrained diffusion motion generators that differ only in the prediction horizon

n

, with

m=n-8

frames executed per re-plan. Evaluated with the frozen tracker over

16

rollouts on each motion. Values are mean

\pm

SEM over motions.

Panel of Figure 21 — Fig. 7 : Goal grid for (a) the pretrained Student and (b) Student $+$ SD:DPO, on the same episodes and color scales. Each map shows the mean final position error (left) and orientation error (right) of the object over the rollouts in which it was lifted by more than $12$ cm: $83\%$ in (a) and $78\%$ in (b), against $90\%$ for the final checkpoint. An arrow at the end of a color bar marks goal positions whose mean lies beyond its range, up to $0.48$ m in (a) and $46.6^{\circ}$ in (b).

Fig. 8 : Robot

x-y

trajectories for new goals chosen outside the demonstration set, for two different motions (a, b and c, d). In (a,c), we displace the goal by

0.5

1.0

m along

x

and

y

directions, while in (b,d), we rotate the goal pose around

z

axis by

\pm 90\deg

. We use different colors for new goals chosen away from the goal in the demonstration goal. The plots were generated using the Student

+

SD:DPO

+

DA:PPO model.

Humanoid Loco-Manipulation

Humanoid loco-manipulation studies how robots coordinate walking, balance, and whole-body interaction to move and handle objects.

See all 86 papers

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning
Mar 14, 2026Zhaoyuan Gu, Yipu Chen, Zimeng Chai +11Humanoid Loco-Manipulation Diffusion Policy
Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA 30332 USA
Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking
Jun 25, 2026Pranav Debbad, Kanish Thiagarajan, Victor Dhédin +2Humanoid Loco-Manipulation Robot Imitation Learning
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany.
LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation
Aug 7, 2026Cheng Guo, Mingzhe Ni, Angelo Cangelosi +1Humanoid Loco-Manipulation Latent Action Learning
Department of Computer Science, University of Manchester, Manchester, UK · Human-Robot Interfaces and Interaction Laboratory, Italian Institute of Technology, Genoa, Italy

Mar 14, 2026J/K move · Enter open · S save

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

Zhaoyuan Gu, Yipu Chen, Zimeng Chai, Alfred Cueva+10

Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA 30332 USA

Humanoid Loco-Manipulation Diffusion Policy Long-Horizon Robotic Manipulation Robot Policy Generalization RL for Diffusion Models

Open paper AskPDF arXiv

Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks. While diffusion policies (DPs) show promise for learning from demonstrations, deploying them on humanoids poses critical challenges: the motion planner trained offline is decoupled from the loco-manipulation controller, leading to poor command tracking, compounding distribution shift, and task failures. The common approach of scaling demonstration data is prohibitively expensive for high-dimensional humanoid systems. To address this challenge, we present REFINE-DP (REinforcement learning FINE-tuning of Diffusion Policy), a hierarchical framework that jointly optimizes a DP motion planner and an RL-based loco-manipulation controller. The DP is fine-tuned via a PPO-based diffusion policy gradient to improve task success rate, while the controller is simultaneously updated to accurately track the planner's evolving command distribution, reducing the distributional mismatch that degrades motion quality. We validate REFINE-DP on a humanoid robot performing loco-manipulation tasks, including door traversal and long-horizon object transport. REFINE-DP achieves an over 90% success rate in simulation, even in out-of-distribution cases not seen in the pre-training data, and enables real-world execution without privileged state information. Our proposed method substantially outperforms pre-trained DP baselines and demonstrates that RL fine-tuning is key to reliable humanoid loco-manipulation. https://refine-dp.github.io/REFINE-DP/

Abstract

Humanoid Loco-Manipulation

Explore similar work

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking

LUCID: Latent-Skill Unified Control via Imagined Dynamics for Long-Horizon Humanoid Loco-Manipulation