Recent advances in robot learning have driven rapid progress in humanoid control, enabling increasingly agile behaviors in the real world. However, humanoid control networks are typically obtained through conventional training procedures, while comparatively little attention has been given to directly modeling the distribution of their learned weights and generating functional networks in weight space. In this work, we investigate whether high-performing humanoid control networks can be generatively modeled directly through their weights. We adopt a latent diffusion framework that couples a linear autoencoder with a Diffusion Transformer. The autoencoder compresses actor network parameters into a compact latent space, and the Diffusion Transformer models the distribution of these latents. Trained on networks collected from reinforcement learning for the Unitree G1 robot, our approach generates deployable networks that perform various motions in the real world, including walking, dancing, and spin kick. Beyond generation, we further study the learned weight space to better understand the structure captured by these humanoid control networks. We identify a structural principle in the first-layer weights that consistently reflects the robot's kinematic prior. Notably, this kinematic prior emerges under both Reinforcement Learning (RL) and Imitation Learning (IL) training, suggesting that it is not specific to a particular optimization process but may instead reflect common knowledge associated with humanoid embodiment. Together, these results demonstrate the feasibility of generative modeling in humanoid network weight space, while providing initial insights into the information encoded by their learned weights. Additional results are available on our project website: https://humanoid-weight-diffusion.github.io/
Figures & tables
Fig. 1 : Real-world deployment of networks with generated weights. Top: A walking motion. Middle: A dancing motion. Bottom: A spin kick motion. For each motion, the corresponding network weights are denoised by a motion-specific diffusion model and subsequently deployed zero shot in the real world.
Fig. 2 : Method overview. (a) We collect actor weights from 750 training runs with random seeds and retain eight post-convergence networks from each run. Each network is a three-layer MLP. (b) The cluster-wise linear autoencoder is implemented using single-layer encoders and decoders. (c) A diffusion transformer is trained with x -pred loss to model the latent distribution. During inference, the model generates novel network weights that can be deployed in the real world.
Dataset
DiT (mean)
DiT (best 90%)
DiT (best)
Motion
Succ ↑
TE ↓
Succ ↑
TE ↓
Succ ↑
TE ↓
Succ ↑
TE ↓
Walking
0.999±0.001
0.162±0.003
0.993±0.023
0.191±0.020
0.997±0.003
0.186±0.013
1.000
0.161
Dancing
0.998±0.004
0.037±0.001
0.941±0.213
0.042±0.007
0.996±0.006
0.040±0.003
1.000
0.036
Spin kick
0.997±0.004
0.046±0.002
0.943±0.289
0.053±0.007
0.993±0.009
0.051±0.004
1.000
0.044
Dataset, DiT-mean and DiT best 90% results are reported as mean ± std. DiT-best-90% is the mean over the 90 best of the 100 generated samples. DiT best (best of 100 samples) corresponds to the per-metric optimum. Samples are ranked separately for each metric. TE denotes tracking error: MPKPE (m) for motion tracking, and mean linear velocity command error (m/s) for walking. For walking, the command includes step changes, so the tracking error also captures transient errors during command transitions.
TABLE I : Generated Networks vs. Training Data
Motion
embedding=512
embedding=768
embedding=1024
Prog ↑
TE ↓
Prog ↑
TE ↓
Prog ↑
TE ↓
Walking
0.643±0.373
0.513±0.284
0.890±0.253
0.298±0.139
0.997±0.010
0.191±0.020
Dancing
0.536±0.231
0.076±0.012
0.651±0.243
0.071±0.012
0.965±0.154
0.042±0.007
Spin kick
0.296±0.122
0.082±0.008
0.873±0.198
0.067±0.010
0.969±0.136
0.053±0.007
All results are reported as mean ± std over diffusion samples. Prog is task progress in the range [0,1] : for motion tracking, it is the fraction of reference motion frames completed; for walking, it denotes the fraction of the max episode duration (20 s) survived. TE denotes tracking error: MPKPE (m) for motion tracking, and mean linear velocity command error (m/s) for walking.
TABLE II : Effect of Transformer Embedding Dimension
Walking
Dancing
Spin kick
Group
σ
Succ ↑
TE ↓
σ
Succ ↑
TE ↓
σ
Succ ↑
TE ↓
Training
0.00
0.999
0.162
0.00
0.998
0.037
0.00
0.997
0.046
Noise σ=0.01
0.01
0.999
0.161
0.01
0.997
0.036
0.01
0.997
0.047
Noise σ=0.05
0.05
0.998
0.172
0.05
0.990
0.042
0.05
0.984
0.052
Noise σ=0.10
0.10
0.959
0.226
0.10
0.629
0.057
0.10
0.650
0.068
Noise, matched
0.110
0.914
0.248
0.078
0.865
0.051
0.090
0.783
0.064
TABLE III : Comparison between diffusion-generated networks and noise-perturbed training networks across motions.
Fig. 3 : t-SNE visualization . Training and generated network weights for the spin kick motion.
Fig. 4 : Kinematic prior in weights . (a) A visualization of the first-layer weight heat map for the walking motion, which reveals a structural principle that certain input regions exhibit larger weight magnitudes. (b) Joint-wise weight distributions across motions demonstrate that hip pitch, knee, and waist joints are assigned higher importance, suggesting the presence of a kinematic prior.
Fig. 5 : First-layer weight comparison between RL and IL for walking . Top: RL-trained weight heat map. Bottom: IL-trained weight heat map, distilled from the top RL network. While trained via different optimization objectives, they both exhibit kinematic prior in the first-layer weights.
Fig. 6 : Kinematic prior for both RL and IL networks. For each motion, the dashed and solid curves largely overlap after distillation, indicating that the first-layer kinematic prior is preserved in the IL network.
RL / IL
Walking
Dancing
Spin kick
Succ
0.9994 / 0.9979
0.9973 / 0.9948
0.9972 / 0.9919
TE
0.1599 / 0.1609
0.0340 / 0.0348
0.0466 / 0.0478
TABLE IV : Simulation results for RL teachers and their distilled IL students.
Fig. 7 : Effect of domain randomization on first-layer weights for dancing. First-layer weight heat maps of RL networks trained from the same initialization (a) with DR and (b) without DR, together with (c) their element-wise difference. Despite their overall similarity, the structured weight difference resembles the RL-trained weights.
Setting
Value
Simulation
mjlab (MuJoCo)
Robot
Unitree G1
Control mode
Velocity-command locomotion / Motion tracking
RL algorithm
PPO (RSL-RL)
Parallel environments
4096
Max iterations
4500
TABLE V : RL policy training configuration (locomotion: walking / motion-tracking: dancing, spin kick).
Setting
Value
Architecture
Type
Two-cluster linear MLP
Cluster input dims (Walking)
12800,20253
Cluster input dims (Dancing, Spin kick)
19840,20253
Input dimension L (Walking)
33053
Input dimension L (Dancing, Spin kick)
40093
Per-cluster latent dim
1024
TABLE VI : Autoencoder configuration: architecture and training hyperparameters for the walking, dancing, and spin kick motions.
Motion
Config
MSE
Walking
2-cluster
1.87×10−4
1-cluster (n=2048)
1.05×10−3
Spin kick
2-cluster
1.07×10−4
1-cluster (n=2048)
5.53×10−4
Dancing
2-cluster
6.36×10−5
1-cluster (n=2048)
3.10×10−4
TABLE VII : Cluster-structure ablation on three motions. MSE is the per-sample mean of reconstruction error on each dataset.
Component
Value
Architecture
Input dimension
2048
Sequence length
128
Patch size
16
Embedding dimension
1024
Transformer depth
8
Attention heads
8
TABLE VIII : DiT configuration: architecture and training/sampling schedule.
Fig. 8 : t-SNE Visualization. Training, generated and deployed network weights for the walking and dancing motions.
Fig. 9 : Subspace similarity across motions. Applying linear CKA to measure subspace similarity on shared inputs across motions, we observe near-perfect alignment within each motion, indicating stable knowledge extraction. Cross motion similarity also remains high, suggesting substantial shared knowledge with minor motion-specific adaptation.
Fig. 10 : First-layer heat maps . (a) Dancing weight heat map. (b) Spin kick weight heat map. The first-layer weights consistently exhibit the structural principle across joints.
Buffer
Nenv×T×R
Samples
Obs
Walking
1024×500×2
1,024,000
0 99
Dancing
512×892×2
913,408
154
Spin kick
512×232×8
950,272
154
TABLE IX : Distillation buffers collected from RL rollouts.
Setting
Value
Objective
MSE
Optimizer
AdamW
Learning rate
3×10−4→1×10−5 (cosine)
Weight decay
1×10−5
Batch size
4096
Epochs
300
TABLE X : Distillation configuration. Settings are shared across all three motions
Fig. 11a : First-layer weight comparison between RL and IL for dancing.
Fig. 11b : First-layer weight comparison between RL and IL for spin kick.
Developing robust autonomous loco-manipulation skills for humanoids remains an open problem in robotics. While RL has been applied successfully to legged locomotion, applying it to complex, interaction-rich manipulation tasks is harder given long-horizon planning challenges for manipulation. A recent approach along these lines is DreamControl, which addresses these issues by leveraging off-the-shelf human motion diffusion models as a generative prior to guide RL policies during training. In this paper, we investigate the impact of DreamControl's motion prior and propose an improved framework that trains a guided diffusion model directly in the humanoid robot's motion space, aggregating diverse human and robot datasets into a unified embodiment space. We demonstrate that our approach captures a wider range of skills due to the larger training data mixture and establishes a more automated pipeline by removing the need for manual filtering interventions. Furthermore, we show that scaling the generation of reference trajectories is important for achieving robust downstream RL policies. We validate our approach through extensive experiments in simulation and on a real Unitree-G1.
Generative modeling of neural network parameters is often tied to architectures because standard parameter representations rely on known weight-matrix dimensions. Generation is further complicated by permutation symmetries that allow networks to model similar input-output functions while having widely different, unaligned parameterizations. In this work, we introduce Neural Network Diffusion Transformers (NNiTs), which generate weights in a width-agnostic manner by tokenizing weight matrices into patches and modeling them as locally structured fields. We establish that Graph HyperNetworks (GHNs) with a convolutional neural network (CNN) decoder structurally align the weight space, creating the local correlation necessary for patch-based processing. Focusing on Multilayer Perceptrons (MLPs), where permutation symmetry is especially apparent, NNiTs generate fully functional networks across a range of architectures. Our approach jointly models discrete architecture tokens and continuous weight patches within a single sequence model. On ManiSkill3 robotics tasks, NNiT achieves >85% success on architecture topologies unseen during training, while baseline approaches fail to generalize; the same pipeline also generalizes to MNIST classification beyond the robotic control setting.
Jiwoo Kim, Swarajh Mehta, Hao-Lun Hsu +3
Department of Electrical and Computer Engineering, Duke University, NC, USA · Department of Computer Science, Duke University, NC, USA · Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, MA, USA
Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered. We study continual learning for humanoid whole-body motion, where a single controller must acquire skills from a sequential task stream without revisiting past data. We introduce Similarity-guided LoRA-PNN, a progressive neural network (PNN) policy that prevents catastrophic forgetting by construction while reusing knowledge across skills through lightweight low-rank adaptation. A two-level motion-similarity measure, built from dynamic time warping aggregated by optimal transport, decides which prior skill to build on and how much new capacity to allocate, yielding strong forward transfer and large efficiency gains. Across six sequentially learned skill categories, our similarity-guided LoRA policy attains the best forward transfer (0.125 vs. 0.079) and the highest average accuracy among all methods, while saving up to 94.5% of trainable parameters and 40.8% of training time. The resulting controller reaches 96.13% sim-to-sim transfer and is deployed on a physical Unitree G1. Our code is available at https://anonymous.4open.science/r/continual-humanoid-learning-35D3.