Modeling and Understanding Neural Weights of Humanoid Policies with Diffusion

Huixuan Cheng; Bike Zhang; Shankar Sastry; Koushil Sreenath

cs.ROOct 14, 2026

Modeling and Understanding Neural Weights of Humanoid Policies with Diffusion

Authors: Huixuan Cheng, Bike Zhang, Shankar Sastry, Koushil Sreenath

Organizations: The authors are with the University of California, Berkeley.

Abstract

Recent advances in robot learning have driven rapid progress in humanoid control, enabling increasingly agile behaviors in the real world. However, humanoid control networks are typically obtained through conventional training procedures, while comparatively little attention has been given to directly modeling the distribution of their learned weights and generating functional networks in weight space. In this work, we investigate whether high-performing humanoid control networks can be generatively modeled directly through their weights. We adopt a latent diffusion framework that couples a linear autoencoder with a Diffusion Transformer. The autoencoder compresses actor network parameters into a compact latent space, and the Diffusion Transformer models the distribution of these latents. Trained on networks collected from reinforcement learning for the Unitree G1 robot, our approach generates deployable networks that perform various motions in the real world, including walking, dancing, and spin kick. Beyond generation, we further study the learned weight space to better understand the structure captured by these humanoid control networks. We identify a structural principle in the first-layer weights that consistently reflects the robot's kinematic prior. Notably, this kinematic prior emerges under both Reinforcement Learning (RL) and Imitation Learning (IL) training, suggesting that it is not specific to a particular optimization process but may instead reflect common knowledge associated with humanoid embodiment. Together, these results demonstrate the feasibility of generative modeling in humanoid network weight space, while providing initial insights into the information encoded by their learned weights. Additional results are available on our project website: https://humanoid-weight-diffusion.github.io/

Figures & tables

Fig. 1 : Real-world deployment of networks with generated weights. Top: A walking motion. Middle: A dancing motion. Bottom: A spin kick motion. For each motion, the corresponding network weights are denoised by a motion-specific diffusion model and subsequently deployed zero shot in the real world.

Fig. 2 : Method overview. (a) We collect actor weights from 750 training runs with random seeds and retain eight post-convergence networks from each run. Each network is a three-layer MLP. (b) The cluster-wise linear autoencoder is implemented using single-layer encoders and decoders. (c) A diffusion transformer is trained with $x$ -pred loss to model the latent distribution. During inference, the model generates novel network weights that can be deployed in the real world.

	Dataset		DiT (mean)		DiT (best 90%)		DiT (best)
Motion	Succ $\uparrow$	TE $\downarrow$	Succ $\uparrow$	TE $\downarrow$	Succ $\uparrow$	TE $\downarrow$	Succ $\uparrow$	TE $\downarrow$
Walking	$0.999\pm 0.001$	$0.162\pm 0.003$	$0.993\pm 0.023$	$0.191\pm 0.020$	$0.997\pm 0.003$	$0.186\pm 0.013$	1.000	0.161
Dancing	$0.998\pm 0.004$	$0.037\pm 0.001$	$0.941\pm 0.213$	$0.042\pm 0.007$	$0.996\pm 0.006$	$0.040\pm 0.003$	1.000	0.036
Spin kick	$0.997\pm 0.004$	$0.046\pm 0.002$	$0.943\pm 0.289$	$0.053\pm 0.007$	$0.993\pm 0.009$	$0.051\pm 0.004$	1.000	0.044
Dataset, DiT-mean and DiT best 90% results are reported as mean $\pm$ std. DiT-best-90% is the mean over the $90$ best of the $100$ generated samples. DiT best (best of $100$ samples) corresponds to the per-metric optimum. Samples are ranked separately for each metric. TE denotes tracking error: MPKPE (m) for motion tracking, and mean linear velocity command error (m/s) for walking. For walking, the command includes step changes, so the tracking error also captures transient errors during command transitions.

TABLE I : Generated Networks vs. Training Data

Motion	embedding=512		embedding=768		embedding=1024
	Prog $\uparrow$	TE $\downarrow$	Prog $\uparrow$	TE $\downarrow$	Prog $\uparrow$	TE $\downarrow$
Walking	$0.643\pm 0.373$	$0.513\pm 0.284$	$0.890\pm 0.253$	$0.298\pm 0.139$	$\boldsymbol{0.997\pm 0.010}$	$\boldsymbol{0.191\pm 0.020}$
Dancing	$0.536\pm 0.231$	$0.076\pm 0.012$	$0.651\pm 0.243$	$0.071\pm 0.012$	$\boldsymbol{0.965\pm 0.154}$	$\boldsymbol{0.042\pm 0.007}$
Spin kick	$0.296\pm 0.122$	$0.082\pm 0.008$	$0.873\pm 0.198$	$0.067\pm 0.010$	$\boldsymbol{0.969\pm 0.136}$	$\boldsymbol{0.053\pm 0.007}$
All results are reported as mean $\pm$ std over diffusion samples. Prog is task progress in the range $[0,1]$ : for motion tracking, it is the fraction of reference motion frames completed; for walking, it denotes the fraction of the max episode duration (20 s) survived. TE denotes tracking error: MPKPE (m) for motion tracking, and mean linear velocity command error (m/s) for walking.

TABLE II : Effect of Transformer Embedding Dimension

	Walking			Dancing			Spin kick
Group	$\sigma$	Succ $\uparrow$	TE $\downarrow$	$\sigma$	Succ $\uparrow$	TE $\downarrow$	$\sigma$	Succ $\uparrow$	TE $\downarrow$
Training	$0.00$	$0.999$	$0.162$	$0.00$	$0.998$	$0.037$	$0.00$	$0.997$	$0.046$
Noise $\sigma{=}0.01$	$0.01$	$0.999$	$0.161$	$0.01$	$0.997$	$0.036$	$0.01$	$0.997$	$0.047$
Noise $\sigma{=}0.05$	$0.05$	$0.998$	$0.172$	$0.05$	$0.990$	$0.042$	$0.05$	$0.984$	$0.052$
Noise $\sigma{=}0.10$	$0.10$	$0.959$	$0.226$	$0.10$	$0.629$	$0.057$	$0.10$	$0.650$	$0.068$
Noise, matched	$0.110$	$0.914$	$0.248$	$0.078$	$0.865$	$0.051$	$0.090$	$0.783$	$0.064$

TABLE III : Comparison between diffusion-generated networks and noise-perturbed training networks across motions.

Fig. 3 : t-SNE visualization . Training and generated network weights for the spin kick motion.

Fig. 4 : Kinematic prior in weights . (a) A visualization of the first-layer weight heat map for the walking motion, which reveals a structural principle that certain input regions exhibit larger weight magnitudes. (b) Joint-wise weight distributions across motions demonstrate that hip pitch, knee, and waist joints are assigned higher importance, suggesting the presence of a kinematic prior.

Fig. 5 : First-layer weight comparison between RL and IL for walking . Top: RL-trained weight heat map. Bottom: IL-trained weight heat map, distilled from the top RL network. While trained via different optimization objectives, they both exhibit kinematic prior in the first-layer weights.

Fig. 6 : Kinematic prior for both RL and IL networks. For each motion, the dashed and solid curves largely overlap after distillation, indicating that the first-layer kinematic prior is preserved in the IL network.

RL / IL	Walking	Dancing	Spin kick
Succ	0.9994 / 0.9979	0.9973 / 0.9948	0.9972 / 0.9919
TE	0.1599 / 0.1609	0.0340 / 0.0348	0.0466 / 0.0478

TABLE IV : Simulation results for RL teachers and their distilled IL students.

Fig. 7 : Effect of domain randomization on first-layer weights for dancing. First-layer weight heat maps of RL networks trained from the same initialization (a) with DR and (b) without DR, together with (c) their element-wise difference. Despite their overall similarity, the structured weight difference resembles the RL-trained weights.

Setting	Value
Simulation	mjlab (MuJoCo)
Robot	Unitree G1
Control mode	Velocity-command locomotion / Motion tracking
RL algorithm	PPO (RSL-RL)
Parallel environments	$4096$
Max iterations	$4500$

TABLE V : RL policy training configuration (locomotion: walking / motion-tracking: dancing, spin kick).

	Setting	Value
Architecture	Type	Two-cluster linear MLP
	Cluster input dims (Walking)	$12800,\,20253$
	Cluster input dims (Dancing, Spin kick)	$19840,\,20253$
	Input dimension $L$ (Walking)	$33053$
	Input dimension $L$ (Dancing, Spin kick)	$40093$
	Per-cluster latent dim	$1024$

TABLE VI : Autoencoder configuration: architecture and training hyperparameters for the walking, dancing, and spin kick motions.

Motion	Config	MSE
Walking	2-cluster	$\mathbf{1.87\times 10^{-4}}$
Walking	1-cluster (n=2048)	$1.05\times 10^{-3}$
Spin kick	2-cluster	$\mathbf{1.07\times 10^{-4}}$
Spin kick	1-cluster (n=2048)	$5.53\times 10^{-4}$
Dancing	2-cluster	$\mathbf{6.36\times 10^{-5}}$
Dancing	1-cluster (n=2048)	$3.10\times 10^{-4}$

TABLE VII : Cluster-structure ablation on three motions. MSE is the per-sample mean of reconstruction error on each dataset.

	Component	Value
Architecture	Input dimension	$2048$
	Sequence length	$128$
	Patch size	$16$
	Embedding dimension	$1024$
	Transformer depth	$8$
	Attention heads	$8$

TABLE VIII : DiT configuration: architecture and training/sampling schedule.

Panel of Figure 16 — Fig. 8 : t-SNE Visualization. Training, generated and deployed network weights for the walking and dancing motions.

Fig. 9 : Subspace similarity across motions. Applying linear CKA to measure subspace similarity on shared inputs across motions, we observe near-perfect alignment within each motion, indicating stable knowledge extraction. Cross motion similarity also remains high, suggesting substantial shared knowledge with minor motion-specific adaptation.

Panel of Figure 18 — Fig. 10 : First-layer heat maps . (a) Dancing weight heat map. (b) Spin kick weight heat map. The first-layer weights consistently exhibit the structural principle across joints.

Buffer	$N_{\mathrm{env}}\!\times\!T\!\times\!R$	Samples	Obs
Walking	$1024\!\times\!500\!\times\!2$	1,024,000	0 99
Dancing	$512\!\times\!892\!\times\!2$	913,408	154
Spin kick	$512\!\times\!232\!\times\!8$	950,272	154

TABLE IX : Distillation buffers collected from RL rollouts.

Setting	Value
Objective	MSE
Optimizer	AdamW
Learning rate	$3\times 10^{-4}\rightarrow 1\times 10^{-5}$ (cosine)
Weight decay	$1\times 10^{-5}$
Batch size	4096
Epochs	300

TABLE X : Distillation configuration. Settings are shared across all three motions

Fig. 11a : First-layer weight comparison between RL and IL for dancing.

Fig. 11b : First-layer weight comparison between RL and IL for spin kick.

Term	Randomized quantity	Walking	Dancing	Sampling
Foot friction	Sliding friction coeff. of foot geoms	$\mathcal{U}(0.3,\,1.2)$	$\mathcal{U}(0.3,\,1.2)$	per env
Torso CoM offset	CoM position of torso link, per axis	$\pm(2.5,2.5,3.0)$ cm	$\pm(2.5,5.0,5.0)$ cm	per env, per axis
Encoder bias	Constant offset on measured joint pos.	$\pm 0.015$ rad	$\pm 0.010$ rad	per env, per joint

Table 23

Explore similar work

DreamControl-v2: Simpler and Scalable Autonomous Humanoid Skills via Trainable Guided Diffusion Priors
Mar 31, 2026Sudarshan Harithas, Sangkyung Kwak, Pushkal Katara +6Robot Skill Learning Humanoid Robot Control
General Robotics, USA · Brown University, USA · University of California, Berkeley, USA
NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces
Feb 26, 2026Jiwoo Kim, Swarajh Mehta, Hao-Lun Hsu +3Diffusion Transformer Neural Network Generalization
Department of Electrical and Computer Engineering, Duke University, NC, USA · Department of Computer Science, Duke University, NC, USA · Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, MA, USA
Continual Humanoid Motion Learning
Oct 3, 2026Zhewen He, Hao Huang, Geeta Chandra Raju Bethala +4Continual Robot Learning Humanoid Motion Tracking
New York University Abu Dhabi, Abu Dhabi, United Arab Emirates. · Fudan University, Shanghai, China.

Mar 31, 2026J/K move · Enter open · S save

DreamControl-v2: Simpler and Scalable Autonomous Humanoid Skills via Trainable Guided Diffusion Priors

Sudarshan Harithas, Sangkyung Kwak, Pushkal Katara, Srujan Deolasee+5

General Robotics, USA · Brown University, USA · University of California, Berkeley, USA

Robot Skill Learning Humanoid Robot Control Humanoid Loco-Manipulation Robotic RL Robot Motion Generation Cross-Embodiment Robot Learning

Open paper AskPDF arXiv

Developing robust autonomous loco-manipulation skills for humanoids remains an open problem in robotics. While RL has been applied successfully to legged locomotion, applying it to complex, interaction-rich manipulation tasks is harder given long-horizon planning challenges for manipulation. A recent approach along these lines is DreamControl, which addresses these issues by leveraging off-the-shelf human motion diffusion models as a generative prior to guide RL policies during training. In this paper, we investigate the impact of DreamControl's motion prior and propose an improved framework that trains a guided diffusion model directly in the humanoid robot's motion space, aggregating diverse human and robot datasets into a unified embodiment space. We demonstrate that our approach captures a wider range of skills due to the larger training data mixture and establishes a more automated pipeline by removing the need for manual filtering interventions. Furthermore, we show that scaling the generation of reference trajectories is important for achieving robust downstream RL policies. We validate our approach through extensive experiments in simulation and on a real Unitree-G1.

Abstract

Explore similar work

DreamControl-v2: Simpler and Scalable Autonomous Humanoid Skills via Trainable Guided Diffusion Priors

NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces

Continual Humanoid Motion Learning