Walk Like This: Stylized Locomotion for Physics-Based Characters through Example-Driven Diffusion

Joshua Aurand; Agon Serifi; Sammy Christen; Ruben Grandia; Espen Knoop; Marco Hutter; Moritz Bächer

cs.ROOct 13, 2026

Walk Like This: Stylized Locomotion for Physics-Based Characters through Example-Driven Diffusion

Authors: Joshua Aurand, Agon Serifi, Sammy Christen, Ruben Grandia, Espen Knoop, Marco Hutter, Moritz Bächer

Organizations: Disney Research Zürich, Switzerland · ETH Zürich, Switzerland

Abstract

We present a real-time framework for expressive legged locomotion that enables independent control over velocity and style. Unlike existing data-driven controllers, which typically require explicit style labels for training, our method efficiently leverages large-scale unstructured motion data and generalizes to unseen styles. We disentangle style from functional locomotion through a tailored sampling strategy used to train a style encoder whose output conditions an autoregressive diffusion model. During inference, the user provides a style reference clip and a joystick velocity command. The diffusion model then generates a kinematic reference tracked by a reinforcement learning controller. To improve user controllability and stylization strength, we propose an attention biasing mechanism and a velocity command correction strategy. We demonstrate our method on a physics-based humanoid character in simulation and on a physical robot, showcasing robust real-time control and continuous style modulation. The results can be best seen in our supplemental video (https://www.youtube.com/watch?v=roFnS6N7nJ4).

Figures & tables

Figure 1: We introduce a real-time controller for physics-based characters that lets users steer the character with joystick commands while conditioning on style from unseen examples of arbitrary duration and various sources (e.g., MoCap, video).

Approach	RT	Gen	Uns Sty	Phys	Joy Ctrl
Style Transfer [ 1 , 14 , 54 , 39 , 16 , 2 , 32 , 55 ]	✗	$\thicksim$	✓	✗	✗
Implicit Rewards [ 24 , 30 ]	✓	$\thicksim$	✗	✓	$\thicksim$
“Physical Diffusion” [ 35 , 43 , 50 , 49 ]	$\thicksim$	✓	✗	✓	$\thicksim$
CAMDM [ 3 ]	✓	✓	✗	✗	✓
Ours	✓	✓	✓	✓	✓

Table 1: Comparison to Related Work. Real-time (RT), generative (Gen), unseen styles (Uns Sty), physics-based model (Phys), joystick control (Joy Ctrl). Legend: ✓ Yes

\thicksim

Partially ✗ No

Figure 2: Overview. We train an autoregressive diffusion model on unstructured motion datasets. We explicitly encode style in the style encoder, enabling generalization to unseen style references of varying length at inference. During inference , users can control the robot’s velocity through joystick commands, while selecting a desired style clip. The motion diffusion model’s output is autoregressively rolled out and passed to an RL-based tracking controller.

Figure 3: Style Encoding. Generated motions conditioned on interpolated style latents of styles from the 100STYLE dataset (top and middle row) and latents computed from unseen motions extracted from videos (bottom row).

Training Data	Loop Mode	Disturbances	FID $\downarrow$	Style Acc.	MPJAE	Falls
			FID $\downarrow$	[%] $\uparrow$	[ $\deg$ ] $\downarrow$	[#/ $\mathrm{min}$ ] $\downarrow$
MoCap	Open	✗	0.16	92.0	5.2	1.104
Generated	Open	✗	0.35	85.1	8.3	0.001
Generated	Closed	✗	0.45	79.6	7.8	0.001
Generated	Open	✓	0.84	62.9	16.0	3.812
Generated	Closed	✓	0.95	58.2	15.9	4.431

Table 3: RL Tracking Policy. We evaluate the effect of training data and loop mode on the performance of the tracking policy. Additionally, we compare open- and closed-loop performance under strong external disturbances.

Figure 4: G1 Robot Deployment. Left: Continuous joystick control with dynamic style modulation. Right: Demonstration of the framework generating five distinct motion styles.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
$d_{\text{model}}$	256
FeedForward Size	1024
# Layers	4
# Heads	4
Dropout Rate	0.2
Activation Fn.	GELU

Appendix

Table 7

Figure A.1: Smoothing Scheme. To achieve smooth transitions between consecutive diffusion calls, we predict the next

F

states

\mathbf{M}_{\text{fut}}^{\text{nxt}}

at the current timestep

t

, then blend the first

K

states in

\mathbf{M}_{\text{fut}}^{\text{nxt}}

with the last

K

states of the previous prediction

\mathbf{M}_{\text{fut}}^{\text{prv}}

, then copy the next

F-2K

states from

\mathbf{M}_{\text{fut}}^{\text{nxt}}

before incrementing the timestep

t:=t+(F-K)

and repeating the process. More formally, we weight state

\mathbf{m}^{\text{nxt}}_{i}

w=\nicefrac{{i}}{{(K+1)}}

and state

\mathbf{m}^{\text{prv}}_{(F-K)+i}

1-w

to form the blended states

\mathbf{m}_{t+i}

for

i=1,\dots,K

. The visualization shows the

F=6,K=2

case.

Tracking
Name	Error / Term	$\sigma$	Humanoid	G1 Robot
Root position	$\\|\mathbf{x}-\hat{\mathbf{x}}\\|^{2}$	0.1	1.0	2.0
Root orientation	$\\|\mathbf{r}\ominus\hat{\mathbf{r}}\\|_{F}^{2}$	0.5	1.0	2.0
Root lin. velocity	$\\|\dot{\mathbf{x}}-\hat{\dot{\mathbf{x}}}\\|^{2}$	1.0	1.0	2.0
Root ang. velocity	$\\|\bm{\omega}-\hat{\bm{\omega}}\\|^{2}$	1.0	1.0	2.0
Upper body joints	$\\|\mathbf{q}^{u}-\hat{\mathbf{q}}^{u}\\|^{2}$	0.5	5.0	5.0

Appendix

Table A.1: RL Reward Terms. Reward terms and weights for the Humanoid and the G1 robot. Tracking and joystick rewards use an exponential kernel

\exp(-e/\sigma)

on the listed squared error

e

. We drop the time index

t

for conciseness. Norms are Euclidean unless specified otherwise.

h_{f}

measures the foot-ground distance while

c_{f}^{\text{sim}},c_{f}^{\text{ref}}

are simulation or reference provided contact flags.

Parameter	Value
$\varepsilon_{\text{clip}}$	0.2
Value Loss Coefficient	1.0
Entropy Coefficient	0.005
Discount Factor $\gamma$	0.99
GAE Factor $\lambda$	0.99
Desired KL Divergence	0.01

Appendix

Table 10

Additive Observation Noise
Term	Unit	Humanoid	Robot
Projected Gravity	$\text{\,}\mathrm{rad}$	$-$	$\mathcal{U}(-0.05,0.05)$
Root Angular Velocity	$\text{\,}\mathrm{rad}\text{\,}{\mathrm{s}}^{-1}$	$-$	$\mathcal{U}(-0.2,0.2)$
Joint Position	$\text{\,}\mathrm{rad}$	$-$	$\mathcal{U}(-0.01,0.01)$
Joint Velocity	$\text{\,}\mathrm{rad}\text{\,}{\mathrm{s}}^{-1}$	$-$	$\mathcal{U}(-1.5,1.5)$
Physics Randomization

Appendix

Table A.2: Domain Randomization. Randomization ranges applied during policy training. A dash indicates that the term is not randomized for the respective embodiment.

Figure A.2: Interactive Control. Example of interactive control where a physics-based humanoid follows velocity commands in various styles. The green dots indicate the trajectory generated from the given velocity commands.

Same window training	Different window training (Ours)

Appendix

Figure A.4: Sampling Scheme. We show the first frame of 5 consecutively predicted motion windows of models trained with and without our proposed sampling scheme. Without our sampling approach (“same window training”), the model collapses and keeps repeating the reference motion. With our sampling approach (“different window training”), the model learns to extract the style characteristic from the reference motion.

Unmodified	CFG	Attention Bias

Appendix

Figure A.5: Attention Bias Quality. We compare unmodified inference with CFG and attention biasing for improved stylization.

Figure A.6: Attention Bias Sensitivity. We show how varying bias parameters affect performance by varying one weight at a time while keeping the other fixed at 0. Increasing either weight to large magnitudes worsens command adherence while increasing generation artifacts. Large positive

w_{\text{pst}}

weights specifically result in frozen poses, which is why foot sliding is at zero.

Figure A.7: Unseen Style. Generation of motion with unseen styles from 100STYLE, shown as the green snapshots.

Figure A.8: Velocity Correction. Qualitative analysis of applying velocity correction over a number of different trajectories.

Style-Conditioned Motion Generation

See all 9 papers

MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles
May 30, 2025Mingyi Shi, Wei Liu, Jidong Mei +4Human Motion Generation Motion Retargeting
The University of Hong Kong, Hong Kong · Shandong University, China · Hong Kong University of Science and Technology, Hong Kong +2
EMoG: Emotion-Modulated Gait Generation for Expressive Humanoid Locomotion
Sep 13, 2026Yi Lu, Tianhao Jiang, Honglong Tian +6Humanoid Robot Locomotion Human Motion Generation
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China. · School of Intelligence Science and Technology, Nanjing University, Suzhou 215163, China. · School of Artiﬁcial Intelligence, Nanjing University, Nanjing 210023, China. +1
Predictive Style Matching: Natural and Robust Humanoid Locomotion
Jun 5, 2026Simeon Nedelchev, Ekaterina Chaikovskaia, Egor Davydenko +2Humanoid Robot Locomotion Reward Shaping
Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Moscow Region, Russia · Innopolis University, Innopolis, Russia · Sber Robotics Center, Moscow, Russia

May 30, 2025J/K move · Enter open · S save

MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles

Mingyi Shi, Wei Liu, Jidong Mei, Wangpok Tse+3

The University of Hong Kong, Hong Kong · Shandong University, China · Hong Kong University of Science and Technology, Hong Kong+2

Human Motion Generation Motion Retargeting Persona-Conditioned Generation

Open paper AskPDF arXiv

We present MotionPersona, a generative framework for character-aware locomotion control, in which the motion for a command depends on the captured persona, the body shape, and the character's style. Unlike style, which one performer can vary at will, persona and body shape are coupled in capture: each performer is observed in only one body. The captured data therefore cannot uniquely determine which motion characteristics should follow the persona and which should change with the body, leaving unseen persona-body combinations unconstrained. We capture 48 performers, aged 5 to 68, under the same nine styles and seven commands, 44 of them with persona annotation. From this repeated-measures design, we identify two robust associations between body shape and gait. These measurements guide a cross-body specification of which characteristics should change and which should be preserved. We implement this specification through a physically informed retargeting pipeline, producing cross-body training data while penalizing penetration and foot skating. On this data we train a single generative controller. A shape-aware VAE compresses each motion block into a few latent tokens and renders them on a conditioned target body under explicit geometric supervision; over these tokens, a latent flow-matching prior generates persona- and style-conditioned motion in two sampling steps. The controller covers all captured personas, a wide family of SMPL-X target bodies, and nine styles in one model, and runs at 27 ms per block on two threads of a laptop CPU. We verify the framework at every stage, following the same gait descriptors from captured to retargeted to generated motion and sweeping each axis in isolation. To our knowledge, this is the first real-time locomotion controller that carries part of a captured persona's performer-specific variation across independently selected body shapes and styles.

Abstract

Style-Conditioned Motion Generation

Explore similar work

MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles

EMoG: Emotion-Modulated Gait Generation for Expressive Humanoid Locomotion

Predictive Style Matching: Natural and Robust Humanoid Locomotion