Weakly Supervised Discovery of Motion Styles for Compositional Robot Control

Heedo Woo; Antonio Marino; Eduardo Sebastián; Amanda Prorok

cs.ROOct 15, 2026

Weakly Supervised Discovery of Motion Styles for Compositional Robot Control

Authors: Heedo Woo, Antonio Marino, Eduardo Sebastián, Amanda Prorok

Organizations: Department of Computer Science and Technology, University of Cambridge

Abstract

When a robot possesses more degrees of freedom than a task requires, multiple trajectories can achieve the same objective using distinct motion styles. Compositional robot control aims to preserve a selected style while simultaneously executing an independent task command, effectively balancing these potentially competing objectives. In this paper, we present CODES, a framework for compositional robot control that discovers a coherent style repertoire from diverse recordings and composes it with independent commands. First, motion retargeting maps the recordings to motions executable by the target robot; next, weakly supervised consensus encoding and density-based clustering identify styles within and across recordings without a prescribed count; finally, reinforcement learning trains a policy to compose the discovered styles with independent commands. We evaluate CODES on a Unitree Go2 quadruped across two simulators and physical hardware. Compared to baselines, CODES significantly improves kinematic coherence. Furthermore, our controller successfully tracks velocity commands that generalize beyond the support of the recorded data, exhibiting robust and distinguishable style characteristics during both cross-simulator and real-world hardware deployments. Project website: https://www.proroklab.org/compositional-motion-styles/

Figures & tables

Fig. 1: Learning and composing motion styles for quadruped control. (A) Motions are retargeted to a common representation compatible with the robot’s kinodynamics; (B) a weakly-supervised method based on consensus affinity and HDBSCAN identifies a style repertoire; and (C) a policy trained with reinforcement learning composes a style reference from the repertoire and an independent task command for robot control. This makes recorded motion characteristics available under independently selected commands.

Method	$K$	$K_{\mathrm{eff}}$	Loc. $K$	Coverage (%)	Core switches (/min)
CODES (ours)	16	10.19	12	72.8	1.36
Loss ablations
No coarse ( $\lambda_{+}=\lambda_{-}=0$ )	15	8.92	10	90.8	0.79
Separation only ( $\lambda_{+}=0$ )	15	7.40	9	91.3	1.01
Uniform attraction ( $\alpha_{y}=1$ )	7	3.42	7	95.9	0.00
No reconstruction ( $\lambda_{x}=\lambda_{\Delta}=0$ )	3	1.20	3	93.6	0.00

TABLE I: Decomposition Ablations and Baselines

Coverage and balance controls
Method	Coverage (%)	Silhouette $\uparrow$	Cross-recording AUC $\uparrow$
CODES (ours)	72.8	0.329	0.832
No coarse terms	90.8	0.264	0.826
VQ-VAE	100.0	-0.043	0.562
CODES, full coverage	100.0	0.246	0.790
VQ-VAE, balanced	100.0	0.022	0.680

TABLE II: Coherence in a Common Kinematic Space

Fig. 2: Command and style fidelity across forward and lateral velocity sweeps, with the other commanded components set to zero. (a) Fraction of three trials satisfying the completion and tracking criteria. (b) Mean reference consensus cosine, shown on an asinh scale clipped below $-0.1$ . In reviewed rollouts, values near $0.5$ usually retain recognizable reference posture and limb coordination, whereas values near $0$ show weaker resemblance; this learned score is not a calibrated perceptual distance. Orange outlines show estimated recorded support; crosses mark early termination. Tracking extends beyond support, but reference resemblance decreases.

Fig. 3: Style switching on hardware under a fixed task command. Deployment of the CODES policy on the Unitree Go2 at a constant $0.5$ m/s forward command with zero lateral and yaw-rate components, while the selected reference switches among C04, C00, C14, and C11. The colored band indicates the active style, and dashed lines the switch instants. The policy, reference bank, and state estimation run onboard, without external motion capture or offboard inference. The top row shows a time-lapse composite of the traversal, with numbered markers at successive instants of the run. The panels below report, from top to bottom, the estimated and commanded forward velocity $v_{x}$ ; the reference consensus cosine of Eq. ( 21 ) with respect to the active reference; the base-relative height $\Delta z$ of the four feet (FL, FR, RL, RR); and the kinematically inferred contact states. The command is identical across all four segments, while foot-swing amplitude, contact timing, and base posture differ, indicating that the reference changes how the command is executed rather than which command is tracked. Videos of the hardware execution are provided on the project website.

Dictionary	$K$	Median $E^{xy}$ (m/s) $\downarrow$	Complete (%) $\uparrow$	Success (%) $\uparrow$
CODES (ours)	16	0.114	98.1	74.2
No coarse terms	15	0.106	99.3	89.8
VQ-VAE	17	0.142	99.2	66.8

TABLE III: Command Fidelity in Simulation against Baselines

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Learning Expressive and Compositional Motion Representation via Spectral Skills
Sep 29, 2026Feiyang Wu, Chenxiao Gao, Chen Yang +3Robot Skill Learning Humanoid Robot Control
Georgia Institute of Technology
Predictive Style Matching: Natural and Robust Humanoid Locomotion
Jun 5, 2026Simeon Nedelchev, Ekaterina Chaikovskaia, Egor Davydenko +2Humanoid Robot Locomotion Reward Shaping
Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Moscow Region, Russia · Innopolis University, Innopolis, Russia · Sber Robotics Center, Moscow, Russia
Whole-Body Humanoid Control via Stochastic Motion Tokenization
Oct 27, 2026Michael Drolet, Oleg Arenz, Jan PetersHumanoid Robot Control Humanoid Motion Tracking
Technical University of Darmstadt · German Research Center for AI (DFKI) · Robotics Institute Germany (RIG)

Sep 29, 2026J/K move · Enter open · S save

Learning Expressive and Compositional Motion Representation via Spectral Skills

Feiyang Wu, Chenxiao Gao, Chen Yang, Ye Zhao+2

Georgia Institute of Technology

Robot Skill Learning Humanoid Robot Control Predictive Representation Learning Motion Representation Learning Compositional Representation Learning

Open paper AskPDF arXiv

Robotic foundation models offer a promising path toward general-purpose humanoid robot control, often through hierarchical architectures. However, their effectiveness depends on the command interface between the planner and the controller, which must support accurate execution while remaining easy to predict, and ideally allow new behaviors to be composed from prior ones. In this work, we introduce spectral skills, a latent representation of this interface that meets these requirements through predictive representation learning. By design, spectral skills compactly encode short motion segments and are learned by predicting subsequent motion rather than reconstructing the encoder input. On a 29-DoF humanoid, a controller conditioned on spectral skills reduces global tracking error by 62% relative to the state of the art. The same frozen controller chains independently encoded skills without a separate transition policy. It also composes new behaviors by adding orthogonal directions to any compatible base skill, producing combinations unseen in the training data. We demonstrate tracking, chaining, and composition, as well as control through a language-conditioned planner, on Unitree G1 hardware. Project page: https://spectral-skill.github.io

Abstract

Robotic Control

Explore similar work

Learning Expressive and Compositional Motion Representation via Spectral Skills

Predictive Style Matching: Natural and Robust Humanoid Locomotion

Whole-Body Humanoid Control via Stochastic Motion Tokenization