A latent control model for realistic rodent motion

Aidan Sirbu; Charles Y. Zhang; Yuanjia Yang; Bence P. Ölveczky; Talmo D. Pereira; Blake A. Richards

cs.AIOct 19, 2026

A latent control model for realistic rodent motion

Authors: Aidan Sirbu, Charles Y. Zhang, Yuanjia Yang, Bence P. Ölveczky, Talmo D. Pereira, Blake A. Richards

Organizations: Google, Paradigms of Intelligence Team · Mila – Quebec AI Institute · School of Computer Science, McGill University · Harvard University ·

Abstract

The development of embodied models of animal behavior is a major goal of the neuro-AI community. Embodied models will enhance our ability to make direct comparisons between biological and artificial agents. To make such comparisons effective, though, we need embodied models that actually move like animals. Previous work has developed virtual animal bodies and techniques for training realistic motion through imitation. However, to date, there is no system for embodied animal modeling that produces natural movements both during trained goal-directed behavior, and during exploration driven by random generation. Here, we introduce Simulated Control via Augmented Motor Priors for Embodied Reinforcement learning (SCAMPER), a system for training an embodied virtual rat model, enabling natural movements during both goal-directed behavior and random exploration. SCAMPER uses a variational architecture with two distinct priors, a motion prior for control and a behavior prior for exploration. We show that combining both of these priors allows SCAMPER to generate realistic behavior during exploration and exploit the task structure effectively. SCAMPER can also learn from sparse rewards as a result of the diverse movements induced by the behavior prior, allowing the system to model realistic sparse-reward tasks from experimental neuroscience. Altogether, SCAMPER provides the neuro-AI community with an effective new system for training embodied models of rodents on realistic tasks.

Figures & tables

Figure 1: Training stages of SCAMPER. (1) The expert imitator is trained with mocap data. (2) The latent controller is trained via online distillation. (3) The behavior prior is distilled from the frozen encoder. Finally, (4) the model can be trained with RL to perform different tasks.

Figure 2: SCAMPER is behaviorally diverse. (a) Renderings of behavior-prior rollouts at increasing sampling noise. (b) Vendi score vs. latent sampling noise $\sigma=\exp(\text{logvar}/2)$ , for the behavior prior ( blue ), motion prior ( purple ), and the training data ( dashed gray ); data shown is bootstrapped means with 95% CIs. (c) Per-rollout net displacement across $\sigma$ ; violins with interquartile lines, gray violin marks training data. (d) Spatial occupancy at $\sigma=0.22$ shown as 2D histograms of body location (log-scaled frame counts) for the training data, motion prior, and behavior prior (see Fig. 9 for all noise levels).

Figure 3: SCAMPER preserves motion realism during random behavior. (a) Kernel Inception Distance (KID) between random rollouts and the training data behavior distributions across sampling noise

\sigma=\exp(\text{logvar}/2)

, for the behavior prior ( blue ) and motion prior ( purple ). Data shown is means

\pm

1 s.d. (b) Empirical cumulative distribution function (ECDF) of root height at

\sigma=0.22

, compared against the training data ( dashed gray ).

Figure 4: SCAMPER produces realistic gaits at varying speeds. (a) Actual vs. target speed for SCAMPER ( blue ), MIMIC-MJX ( red ), and the from-scratch model ( gray ); lines show means, shaded bands show $\pm 1$ s.d. across seeds, dashed line indicates perfect tracking. The training-data speed distribution is shown in purple on the right. (b) Renderings of SCAMPER walking, trotting, and galloping at increasing target speeds. (c) Stance-phase plots for each limb (RF, LF, RH, LH) across walk, trot, and gallop gaits, comparing empirical rat data ( purple ; data from [ 39 ] ) to SCAMPER ( blue ); bars show stance duration, error bars show variability across strides.

	Walk			Trot			Gallop
Method	Coord.	Pattern	All	Coord.	Pattern	All	Coord.	Pattern	All
SCAMPER	1.49	0.95	0.59	1.27	1.40	1.19	1.35	2.29	1.91
MIMIC-MJX	1.83	1.68	1.28	5.15	3.62	2.72	1.96	2.51	2.74
Scratch	4.83	6.55	3.44	7.84	14.69	11.01	5.93	8.28	5.91

Table 1: Per-metric distance to empirical rat data, computed as the absolute difference in means divided by empirical standard deviation,

|\mu_{emp}-\mu_{virtual}|/\sigma_{\text{emp}}

(

\downarrow

). Values shown are the average across metrics in each category: Coordination (temporal symmetry, inter-limb phase couplings), Pattern (phase couplings, duty factors), and All (18 gait metrics). (See Table 11 for all metrics)

Figure 5: SCAMPER expands the precision and stability of learnable rearing behaviors. (a) Example rollouts of a walk-then-rear policy, with the target rearing height indicated by the red box, for policies trained using SCAMPER ( top ), MIMIC-MJX ( middle ), and trained from scratch ( bottom ). (b) Task success rate over simulation steps for each method; lines show the mean across seeds and bands show $\pm 1$ s.d. Only the SCAMPER policy solves the task consistently within the training budget.

Figure 6: SCAMPER enables efficient foraging under increasingly sparse reward. (a) Arena forage task: treats ( blue circles ) are spawned uniformly within an annulus bounded by the minimum spawn radius ( red dashed ) and the arena boundary ( orange ), with a scent field (colorbar) guiding the rodent from its start position ( white ). (b) Number of treats collected per episode as a function of the minimum spawn radius, for policies trained using SCAMPER ( blue ), a motion-prior-only variant ( purple ), MIMIC-MJX ( red ), and trained from scratch ( gray ). Lines show the mean across seeds and bands show $\pm 1$ s.d.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Group	Hyperparameter	Value
Environment	Solver	Newton
	Solver iterations	5
	Line-search iterations	5
	Max contacts ( $n_{\text{aconmax}}$ )	256
	Max constraint rows ( $n_{\text{jmax}}$ )	128
	Simulation timestep	$2\times 10^{-3}$ s (500 Hz)

Appendix

Table 2: Hyperparameters used for Stage 1 (train expert imitator) of the SCAMPER training pipeline.

Group	Hyperparameter	Value
Distillation loss	AR(1) smoothness weight $\alpha$	$5\times 10^{-3}$
	Gradient clip threshold (global norm)	$10.0$
	Encoder / prior logvar clamp	$[-5,0]$
KL schedule	Schedule type	linear ramp, then hold
	Start weight $\lambda_{0}$	$1\times 10^{-3}$
	End weight $\lambda_{1}$	$1\times 10^{-4}$

Appendix

Table 3: Hyperparameters used for Stage 2 (online distillation) of the SCAMPER training pipeline. Environment, termination, and domain-randomization settings are inherited from Stage 1 (Table 2 ).

Group	Hyperparameter	Value
Distillation loss	KL weight	$1.0$
	Gradient clip threshold (global norm)	$10.0$
Network architecture	Latent size (inherited from Stage 2)	$16$
	Behavior prior hidden sizes	$[2048,2048,1024,512]$
Training	Parallel environments	$4{,}096$
	Total environment steps	$2\times 10^{9}$

Appendix

Table 4: Hyperparameters used for Stage 3 (behavior prior distillation) of the SCAMPER training pipeline. Environment settings are inherited from Stage 1 (Table 2 ); the encoder, decoder, and motion prior are loaded frozen from Stage 2 (Table 3 ). Only the behavior prior is updated.

Group	Hyperparameter	Value
Task / environment	Episode length	$2{,}000$ control steps
	Evaluation interval	$1.5\times 10^{7}$ env steps
	Number of treats per episode	$10$
	Treat sphere radius	$0.06$ m
	Treat height	$0.03$ m
	Minimum spawn radius	Variable

Appendix

Table 5: Hyperparameters used to train SCAMPER on the arena foraging task (Stage 4: transfer learning). The decoder, motion prior, and behavior prior are loaded frozen from the Stage 3 checkpoint (Table 4 ); only the task policy and critic are trained at this stage.

Group	Hyperparameter	Value
Task / environment	Episode length	$1{,}000$ control steps
	Evaluation interval	$1.5\times 10^{7}$ env steps
	Target forward speed	Variable
	Forward-velocity reward weight	$1.0$
Network architecture	Task policy hidden sizes	$[2048,1024,512]$
	Critic hidden sizes	$[2048,1024,1024,512]$

Appendix

Table 6: Hyperparameters used to train SCAMPER on the locomotion task. The motion prior

p_{\psi}

and decoder

p_{\theta}

are loaded frozen from the Stage 2 checkpoint (Table 3 ); only the task policy and critic are trained at this stage. The policy outputs a residual

\boldsymbol{\delta}_{t}

added to the (deterministic) motion-prior mean.

Group	Hyperparameter	Value
Task / environment	Episode length	$500$ control steps
	Evaluation interval	$1.5\times 10^{7}$ env steps
	Target position $(x^{\star},y^{\star},z^{\star})$	$(1.0,\,0.0,\,0.2)$ m
	$xy$ -tolerance for success	$0.01$ m
	$z$ -tolerance for success	$0.02$ m
	Required hold duration	$1.0$ s

Appendix

Table 7: Hyperparameters used to train SCAMPER on the walk-rear task. The motion prior

p_{\psi}

and decoder

p_{\theta}

are loaded frozen from the Stage 2 checkpoint (Table 3 ); only the task policy and critic are trained. The policy outputs a residual

\boldsymbol{\delta}_{t}

added to the (deterministic) motion-prior mean.

Group	Hyperparameter	Value
Architecture	Encoder hidden sizes	$[512,256]$
	Decoder hidden sizes	$[256,512]$
	Latent dimensionality $d$	$64$
	Activation	SiLU
	Layer normalization	yes
Regularization	Dropout rate	$0.1$

Appendix

Table 8: Hyperparameters for the VAE feature extractor (“inception network”) used to compute KID.

Figure 7: (a) KL divergence to the encoder marginal for the motion and behavior priors. (b) Between-goal and within-goal shares of the encoder’s latent variance at fixed proprioception. (c) Goal decoding accuracy from the latent, compared with chance. (d) Goal sensitivity across Stage-2 training, with the KL coefficient

\lambda

on the right axis. Error bars are 95% clip-level bootstrap CIs.

Figure 8: Diversity at matched latent scale. Vendi score of random generations vs. the realized per-dimension latent s.d. ( a ) and mean one-step latent displacement ( b ) at various levels of sampling noise (

\sigma

). Stars: deterministic (

\sigma=0

). 95% bootstrap CIs shown as shaded regions.

Figure 9: Spatial occupancy of all tested sampling noise levels plotted as 2D histograms of body location (log-scaled frame counts) for the motion prior and behavior prior.

Figure 10: Empirical cumulative distribution function of root height at all tested sampling noise levels compared against the training data reference ( dashed gray ).

	Wasserstein Distance ( $\downarrow$ )
Prior Networks	$\sigma{=}0$	$\sigma{=}0.14$	$\sigma{=}0.22$	$\sigma{=}0.37$	$\sigma{=}0.61$	$\sigma{=}0.78$	$\sigma{=}1.00$
Behavior Prior	.0085	.0054	.0045	.0092	.0106	.0129	.0162
Motion Prior	.0116	.0121	.0115	.0099	.0126	.0171	.0173

Appendix

Table 9: Wasserstein distance between empirical root-height distributions of prior rollouts and the training data, across sampling noise levels

\sigma

. Corresponds to results from Figure 10 .

Figure 11: Rendering of the arena foraging task.

Figure 12: SCAMPER’s training progression on the arena foraging task. Top-down view of trajectories the rodent takes along with the evolution of the gate logit ( $\beta_{t}$ ) and its resulting binarization ( $b_{t}$ ). Here when a treat is collected it is still displayed in the arena for visualization purposes, but in practice it disappear upon successful collection. $b_{t}=0$ results in stochastic sampling of the behavior prior. $b_{t}=1$ results in residual control of the motion prior. During early training the rodent relies heavily on the behavior prior to explore its environment. During mid training, the rodent begins to gate rapidly between both priors as it begins to exploit the reward signals from the collection of treats. During late training the rodent mostly uses residual control of the motion prior to effectively exploit the task structure.

Variant	Setting	Mean treats collected
Fixed schedule (warmup / ramp)	$0.05$ / $0.25$	$9.94\pm 0.03$
	$0.10$ / $0.60$	$0.00\pm 0.00$
	$0.25$ / $0.65$	$0.00\pm 0.00$
Linear interpolation ( $\rho$ )	$0.25$	$9.91\pm 0.04$
	$0.50$	$0.00\pm 0.00$
	$0.75$	$0.00\pm 0.00$

Appendix

Table 10: Alternatives to the adaptive binary gate in the foraging task. Values are the mean number of treats collected in the hardest condition (minimum treat radius

1.5

m). Schedule fractions are of total training steps.

\alpha

is the weight on the motion prior.

Figure 13: Stance-phase plots for each limb (RF, LF, RH, LH) across walk, trot, and gallop gaits, comparing empirical data ( purple ; [ 39 ] ) to the base SCAMPER model (MP, blue ), the behavior-prior-controlled SCAMPER model (BP, dark blue ), MIMIC-MJX ( red ), and the from-scratch model ( gray ); bars show stance duration, error bars show variability across strides.

		Empirical			SCAMPER (MP)			SCAMPER (BP)			MIMIC-MJX			Scratch			$\|\Delta\mu\|/\sigma_{\text{emp}}$ ( $\downarrow$ )
Gait	Metric																SCAMPER (MP)	SCAMPER (BP)	MIMIC-MJX	Scratch
Walk	Stride Duration (ms)	308	$\pm$	91	332	$\pm$	16	313	$\pm$	34	374	$\pm$	23	255	$\pm$	293	0.27	0.06	0.73	0.58
	Step Frequency (Hz)	3.46	$\pm$	0.75	3.02	$\pm$	0.15	3.22	$\pm$	0.33	2.69	$\pm$	0.26	4.61	$\pm$	2.62	0.58	0.31	1.02	1.53
	Stance LH (ms)	182	$\pm$	82	191	$\pm$	21	178	$\pm$	26	221	$\pm$	36	223	$\pm$	295	0.11	0.05	0.48	0.50
	Stance RH (ms)	175	$\pm$	66	183	$\pm$	9	208	$\pm$	59	247	$\pm$	31	80	$\pm$	0	0.12	0.51	1.09	1.45
	Stance LF (ms)	171	$\pm$	71	172	$\pm$	32	137	$\pm$	11	153	$\pm$	15	20	$\pm$	10	0.02	0.48	0.25	2.12

Appendix

Table 11: Full gait metric comparison to empirical rat data. Empirical, SCAMPER (MP), SCAMPER (BP), MIMIC-MJX, and Scratch columns show mean

\pm

std.

|\Delta\mu|/\sigma_{\text{emp}}

is the absolute difference in means normalized by empirical standard deviation.

Latent Action Learning

Latent action learning discovers compact representations of behavior from observations or trajectories to support control, planning, and transfer.

See all 113 papers

Position: Good Embodied Reward Models Need Bad Behavior Data
May 31, 2026Ran Tian, Yilin Wu, Andrea BajcsyReward Modeling Preference Alignment
UC Berkeley · NVIDIA Research · Carnegie Mellon University
ROOT: Discovering Rewards for User-Specified Embodied Behaviors
Oct 3, 2026Eren Sadikoglu, Aditya Taparia, Xinyuan Liu +1Reward Design for RL Robotic RL
Robotics and Autonomous Systems, Arizona State University, Tempe, Arizona, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA
Embodiment Shapes Rolling Behavior in a Multimodal Infant Model
Jun 16, 2026Leon Philipp, Francisco M. López, Jochen TrieschHuman Behavior Simulation Embodied AI
Frankfurt Institute for Advanced Studies, Germany · University of New South Wales, Australia · Goethe University Frankfurt, Germany

May 31, 2026J/K move · Enter open · S save

Position: Good Embodied Reward Models Need Bad Behavior Data

Ran Tian, Yilin Wu, Andrea Bajcsy

UC Berkeley · NVIDIA Research · Carnegie Mellon University

Reward Modeling Preference Alignment Reward Model Evaluation

Open paper AskPDF arXiv

This position paper argues that to obtain reliable embodied reward models, the community must invest in ``bad'' robot data: failed, suboptimal, error-prone, and even hazardous behaviors. While reward models are central to any foundation model's lifecycle, today's embodied reward models are trained primarily on successful behaviors. We analyze three state-of-the-art embodied reward models and find that they systematically over-reward behaviors that real human evaluators would penalize, including unsafe interactions, poor execution, and shortcut strategies that only superficially satisfy tasks. We attribute these failures to a key data gap: the scarcity of negative embodied data which is costly to collect and often filtered out or withheld in existing robotics datasets. Furthermore, we show that even modest exposure to real bad behavior data can improve alignment with human preferences and reduce costly false positives. We therefore call on the embodied AI community to curate and release their bad robot data, build synthetic bad data generation engines, develop more decentralized physical evaluation systems, and design benchmarks for fine-grained embodied reward model evaluations.

Abstract

Latent Action Learning

Explore similar work

Position: Good Embodied Reward Models Need Bad Behavior Data

ROOT: Discovering Rewards for User-Specified Embodied Behaviors

Embodiment Shapes Rolling Behavior in a Multimodal Infant Model