We introduce Continuous-Action World Models (CoAWM), a framework for learning cross-embodiment spatiotemporal dynamics priors that support robot action generation across embodiments and execution frequencies. At its core is a shared latent function space that captures the spatial and temporal structure of robot actions. We learn this space through two stages. First, function pretraining brings trajectories from different embodiments and sampling frequencies into a common representation, learning shared dynamics priors through action reconstruction and consistency across temporal sampling views. By accommodating heterogeneous robot trajectories, this stage broadens the range of data sources available for pretraining and facilitates scaling to larger and more diverse datasets. Second, we connect the pretrained function space to a world-model backbone, enabling observations, language instructions, and the current robot state to condition the prediction of a latent action function. A single decoder takes any query time and returns the corresponding end-effector action. By separating shared dynamics priors from task-conditioned action selection and temporal sampling, our approach provides a unified foundation for transferring robot experience across embodiments and control frequencies.
Figures & tables
Figure 1: Overview of continuous action-function learning and deployment. Training Stage A learns shared spatiotemporal priors from typed trajectories across embodiments and temporal sampling grids. Training Stage B connects observation-conditioned latent prediction to the learned function space using latent and physical reconstruction supervision. At inference, current multimodal observations and visual features condition latent prediction; the function decoder combines the predicted latent with physical query times and observed context to produce embodiment-specific control commands.
Figure 2: An action interval and its sampling grid are different objects. Schematic of a continuous scalar position reference. The first two panels query the same function over the same physical duration with different grids. The third changes the duration: the geometric progression is preserved, but its velocity and acceleration change. A reusable function representation must retain physical timing while allowing the execution grid to vary.
Figure 3: Mean position and rotation reconstruction errors of the function autoencoder on 8,192 held-out evaluation windows used for model selection.
Figure 4: Initial execution quality during the first 1 s across 800 paired trials in 16 tasks. The highlighted percentages show the reduction in translation jerk RMS, the reduction in rotation jerk RMS, and the retained translation speed RMS relative to LaWAM, respectively. Metrics describe actual end-effector motion, with equal task weighting and both arms pooled.
Figure 5: (a) CoAWM success rates at six execution frequencies, evaluated with 150 trials per frequency across 50 tasks (three trials per task), with the execution segment and replanning period fixed at 1.2 s. (b) Five selected tasks from a separate 16-task evaluation with 50 trials per task. The panels therefore use different evaluation cohorts and denominators.
Figure 6: Estimated positive mechanical work for the 13 jointly successful scenes among 16 prespecified scenes, computed from PD torque estimates and measured joint velocities. Open blue points denote LaWAM and filled red points denote CoAWM; connecting segments identify paired scenes. The two time columns report physical execution duration in seconds using the same method colors.