Accurately anticipating how complex, diverse scenes will evolve requires models that represent uncertainty, simulate along extended interaction chains, and efficiently explore many plausible futures. Yet most existing approaches rely on dense video or latent-space prediction, expending substantial capacity on dense appearance rather than on the underlying sparse trajectories of points in the scene. This makes large-scale exploration of future hypotheses costly and limits performance when long-horizon, multi-modal motion is essential. We address this by formulating the prediction of open-set future scene dynamics as step-wise inference over sparse point trajectories. Our autoregressive diffusion model advances these trajectories through short, locally predictable transitions, explicitly modeling the growth of uncertainty over time. This dynamics-centric representation enables fast rollout of thousands of diverse futures from a single image, optionally guided by initial constraints on motion, while maintaining physical plausibility and long-range coherence. We further introduce OWM, a benchmark for open-set motion prediction based on diverse in-the-wild videos, to evaluate accuracy and variability of predicted trajectory distributions under real-world uncertainty. Our method matches or surpasses dense simulators in predictive accuracy while achieving orders-of-magnitude higher sampling speed, making open-set future prediction both scalable and practical. Project page: http://compvis.github.io/myriad.
Figures & tables
Figure 2: Motion Token Construction. The fourier-embedded motion Δxt(i) (alternatively a zero-vector) is combined with a per-trajectory unique randomized trajectory identifier idtraj(i) and the local image features, retrieved at the current xt(i) and original position xt=0(i) , providing information about what it is and local context.
Figure 3: Positional Encoding Scheme. We encode the current and original spatial position of each token, alongside its time . Motion tokens attend to each other and to image tokens.
Figure 4: Fast Reasoning Blocks. (a) Previous methods [ 9 , cf.] use normal transformer layers, incurring significant overhead due to the multitude of operations performed per block. (b) Our fused layers reduce complexity significantly, improving efficiency.
Figure 5: Our attention mask.
Figure 6: Posterior FM Head. Left: Our FM Head consists of multiple FFN blocks conditioned on zt(i) and flow matching time τ via adaptive norms [ 47 ] . We set up the conditioning mechanism such that every component can be cached, reducing computations. Right: Our multiscale, tanh-saturated input stack helps stabilize behavior when modeling motion with heavy-tailed behavior.
Figure 7: Value distribution.
Figure 8: OWM Composition. We curate OWM to cover a wide variety of settings. Top: dataset statistics. Bottom: some examples.
Figure 9: (a) Given different input pokes (initial motion), our model produces different motions (visualized as green lines) that adhere to constraints of the environment. (b) Our model predicts coherent motion for multiple points on the same object.
Table 9
Table 2: Planning Billiard Shots through Future Exploration. Left: We compare the Accuracy of landing a ball at a randomly selected goal position in a billiard simulation by unrolling potential futures starting from varying cue ball impulses. Under a fixed compute budget, our model surpasses dense world models from scratch using the same data. This is enabled by our methods’ low Latency , enabling us to sample a large number of potential futures. Right: We visualize results w.r.t. final target error and show its evolution over planning time for our model and an I2V baseline.
Figure 11: Planning a Billiard Shot. We search for a plan to move the red ball to the goal (top left). Our model derives a plan (top right) by predicting motion for different initial actions. Executing the action moves the ball to the desired location (bottom).
Figure 12: Posterior Uncertainty vs. Error. Starting around pixel-level ( 5121 ), our model’s posterior uncertainty is well-correlated ( green line) with true error.
Table 3: Posterior Parametrization Ablation. Substituting previously used GMM-based heads with flow matching heads leads to significant improvements in accuracy and increases convergence substantially. Adding our scale cascade improves accuracy further.
Figure A : Qualitative comparison on OWM: Our model produces motion samples that are qualitatively on par with much larger models such as WAN2.2 and MAGI-1.
Figure B : Qualitative comparison on Physics-IQ: While video generation models fail to capture the complexity of object interactions and predict simplified or no motion, our approach captures realistic physical interactions.
Figure C : Qualitative comparison on Physion: For simplified rigid body settings in Physion, video generation models hallucinate overly complex motion, while our approach is able to capture physical dynamics.
Figure D : Context informed samples: The above samples show our model’s ability to take appearance information into account when predicting motion. In both comparisons, images were sampled with the same initial poke. Images where edited using nano banana for high similarity in appearance.
Figure E : Hinted Samples Our model is capable of producing complex, coherent, and appearance-informed motion given an initial motion hint.
Figure F : Un-hinted Samples Given only appearance conditioning, our approach is able to produce physically correct and coherent motion, also show ing more complex understanding, such as that cars at an intersection should not move when pedestrians are blocking their path.
Figure G : Partially-hinted Samples Given only a single poke conditioning per example, our model produces coherent motion for queries on the same or linked objects. The Jenga example highlights that our model is able to capture multi-modality if two outcomes are possible given the same initial motion hint.
Figure H : Qualitative samples on our billiard simulation. The respective top row shows our model’s prediction given an initial impulse for the ball marked in red, where we visualize the predicted trajectory state using a frame-wise renderer; the lower row shows an overlay of the ground truth simulation with the prediction to enable comparisons. Our model can successfully predict the observed motion up to minor stochastic details.