Organizations: Centre for Quantum Technologies, Nanyang Technological University, Singapore · Nanyang Quantum Hub, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
World models provide digital simulacra of the true world, allowing agents to be trained and tested before costly real-world deployment. At each time step, they receive an action and generate an observation and reward matching the statistics of the true world. In complex environments where present outcomes depend on events far in the past, this requires memory. One might expect that, by increasing memory, we can always build a model accurately enough to align the optimal agent policies of the real and virtual worlds. We show that this is false for classical world models, even when the true world itself is classical. We construct true worlds for which every finite classical model fails along the same possible trajectory: it either loses the ability to distinguish actions when the true world clearly prefers one, or repeatedly assigns the highest expected reward to suboptimal actions. Its expected-reward estimates also retain a nonvanishing average error. In contrast, each such true world admits a quantum world model using a single qutrit that reproduces it exactly: its reward estimates and preferred actions always match those of the true world, ensuring that the optimal policies of the real and virtual worlds remain perfectly aligned.
Figures & tables
Figure 1: Classical and quantum world models . The true world represents reality (a). At each time step, it receives an action at and returns an outcome yt+1 that consists of an observation and reward. A world model reproduces this input–output behavior through recurrent internal dynamics, updating its memory from one simulated step to the next. In a classical world model (b), a classical register with a finite number of bits carries the memory, and it evolves under classical stochastic updates. In a quantum world model (c), the memory is encoded in a finite number of qubits and evolves through quantum operations.
Figure 2: The resettable FRDN clock world. The displayed signs illustrate a representative payoff regime: Wait offers a reward +RW if the clock advances but risks a failure cost −CW ; Maintain accepts the known cost −CM of an immediate reset; and Probe also resets, with reward +RP or cost −CP determined by whether the clock would have advanced. The diagram thus contrasts risky continuation under Wait, a certain outcome under Maintain, and an outcome-dependent bet under Probe, creating a nontrivial decision problem.
Figure 3: Conditional Tick probabilities. Probability of one additional Tick after ht for λ=0.4 and α=π/2 . The gray curve is the true-world probability, which the qutrit world model reproduces exactly. The colored curves show the predictions of the fitted classical models with memory dimensions N=5 (blue), N=50 (orange), and N=500 (green). Increasing N allows the model to reproduce the nonrepeating dependence on the clock age over a larger initial range. At later ages, the predictions approach the finite limiting behavior imposed by finite classical memory.
Figure 4: Cumulative wrong-action fraction under the reward setting in Appendix B.4 . Blue, orange, and green correspond to N=5 (blue), N=50 (orange), and N=500 (green), respectively. The qutrit world model has zero wrong actions for every T ; its curve coincides with the horizontal axis and is omitted for clarity.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Conditional Tick probabilities of different models at system dimension N=100 . Probability of one additional Tick after ht for λ=0.4 and α=π/2 .
Figure 6: Cumulative gaps in predicted conditional probabilities for models of different sizes. (a) Age counter, (b) Rational phase cycle model, and (c) fitted HMM. The red dashed line corresponds to the asymptotic bound κFRDN defined in Eq.( 152 ). Please refer to Appendix B.3 ) for detailed information.
Figure 7: Cumulative wrong-action fraction for models of different sizes. (a) Age-counter model, (b) Rational phase cycle model, and (c) fitted HMM.
Figure 8: Mean decision loss of different models with different sizes. (a) Age-counter model, (b) Rational phase cycle model, and (c) fitted HMM.
World models learn latent states that summarize interaction histories, evolve over time, and support prediction, simulation, or planning. Most existing world models represent these states using classical vectors, probability distributions, recurrent hidden states, or transformer activations. In this paper, we introduce Quantum-Structured World Models (QSWMs), a quantum-inspired framework for predictive world modeling with structured latent states, latent transition operators, and measurement-inspired decoding maps. We study whether mathematical structures inspired by quantum theory, such as complex-valued representations and density-matrix-like latents, provide useful inductive biases for world modeling. We establish three foundational properties: classical inclusion, predictive sufficiency, and structured compactness. We then instantiate complex-valued and density-matrix-like QSWM variants and evaluate them on elementary cellular automata against strong classical baselines. Results show promising local predictive potential for complex-valued QSWMs, while also revealing limitations in long-horizon rollout, density-matrix variants
Hailong Jiang, Emran Hossain, Feng Yu +3
Youngstown State University, Youngstown, OH, USA · Kent State University · George Washington University
A single observation can admit several plausible futures, making uncertainty an essential part of world modelling. However, quantum generative models remain underexplored for this task. We introduce Quantum-JEPA, a latent-action JEPA framework that learns distributions of future representations using a conditional quantum Born prior. Circuit measurements generate binary latent actions, which a shared predictor maps to alternative future representations. We train the quantum prior through the quality and diversity of these predictions using a sampled energy objective. Our model learns distinct continuations on constructed multimodal tasks and captures latent-action correlations that support coordinated multi-agent futures. On D3IL, a benchmark of diverse human demonstrations, our trained models generate multiple obstacle-crossing routes from a common starting point without observed motion. On the controlled tasks, our classical and quantum prior implementations improve prediction validity and distributional fidelity over the evaluated variational baselines. Our quantum priors combine competitive predictive quality with a parameter-efficient sampling structure: they avoid an exponentially sized categorical output layer while supporting correlated latent actions. We further validate finite-shot parameter-shift training, including training with readout noise. Unlike most prior work, which focuses on simulation, we also verify the feasibility of inference with trained priors on a real QPU. These results establish a trainable quantum component for JEPA future prediction and a basis for exploring quantum-enhanced world models at larger scales.
Zhiyuan Liu, Sicheng Liu, Haiyu Wu
Qinous Technology Co., Ltd., Beijing, China · Tsinghua University, Beijing, China · University of Notre Dame, Notre Dame, IN, USA
Off-policy reinforcement learning (RL) has become increasingly sample-efficient, enabling applications such as RL fine-tuning of Vision-Language-Action models into reliable, high-performing policies. World models offer a further lever for sample efficiency, as they predict state changes rather than actions alone, but their success has largely been confined to supervised policy learning. Prior model-based RL methods often optimize the policy or value function directly on imagined rollouts, which is prone to compounding bias and struggles to scale to large, high-dimensional problems such as real-world robotics, a problem that worsens with task horizon and visual complexity. In this work, we instead ask whether we can leverage world models directly on top of standard Q-learning to improve performance, while remaining trained and grounded in the real, online setting. We propose QWM, a framework that leverages world models to perform test-time search over imagined trajectories on top of Q-learning to select high-value actions during both online rollouts and evaluation. Since the policy and value function are trained only on real transitions, QWM avoids compounding model bias while still gaining the sample-efficiency benefits of predictive search. On challenging manipulation benchmarks Robomimic and LIBERO, QWM significantly outperforms strong prior state-of-the-art methods on both sample efficiency and performance.