Spacetime Emergence in the Memory of a Learning Agent
Authors: Philip A. LeMaitre, T. Rick Perche
Organizations: University of Innsbruck, Institute for Theoretical Physics, Technikerstrasse 21a, A-6020 Innsbruck, Austria · Nordita, KTH Royal Institute of Technology and Stockholm University, Hannes Alfvéns väg 12, 23, SE-106 91 Stockholm, Sweden
We show that the causal structure of spacetime can emerge within the memory of a learning agent performing a quantum information task. We train a model-based reinforcement-learning agent to maximize the quantum mutual information between localized quantum probes interacting with a quantum field in Minkowski spacetime, without directly providing it with information regarding the spacetime metric or causal structure. In learning to accomplish this task, the reinforcement-learning agent constructs an internal predictive model whose planner develops the characteristic null structure of spacetime, allowing us to reconstruct the causal structure of the Minkowski metric with high accuracy. Within this task, spacetime geometry is obtained from an agent's successful predictions and actions, providing a first step toward showing that spacetime concepts emerge from operational quantum experience.
Figures & tables
Figure 1: A schematic of the agent architecture and learning pipeline. Detector 1 is fixed at the origin and the (x,t) grid represents the spacetime smearing centres of detector 2. The thumbnail is Fig. 2 , showing the transformed quantum mutual information contained in the reward (Eq. ( 45 )) received by the agent. All eight local actions remain available during planning and exploration, with proposed off-grid moves being clipped by the environment and given a boundary penalty. Here eat is the one-hot representation of the action, and numbers beside network symbols specify hidden-layer widths, while dashed connections show step-replay learning with shared network parameters. The RL agent chooses these actions by initially using the encoder to map the normalized observation ot to the latent state zt=EθE(ot)∈R20 , then the residual dynamics predicts zt+1=zt+FθF(zt,eat) which is then fed into the reward model that predicts rt+1=RθR(zt,zt+1,eat) . Since the planning horizon is H=1 , the planner score is simply Jθ(st,at)=rt+1 . Reward training uses zt+1=EθE(ot+1) , the encoded observed next state, whereas planning uses the model-predicted state zt+1 . The decoder acts separately on encoded and predicted latent states. Its two displayed outputs are the reconstructed observation ot=DθD(zt) and predicted next observation ot+1=DθD(zt+1) ; coordinate reconstruction is also applied to the encoded next observation zt+1 . The seven loss constraints Eqs. ( 49 ),( 51 ), and ( 54 )- ( 57 ) are shown alongside their corresponding predictions, with sg denoting stop-gradient and zt← the reverse-action prediction obtained from zt+1 . The λj denote the corresponding loss weights. The purple branch first averages the planner action-score fields from the five trained agents into J(s,a) and then forms the scalar softmax-average field S . A cell enters the candidate lightcone mask when its score exceeds that of at least 70% of the eight neighbouring cells in a radius-1 square. Cells without a complete neighbourhood are rejected. Least squares then fits the quadratic form to the extracted branch points, with displacements measured relative to the fixed detector and the conformal freedom fixed by gxx=1 , yielding a representative of the conformal metric in (t,x) coordinate order.
Figure 2: The transformed quantum mutual information landscape Eq. ( 44 ), navigated by the agent. The plot shows the broad central high-information region corresponding to small detector separation and the diagonal ridge structure associated with the light cone that the RL agent samples only through the reward.
Figure 3: Learning curves for the model. On top is the episode return and below that is the episode length over 4000 episodes, both showing rapid convergence to the optimal policy and stability thereafter. Each plot is an average over 5 agents with the shaded regions representing the standard deviation around the average curve with respect to the 5 agents.
Figure 4: Average planner memory output plots. Left: softmax expected-action field using β=0.7 and normalized arrows, with the choice of β used only for visualization and not entering into training or planning. Right: averaged best planner score B at each detector placement. The trained planner maps the detector-generated reward landscape to a local action flow and a score surface that retain the underlying spacetime features (especially visible are the diagonal lightcone arms) inherited from the field correlations appearing in the quantum mutual information Eq. ( 10 ).
Figure 5: Internal diagnostics for the trained planner. Panels (a) and (b) show the first two principal components of the latent encoder; panel (c) shows the mean one-step latent dynamics error; panel (d) shows the current-state decoder reconstruction error; panel (e) shows the planner action gap; and panel (f) shows the planner-step reward on the selected action. Together, these quantities describe the latent representation, its one-step predictive dynamics, its connection to the observed detector-placement space, and the action preferences produced by the planner.
Figure 6: Reconstruction of the conformal metric from the scalar softmax-average planner score S . Left: local neighbour-comparison mask obtained by retaining cells whose S value exceeds at least 70% of the values in a radius- 1 neighbourhood. Right: quadratic form displaying Lorentzian signature fitted to the extracted lightcone branches with the conformal convention gxx=1 . The resulting null directions closely reproduce those of conformally flat (1+1) -dimensional Minkowski spacetime.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Supplementary internal diagnostics for the trained planner. Plot (a) shows the argmax action splitting into each action direction; plots (b) and (c) show the decoded coordinates; plot (d) shows the latent norm; plot (e) shows the dynamics error; plot (f) shows the mean predicted latent step norm; plots (g)—(i) show next-state reconstruction errors; and plots (j)—(l) show reward model projections.
A recent line of work measures causal emergence in reinforcement learning agents through Integrated Information Decomposition, reporting that Φr grows with training and tracks reward improvement. For active inference, this raises the question of how reward-free predictive organization relates to such information-theoretic signatures. I test this within an active inference agent whose architecture separates a fast perception latent z from a slow global latent g, where g is driven by prediction error and structurally decoupled from policy gradients. In a reward-free environmental regime-switching protocol, Φr concentrates in g; its aggregate magnitude is largely architectural and decreases with training. The substantive effect of learning becomes legible only at the atom-compositional level: decoupling flips sign from negative to positive and becomes regime-invariant under environmental change, while downward causation carries the regime-dependent adjustment. These results identify g as the architectural locus of Φr-relevant temporal organization in an active inference agent, and argue against reading scalar Φr as a direct index of learned integration.
Hongju Pae
Active Inference Institute, Crescent City, CA, USA
The situated view of cognition holds that intelligent behavior depends not only on internal memory, but on an agent's active use of environmental resources. Here, we begin formalizing this intuition within Reinforcement Learning (RL). We introduce a mathematical framing for how the environment can functionally serve as an agent's memory, and prove that certain observations, which we call artifacts, can reduce the information needed to represent history. We corroborate our theory with experiments showing that when agents observe spatial paths, the amount of memory required to learn a performant policy is reduced. Interestingly, this effect arises unintentionally, and implicitly through the agent's sensory stream. We discuss the implications of our findings, and show they satisfy qualitative properties previously used to ground accounts of external memory. Moving forward, we anticipate further work on this subject could reveal principled ways to exploit the environment as a substitute for explicit internal memory.
John D. Martin, Fraser Mince, Esra'a Saleh +1
Openmind Research Institute · University of Alberta · Cohere Labs Community +3
We introduce a continuous metric field framework trained by a single causal contrastive loss. The framework encodes a scene into coefficients of a fixed symmetric matrix basis, assembles them into a Lie algebra element, and exponentiates the result to a Riemannian or Lorentzian metric. Across dimensions, this field discovers the full spectrum of geometric structures: from obstacle-avoiding geodesics in robot navigation across planar and manipulator configuration spaces, to event horizons of black holes in Lorentzian spacetime. Extensive zero-shot generalization studies demonstrate that the field captures transferable geometric structure rather than memorizing specific configurations. In the black hole setting, the causal loss spontaneously evolves genuine black-hole-like structures with the correct Lorentzian signature. The same loss, the same architecture, and the same training protocol produce the full range of geometric phenomena across dimensions. The field knows geometry, and geometry knows physics.
Chenghao Xu
School of Artificial Intelligence and Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China