Spacetime Emergence in the Memory of a Learning Agent
Authors: Philip A. LeMaitre, T. Rick Perche
Organizations: University of Innsbruck, Institute for Theoretical Physics, Technikerstrasse 21a, A-6020 Innsbruck, Austria · Nordita, KTH Royal Institute of Technology and Stockholm University, Hannes Alfvéns väg 12, 23, SE-106 91 Stockholm, Sweden
We show that the causal structure of spacetime can emerge within the memory of a learning agent performing a quantum information task. We train a model-based reinforcement-learning agent to maximize the quantum mutual information between localized quantum probes interacting with a quantum field in Minkowski spacetime, without directly providing it with information regarding the spacetime metric or causal structure. In learning to accomplish this task, the reinforcement-learning agent constructs an internal predictive model whose planner develops the characteristic null structure of spacetime, allowing us to reconstruct the causal structure of the Minkowski metric with high accuracy. Within this task, spacetime geometry is obtained from an agent's successful predictions and actions, providing a first step toward showing that spacetime concepts emerge from operational quantum experience.
Figures & tables
Figure 1: A schematic of the agent architecture and learning pipeline. Detector 1 is fixed at the origin and the (x,t) grid represents the spacetime smearing centres of detector 2. The thumbnail is Fig. 2 , showing the transformed quantum mutual information contained in the reward (Eq. ( 45 )) received by the agent. All eight local actions remain available during planning and exploration, with proposed off-grid moves being clipped by the environment and given a boundary penalty. Here eat is the one-hot representation of the action, and numbers beside network symbols specify hidden-layer widths, while dashed connections show step-replay learning with shared network parameters. The RL agent chooses these actions by initially using the encoder to map the normalized observation ot to the latent state zt=EθE(ot)∈R20 , then the residual dynamics predicts zt+1=zt+FθF(zt,eat) which is then fed into the reward model that predicts rt+1=RθR(zt,zt+1,eat) . Since the planning horizon is H=1 , the planner score is simply Jθ(st,at)=rt+1 . Reward training uses zt+1=EθE(ot+1) , the encoded observed next state, whereas planning uses the model-predicted state zt+1 . The decoder acts separately on encoded and predicted latent states. Its two displayed outputs are the reconstructed observation ot=DθD(zt) and predicted next observation ot+1=DθD(zt+1) ; coordinate reconstruction is also applied to the encoded next observation zt+1 . The seven loss constraints Eqs. ( 49 ),( 51 ), and ( 54 )- ( 57 ) are shown alongside their corresponding predictions, with sg denoting stop-gradient and zt← the reverse-action prediction obtained from zt+1 . The λj denote the corresponding loss weights. The purple branch first averages the planner action-score fields from the five trained agents into J(s,a) and then forms the scalar softmax-average field S . A cell enters the candidate lightcone mask when its score exceeds that of at least 70% of the eight neighbouring cells in a radius-1 square. Cells without a complete neighbourhood are rejected. Least squares then fits the quadratic form to the extracted branch points, with displacements measured relative to the fixed detector and the conformal freedom fixed by gxx=1 , yielding a representative of the conformal metric in (t,x) coordinate order.
Figure 2: The transformed quantum mutual information landscape Eq. ( 44 ), navigated by the agent. The plot shows the broad central high-information region corresponding to small detector separation and the diagonal ridge structure associated with the light cone that the RL agent samples only through the reward.
Figure 3: Learning curves for the model. On top is the episode return and below that is the episode length over 4000 episodes, both showing rapid convergence to the optimal policy and stability thereafter. Each plot is an average over 5 agents with the shaded regions representing the standard deviation around the average curve with respect to the 5 agents.
Figure 4: Average planner memory output plots. Left: softmax expected-action field using β=0.7 and normalized arrows, with the choice of β used only for visualization and not entering into training or planning. Right: averaged best planner score B at each detector placement. The trained planner maps the detector-generated reward landscape to a local action flow and a score surface that retain the underlying spacetime features (especially visible are the diagonal lightcone arms) inherited from the field correlations appearing in the quantum mutual information Eq. ( 10 ).
Figure 5: Internal diagnostics for the trained planner. Panels (a) and (b) show the first two principal components of the latent encoder; panel (c) shows the mean one-step latent dynamics error; panel (d) shows the current-state decoder reconstruction error; panel (e) shows the planner action gap; and panel (f) shows the planner-step reward on the selected action. Together, these quantities describe the latent representation, its one-step predictive dynamics, its connection to the observed detector-placement space, and the action preferences produced by the planner.
Figure 6: Reconstruction of the conformal metric from the scalar softmax-average planner score S . Left: local neighbour-comparison mask obtained by retaining cells whose S value exceeds at least 70% of the values in a radius- 1 neighbourhood. Right: quadratic form displaying Lorentzian signature fitted to the extracted lightcone branches with the conformal convention gxx=1 . The resulting null directions closely reproduce those of conformally flat (1+1) -dimensional Minkowski spacetime.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Supplementary internal diagnostics for the trained planner. Plot (a) shows the argmax action splitting into each action direction; plots (b) and (c) show the decoded coordinates; plot (d) shows the latent norm; plot (e) shows the dynamics error; plot (f) shows the mean predicted latent step norm; plots (g)—(i) show next-state reconstruction errors; and plots (j)—(l) show reward model projections.
School of Artificial Intelligence and Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China