Topological Effects in Neural Network Field Theory
Authors: Christian Ferko, James Halverson, Vishnu Jejjala, Brandon Robinson
Organizations: The NSF Institute for Artificial Intelligence and Fundamental Interactions (IAIFI) and Department of Physics, Northeastern University, Boston, MA 02115, USA · Mandelstam Institute for Theoretical Physics, School of Physics, and NITheCS, University of the Witwatersrand, Johannesburg, WITS 2050, South Africa · Institute of Physics, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, Netherlands
Neural network field theory formulates field theory as a statistical ensemble of fields defined by a network architecture and a density on its parameters. We extend the construction to topological settings via the inclusion of discrete parameters that label the topological quantum number. We recover the Berezinskii--Kosterlitz--Thouless transition, including the spin-wave critical line and the proliferation of vortices at high temperatures. We also verify the T-duality of the bosonic string, showing invariance under the exchange of momentum and winding on S1, the transformation of the sigma model couplings according to the Buscher rules on constant toroidal backgrounds, the enhancement of the current algebra at self-dual radius, and non-geometric T-fold transition functions.
Figures & tables
Figure 1: Spin-wave correlator C(r) below Tc (log-log). Solid lines: data. Dashed lines: predicted power law r−b2 .
b
ηexp=b2
ηfit
R2
0.050
0.00250
0.00252
0.99196
0.089
0.00791
0.00796
0.99196
0.158
0.02500
0.02517
0.99196
0.281
0.07906
0.07963
0.99198
0.500
0.25000
0.25204
0.99204
Table 1: Below- Tc power law exponents. ηfit agrees with b2 to better than 1% throughout, confirming ( 44 ).
Figure 2: Spin-wave correlator above Tc (log-log). The correlator remains power law because the RFF field is Gaussian by construction: the spin-wave sector alone has no disordered phase.
Figure 3: Correlation length ξ extracted from the full two-point function via ( 48 ). R2 values of the per-point fit ( 48 ) are annotated. Dashed curve: BKT essential singularity fit ( 49 ) ( R2=0.98 ).
Figure 4: Vortex density ρv vs. b . The vertical dashed line marks bc=1/2 . Below Tc the vortex density is consistent with zero; above Tc it rises steeply as vortices unbind.
Figure 5: Vortex angle field θv(x)mod2π for nine values of b . Red (blue) markers denote vortices (antivortices). At small b the field is featureless; as b increases past bc=1/2 , vortex–antivortex pairs appear and eventually form a dense plasma.
Figure 6: Vortex–antivortex pair correlation g+−(r) . At large T , b→∞ and g+−=1 for all r , since the unbinding makes it an uncorrelated Coulomb gas (dashed line). At lower temperatures, we expect excess (deficit) probability at low (large) r , which is reflected in the indigo ( b=0.60 ) and blue ( b=0.75 ) curves, due to the logarithmic potential in Pvort . The noisy statistics is due to a paucity of vortices.
Figure 7: Renormalized stiffness KR vs. b . Gray dashed: bare coupling K0=1/(2πb2) . Blue dotted: universal value 2/π . Red dashed: bc=1/2 . Below bc , KR≈K0 ; above bc , vortex screening drives KR sharply downward.
Figure 8: Helicity modulus ΥR=2πb2KR vs. temperature T=2πb2 . Blue dashed: Nelson–Kosterlitz line ΥR=2T/π . Red dashed: Tc=π/2 . The data cross the NK line near Tc as predicted, and ΥR collapses toward zero in the disordered phase.
Observable
b<bc (ordered)
b=bc (critical)
b>bc (disordered)
ηsw
b2<1/4
1/4
b2>1/4
G2(r)
∼r−b2
∼r−1/4
∼r−b2e−r/ξ
ξ
∞
∞
finite, ∼ec/b2−bc2
ρv
≈0
onset
>0 , increasing
KR
≈K0=1/(2πb2)
2/π
≪1/(2πb2)
ΥR
above NK line
on NK line
below NK line
Table 2: Observable signatures across the BKT transition in NN-FT.
One aim of neural network field theory is to put a quantum or effective field theory on a computer, with the network ensemble itself as the theory. We ask how far that aim can be pushed for a function class regular enough to be computed with. Our main result is a no-go theorem with assumptions that hold for standard network architectures. We use it to separate four versions of neural network field theory, according to whether the defining object is the finite width ensemble or its infinite width limit, and whether the target we want to compute is a quantum or an effective field theory. Neither finite width interpretation is straightforwardly consistent. For finite width ensembles with finite variance at each point, the QFT interpretation fails reflection positivity, while the EFT interpretation establishes no scale separation by which the positivity violation can be placed outside its domain of validity. Of the two limit versions, one can be simulated in full and the other only in part, as only its smeared correlators are computable with a controlled error. As such, at the level of a controlled numerical computation, the QFT and EFT versions cannot be distinguished. One dimension escapes the obstruction, yet reflection positivity is shown to still fail there at every finite width for the cosine network. Two escapes from the theorem remain, giving up either finite variance at a point or exact rotation invariance, and we discuss both of these possibilities.
Thomas R. Harvey
The NSF AI Institute for Artificial Intelligence and Fundamental Interactions. Center for Theoretical Physics, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA.
Under mild assumptions, any quantum mechanical (QM) model or quantum field theory (QFT) admits a representation in terms of an ensemble of neural networks with countably many random parameters. We investigate the features of NN-QM and NN-FT models with finitely many parameters, such as a feedforward network of width N<∞. We find that, generically, such models must violate one of the properties of conventional Euclidean QFTs, such as reflection positivity or cluster decomposition. We present several complementary ways of understanding which features can and cannot be preserved at finite N, both in QM and in QFT.
Christian Ferko, Aaron Mutchler
Department of Physics, Northeastern University, Boston, MA 02115, USA · The NSF Institute for Artificial Intelligence and Fundamental Interactions
We modify the NN/QFT duality [1] to incorporate the layerwise permutation symmetry of the network, resulting in a (0+1)-dimensional lattice gauge theory, in which each layer of N neurons acts as an N-component lattice site, and the weight matrices play the role of gauge fields living on the links. In this framework, we compute the tree-level neuron-neuron propagator which describes the evolution of layer variance in the network, and develop the Feynman diagram machinery to compute interactions in the perturbative expansion in 1/N. In particular, we obtain a recursive expression for all corrections to the exact propagator at O(1), representing statistical fluctuations in the ensemble of networks, including infinitely-many loop diagrams mediating the interactions from previous layers. We also present a preliminary analysis of neuron scattering amplitudes that contribute order-by-order in 1/N, which provides a field-theoretic framework for studying higher-point correlations, and by extension information propagation, in deep networks. We remark on some interesting directions for future work at the intersection of neural networks and quantum field theory.
Ro Jefferson, Shradha Ramakrishnan
Institute for Theoretical Physics, and Department of Information and Computing Sciences, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands