cs.CLApr 1, 2026

Phase transition on a context-sensitive random language model with short range interactions

Authors: Yuma Toji, Jun Takahashi, Vwani Roychowdhury, Hideyuki Miyahara

Organizations: Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Hokkaido 060-0814, Japan · Institute for Solid State Physics, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8581, Japan · Henry Samueli School of Engineering and Applied Science, University of California, Los Angeles, California 90095

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Since the random language model was proposed by E. DeGiuli [Phys. Rev. Lett. 122, 128301], language models have been investigated intensively from the viewpoint of statistical mechanics. Recently, the existence of a Berezinskii--Kosterlitz--Thouless transition was numerically demonstrated in models with long-range interactions between symbols. In statistical mechanics, it has long been known that long-range interactions can induce phase transitions. Therefore, it has remained unclear whether phase transitions observed in language models originate from genuinely linguistic properties that are absent in conventional spin models. In this study, we construct a random language model with short-range interactions and numerically investigate its statistical properties. Our model belongs to the class of context-sensitive grammars in the Chomsky hierarchy and allows explicit reference to contexts. We find that a phase transition occurs even when the model refers only to contexts whose length remains constant with respect to the sentence length. This result indicates that finite-temperature phase transitions in language models are genuinely induced by the intrinsic nature of language, rather than by long-range interactions.

Figures & tables

Figure 1: Ranking of symbol appearance frequencies for (upper) K=100K=100 and (lower) K=500K=500 , plotted on log–log scales. The critical temperature in both cases is kBT≃0.42k_{\mathrm{B}}T\simeq 0.42 . We set J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . The curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 2: A schematic diagram of the vectors ek\bm{e}_{k} for K=2,3,K=2,3, and 44 . They sum to the zero vector, and the inner products of any two different vectors are constant.
Figure 3: Temperature dependence of (upper) the magnetization, Eq. ( 3.1 ), (middle) the susceptibility, Eq. ( 3.4 ), and (lower) the Binder parameter, Eq. ( 3.5 ). We set K=20,J=1.0,q=10−2.0,t=0.0K=20,J=1.0,q=10^{-2.0},t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 4: The correlation function, Eq. ( 3.7 ), with i=⌊N/4⌋i=\lfloor N/4\rfloor and j=⌊3N/4⌋−1j=\lfloor 3N/4\rfloor-1 . We set K=20,J=1.0,q=10−2.0,t=0.0K=20,J=1.0,q=10^{-2.0},t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.1,0.2,…,2.0k_{\mathrm{B}}T=0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 5: Finite-size scaling of χ~\tilde{\chi} with Tc=0.24,ν=2.50T_{\mathrm{c}}=0.24,\nu=2.50 , and γ=2.00\gamma=2.00 . We set K=20,J=1.0,q=10−2.0,t=0.0K=20,J=1.0,q=10^{-2.0},t=0.0 , and ϵ=0.00\epsilon=0.00 . We varied N=64,128,…,4096N=64,128,\dots,4096 .
Figure 6: Phase diagram for K=20,J=1.0,t=0.0K=20,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter qq , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
Figure 7: Phase diagram for K=20,J=1.0K=20,J=1.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter tt , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
Appendix figures & tables44 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Phase diagram for K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter qq , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
Figure 9: Temperature dependence of the magnetization, Eq. ( 3.1 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 10: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 11: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 12: The system-size dependence of the magnetization, Eq. ( 3.1 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 13: The system-size dependence of the susceptibility, Eq. ( 3.4 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 14: The system-size dependence of the Binder parameter, Eq. ( 3.5 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 15: The histogram of the magnetization. The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2K=2 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 16: Typical configurations of symbols. The upper panel shows an example sentence at a temperature below the transition ( kBT=0.001k_{\mathrm{B}}T=0.001 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The middle panel shows an example sentence at a temperature just above the transition ( kBT=0.20k_{\mathrm{B}}T=0.20 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The lower panel shows an example sentence at a temperature above the transition ( kBT=0.40k_{\mathrm{B}}T=0.40 ) with magnetization 0.00≤M<0.020.00\leq M<0.02 . We set K=2K=2 , J=1.0J=1.0 , q=10−2.0q=10^{-2.0} , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 .
Figure 17: Typical configurations of symbols. The upper panel shows an example sentence at a temperature below the transition ( kBT=0.14k_{\mathrm{B}}T=0.14 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The middle panel shows an example sentence at a temperature just above the transition ( kBT=0.34k_{\mathrm{B}}T=0.34 ) with magnetization 0.72≤M<0.740.72\leq M<0.74 . The lower panel shows an example sentence at a temperature above the transition ( kBT=0.54k_{\mathrm{B}}T=0.54 ) with magnetization 0.00≤M<0.020.00\leq M<0.02 . We set K=2K=2 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 .
Figure 18: The correlation function , Eq. ( 3.7 ), with i=⌊N/4⌋i=\lfloor N/4\rfloor and j=⌊3N/4⌋−1j=\lfloor 3N/4\rfloor-1 . The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.1,0.2,…,2.0k_{\mathrm{B}}T=0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 19: The mutual information I(⌊N/4⌋,⌊3N/4⌋−1)I(\lfloor N/4\rfloor,\lfloor 3N/4\rfloor-1) . The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=2,J=1.0,t=0.0K=2,J=1.0,t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.1,0.2,…,2.0k_{\mathrm{B}}T=0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 20: The finite-size scaling plot. The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We plot χ~(T,N)/Nγ/ν\tilde{\chi}(T,N)/N^{\gamma/\nu} versus N1/νtN^{1/\nu}t . The critical exponents are set to ν=3.25\nu=3.25 and γ=3.00\gamma=3.00 for q=10−2.0q=10^{-2.0} , and to ν=3.50\nu=3.50 and γ=3.00\gamma=3.00 for q=10−1.0q=10^{-1.0} . We set K=2K=2 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for N=64,128,256,512,1024,2048,4096N=64,128,256,512,1024,2048,4096 , and the curves for different NN are overlaid.
Figure 21: The qq -dependence of the critical exponents ν\nu and γ\gamma . We set K=2K=2 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for N=64,128,256,512,1024,2048,4096N=64,128,256,512,1024,2048,4096 , and the curves for different NN are overlaid.
tt0.30.50.7
q=10−2.0q=10^{-2.0}0.300.26-
q=10−1.0q=10^{-1.0}0.360.28-
Appendix
Table 1: The critical temperature of each parameter set. The symbol ‘-’ means no phase transitions occur in the parameter setting.
Figure 22: Temperature dependence of the magnetization, Eq. ( 3.1 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−2.0q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 23: Temperature dependence of the magnetization, Eq. ( 3.1 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 24: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−2.0q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 25: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 26: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−2.0q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 27: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=2K=2 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 28: Phase diagram for K=2K=2 , J=1.0J=1.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter tt , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
Figure 29: Phase diagram for K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter qq , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
Figure 30: Temperature dependence of the magnetization, Eq. ( 3.1 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 31: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 32: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 33: The system-size dependence of the magnetization, Eq. ( 3.1 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 34: The system-size dependence of the susceptibility, Eq. ( 3.4 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 35: The system-size dependence of the Binder parameter, Eq. ( 3.5 ). The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.01,0.1,0.2,…,2.0k_{\mathrm{B}}T=0.001,0.01,0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 36: The histogram of the magnetization. The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.001,0.1,…,2.0k_{\mathrm{B}}T=0.001,0.1,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 37: Typical configurations of symbols. The upper panel shows an example sentence at a temperature below the transition ( kBT=0.001k_{\mathrm{B}}T=0.001 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The middle panel shows an example sentence at a temperature just above the transition ( kBT=0.20k_{\mathrm{B}}T=0.20 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The lower panel shows an example sentence at a temperature above the transition ( kBT=0.40k_{\mathrm{B}}T=0.40 ) with magnetization 0.00≤M<0.020.00\leq M<0.02 . We set K=20K=20 , J=1.0J=1.0 , q=10−2.0q=10^{-2.0} , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 .
Figure 38: Typical configurations of symbols. The first panel shows an example sentence at a temperature below the transition ( kBT=0.20k_{\mathrm{B}}T=0.20 ) with magnetization 0.68≤M<0.700.68\leq M<0.70 . The second panel shows an example sentence at the same temperature ( kBT=0.20k_{\mathrm{B}}T=0.20 ) with magnetization 0.98≤M<1.000.98\leq M<1.00 . The third panel shows an example sentence at a temperature just above the transition ( kBT=0.40k_{\mathrm{B}}T=0.40 ) with magnetization 0.22≤M<0.240.22\leq M<0.24 . The fourth panel shows an example sentence at a temperature above the transition ( kBT=0.60k_{\mathrm{B}}T=0.60 ) with magnetization 0.02≤M<0.040.02\leq M<0.04 . We set K=20,R−=R+=1,J=0,q=10−1.0,t=0.0,s=0.9K=20,R_{-}=R_{+}=1,J=0,q=10^{-1.0},t=0.0,s=0.9 , and ϵ=0.00\epsilon=0.00 . The first and second panels correspond to the same value of kBTk_{\mathrm{B}}T , because the histogram at this temperature exhibits two peaks, and we show configurations corresponding to both peaks. However, the configurations with 0.98≤M<1.000.98\leq M<1.00 appear more frequently than those with 0.68≤M<0.700.68\leq M<0.70 .
Figure 39: The correlation function, Eq. ( 3.7 ), with i=⌊N/4⌋i=\lfloor N/4\rfloor and j=⌊3N/4⌋−1j=\lfloor 3N/4\rfloor-1 . The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.1,0.2,…,2.0k_{\mathrm{B}}T=0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 40: The mutual information I(⌊N/4⌋,⌊3N/4⌋−1)I(\lfloor N/4\rfloor,\lfloor 3N/4\rfloor-1) . The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for various kBT=0.1,0.2,…,2.0k_{\mathrm{B}}T=0.1,0.2,\dots,2.0 , and the curves for different kBTk_{\mathrm{B}}T are overlaid.
Figure 41: The finite-size scaling plot. The top panel is the result for q=10−2.0q=10^{-2.0} , and the bottom panel is the result for q=10−1.0q=10^{-1.0} . We plot χ~(T,N)/Nγ/ν\tilde{\chi}(T,N)/N^{\gamma/\nu} versus N1/νtN^{1/\nu}t . The critical exponents are set to ν=2.50\nu=2.50 and γ=2.00\gamma=2.00 for q=10−2.0q=10^{-2.0} , and to ν=2.75\nu=2.75 and γ=2.00\gamma=2.00 for q=10−1.0q=10^{-1.0} . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for N=64,128,256,512,1024,2048,4096N=64,128,256,512,1024,2048,4096 , and the curves for different NN are overlaid.
Figure 42: The qq -dependence of the critical exponents ν\nu and γ\gamma . We set K=20K=20 , J=1.0J=1.0 , t=0.0t=0.0 , and ϵ=0.00\epsilon=0.00 . We show the results for N=64,128,256,512,1024,2048,4096N=64,128,256,512,1024,2048,4096 , and the curves for different NN are overlaid.
 t=0.3t=0.3t=0.5t=0.5t=0.7t=0.7
q=10−2.0q=10^{-2.0}0.240.18-
q=10−1.0q=10^{-1.0}0.300.24-
Appendix
Table 2: The critical temperature of each parameter set. The symbol ‘-’ means no phase transitions occur in the parameter setting.
Figure 43: Temperature dependence of the magnetization, Eq. ( 3.1 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20,J=1.0,q=10−2.0K=20,J=1.0,q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 44: Temperature dependence of the magnetization, Eq. ( 3.1 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20K=20 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 45: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20,J=1.0,q=10−2.0K=20,J=1.0,q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 46: Temperature dependence of the susceptibility, Eq. ( 3.4 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20K=20 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 47: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20,J=1.0,q=10−2.0K=20,J=1.0,q=10^{-2.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 48: Temperature dependence of the Binder parameter, Eq. ( 3.5 ). The first panel is the result for t=0.3t=0.3 , the second panel is the result for t=0.5t=0.5 , and the third panel is the result for t=0.7t=0.7 . We set K=20K=20 , J=1.0J=1.0 , q=10−1.0q=10^{-1.0} , and ϵ=0.00\epsilon=0.00 . We show the results for various system sizes N=16,32,…,4096N=16,32,\dots,4096 , and the curves for different NN are overlaid in each panel.
Figure 49: Phase diagram for K=20K=20 , J=1.0J=1.0 , and ϵ=0.00\epsilon=0.00 . The horizontal and vertical axes represent the parameter tt , which controls the probability of selecting production rules, and the critical temperature kBTk_{\mathrm{B}}T , respectively.
More in this topic

Language Modeling

Language modeling develops systems that learn patterns in text and other sequences to understand, predict, and generate language.

See all 519 papers

Explore similar work

CardsList

  1. Scaling limit of the Random Language Model

    Jun 26, 2026Eric De GiuliLanguage Model Scaling LawsLanguage Modeling

    Department of Physics, Toronto Metropolitan University, M5B 2K3, Toronto, Canada

  2. Phantom transitions in language model fine-tuning

    May 25, 2026Vaibhav Prakash, Jayasri DontabhaktuniLLM Fine-Tuning

    Department of Physics, Mahindra University, Hyderabad, India

  3. A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning

    May 13, 2026Jason Gaitonde, Frederic Koehler, Elchanan Mossel +2Long-Context Language ModelingLanguage Model Scaling Laws

    Duke University · University of Chicago · Massachusetts Institute of Technology +1