On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networks

Hanyu Zhao; Yang Wu; Yuexian Hou

cs.LGFeb 27, 2026

On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networks

Authors: Hanyu Zhao, Yang Wu, Yuexian Hou

Organizations: Tianjin University, China

Abstract

Inspired by measurement incompatibility and Bell-family inequalities in quantum mechanics, we propose the Non-Classical Network (NCnet), a simple classical neural architecture that stably exhibits non-classical statistical behaviors under typical and interpretable experimental setups. We find non-classicality, measured by the $S$ statistic of CHSH inequality, arises from gradient competitions of hidden-layer neurons shared by multi-tasks. Remarkably, even without physical links supporting explicit communication, one task head can implicitly sense the training task of other task heads via local loss oscillations, leading to non-local correlations in their training outcomes. Specifically, in the low-resource regime, the value of $S$ increases gradually with increasing resources and approaches toward its classical upper-bound 2, which implies that underfitting is alleviated with resources increase. As the model nears the critical scale required for adequate performance, $S$ may temporarily exceed 2. As resources continue to grow, $S$ then asymptotically decays down to and fluctuates around 2. Empirically, when model capacity is insufficient, $S$ is positively correlated with generalization performance, and the regime where $S$ first approaches $2$ often corresponding to good generalization. Overall, our results suggest that non-classical statistics can provide a novel perspective for understanding internal interactions and training dynamics of deep networks.

Figures & tables

Panel of Figure 1 — Figure 1: The framework of the proposed NCnet. (a) XORnet illustrates the basic network structure for modeling the XOR function with ReLU activations. (b) NCnet is constructed by integrating two XORnets. The red node denotes a shared neuron where gradient competition is prone to occur.

NCnet Parameter	Bell Experiment Parameter
Alice’s task $\alpha_{i}$	Alice’s measurement basis $i$
Bob’s task $\beta_{j}$	Bob’s measurement basis $j$
Consistency indicator $A_{i}(B_{j})$	Measurement result $A_{i}(B_{j})$
Correlation of measurement outcomes $C(A_{i},B_{j})$	Correlation $C_{A_{i}B_{j}}$

Table 1: Parameter Mapping Between NCnet and Bell Experiments.

Panel of Figure 2 — Figure 2: Overall results of NCnet with different hidden-layer sizes. (a) Scatter distribution of $S$ obtained from 50 independent runs of NCnet for hidden-layer sizes $n=2,3,4$ . The red dashed line indicates the classical upper-bound of the CHSH statistic, and the blue dot-dashed line marks the Tsirelson bound. (b) Mean correlation values $C(A_{i},B_{j})$ and the corresponding average statistic $S$ for each value of $n$ .

$r$	Task	Multilingual Training				Mixed Reasoning Tasks
$r$	Task	Acc ${}_{A_{i}}$	Acc ${}_{B_{j}}$	$C(A_{i},B_{j})$	$S$	Acc ${}_{A_{i}}$	Acc ${}_{B_{j}}$	$C(A_{i},B_{j})$	$S$
1	$A_{1}B_{1}$	95.44%	91.31%	0.7501	1.502	99.68%	99.82%	0.9899	1.902
	$A_{1}B_{2}$	95.65%	89.34%	0.7192		98.92%	76.16%	0.5144
	$A_{2}B_{1}$	93.68%	91.28%	0.7202		81.79%	98.88%	0.6197
	$A_{2}B_{2}$	93.80%	89.18%	0.6871		79.89%	68.91%	0.2219
2	$A_{1}B_{1}$	97.32%	94.67%	0.8453	1.713	99.92%	99.96%	0.9976	2.191

Table 2: Accuracy Acc

{}_{A_{i}}

(Acc

{}_{B_{j}}

), Correlation

C(A_{i},B_{j})

, and CHSH Statistics

S

for Multilingual Training and Mixed Reasoning Tasks across Varying LoRA Ranks

r

Task	Multilingual Training	Mixed Reasoning Tasks
$A_{1}$	PAWS-X-en	SST-2
$A_{2}$	PAWS-X-fr	CommonsenseQA
$B_{1}$	PAWS-X-ja	MRPC
$B_{2}$	PAWS-X-ko	MathQA

Table 3: Different Task Combinations of Experiments

Figure 3: The trend of the CHSH statistic $S$ as a function of the rank $r$ . The plot includes two sets of experimental results: the blue curve (Multilingual Training) and the orange curve (Mixed Reasoning Tasks). The red dashed line represents the classical upper-bound $S=2$ .

Panel of Figure 4 — Figure 4: The CHSH statistic $S$ convergence across different ranks under Mixed Reasoning Tasks. (a) Higher ranks $r$ lead to faster convergence of $S$ . (b) The bar chart reports $\mu_{\nabla S}$ , the arithmetic mean of the instantaneous slopes of $S$ over epochs 0–80; larger bars indicate a faster convergence rate.

Figure 5: Generalization performance and the CHSH statistic

S

across ranks

r

in Multilingual Training. Bars show task-pair mean accuracy

\overline{\mathrm{Acc}}(A_{i},B_{j})=\frac{\mathrm{Acc}(A_{i})+\mathrm{Acc}(B_{j})}{2}

. The purple curve shows the combination average

\mathrm{Acc}_{\mathrm{comb\_avg}}

(the mean of the four bars) per rank. The red curve denotes

S

value, reflecting the non-classical coupling strength of their learned representations.

$r$	Multilingual Training			Mixed Reasoning Tasks
$r$	Total Params	Trainable Params	Trainable Ratio (%)	Total Params	Trainable Params	Trainable Ratio (%)
1	177,893,380	39,940	0.022	109,522,180	39,940	0.036
2	177,930,244	76,804	0.043	109,559,044	76,804	0.070
4	178,003,972	150,532	0.085	109,632,772	150,532	0.137
8	178,151,428	297,988	0.167	109,780,228	297,988	0.271
16	178,446,340	592,900	0.332	110,075,140	592,900	0.539

Table 4: Parameter Statistics for Different LoRA Rank Parameters

r

Across Experiments

Neural Network Training Dynamics

Neural network training dynamics studies how parameters and representations change during optimization and how those changes shape learning, stability, and generalization.

See all 511 papers

Toward a Physics of Deep Learning and Brains
Sep 26, 2025Arsham Ghavasieh, Meritxell Vila-Minana, Akanksha Khurd +3Phase Transitions in Neural Networks Statistical Physics of Learning
Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, Indiana 47408, USA · Department of Physics, Indiana University, Bloomington, Indiana 47405, USA · Institute for Advanced Study, Princeton, NJ 08540, USA
From Reachability to Learnability: Geometric Design Principles for Quantum Neural Networks
Mar 3, 2026Vishal S. Ngairangbam, Michael SpannowskyQuantum Neural Networks Quantum Machine Learning
Institute for Particle Physics Phenomenology, Department of Physics Durham University, Durham DH1 3LE, United Kingdom · Institute for Theoretical Physics, Karlsruhe Institute of Technology (KIT), Wolfgang-Gaede-Str. 1, 76131 Karlsruhe, Germany
The Bell Boundary of Shared Representations
Oct 12, 2026Yang Wu, Hanyu Zhao, Yazhou Zhang +1Shared Representation Learning Task Interference
Tianjin University, China

Sep 26, 2025J/K move · Enter open · S save

Toward a Physics of Deep Learning and Brains

Arsham Ghavasieh, Meritxell Vila-Minana, Akanksha Khurd, John Beggs+2

Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, Indiana 47408, USA · Department of Physics, Indiana University, Bloomington, Indiana 47405, USA · Institute for Advanced Study, Princeton, NJ 08540, USA

Phase Transitions in Neural Networks Statistical Physics of Learning Neural Network Training Dynamics

Open paper AskPDF arXiv

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie them both? Here we show that the equations used to describe neuronal avalanches in living brains can also be applied to cascades of activity in deep neural networks. These equations are derived from non-equilibrium statistical physics and show that deep neural networks learn best when poised between absorbing and active phases. Because these networks are strongly driven by inputs, however, they do not operate at a true critical point but within a quasi-critical regime -- one that still approximately satisfies crackling noise scaling relations. By training networks with different initializations, we show that maximal susceptibility is a more reliable predictor of learning than proximity to the critical point itself. This provides a blueprint for engineering improved network performance. Finally, using finite-size scaling we identify distinct universality classes, including Barkhausen noise and directed percolation. This theoretical framework demonstrates that universal features are shared by both biological and artificial neural networks.

Abstract

Neural Network Training Dynamics

Explore similar work

Toward a Physics of Deep Learning and Brains

From Reachability to Learnability: Geometric Design Principles for Quantum Neural Networks

The Bell Boundary of Shared Representations