On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networks

Hanyu Zhao; Yang Wu; Yuexian Hou

cs.LGFeb 27, 2026

On Emergences of Non-Classical Statistical Characteristics in Classical Neural Networks

Authors: Hanyu Zhao, Yang Wu, Yuexian Hou

Organizations: Tianjin University, China

Abstract

Inspired by measurement incompatibility and Bell-family inequalities in quantum mechanics, we propose the Non-Classical Network (NCnet), a simple classical neural architecture that stably exhibits non-classical statistical behaviors under typical and interpretable experimental setups. We find non-classicality, measured by the $S$ statistic of CHSH inequality, arises from gradient competitions of hidden-layer neurons shared by multi-tasks. Remarkably, even without physical links supporting explicit communication, one task head can implicitly sense the training task of other task heads via local loss oscillations, leading to non-local correlations in their training outcomes. Specifically, in the low-resource regime, the value of $S$ increases gradually with increasing resources and approaches toward its classical upper-bound 2, which implies that underfitting is alleviated with resources increase. As the model nears the critical scale required for adequate performance, $S$ may temporarily exceed 2. As resources continue to grow, $S$ then asymptotically decays down to and fluctuates around 2. Empirically, when model capacity is insufficient, $S$ is positively correlated with generalization performance, and the regime where $S$ first approaches $2$ often corresponding to good generalization. Overall, our results suggest that non-classical statistics can provide a novel perspective for understanding internal interactions and training dynamics of deep networks.

Figures & tables

Panel of Figure 1 — Figure 1: The framework of the proposed NCnet. (a) XORnet illustrates the basic network structure for modeling the XOR function with ReLU activations. (b) NCnet is constructed by integrating two XORnets. The red node denotes a shared neuron where gradient competition is prone to occur.

NCnet Parameter	Bell Experiment Parameter
Alice’s task $\alpha_{i}$	Alice’s measurement basis $i$
Bob’s task $\beta_{j}$	Bob’s measurement basis $j$
Consistency indicator $A_{i}(B_{j})$	Measurement result $A_{i}(B_{j})$
Correlation of measurement outcomes $C(A_{i},B_{j})$	Correlation $C_{A_{i}B_{j}}$

Table 1: Parameter Mapping Between NCnet and Bell Experiments.

Panel of Figure 2 — Figure 2: Overall results of NCnet with different hidden-layer sizes. (a) Scatter distribution of $S$ obtained from 50 independent runs of NCnet for hidden-layer sizes $n=2,3,4$ . The red dashed line indicates the classical upper-bound of the CHSH statistic, and the blue dot-dashed line marks the Tsirelson bound. (b) Mean correlation values $C(A_{i},B_{j})$ and the corresponding average statistic $S$ for each value of $n$ .

$r$	Task	Multilingual Training				Mixed Reasoning Tasks
$r$	Task	Acc ${}_{A_{i}}$	Acc ${}_{B_{j}}$	$C(A_{i},B_{j})$	$S$	Acc ${}_{A_{i}}$	Acc ${}_{B_{j}}$	$C(A_{i},B_{j})$	$S$
1	$A_{1}B_{1}$	95.44%	91.31%	0.7501	1.502	99.68%	99.82%	0.9899	1.902
	$A_{1}B_{2}$	95.65%	89.34%	0.7192		98.92%	76.16%	0.5144
	$A_{2}B_{1}$	93.68%	91.28%	0.7202		81.79%	98.88%	0.6197
	$A_{2}B_{2}$	93.80%	89.18%	0.6871		79.89%	68.91%	0.2219
2	$A_{1}B_{1}$	97.32%	94.67%	0.8453	1.713	99.92%	99.96%	0.9976	2.191

Table 2: Accuracy Acc

{}_{A_{i}}

(Acc

{}_{B_{j}}

), Correlation

C(A_{i},B_{j})

, and CHSH Statistics

S

for Multilingual Training and Mixed Reasoning Tasks across Varying LoRA Ranks

r

Task	Multilingual Training	Mixed Reasoning Tasks
$A_{1}$	PAWS-X-en	SST-2
$A_{2}$	PAWS-X-fr	CommonsenseQA
$B_{1}$	PAWS-X-ja	MRPC
$B_{2}$	PAWS-X-ko	MathQA

Table 3: Different Task Combinations of Experiments

Figure 3: The trend of the CHSH statistic $S$ as a function of the rank $r$ . The plot includes two sets of experimental results: the blue curve (Multilingual Training) and the orange curve (Mixed Reasoning Tasks). The red dashed line represents the classical upper-bound $S=2$ .

Panel of Figure 4 — Figure 4: The CHSH statistic $S$ convergence across different ranks under Mixed Reasoning Tasks. (a) Higher ranks $r$ lead to faster convergence of $S$ . (b) The bar chart reports $\mu_{\nabla S}$ , the arithmetic mean of the instantaneous slopes of $S$ over epochs 0–80; larger bars indicate a faster convergence rate.

Figure 5: Generalization performance and the CHSH statistic

S

across ranks

r

in Multilingual Training. Bars show task-pair mean accuracy

\overline{\mathrm{Acc}}(A_{i},B_{j})=\frac{\mathrm{Acc}(A_{i})+\mathrm{Acc}(B_{j})}{2}

. The purple curve shows the combination average

\mathrm{Acc}_{\mathrm{comb\_avg}}

(the mean of the four bars) per rank. The red curve denotes

S

value, reflecting the non-classical coupling strength of their learned representations.

$r$	Multilingual Training			Mixed Reasoning Tasks
$r$	Total Params	Trainable Params	Trainable Ratio (%)	Total Params	Trainable Params	Trainable Ratio (%)
1	177,893,380	39,940	0.022	109,522,180	39,940	0.036
2	177,930,244	76,804	0.043	109,559,044	76,804	0.070
4	178,003,972	150,532	0.085	109,632,772	150,532	0.137
8	178,151,428	297,988	0.167	109,780,228	297,988	0.271
16	178,446,340	592,900	0.332	110,075,140	592,900	0.539

Table 4: Parameter Statistics for Different LoRA Rank Parameters

r

Across Experiments

Neural Network Training Dynamics

Neural network training dynamics studies how parameters and representations change during optimization and how those changes shape learning, stability, and generalization.

See all 511 papers

Sep 26, 2025

Toward a Physics of Deep Learning and Brains

Deep neural networks and brains both learn and share superficial similarities: processing nodes are likened to neurons and adjustable weights are likened to modifiable synapses. But can a unified theoretical framework be found to underlie them both? Here we show that the equations used to describe neuronal avalanches in living brains can also be applied to cascades of activity in deep neural networks. These equations are derived from non-equilibrium statistical physics and show that deep neural networks learn best when poised between absorbing and active phases. Because these networks are strongly driven by inputs, however, they do not operate at a true critical point but within a quasi-critical regime -- one that still approximately satisfies crackling noise scaling relations. By training networks with different initializations, we show that maximal susceptibility is a more reliable predictor of learning than proximity to the critical point itself. This provides a blueprint for engineering improved network performance. Finally, using finite-size scaling we identify distinct universality classes, including Barkhausen noise and directed percolation. This theoretical framework demonstrates that universal features are shared by both biological and artificial neural networks.

Arsham Ghavasieh, Meritxell Vila-Minana, Akanksha Khurd +3

Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, Indiana 47408, USA · Department of Physics, Indiana University, Bloomington, Indiana 47405, USA · Institute for Advanced Study, Princeton, NJ 08540, USA

Phase Transitions in Neural Networks Statistical Physics of Learning Neural Network Training Dynamics

Ask PDF

Mar 3, 2026

From Reachability to Learnability: Geometric Design Principles for Quantum Neural Networks

Classical deep networks are effective because depth enables adaptive geometric deformation of data representations. In quantum neural networks (QNNs), however, depth or state reachability alone does not guarantee this feature-learning capability. We study this question in the pure-state setting by viewing encoded data as an embedded manifold in

\mathbb{C}P^{2^n-1}

and analysing infinitesimal unitary actions through Lie-algebra directions. We introduce Classical-to-Lie-algebra (CLA) maps and the criterion of almost Complete Local Selectivity (aCLS), which combines directional completeness with data-dependent local selectivity. Within this framework, we show that data-independent trainable unitaries are complete but non-selective, i.e. learnable rigid reorientations, whereas pure data encodings are selective but non-tunable, i.e. fixed deformations. Hence, geometric flexibility requires a non-trivial joint dependence on data and trainable weights. We further show that accessing high-dimensional deformations of many-qubit state manifolds requires parametrised entangling directions; fixed entanglers such as CNOT alone do not provide adaptive geometric control. Numerical examples validate that aCLS-satisfying data re-uploading models outperform non-tunable schemes while requiring only a quarter of the gate operations. Thus, the resulting picture reframes QNN design from state reachability to controllable geometry of hidden quantum representations.

Vishal S. Ngairangbam, Michael Spannowsky

Institute for Particle Physics Phenomenology, Department of Physics Durham University, Durham DH1 3LE, United Kingdom · Institute for Theoretical Physics, Karlsruhe Institute of Technology (KIT), Wolfgang-Gaede-Str. 1, 76131 Karlsruhe, Germany

Quantum Neural Networks Quantum Machine Learning Quantum Feature Maps Quantum Circuit Trainability

Ask PDF

Oct 12, 2026

The Bell Boundary of Shared Representations

When does a shared representation become insufficient for jointly supporting multiple tasks? This question lies at the heart of multi-task learning models. Although task interference in shared representations has been widely observed, its onset, measurable form, and capacity boundary remain poorly understood. In this paper, we revisit this problem through the lens of Bell's theorem. We show that if four binary task outcomes can be jointly represented by a common shared representation, their correlations must satisfy the classical CHSH bound

|S|\leq2

. We introduce NCnet, a controlled multi-task model for studying how representational capacity affects task compatibility. Experiments reveal a clear capacity-dependent transition. Once nontrivial task fitting emerges, constrained capacity can yield

|S|>2

. As capacity further increases, these exceedances progressively disappear and

S

returns to the bounded region. This systematic pattern provides evidence that limited representational capacity is an important source of the observed incompatibility. We observe similar transitions across CNNs, PLMs, and LLMs, suggesting that the phenomenon extends beyond the controlled NCnet setting. We refer to this capacity-dependent transition as the Bell Boundary, providing a measurable diagnostic for representation bottlenecks, task incompatibility, and capacity allocation in multi-task models.

Yang Wu, Hanyu Zhao, Yazhou Zhang +1

Tianjin University, China

Shared Representation Learning Task Interference Multi-Task Learning Representation Learning

Ask PDF

Abstract

Neural Network Training Dynamics

Explore similar work

Toward a Physics of Deep Learning and Brains

From Reachability to Learnability: Geometric Design Principles for Quantum Neural Networks

The Bell Boundary of Shared Representations