Purin: A Biology-inspired Mechanism for Artificial Neural Networks

Zishu Liu; Chunbo Luo; Christos Grecos

cs.AISep 25, 2026

Purin: A Biology-inspired Mechanism for Artificial Neural Networks

Authors: Zishu Liu, Chunbo Luo, Christos Grecos

Organizations: University of Exeter · University of Wisconsin - Parkside

Abstract

Artificial neural networks (ANNs) usually represent neural transmission with fixed trainable weights during a training batch, which omits short-term changes in synaptic efficacy. In addition, the discrete time-step simulation requires additional temporal processing that many conventional ANN architectures do not use. To overcome these challenges, we propose Purin, a biology-inspired and ANN-compatible mechanism, that introduces synaptic efficacy modulation into conventional convolutional neural networks. Purin uses a time-interval-based abstraction for neural activities, which allows Purin to introduce short- and long-term synaptic efficacy changes without using discrete time-steps. Purin introduces a bounded factor to represent temporary synaptic efficacy changes, together with two weight matrices that represent input-side and output-side efficacy. The weight matrices are updated by backpropagation and interpreted as the long-term synaptic efficacy changes. Experimental results show that after removing the confounding factors in the AlexNet, VGG11, and GoogLeNet architectures, Purin improves the classification accuracies in all three models across the evaluated datasets.

Figures & tables

Figure 1: A schematic diagram of neurons in ANNs and biology. (a) neuron in ANN. (b) neuron in biology.

Figure 2: A flowchart describing how an ANN’s neuron and biological neuron process their inputs. We focus on the biological neuron using chemical synapses. (a) ANN’s neuron’s procedure. (b) biological neuron’s procedure.

Figure 3: An ANN neuron with Purin mechanism. This neuron receives 3 inputs ( $x_{1},x_{2},x_{3}$ ) from the previous layer and generates 3 outputs ( $y_{1},y_{2},y_{3}$ ) to 3 neurons in the next layer.

Model	UCM	AID	CIFAR-100	Oxford102
AlexNet	$64.5\pm 0.9$	$61.3\pm 1.8$	$48.9\pm 0.5$	$17.4\pm 2.3$
AlexNet+P	$72.2\pm 1.6$	$59.4\pm 1.0$	$45.9\pm 0.4$	$26.8\pm 0.9$
VGG11	$51.9\pm 2.4$	$55.4\pm 1.5$	$32.4\pm 0.4$	$7.4\pm 0.9$
VGG11+P	$68.2\pm 2.6$	$57.8\pm 1.4$	$39.8\pm 0.8$	$23.8\pm 0.7$
GoogLeNet	$76.7\pm 2.5$	$63.3\pm 3.9$	$55.0\pm 0.6$	$33.2\pm 3.2$
GoogLeNet+P	$74.0\pm 1.7$	$64.2\pm 2.0$	$52.8\pm 0.5$	$27.6\pm 2.3$

Table 1: OA (%) on different datasets. The average accuracy and the standard deviation are given. "P" means Purin mechanism.

Confounding factor	Models with Purin	Vanilla models
Input rescaling	[-1.0, 1.0]	[0.0, 1.0]
Activation function	Leaky ReLU	ReLU
Use dropout	No	Yes
Use batch normalization	No	Yes

Table 2: Confounding factors between vanilla models and models with Purin.

Model	UCM	AID	CIFAR-100	Oxford102
AlexNet	$61.3\pm 0.6$	$54.1\pm 1.1$	$42.1\pm 0.6$	$16.6\pm 2.2$
AlexNet+P	$72.2\pm 1.6$	$59.4\pm 1.0$	$45.9\pm 0.4$	$26.8\pm 0.9$
VGG11	$51.3\pm 3.4$	$50.2\pm 1.3$	$33.3\pm 1.1$	$10.5\pm 0.7$
VGG11+P	$68.2\pm 2.6$	$57.8\pm 1.4$	$39.8\pm 0.8$	$23.8\pm 0.7$
GoogLeNet	$49.9\pm 3.3$	$53.8\pm 1.5$	$39.2\pm 0.9$	$14.4\pm 2.1$
GoogLeNet+P	$74.0\pm 1.7$	$64.2\pm 2.0$	$52.8\pm 0.5$	$27.6\pm 2.3$

Table 3: Comparisons with matched benchmark models in OA (%). The average accuracy and the standard deviation are given. "P" means Purin mechanism.

Variant	Split weight	$g_{stp}$
A	-	-
B	✓	-
C	✓	✓

Table 4: Ablation variants

Model	Variant	UCM	AID	CIFAR-100	Oxford102
AlexNet	A	$61.3\pm 0.6$	$54.1\pm 1.1$	$42.1\pm 0.6$	$16.6\pm 2.2$
	B	$72.4\pm 0.9$	$58.7\pm 1.2$	$44.5\pm 0.4$	$26.6\pm 1.2$
	C	$72.2\pm 1.6$	$59.4\pm 1.0$	$45.9\pm 0.4$	$26.8\pm 0.9$
VGG11	A	$51.3\pm 3.4$	$50.2\pm 1.3$	$33.3\pm 1.1$	$10.5\pm 0.7$
	B	$65.4\pm 2.5$	$56.7\pm 0.8$	$38.0\pm 0.9$	$23.5\pm 0.6$
	C	$68.2\pm 2.6$	$57.8\pm 1.4$	$39.8\pm 0.8$	$23.8\pm 0.7$

Table 5: OA (%) of ablation variants on four datasets.

Variant	exponential recovery	range
A	-	$g_{stp}\sim U(0.8,1.2)$
B	-	(0.8,1.2)
C	✓	$(0,+\infty)$
D	-	$(0,+\infty)$
E	✓	$(0.8,1.2)$

Table 6: Variants of

g_{stp}

Architecture	Variant	CIFAR-100	UCM
VGG11	A	$37.6\pm 0.4$	$62.1\pm 1.6$
	B	$38.6\pm 0.7$	$65.7\pm 2.3$
	C	$38.9\pm 0.4$	$66.0\pm 1.2$
	D	$38.6\pm 1.0$	$67.2\pm 1.7$
	E	$39.8\pm 0.8$	$68.2\pm 2.6$
GoogLeNet	A	$30.5\pm 0.3$	$58.19\pm 1.7$

Table 7: OA (%) for architectures with different

g_{stp}

variants

Model	CIFAR-100	UCM
GoogLeNet+conv SE block	$38.4\pm 1.1$	$56.4\pm 0.9$
GoogLeNet+SE block	$35.9\pm 1.9$	$57.7\pm 4.3$
GoogLeNet (matched)	$39.2\pm 0.9$	$49.9\pm 3.3$
GoogLeNet+Purin	$52.8\pm 0.5$	$74.0\pm 1.7$

Table 8: OA (%) on CIFAR-100 and UCM for different combinations of matched GoogLeNet

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: The ResNet20-like model used in our experiments. We build residual connections from the first convolution layer to the last layer in the same convolution block.

Model	Accuracy
vanilla ResNet20	$69.53\pm 0.61$
Ours	$69.35\pm 2.64$

Appendix

Table 9: Accuracy (%) on the CIFAR-10 dataset. The average values and standard deviations are given.

Panel of Figure 5 — Figure 5: $g_{stp}$ curves in an evaluation batch in convolution layers and fully connected layers in the Purin+VGG11 model. The last fully connected layer does not use $g_{stp}$ . (a) Convolution_0. (b) Convolution_2. (c) Convolution_4, (d) Convolution_5. (e) Convolution_7. (f) Convolution_8. (g) Convolution_10. (h) Convolution_11. (i) fully connected layer_14. (j) fully connected layer_15.

Model	Extra trainable parameter	Overhead
AlexNet	9344	0.0162%
VGG11	10944	0.0084%
GoogLeNet	7280	0.1199%

Appendix

Table 10: Relative overhead of additional trainable parameters for a dataset with

224\times 224

images, 100 categories.

Neural Processes

Neural Processes learn distributions over functions from observed examples, enabling fast predictions with uncertainty for new inputs and tasks.

See all 56 papers

Perforated Backpropagation: A Neuroscience Inspired Extension to Artificial Neural Networks
Jan 29, 2025Rorry Brenner, Laurent IttiBackpropagation
Perforated AI Inc. · Computer Science and Neuroscience, University of Southern California
Position Paper: Neurotransmitters as a Missing Dimension in Artificial Neural Networks
Sep 17, 2026Yupei Li, Manuel Milling, Berrak Sisman +1Neural Network Training Dynamics
Department of Computing and Chair of Health Informatics Imperial College London and Technical University of Munich London and Munich, United Kingdom and Germany · Chair of Health Informatics Technical University of Munich Munich, Germany · Department of Electrical and Computer Engineering Johns Hopkins Whiting School of Engineering Baltimore, United States +1
Updating the standard neuron model in artificial neural networks
May 19, 2026Raul Mohedano, Thomas Batard, Erik Velasco-Salido +4Neural Network Robustness Neural Network Expressivity
1Spanish National Research Council (CSIC), Madrid, Spain. · Center for Research in Mathematics (CIMAT), Guanajuato, Mexico. · 3Universidad Aut´onoma de Madrid (UAM), Madrid, Spain. +1

Jan 29, 2025J/K move · Enter open · S save

Perforated Backpropagation: A Neuroscience Inspired Extension to Artificial Neural Networks

Rorry Brenner, Laurent Itti

Perforated AI Inc. · Computer Science and Neuroscience, University of Southern California

Backpropagation

Open paper AskPDF arXiv

The neurons of artificial neural networks were originally invented when much less was known about biological neurons than is known today. Our work explores a modification to the core neuron unit to make it more parallel to a biological neuron. The modification is made with the knowledge that biological dendrites are not simply passive activation funnels, but also compute complex non-linear functions as they transmit activation to the cell body. The paper explores a novel system of

perforated'' backpropagation empowering the artificial neurons of deep neural networks to achieve better performance coding for the same features they coded for in the original architecture. After an initial network training phase, additional

dendrite'' nodes are added to the network and separately trained with a different objective: to correlate their output with the remaining error of the original neurons. The trained dendrites are then frozen, and the original neurons are further trained, now taking into account the additional error signals provided by the dendrites. The cycle of training the original neurons and then adding and training dendrites can be repeated several times until satisfactory performance is achieved. Our algorithm was successfully added to modern state-of-the-art PyTorch networks across multiple domains, improving upon original accuracies and allowing for significant model compression without a loss in accuracy.

Abstract

Neural Processes

Explore similar work

Perforated Backpropagation: A Neuroscience Inspired Extension to Artificial Neural Networks

Position Paper: Neurotransmitters as a Missing Dimension in Artificial Neural Networks

Updating the standard neuron model in artificial neural networks