Structured Affinity for Unsupervised Visual Class-Incremental Memory in Deep Artificial Immune Networks

Siphesihle Sithungu

cs.CVAug 20, 2026

Structured Affinity for Unsupervised Visual Class-Incremental Memory in Deep Artificial Immune Networks

Authors: Siphesihle Sithungu

Organizations: University of Johannesburg Johannesburg, South Africa

Abstract

Artificial immune networks (AINs) are naturally memory-forming systems, but conventional visual AINs often rely on flattened vector affinity that ignores spatial structure. This paper studies whether structured, gradient-free immune affinity can make Deep AINs viable as replay-free visual class-incremental representation-memory learners. Visual B-cells are formalized as structured templates, including shifted-template affinity, zero-normalized cross-correlation (ZNCC) filters, and feature-map binding profiles. A repertoire is treated both as memory and as a representation-inducing basis, while depth is obtained by passing binding-profile response maps to subsequent immune layers. The resulting Deep AIN exhibits adaptive latent coordinate reorganization: as new classes arrive, the binding-profile space evolves while retaining recoverable structure for earlier classes. Experiments on sklearn digits, MNIST, Fashion-MNIST, and KMNIST show that preserving response maps is critical. Scalar binding-profile variants underperform, whereas feature-map Deep AINs learn class-discriminative visual memory without replay, label-driven immune updates, or backpropagation through the immune layers. On sklearn digits, downstream probes fitted on the learned binding profiles reach 0.939 final balanced accuracy with logistic regression and 0.902 with 1-nearest-neighbour after all ten classes are encountered, with initial-class retention of 0.978. Adaptive layer-wise scale calibration further improves the two-layer feature-map Deep AIN to 0.978 balanced accuracy. With the same calibration rule, Fashion-MNIST reaches 0.814 and KMNIST reaches 0.853. These probes are external validation tools, not components of the AIN. The results identify structured affinity, response-map preservation, adaptive latent reorganization, and layer-wise scale calibration as key mechanisms for replay-free visual immune memory.

Figures & tables

Figure 1: High-level Deep AIN construction. Each layer transforms its antigenic input into a repertoire response. Later layers learn over earlier response patterns.

Figure 2: Convolution-style ZNCC binding response. A local B-cell filter is evaluated over image patches, producing a response map. The response map may be preserved for deeper layers or pooled into a scalar affinity.

Figure 3: Experimental pipeline. The immune layers are updated online from incoming batches. External probes are fitted only after embeddings are produced, to evaluate the current binding-profile space.

Dataset	Method	Accuracy	Balanced accuracy
Sklearn digits	Flat RBF AIN	0.849	0.848
Sklearn digits	Shifted-template RBF AIN	0.917	0.917
Sklearn digits	Convolution-style ZNCC AIN	0.931	0.931
Sklearn digits	Feature-map Deep AIN	0.850	–
MNIST	Feature-map Deep AIN	0.731	–

Table 1: Multiclass digit diagnostics. Sklearn digits rows report accuracy and balanced accuracy. The MNIST feature-map row reports multiclass accuracy over a 350-example-per-class subset.

Method	Evaluation mode	Balanced accuracy	Initial retention	Current-class accuracy	Memory size / embedding dimension
Static k-means	no update	0.100	1.000	0.000	80
Mini-batch k-means	online centroid update	0.617	0.978	0.678	80
Full refit k-means	refit on seen data	0.928	1.000	0.867	120
Online prototype memory	supervised online prototypes	0.872	1.000	0.800	120
No-replay MLP	current batch only	0.827	0.922	0.767	–
Replay MLP	replay classifier	0.953	0.989	0.933	–

Table 2: Final ten-class performance in the sklearn digits class-incremental visual-memory stream. Values are averaged over three seeds. Probe rows externally validate the learned Deep AIN binding-profile memory.

Method	Memory train	Total train	Total inference	Train/event	Inference/eval
Online prototype memory	0.008	0.008	0.129	0.001	0.0129
Static k-means	0.081	0.081	0.088	0.008	0.0088
Mini-batch k-means	0.165	0.165	0.083	0.017	0.0083
No-replay MLP	0.554	0.554	0.007	0.055	0.0007
Replay MLP	0.954	0.954	0.005	0.095	0.0005
Full refit k-means	1.525	1.525	0.126	0.152	0.0126

Table 3: Training and inference time for the sklearn digits Table 2 protocol. Values are seconds averaged over three seeds. Inference time is the total time across all ten evaluations.

Method	Evaluation mode	Balanced accuracy	Initial retention	Current-class accuracy	Memory size / embedding dimension
Static k-means	no update	0.100	1.000	0.000	80
Mini-batch k-means	online centroid update	0.329	0.867	0.067	80
Full refit k-means	refit on seen data	0.699	0.933	0.756	120
Online prototype memory	supervised online prototypes	0.779	0.911	0.733	120
No-replay MLP	current batch only	0.444	0.100	0.967	–
Replay MLP	replay classifier	0.851	0.900	0.856	–

Table 4: Final ten-class performance in the MNIST class-incremental visual-memory stream. Values are averaged over three seeds. Probe rows externally validate the learned Deep AIN binding-profile memory.

Method	Evaluation mode	Balanced accuracy	Initial retention	Current-class accuracy	Memory size / embedding dimension
Static k-means	no update	0.100	1.000	0.000	80
Mini-batch k-means	online centroid update	0.469	0.700	0.767	80
Full refit k-means	refit on seen data	0.649	0.733	0.900	120
Online prototype memory	supervised online prototypes	0.651	0.833	0.844	120
No-replay MLP	current batch only	0.298	0.000	1.000	–
Replay MLP	replay classifier	0.780	0.756	0.933	–

Table 5: Final ten-class performance in the Fashion-MNIST class-incremental visual-memory stream. Values are averaged over three seeds. Feature-map rows use the same adaptive layer-wise affinity scale rule.

Dataset	Method	Balanced accuracy	Initial retention	Current-class accuracy
Sklearn digits	Replay MLP	0.946	0.989	0.944
Sklearn digits	Feature-map Deep AIN + logistic probe	0.943	0.978	0.933
Sklearn digits	Feature-map Deep AIN + 1NN probe	0.952	1.000	0.944
MNIST	Replay MLP	0.858	0.911	0.944
MNIST	Feature-map Deep AIN + logistic probe	0.862	0.900	0.856
MNIST	Feature-map Deep AIN + 1NN probe	0.857	0.922	0.889

Table 6: Cross-dataset comparison of replay MLP and feature-map Deep AIN binding-profile probes. Values are final ten-class performance averaged over three seeds. Deep AIN rows use the same adaptive layer-wise affinity scale rule on every dataset.

Probe	Balanced accuracy	Initial retention	Current-class accuracy
Nearest centroid	0.612	0.811	0.589
Logistic regression	0.711	0.811	0.644
PCA + RBF-SVM	0.818	0.911	0.778
1-nearest-neighbour	0.853	0.933	0.844

Table 7: KMNIST probe-geometry diagnostic on the same learned feature-map Deep AIN binding-profile space. Values are final ten-class performance averaged over three seeds.

Step	Encountered classes	LogReg balanced accuracy	1NN balanced accuracy	LogReg initial retention	1NN initial retention
0	$\{0\}$	1.000	1.000	1.000	1.000
1	$\{0,1\}$	1.000	1.000	1.000	1.000
2	$\{0,1,2\}$	0.981	0.985	1.000	1.000
3	$\{0,1,2,3\}$	0.953	0.944	0.989	1.000
4	$\{0,1,2,3,4\}$	0.971	0.962	0.989	0.978
5	$\{0,\ldots,5\}$	0.957	0.963	0.989	0.989

Table 8: External probe trajectory on online feature-map Deep AIN binding profiles in the sklearn digits class-incremental stream. Values are averaged over three seeds.

Depth	Scale setting	Probe	Balanced accuracy	Initial retention	Current-class accuracy	Dimension	Interpretation
1	direct layer-1 maps	LogReg	0.977	1.000	0.989	480	strong shallow visual response space
1	direct layer-1 maps	1NN	0.982	1.000	0.978	480	strong shallow visual response space
2	fixed $\sigma_{2}=1.0$	LogReg	0.953	0.978	0.967	270	useful but under-scaled depth
2	fixed $\sigma_{2}=2.0$	LogReg	0.978	0.978	1.000	270	best calibrated two-layer result
2	adaptive NN scale $\times 6$	LogReg	0.968	0.989	0.967	270	principled scale rule, near manual best
3	fixed $\sigma_{2}=2.0,\sigma_{3}=2.0$	LogReg	0.924	0.989	0.933	120	third layer improves with scale but compresses strongly

Table 9: Depth and layer-wise scale calibration in the sklearn digits class-incremental stream. Values are final-step means over three seeds. Probe rows externally validate the learned binding-profile memory.

Class-Incremental Learning

Class-incremental learning studies how models can add new categories over time while retaining the ability to recognize previously learned classes.

See all 162 papers

Oct 12, 2026

Memorization in image classification: test set behavior and scaling laws

Deep neural networks are capable of both learning generalizing solutions on natural tasks and memorizing arbitrary mappings. We argue that understanding the former requires understanding the latter, which is much less studied. In what ways do networks that generalize or memorize differ in their behavior or their training dynamics? Do memorizing networks simply function as a lookup table where input-output mappings are stored independently? We investigate these questions in the context of CNNs trained on image classification tasks with randomized labels or pixels. We first study the behavior of such memorizing networks on unseen images. We find that independently initialized networks trained on the same random labels learn different predictive functions: they make different predictions on test examples, in contrast with generalizing networks. We then examine the effect of dataset size on memorization through the required network capacity and training time to interpolate the training set. Counter-intuitively, we find that the number of presentations of a given example required for complete memorization can decrease with dataset size even when input-output pairs share no common structure. Although memorizing networks do not learn a universal solution, they exhibit interesting behaviors which call for further investigations into the underlying mechanisms.

Jeevesh Attri, Florentin Guth

Center for Data Science New York University · Center for Computational Neuroscience Flatiron Institute · Work done while partially affiliated with Center for Data Science, New York University.

Neural Network Memorization Neural Network Training Dynamics Neural Scaling Laws Neural Network Generalization Convolutional Neural Networks

Ask PDF

Apr 9, 2026

Unsupervised Local Plasticity in a Multi-Frequency VisNet Hierarchy

We introduce an unsupervised visual representation learning system based entirely on local plasticity rules, without labels, backpropagation, or global error signals. The model is a VisNet-inspired hierarchical architecture combining opponent color inputs, multi-frequency Gabor and wavelet feature streams, competitive normalization with lateral inhibition, saliency modulation, associative memory, and a feedback loop. All representation learning occurs through continuous local plasticity applied to unlabeled image streams over 300 epochs. Performance is evaluated using a fixed linear probe trained only at readout time. The system achieves 80.1 percent accuracy on CIFAR-10 and 47.6 percent on CIFAR-100, improving over a Hebbian-only baseline. Ablation studies show that anti-Hebbian decorrelation, free-energy inspired plasticity, and associative memory are the main contributors, with strong synergistic effects. Even without learning, the fixed architecture alone reaches 61.4 percent on CIFAR-10, indicating that plasticity, not only inductive bias, drives most of the performance. Control analyses show that independently trained probes match co-trained ones within 0.3 percentage points, and a nearest-class-mean classifier achieves 78.3 percent without gradient-based training, confirming the intrinsic structure of the learned features. Overall, the system narrows but does not eliminate the performance gap to backpropagation-trained CNNs (5.7 percentage points on CIFAR-10, 7.5 percentage points on CIFAR-100), demonstrating that structured local plasticity alone can learn strong visual representations from raw unlabeled data.

Mehdi Fatan Serj, C. Alejandro Parraga, Xavier Otazu

Computer Vision Centre (CVC), Universitat Autonoma de Barcelona, Spain

Visual Representation Learning Hierarchical Representation Learning Local Learning Hebbian Learning

Ask PDF

Jun 12, 2026

AI Engram: In Search of Memory Traces in Artificial Intelligence

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces analogous to biological memory units remains an open question. This work introduces a geometric framework to identify such "AI engrams" by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters, and show that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning and offer geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Jea Kwon, Dong-Kyum Kim, Jiwon Kim +3

Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Seoul National University (SNU), Seoul, Korea · KAIST, Daejeon, Korea

Neural Representation Geometry Memorization in Language Models Neural Network Memorization

Ask PDF

Abstract

Class-Incremental Learning

Explore similar work

Memorization in image classification: test set behavior and scaling laws

Unsupervised Local Plasticity in a Multi-Frequency VisNet Hierarchy

AI Engram: In Search of Memory Traces in Artificial Intelligence