Organizations: Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea · Center for Neuroscience-inspired AI, KAIST, Daejeon, South Korea
Artificial intelligence systems are increasingly deployed in high-stakes decision-making. In response, prototype-based models in explainable AI have attracted growing attention for their ability to retrieve visually similar examples as faithful explanations. Although these models achieve competitive classification accuracy, they often exhibit degraded explanation quality across datasets with varying characteristics, limiting their practical applicability. We draw on insights from cognitive neuroscience, particularly the human memory system, which encodes input-relevant information into representations that support recognition. We propose an Engram Memory Network (EMN), a model that emulates the ventral visual stream and inferior temporal cortex in the human brain. Central to this model is a Hopfield-inspired memory module that iteratively retrieves representative prototypes. These memory-refined latents are used for both reconstruction and classification, encouraging the selected prototypes to capture input-relevant information across diverse datasets. To address the lack of standardized evaluation for prototype explanations, we assess three complementary aspects: perceptual similarity (LPIPS and DISTS) for perceived resemblance, subclass alignment, measuring whether prototypes match the input's fine-grained category beyond supervised labels, and information preservation (NMI), measuring how much input information is captured by selected prototypes. Experiments show that EMN improves visual similarity by 17.06%, subclass alignment by 104.57%, and NMI by 42.42% on average across CIFAR-10/100, MNIST, HAM10000, and CUB-200, while achieving the highest average classification accuracy (87.62%) among all methods. These findings underscore the effectiveness of neuroscience-inspired explanation mechanisms in advancing the interpretability and reliability of prototype-based models.
Figures & tables
Fig. 1: Queries and selected prototypes by baselines (ProtoVAE [ 10 ] , ProtoPNet [ 1 ] , ProtoViT [ 8 ] , ProtoPFormer [ 7 ] ) and EMN (ours). Each row shows a model’s top-3 retrieved prototypes for four representative queries (A–D); each Query column shows the input image and the three columns on the right shows its retrieved prototypes. In these examples, EMN retrieves prototypes that appear visually clearer and more class-consistent, consistent with the improved explanation metrics reported in Sec. IV .
Fig. 2: (a–c) Visual pathway from ventral stream to inferior temporal cortex (ITC), enabling associative memory and categorization with feedback to early visual areas. (d–f) EMN models this process via an encoder, memory-guided refinement, and a decoder.
Model
LPIPS [ 14 ]
DISTS [ 15 ]
top1
top3
top1
top3
(CIFAR-10 [ 12 ] )
ProtoVAE [ 10 ]
7.50 ± 0.06
7.50 ± 0.07
6.75 ± 0.11
6.76 ± 0.11
ProtoPNet [ 1 ]
6.95 ± 0.19
7.00 ± 0.32
4.07 ± 0.13
4.02 ± 0.12
ProtoViT [ 8 ]
6.26 ± 0.10
6.31 ± 0.07
3.67 ± 0.03
3.69 ± 0.03
ProtoPFormer [ 7 ]
6.23 ± 0.02
6.24 ± 0.02
3.68 ± 0.01
3.69 ± 0.02
TABLE I: Visual similarity (LPIPS [ 14 ] and DISTS [ 15 ] ) comparison across datasets. Results are reported as mean ± std. ( ×10−1 ; lower is better; bold indicates best).
Method
top1
top3
ProtoVAE [ 10 ]
-
-
ProtoPNet [ 1 ]
2.28 ± 0.91
1.95 ± 0.47
ProtoViT [ 8 ]
15.93 ± 6.55
14.03 ± 7.04
ProtoPFormer [ 7 ]
19.49 ± 1.60
19.42 ± 1.38
EMN (ours)
39.87 ± 0.72
36.07 ± 0.36
TABLE II: Subclass Alignment (%) on CIFAR-100S (higher is better; chance level is 1%).
Model
Acc.
Model
Acc.
(CIFAR-10 [ 12 ] )
(MNIST [ 11 ] )
ProtoVAE [ 10 ]
81.59 ± 0.36
ProtoVAE [ 10 ]
99.05 ± 0.02
ProtoPNet [ 1 ]
94.30 ± 0.12
ProtoPNet [ 1 ]
99.06 ± 0.10
ProtoViT [ 8 ]
97.63 ± 0.08
ProtoViT [ 8 ]
99.09 ± 0.09
ProtoPFormer [ 7 ]
97.85 ± 0.06
ProtoPFormer [ 7 ]
99.41 ± 0.05
EMN (ours)
96.39 ± 0.27
EMN (ours)
99.19 ± 0.08
TABLE III: Classification accuracy (%) across all datasets (higher is better; bold indicates best).
Model
top1
top3
(CIFAR-10 [ 12 ] )
ProtoVAE [ 10 ]
0.07 ± 0.02
2.41 ± 1.37
ProtoPNet [ 1 ]
0.07 ± 0.10
0.08 ± 0.12
ProtoViT [ 8 ]
0.60 ± 0.08
0.50 ± 0.05
ProtoPFormer [ 7 ]
0.55 ± 0.05
0.55 ± 0.04
EMN (ours)
1.26 ± 0.02
1.21 ± 0.02
TABLE IV: NMI(X;P) [ 16 ] scores across all datasets ( ×10−1 ; higher is better; bold indicates best).
Fig. 3: Examples of prototype-based explanations across datasets: (from top to bottom) MNIST [ 11 ] , HAM10000 [ 19 ] , and CUB-200 [ 13 ] . The Query A-D columns show input queries, and the Prototype A-D column groups display the corresponding top-3 retrieved prototypes.
Setting
Acc ↑
LPIPS [ 14 ] ↓
DISTS [ 15 ] ↓
NMI (X;P) [ 16 ] ↑
N=200
96.63 ± 0.20
5.75 ± 0.02
3.39 ± 0.01
1.06 ± 0.04
N=500
96.63 ± 0.30
5.59 ± 0.02
3.30 ± 0.01
1.17 ± 0.02
N=2000
96.69 ± 0.25
5.40 ± 0.02
3.22 ± 0.01
1.21 ± 0.03
N=5000
96.69 ± 0.19
5.34 ± 0.03
3.18 ± 0.02
1.28 ± 0.02
K=1
96.17 ± 0.24
5.50 ± 0.04
3.33 ± 0.02
1.24 ± 0.02
K=3
96.69 ± 0.25
5.40 ± 0.02
3.22 ± 0.01
1.21 ± 0.03
TABLE V: Sensitivity analysis on CIFAR-10 [ 12 ] . N : number of prototypes; K : target entropy for Lsep ; d2 : categorical latent dimension. Acc (%) higher is better; LPIPS [ 14 ] , DISTS [ 15 ] ( ×10−1 ) lower is better; NMI [ 16 ] ( ×10−1 ) higher is better.
Setting
Acc ↑
LPIPS [ 14 ] ↓
DISTS [ 15 ] ↓
NMI (X;P) [ 16 ] ↑
EMN (ours)
96.69 ± 0.25
5.40 ± 0.02
3.22 ± 0.01
1.21 ± 0.03
w/o Lrec
96.67 ± 0.28
5.39 ± 0.02
3.22 ± 0.01
1.24 ± 0.02
w/o Ldiv
96.44 ± 0.33
5.43 ± 0.03
3.31 ± 0.02
1.27 ± 0.04
w/o Lsep
97.37 ± 0.17
5.48 ± 0.02
3.26 ± 0.01
1.17 ± 0.01
w/o Lrec,Ldiv
96.36 ± 0.19
5.41 ± 0.04
3.30 ± 0.02
1.27 ± 0.05
w/o Lrec,Lsep
97.38 ± 0.15
5.57 ± 0.02
3.32 ± 0.00
1.16 ± 0.02
TABLE VI: Ablation study on loss components (CIFAR-10 [ 12 ] ). Acc (%) higher is better; LPIPS [ 14 ] , DISTS [ 15 ] ( ×10−1 ) lower is better; NMI [ 16 ] ( ×10−1 ) higher is better.