Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers
Organizations: Arts et Métiers Institute of Technology
Abstract
Relevance propagation, which redistributes a network's prediction backward through the layers, is usually treated as a heuristic. We show that it is the solution of a single variational problem: Layer-wise Relevance Propagation (LRP) is the divergence-free flow on the active computational graph of zero relative entropy to the backward transition law of the forward activation flow; for a non-negative reference, this is a relative entropy between Markov measures on backward paths. The rules LRP, LRP and LRP follow from this principle by the choice of a reference, and are instances of one rule, LRPN, whose weights may vary from node to node, for instance with the node's activity, in the spirit of Hebb's principle. The principle rests on a measure-theoretic formalization of feed-forward networks as directed acyclic graphs of Filippov operators, which covers piecewise-smooth and discontinuous architectures (e.g. hard-routed Mixture-of-Experts) with a Clarke-Filippov generalized-gradient calculus; it also characterizes backpropagation as the adjoint equation of a Lagrangian relaxation. On networks of ReLU and max-pooling nodes, LRP is GradientInput, with an explicit jump across a kink where LRP stays continuous. Taking the entropy edge by edge yields a new rule, Sinkhorn Relevance Propagation. Aggregated over an input region, the flow of a non-negative reference tells, with exact accounting, which hidden units carry its evidence, the conditioned flow being a Schrödinger bridge. On VGG-16, the conservation identities hold to round-off, and the rules are told apart by the sign of their maps, the share of the output left to the bias nodes, faithfulness, localization and class dependence; on ConvNeXt-Tiny, LRPN conserves the output and its Hebbian weights give the highest ABPC of the rules tested.
Figures & tables
| rule | cons. error | bias (%) | neg. (%) | time (s) | ABPC | mass | |
|---|---|---|---|---|---|---|---|
| LRP 0 | 92 | 50 | 0.11 | 0.053 | 0.736 | 0.76 | |
| LRP | 16 | 0 | 0.16 | 0.431 | 0.816 | 1.00 | |
| LRPN 2,-1 | 32 | 20 | 0.26 | 0.431 | 0.855 | 1.00 | |
| LRPN | 10 | 0 | 0.24 | 0.419 | 0.785 | 1.00 | |
| SRP | 9 | 0 | 22 | 0.329 | 0.642 | 1.00 | |
| SRP 1 | 40 | 2 | 93 | 0.427 | 0.820 | 1.00 |
| LRP | LRPN | |
| area of the region (% of the image) | ||
| head | 21 (13–31) | |
| body | 26 (21–34) | |
| background | 49 (39–58) | |
| share of carried by the region (%) | ||
| head | 42 (32–50) | 42 (31–50) |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| stage | layers | output size |
|---|---|---|
| input | – | |
| block 1 | 2 convolutions, channels; max-pooling | |
| block 2 | 2 convolutions, channels; max-pooling | |
| block 3 | 3 convolutions, channels; max-pooling | |
| block 4 | 3 convolutions, channels; max-pooling | |
| block 5 | 3 convolutions conv5_1–conv5_3, channels; max-pooling |
| stage | layers | output size |
|---|---|---|
| input | – | |
| stem | convolution of stride , channels; LN | |
| stage 1 | 3 blocks, | |
| stage 2 | LN, convolution of stride ; 3 blocks, | |
| stage 3 | LN, convolution of stride ; 9 blocks, | |
| stage 4 | LN, convolution of stride ; 3 blocks, |
Layer-Wise Relevance Propagation
Layer-wise relevance propagation (LRP) traces model predictions backward to show which inputs and internal features contributed to them.
See all 18 papers