cs.LGNov 3, 2026

Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers

Authors: Olivier Gibaru, Eric Nyiri

Organizations: Arts et Métiers Institute of Technology

Abstract

Relevance propagation, which redistributes a network's prediction backward through the layers, is usually treated as a heuristic. We show that it is the solution of a single variational problem: Layer-wise Relevance Propagation (LRP) is the divergence-free flow on the active computational graph of zero relative entropy to the backward transition law of the forward activation flow; for a non-negative reference, this is a relative entropy between Markov measures on backward paths. The rules LRP0_0, LRPz+_{z^+} and LRPα,β_{α,β} follow from this principle by the choice of a reference, and are instances of one rule, LRPNα,β_{α,β}, whose weights may vary from node to node, for instance with the node's activity, in the spirit of Hebb's principle. The principle rests on a measure-theoretic formalization of feed-forward networks as directed acyclic graphs of Filippov operators, which covers piecewise-smooth and discontinuous architectures (e.g. hard-routed Mixture-of-Experts) with a Clarke-Filippov generalized-gradient calculus; it also characterizes backpropagation as the adjoint equation of a Lagrangian relaxation. On networks of ReLU and max-pooling nodes, LRP0_0 is Gradient×\timesInput, with an explicit jump across a kink where LRPz+_{z^+} stays continuous. Taking the entropy edge by edge yields a new rule, Sinkhorn Relevance Propagation. Aggregated over an input region, the flow of a non-negative reference tells, with exact accounting, which hidden units carry its evidence, the conditioned flow being a Schrödinger bridge. On VGG-16, the conservation identities hold to round-off, and the rules are told apart by the sign of their maps, the share of the output left to the bias nodes, faithfulness, localization and class dependence; on ConvNeXt-Tiny, LRPNα,β_{α,β} conserves the output and its Hebbian weights give the highest ABPC of the rules tested.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagation

    Sep 28, 2026Ping Xiong, Shanglin Li, Yi Ding +2Neural Network InterpretabilityLayer-Wise Relevance Propagation

  2. On the Principles of Deep Feedforward ReLU Networks

    Jul 8, 2026Changcun HuangReLU Neural NetworksNeural Network Interpretability

  3. Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures

    May 30, 2026Ping Xiong, Thomas Schnake, Grégoire Montavon +2Feature AttributionNeural Network Interpretability