Higher Order Component Attribution via Learned Surrogates in High Energy Physics
Organizations: Imperial College London, United Kingdom · Institute for Theoretical Physics (ITP), Karlsruhe Institute of Technology, Germany · Institute for Quantum Materials and Technologies (IQMT), Karlsruhe Institute of Technology, Germany · Institute of Experimental Particle Physics (ETP), Karlsruhe Institute of Technology, Germany
Abstract
Experimental High Energy Physics (HEP) has adopted the transformer architecture for identification of jets, particularly the state-of-the-art Particle Transformer (ParT) which injects explicit information about particle correlations into the attention mechanism. While able to achieve state-of-the-art results in accuracy, interpretability to its effectiveness has been limited. In this paper we propose component attribution methods using Kolmogorov-Arnold Networks (KANs) that directly tie back to the physical descriptors of jets, to probe whether mechanisms such as Multi-Head Attention (MHA), or the ParT architecture, are able to learn these as discriminating features.
Figures & tables
| MHA | ParT | |||
|---|---|---|---|---|
| Blocks | Best | Mean std | Best | Mean std |
| 1 | 54.2 | 55.3 | ||
| 2 | 72.8 | 80.5 | ||
| 4 | 77.7 | 82.2 | ||
| 8 | 81.9 | 82.8 | ||
| Model | Blocks | (1-prong) | (2-prong) | (3-prong) |
|---|---|---|---|---|
| MHA | 1 | 1.000 (0.999) | 1.000 (0.999) | 1.000 (0.998) |
| 2 | 0.999 (0.997) | 0.998 (0.996) | 0.998 (0.996) | |
| 4 | 0.927 (0.893) | 0.866 (0.842) | 0.908 (0.878) | |
| 8 | 0.875 (0.776) | 0.852 (0.725) | 0.878 (0.749) | |
| ParT | 1 | 1.000 (0.999) | 1.000 (0.999) | 0.999 (0.998) |
| 2 | 0.995 (0.970) | 0.999 (0.998) | 0.995 (0.968) |
Transformer Interpretability
Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.
See all 356 papers