Methods used in mechanistic interpretability have revealed human-aligned concepts inside language and image models. In this paper, we search for physics concepts inside a neural a generative collider physics model trained on real LHC data (ShellFlow). We discover features inside ShellFlow that represent resonances, such as the Z, J/ψ, and Υ. The network was never told which muons come from a resonance, it was only trained to generate the kinematics of the final-state particles. We use these features to alter aspects of the Z boson: we change its mass between ∼80 and 105 GeV, and removing such a feature from the neural network produces a broader Z peak. In both cases the rest of the invariant mass spectrum remains mostly unchanged. The feature was built from the gradients of the neural network, and its selectivity was built using pattern recognition methods. A generative particle physics model trained on unlabeled data can be modified internally to change a physical quantity.
Figures & tables
Figure 1: Feynman diagram of the production of a μ−μ+ pair, through an intermediate vector boson V=J/ψ,Υ,Z,γ . X denotes the proton remnants and any additional radiation.
Figure 2: One transformer block of ShellFlow , block L (dotted frame). The steering vector is added to the residual stream at the output of the block. Inset: the output tokens of one event; only the μ− and μ+ tokens receive the steering vector, every other token is left unchanged.
Class
mμμ [GeV]
Events used for the gradients
Z
[81.19,101.19]
181,987
J/ψ
[2.9,3.3]
224,901
Υ
[9.2,10.6]
50,528
rest
elsewhere above 0.3
542,584
total
1,000,000
Table 1: The selection criteria of our classes of muon pairs, defined from the invariant mass of the generated pair of one seed. All events have exactly one pair of opposite-sign muons. The Υ class contains all three Υ resonances, as ShellFlow cannot properly generate three different resonances.
Figure 3: The steering of the Z boson mass. Top: dimuon invariant mass of one million muon pairs, compared with real data (grey), baseline generation of ShellFlow (black, dashed), and the results from adding a steering vector at block 1 with α=±1,±2 (blue to green). Middle: steered over baseline. Bottom: zoom-in over the different peaks: J/ψ , Υ , and Z bosons.
Figure 4: Muon kinematics under Z steering. 1,000,000 validation events of every final state, the Z steering vector at block 1 added to every muon: transverse momentum, pseudorapidity, and azimuth of the muons. Real data (grey), baseline generation (black, dashed), and steered with α=−2,−1,+1,+2 (blue to green); ratio panels: steered over baseline, drawn in every bin with baseline muons, each panel on its own scale. Every other particle type is in App. D.1 .
Figure 5: Fitted pole and width for different α as steered in block L=1 , from the lineshape fit of App. E . (a) Pole mass M . (b) Width Γ . The point at α=0 is the baseline fit. Dotted: the baseline value times exp(α∑kGˉZ(tk)⊤ξZ(tk)) , with the sum equal to 0.122, as predicted by Eq. ( 9 ). Error bars: standard deviation over three seeds of the sampler.
Figure 6: The Z steering quality features as a function of flow time t . (a) Gradient response of the Z , SZ(tk) , (b) the cosine similarity between the class-mean gradient of the Z and the steering vector, cos(GˉZ(tk),ξZ(tk)) , and (c) the largest eigenvalue of Eq. ( 18 ) for the Z , λZ(tk) .
L
∑kSZ(tk)
⟨cos(GˉZ,ξZ)⟩k
⟨λZ⟩k
0
1.4×10−1
1.4×10−1
0.92
1
1.2×10−1
1.5×10−1
0.91
2
7.7×10−2
1.4×10−1
0.91
3
4.2×10−2
1.1×10−1
0.93
4
1.1×10−2
5.1×10−2
0.95
5
2.6×10−5
2.4×10−4
0.93
Table 2: The Z steering vector at each block L . ∑kSZ(tk) is the first-order shift of logmμμ of the Z per unit α (Eq. ( 21 )), and ⟨⋅⟩k is the average over the 200 Euler steps.
Figure 7: Result of the generated muon spectrum from the ablation in Eq. ( 24 ) (blue), compared with the output from ShellFlow (black, dashed), and real data (grey). The inset shows the Z peak in linear scale, and the lower panel the ratio of ShellFlow to the ablation.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: The steering of the Z boson with the difference-of-means steering vector, as in Fig. 3 . Top: dimuon invariant mass of one million muon pairs, compared with real data (grey), baseline generation of ShellFlow (black, dashed), and the results from adding the steering vector at block 1 with α=±0.25,±0.5 . Middle: steered over baseline. Bottom: zoom-in over the J/ψ , Υ , and Z peaks.
Figure 9: The same as Fig. 8 for the Fisher steering vector, with α=±3,±4 (blue to green).
Figure 10: Fitted pole (a) and width (b) of the Z boson against α for the three steering vectors at block 1, as in Fig. 5 ; dotted: the first-order prediction of each. The Fisher steering vector skews the Z peak, so its fits describe the peak less well.
Channels
Architecture
Training
embedding
MLP
heads
blocks
parameters
FLOPs
data
events
epochs
lr
GPUs
ShellFlow
128
512
4
6
3.02M
5.5M
1LMET30 + 2to4lep
783M
30
6⋅10−4
8 A100
ShellFlow-B
128
512
4
6
3.02M
5.5M
2to4lep
268M
100
3⋅10−4
4 A100
ShellGATr
32 v + 96 s
128 v + 384 s
4
6
3.09M
6.2M
2to4lep
268M
100
3⋅10−4
4 B200
Appendix
Table 3: The three networks. ShellGATr counts its channels in four-vectors (v) and scalars (s). FLOPs: one forward pass per particle, measured at 8 particles per event.
Figure 11: Dimuon invariant mass of ShellFlow-B for the Z steering vector at block L=1 , for α=±1,±2 .
Figure 12: The same as Fig. 11 for ShellGATr , for α=±2,±4 .
Figure 13: Fitted pole (a) and width (b) against α for the Z steering vector at block L=1 of the three networks, as in Fig. 5 , mean of three seeds. Dotted: the baseline value of each network times exp(α∑kSZ(tk)) .
∑kSZ(tk)
⟨cos(GˉZ,ξZ)⟩k
⟨λZ⟩k
ShellFlow
0.122
0.147
0.91
ShellFlow-B
0.140
0.108
0.92
ShellGATr
0.047
0.111
0.91
Appendix
Table 4: The Z steering vector at block L=1 of the three networks, as in Table 2 .
Figure 14: Kinematics of the electrons, photons, jets, large-R jets, and hadronic taus after the steering on muon tokens at block L=1 that moves the Z boson peak. Same events and steering as in Fig. 4 : 1,000,000 validation events. Columns: transverse momentum, pseudorapidity, azimuth, energy, and, for the massive particles, mass; 60 bins per panel. Real data (grey), baseline generation (black, dashed), and steered with α=−2,−1,+1,+2 (blue to green). Ratio panels: steered over baseline.
Figure 15: The Z steering vector at block L=0 (left, α=±1,±2 ) and L=3 (right, α=±2,±3 ), as in Fig. 3 . Top: dimuon invariant mass of one million muon pairs, real data (grey), baseline generation of ShellFlow (black, dashed), and steered (blue to green). Middle: steered over baseline. Bottom: zooms on the J/ψ , Υ , and Z .
Figure 16: Fitted pole and width against α for the Z steering vector at blocks L=0 , 1, and 3, as in Fig. 5 . Dotted: the baseline value times exp(α∑kSZ(tk)) of each block (Table 2 ).
Figure 17: Steered generation compared to the baseline ShellFlow generation for different values of α=−2,−1,+1,+2 . The filled regions show the output of the steered model, and the red contour lines show the predicted steering of Eq. ( 20 ). The star is the result from Eq. ( 21 ).
Figure 18: The steering of the J/ψ mass, as in Fig. 3 : dimuon invariant mass of one million muon pairs, compared with real data (grey), baseline generation of ShellFlow (black, dashed), and the results from adding the J/ψ steering vector at block 1 with α=±1,±2 (blue to green).
Figure 19: The same as Fig. 18 for the Υ steering vector.
Class C
∑kSC(tk)
Z
0.122
J/ψ
0.063
Υ
0.123
Appendix
Table 5: ∑kSC(tk) of each class at block L=1 .
Figure 25Figure 26Figure 27
Figure 20: Lineshape fits of the Z peak. Real data (grey), baseline generation of ShellFlow (black, dashed), and the generation steered at block 1 with α=−2,−1,+1,+2 (blue to green), one seed. Each sample is drawn in the 100 bins of its own fit window, with the fitted lineshape as a thin line. Bottom: pulls, (ni−νi)/νi .
Figure 21: The remaining parameters of the lineshape fit against α , for the Z steering vector at block 1: (a) detector resolution σdet , (b) luminosity exponent ρ , and (c) deviance per degree of freedom. Mean over three seeds, with their standard deviation as bars. Dotted in (a): the baseline value times exp(α∑kGˉZ(tk)⊤ξZ(tk)) , a pure shift of logmμμ , as in Fig. 5 . At α=−3 the fit sets σdet to zero and puts all of the broadening into Γ .
Sample
M [GeV]
Γ [GeV]
σdet [GeV]
ρ
D/dof
PDG
91.1879±0.0020
2.4955±0.0023
–
–
–
.
Real data
90.979±0.009
2.87±0.03
1.60±0.02
−0.32±0.06
1.3
Baseline generation
90.573±0.009
2.41±0.02
1.74±0.02
−1.01±0.07
2.0
.
α=−2
70.299±0.011
4.37±0.03
1.25±0.03
0.76±0.05
15.1
Appendix
Table 6: Lineshape fits of the Z peak. Values are the mean over three seeds; uncertainties are statistical, from ΔD=1 . D/dof is the deviance per degree of freedom (95). PDG: world average of the Particle Data Group ( Particle Data Group, 2026 ) .
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.
Bonn-Aachen Institute of Technology, University of Bonn, Germany · Physics Institute III A, RWTH Aachen University, Germany · Institute of Physics, University of Bonn, Germany +1