Contrastive Learning for Interpretable Anomaly Detection at Collider Experiments

Haoyi Jia; Sagar Addepalli; Julia Gonski

cs.LGAug 13, 2026

Contrastive Learning for Interpretable Anomaly Detection at Collider Experiments

Authors: Haoyi Jia, Sagar Addepalli, Julia Gonski

Organizations: Department of Physics, Stanford University, 450 Jane Stanford Way, Stanford, CA 94305 · SLAC National Accelerator Laboratory, 2575 Sand Hill Rd, Menlo Park, CA 94025

Abstract

Generic event-level anomaly detection for collider physics has two recurring problems: anomaly scores are hard to interpret, and they correlate strongly with energy scale and object multiplicity. We present Organized Representation via Contrastive learning for Anomaly detection (ORCA), a two-stage framework that first learns an embedding space via supervised contrastive learning across a diverse set of physics processes, then runs a standard autoencoder in that space to generate event-level anomaly scores. On a simulated dataset consistent with conditions at the High-Luminosity Large Hadron Collider, ORCA delivers significant gains in both breadth and depth of sensitivity to new physics signals with respect to a baseline autoencoder architecture. Beyond improved sensitivity, the contrastive embedding makes the anomalous sample interpretable: because known processes occupy distinct regions of the space, a maximum-likelihood template fit to the embedding distributions can attribute events in an anomalous sample to template physics processes with quantified uncertainties. We demonstrate that the fit accurately recovers injected signal yields, including for signals excluded from the training of the embedding, and characterizes signals absent from the template library through the known processes they most resemble. These results establish ORCA as a route to interpretable anomaly detection-based searches at colliders, where the embedding geometry carries higher dimensional physics information compared to standard one-dimensional output fits, enhancing downstream statistical analysis.

Figures & tables

Figure 1: Overview of the two-stage ORCA framework. In the first stage, an embedder consisting of an encoder and a projection head is trained with a supervised contrastive loss to produce a representation space in which the Standard Model process groups are separated from one another. In the second stage, an anomaly detection model, here an autoencoder, is trained unsupervised on background data embedded into this representation, and its reconstruction error serves as the anomaly score. The projection head is used only during the first-stage training.

Figure 2: Architecture of the Stage 1 ORCA embedder model. Each reconstructed object (jets, electrons, muons, photons, and MET) is mapped to a common 128-dimensional token by a per-type embedding network with shared weights across objects of the same type. The 23 tokens, augmented with learned positional embeddings, pass through four Transformer encoder blocks and a multi-head attention-pooling layer that aggregates them into a single event representation

\vec{r}\in\mathbb{R}^{128}

, normalized onto the unit hypersphere. A projection head maps

\vec{r}

to an 8-dimensional vector on which the supervised contrastive loss (

\tau=0.07

) is evaluated; a variance regularizer is applied to

\vec{r}

. The projection head is used only during training, and downstream anomaly detection operates on

\vec{r}

Group	Processes
Single vector boson ( $V$ )	$Z\to\nu\bar{\nu}$ , $Z\to q\bar{q}$ ( $q=u,d,s$ ), $Z\to b\bar{b}$ , $Z\to c\bar{c}$ ,
Single vector boson ( $V$ )	$W\to\ell\bar{\nu}$ , $W\to q\bar{q^{\prime}}$ ( $q,q^{\prime}=u,d,c,s$ )
Diboson ( $VV$ )	$WW$ , $WZ$ , $ZZ$ (all-leptonic, semi-leptonic, all-hadronic)
Triboson ( $VVV$ )	$VVV$ (all-leptonic, semi-leptonic, all-hadronic)
Drell–Yan (DY)	$Z/\gamma^{*}\to\ell\ell$
Low-mass resonance ( $\Upsilon$ )	$\Upsilon\to\ell\ell$

Table 1: The 48 Standard Model processes used to train the embedder (right), merged into the 14 groups that define the labels of the supervised contrastive loss (left).

Figure 3: Architecture of the Stage 2 ORCA autoencoder model. The 128-dimensional latent representation

z

produced by the contrastive encoder is standardized and passed through an autoencoder with encoder layers of sizes 128, 64, and 32, followed by a 16-dimensional bottleneck. A symmetric decoder reconstructs the input representation,

\hat{z}\in\mathbb{R}^{128}

. The per-event anomaly score is defined as the mean squared reconstruction error,

s=\langle(z-\hat{z})^{2}\rangle

Figure 4: Primary component analysis in six dimensions of the standard autoencoder (top) and ORCA Stage 1 embedding model output space (bottom). Only a focused subset of all training signal models is shown. The capacity of the contrastive learning in the ORCA approach allows for more distinguished classes of signals in the latent space, ultimately providing more information to a downstream autoencoder training.

Figure 5: The relative percent improvement of the contrastive method with respect to the standard autoencoder in terms of AUC. All groups defined in Table 1 are shown except QCD due to its proximity to the background.

Figure 6: The relative percent improvement of the contrastive method with respect to the standard autoencoder in terms of TPR at a fixed FPR of 10 -4 . All groups defined in Table 1 are shown except QCD due to its proximity to the background.

Panel of Figure 7 — Figure 7: ROC curves comparing ORCA (solid) and the standard autoencoder (dash), shown for four representative process groups: Di-Higgs (top left), Gluon-gluon fusion Higgs (top right), Two Vector Boson (bottom left), and Vector Boson Fusion Higgs (bottom right). The signal efficiency is shown as a function of the background efficiency, evaluated against minimum-bias data. The AUC for both models is provided parenthetically in the legend for each process. The dashed vertical line indicates the background efficiency working point of $10^{-4}$ used for the fixed background rejection comparison.

Panel of Figure 8 — Figure 8: Fitted event yields in the holdout test for four groups: Di-Higgs (top left), Single Vector Boson (top right), Low-mass resonance $\Upsilon$ (bottom left), and Gluon–gluon fusion Higgs (bottom right); in each case the group is excluded from training and 200 of its AD-passing events are injected. Within each panel, each row is one fit parameter: the background yield (shaded gray) and the per-group signal yields, with the injected group highlighted. Blue bars show the blind fit (no template for the injected group); orange bars the informed fit (its template included). Error bars are the sandwich estimates; yields at the boundary $\nu_{i}=0$ carry no interval. Vertical dotted lines mark the true background yield (red) and the injected yield (green).

Panel of Figure 9 — Figure 9: Signal recovery, defined as the fitted signal yield divided by the number of injected events, as a function of the number of injected signal events for four representative process groups: Di-Higgs (top left), Three Vector Boson (top right), Top pair (bottom left), and VBF Single Higgs (bottom right). Fits are performed in the standard autoencoder embedding (blue circles), the contrastive learning embedding (orange squares), and on the one-dimensional contrastive learning anomaly score (gray crosses). The dashed red line indicates exact recovery of the injected signal.

Explore similar work

Oct 9, 2026

Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC

Anomaly detection, the identification of collision events that deviate from Standard Model expectations, is one of the most promising model-agnostic strategies for new-physics searches at the Large Hadron Collider. Searching for rare events in learned low-dimensional representations removes the need to commit to hand-crafted observables a priori, but sacrifices the immediate physical interpretation those observables provide. For anomaly detection to enter a discovery pipeline, flagged events must therefore be diagnosed locally, both to distinguish potential new physics from mismodeling or detector effects and to identify observables for targeted follow-up searches. We evaluate supervised and self-supervised neural embedding strategies on COLLIDE-2V, a 750-million-event dataset simulated under realistic High-Luminosity LHC conditions, using the reconstruction error of an autoencoder trained in the learned latent space as the anomaly score. Across the configurations and benchmark signals considered, supervised objectives, which exploit simulation process labels during training, generally outperform their self-supervised counterparts and, in their best configurations, improve on an autoencoder trained directly on kinematic features, raising the signal efficiency at 10% background acceptance on all held-out processes. We further introduce a local diagnostic pipeline that fits a Gaussian Mixture Model to the SM latent space, profiles each component through high-level observables without process labels, localizes the flagged events, and ranks observables by their local discriminating power. The ranking recovers the defining kinematics of the held-out processes, while correlating the anomaly score with the same observables quantifies how closely the score tracks them, revealing cases where the most discriminating observable is not the one the score follows most closely.

Donatella Genovese, Philip Ploner, Simone Scardapane +2

Department of Computer, Control and Management Engineering, Sapienza University of Rome · Department of Physics, ETH Zurich · Department of Information Engineering, Electronics and Telecommunications, Sapienza University of Rome +1

Anomaly Detection Interpretable Anomaly Detection Reconstruction-Based Anomaly Detection High-Energy Physics Representation Learning

Ask PDF

Jun 11, 2026

Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches

Machine-learned anomaly detection is reshaping searches for new physics, but it has outrun the statistics used to interpret it. A raw anomaly score has no calibrated meaning, a model that scans many regions inflates the look-elsewhere effect, and the asymptotic significances the field relies on are blind to the background mismodelling that anomaly detectors are especially prone to. We propose a calibration layer, built on conformal prediction, that turns any anomaly score into a defensible significance with distribution-free, finite-sample guarantees. Conformal prediction converts scores into valid local p-values, weighted and Mondrian variants repair the sideband-to-signal-region exchangeability failures that resonant searches suffer, and a Gross-Vitells step carries the result through to a look-elsewhere-aware global significance. The layer does two things at once. It exposes miscalibration that the standard pipeline cannot see, and it corrects it without retraining the detector. On public LHC Olympics data, a classifier develops a substructure-mass correlation that makes sideband-calibrated background p-values anti-conservative. Taken at face value, this manufactures a

\sim 46σ

excess from background sculpting alone, which the label-free weighted correction removes, restoring an honest null. When run as a blind wide-mass bump hunt, the standard asymptotic and unweighted procedures fabricate

\gtrsim10σ

excesses and

\approx5σ

excesses even in signal-free windows, while the conformal layer raises no false alarms and its global false-positive rate is verified on background-only pseudoexperiments. The result is an auditable, detector-agnostic path from an uncalibrated score to a trials-factor-aware significance, ready to be folded into experimental anomaly searches.

Jack Y. Araz, Michael Spannowsky

Department of Physics and Astronomy, University College London, London, WC1E 6B, UK · Department of Engineering, City St. George’s, University of London, London, EC1V 0HB, UK · Institute for Theoretical Physics, Campus Süd, Karlsruhe Institute of Technology (KIT), D-76128 Karlsruhe, Germany +1

Post-Hoc Calibration Conformal Prediction Anomaly Detection

Ask PDF

Apr 8, 2026

Quantum-Inspired Tensor Network Autoencoders for Anomaly Detection: A MERA-Based Approach

We investigate whether a multiscale tensor-network architecture can provide a useful inductive bias for reconstruction-based anomaly detection in collider jets. Jets are produced by a branching cascade, so their internal structure is naturally organised across angular and momentum scales. This motivates an autoencoder that compresses information hierarchically and can reorganise short-range correlations before coarse-graining. Guided by this picture, we formulate a MERA-inspired autoencoder acting directly on ordered jet constituents. To the best of our knowledge, a MERA-inspired autoencoder has not previously been proposed, and this architecture has not been explored in collider anomaly detection. We compare this architecture to a dense autoencoder, the corresponding tree-tensor-network limit, and standard classical baselines within a common background-only reconstruction framework. The paper is organised around two main questions: whether locality-aware hierarchical compression is genuinely supported by the data, and whether the disentangling layers of MERA contribute beyond a simpler tree hierarchy. To address these questions, we combine benchmark comparisons with a training-free local-compressibility diagnostic and a direct identity-disentangler ablation. The resulting picture is that the locality-preserving multiscale structure is well matched to jet data, and that the MERA disentanglers become beneficial precisely when the compression bottleneck is strongest. Overall, the study supports locality-aware hierarchical compression as a useful inductive bias for jet anomaly detection.

Emre Gurkanli, Michael Spannowsky

Department of Physics, Sinop University, Turkiye · Institute for Theoretical Physics, Campus Süd, Karlsruhe Institute of Technology (KIT), D-76128 Karlsruhe, Germany · Institute for Quantum Materials and Technologies, Karlsruhe Institute of Technology, Karlsruhe 76131, Germany

Anomaly Detection Autoencoders Reconstruction-Based Anomaly Detection Hierarchical Representation Learning High-Energy Physics

Ask PDF