stat.APOct 29, 2026

Model-Agnostic Search for New Physics in the Presence of Imperfect Background by Smearing the Data

Authors: Soheun Yi, John Alison, Mikael Kuusela

Organizations: Department of Statistics and Data Science, Carnegie Mellon University · Department of Physics, Carnegie Mellon University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In the search for new particles in high-energy physics, it is necessary to first define the signal region (SR) enriched in signal events, and then compare events in this region with the expected background distribution. While most existing search methods rely on prior domain knowledge of either the signal or the background, there has been less effort to relax both assumptions simultaneously, as methods that weaken one assumption typically rely on strong knowledge of the other. We propose Model-Agnostic Search with Smearing and Imperfect Background (MASSIB), a method that addresses this challenge by learning the density ratio of events relative to a misspecified background distribution using modern tools from machine learning, and defining the SR as the region with high density ratio. Recognizing that a naive application of this idea introduces significant bias in background estimation when interpolating from the control region to the SR, MASSIB mitigates this issue by introducing a "smeared" density ratio, which can be learned efficiently through data-level manipulation and used to define an SR that allows for accurate background estimation. By applying it to various types of simulated Large Hadron Collider (LHC) events with realistic background mismodeling, we demonstrate that MASSIB identifies a data-driven SR that achieves high signal concentration consistently across different types of signals while avoiding bias in background estimation. The method does this without making strong assumptions about either the signal or background distribution. We then show that this robustness extends to the validity and strong statistical power of the resulting hypothesis tests for signal detection, demonstrating the potential of this approach for future model-agnostic searches at the LHC.

Figures & tables

Fig 1: Classification of methods by how strong their assumptions on the signal and background are. Traditional methods, including Higgs boson detection ( ATLAS, 2012 ; CMS, 2012 ) , rely on strong assumptions about both the signal and background. Some recent methods relax the assumptions on either the signal ( Chakravarti et al., 2023 ) or the background ( Manole et al., 2024 ) , but still impose a strong assumption on the other component. In contrast, this work introduces a new approach built upon weaker assumptions on both the signal and background.
Fig 2: A contingency table explaining the ABCD method. The four regions AA , BB , CC , and DD are defined based on whether an event belongs to the imperfect background or the target events and whether it is in the SR or CR. Given that the imperfect background does not include any signal events and that the two axes are uncorrelated, we can estimate the number of background events in region DD by ND≈NBNC/NAN_{D}\approx N_{B}N_{C}/N_{A} .
Fig 3: An overview of MASSIB. The procedure begins by estimating the density ratio γ\gamma between the data and imperfect background (IB) in the first split of the dataset. Then, the representation ζ\zeta is smeared and used to learn the density ratio γ~\widetilde{\gamma} between the smeared data and smeared IB in the same split. The SR and CR are then defined by thresholding on ψ=γ/γ~\psi=\gamma/\widetilde{\gamma} . Next, the density ratio of the data and IB in the CR, denoted as γCR\gamma_{\text{CR}} , is estimated. Finally, the estimated background γCRpIB\gamma_{\text{CR}}p_{\text{IB}} is compared with the data distribution pdatap_{\text{data}} within the SR via hypothesis tests.
Fig 4: Abstraction of the Four-vs-Three (FvT) classifier architecture ( Manole et al., 2024 ) . xx denotes the original input features, ζ=ζ(x)\zeta=\zeta(x) is the representation, and γ^\widehat{\gamma} is the output of the classifier predicting the density ratio of 4b4b events to 3b3b events.
Fig 5: Histograms of classifier outputs trained on a dataset with signal ratio ϵ=0.01\epsilon=0.01 . Even when trained on the same training dataset, due to the stochasticity of the training, a classifier may or may not capture signal events by identifying a region where the HH→4b\mathrm{HH}\to 4b signal events dominate over the background 4b4b events, as illustrated in the left and right panels.
Fig 6: Proportion of signal events ( S(Xs)S({\mathcal{X}}_{s}) ) versus that of 4b4b events ( P4b(Xs)P_{4b}({\mathcal{X}}_{s}) ) in the SR ( Xs{\mathcal{X}}_{s} ) with varying thresholds for ensembles of ψ^\widehat{\psi} . The signal ratio is set to ϵ=0.01\epsilon=0.01 and the noise scales are set to η=∞\eta=\infty (i.e., ψ^=γ^\widehat{\psi}=\widehat{\gamma} is an ensemble maximum of density ratios) and η=2.0\eta=2.0 . The maximum and mean of ψ^\widehat{\psi} for 1515 different training instances are used to define the SR. The plot illustrates that the ensemble maximum is more effective than the ensemble mean in enriching the signal events in the SR for both smearing and non-smearing cases.
Fig 7: In the signal-free case ( ϵ=0\epsilon=0 ), defining the SR and CR by thresholding γ^\widehat{\gamma} reduces their overlap in terms of γ\gamma (first column, top two panels). Consequently, a classifier trained in the CR cannot reliably learn the high values of γ\gamma occurring in the SR: γ^CR\widehat{\gamma}_{\mathrm{CR}} is calibrated in the CR but poorly calibrated in the SR (first column, bottom panel). Defining the regions by thresholding ψ^\widehat{\psi} , rather than γ^\widehat{\gamma} , increases the overlap and improves calibration in the SR (second column). At the smaller noise scale η=0.1\eta=0.1 , ψ^\widehat{\psi} is close to 11 across most of the range of γ^\widehat{\gamma} , producing still greater overlap between the SR and CR (third column). In the calibration panels, the red curve plots the event-weighted mean of γ^CR\widehat{\gamma}_{\mathrm{CR}} against the event-weighted mean of γ^\widehat{\gamma} within each of 5050 equal-count (quantile) bins of γ^\widehat{\gamma} , and the dashed black line is the identity.
Fig 8: Quantiles of ψ^\widehat{\psi} versus the normalized difference between the true and estimated 4b4b event counts in the SR in the signal-free case ( ϵ=0\epsilon=0 ). Events in the SR are binned into 6464 quantiles of ψ^\widehat{\psi} , and the mean ±\pm standard deviation of the normalized difference between the sum of weights of background 4b4b events and its estimate are shown across 5050 replications. When ψ^=γ^\widehat{\psi}=\widehat{\gamma} (left), bias grows with ψ^\widehat{\psi} , inducing poor calibration of γ^CR\widehat{\gamma}_{\text{CR}} in the SR. When smearing with noise scale η=2.0\eta=2.0 (right), the bias is reduced and the calibration of γ^CR\widehat{\gamma}_{\text{CR}} in the SR is improved. Here, η\eta denotes the noise scale used for smearing; see Algorithm 2 .
Fig 9: Histogram and ratio of estimated background (i.e., reweighted 3b3b ) and true 4b4b background. The choice of space on which to train the density ratio in the CR impacts the estimated background distribution (i.e., counts of reweighted 3b3b events in each bin) for 4b4b events. Learning the density ratio on the original space (left) provides a reasonable estimate of the background 4b4b distribution. Training the density ratio on the representation space (right), on the other hand, introduces a strong bias toward higher values in the presence of signal events.
Fig 10: t-SNE plots of original features (left) and representations (right) in the CR with colors based on types of events. Signal events in the representation space are more concentrated in a specific area compared to the original space, leading to a larger bias in the background estimator when it is learned based on the representations.
Fig 11: A toy example (left panel) shows why the choice of noise scale η\eta should not be too small. When η\eta is small, the smeared density ratio γ~\widetilde{\gamma} still preserves the high-frequency signal feature (third row, left panel), so thresholding ψ=γ/γ~\psi=\gamma/\widetilde{\gamma} fails to identify a region rich in signal events. Indeed, choosing a smaller η\eta causes signal events to be less concentrated in the tail of ψ\psi (first row, right panel). Consequently, signal events are dominated by background events over the entire range of ψ\psi (second row, right panel), resulting in a loss of power for the hypothesis test.
Fig 12: The SR efficiency of the non-resonant HH→4b\mathrm{HH}\to 4b signal with signal ratio ϵ=0.01\epsilon=0.01 (left) and ϵ=0.02\epsilon=0.02 (right) for different noise scales η\eta . Dashed lines represent the average measured across 100100 repeated experiments, and the shaded regions represent the corresponding standard deviations. The noise scales give comparable concentration overall. On the left side of the curves, where P4b(Xs)P_{4b}({\mathcal{X}}_{s}) is small and the SR consists of the highest- ψ\psi events, η=0.5\eta=0.5 shows the least concentration and overtakes the larger scales only as the SR widens.
Fig 13: Comparison with the baseline SR proposed by Bryant (2018) . Dashed lines represent the average measured across 100100 repeated experiments, and the shaded region represents the standard deviation of our method. While the baseline, which has access to a priori knowledge of the location of the signal, shows better performance when P4b(Xs)P_{4b}({\mathcal{X}}_{s}) is large, our method matches or exceeds it once P4b(Xs)P_{4b}({\mathcal{X}}_{s}) is small.
Fig 14: Percentage of experiments rejecting the null hypothesis of ( 2 ) with non-resonant HH→4b\mathrm{HH}\to 4b signal for the hypothesis test at significance level α=0.05\alpha=0.05 with noise scale η=2.0\eta=2.0 . Error bars are Clopper–Pearson 95%95\% intervals based on the 100100 repetitions. ϵ=0\epsilon=0 corresponds to the null case, for which the observed rejection rates are below the nominal level α=0.05\alpha=0.05 .
Fig 15: The SR efficiency of the resonant X→HH→4bX\to\mathrm{HH}\to 4b signal at 400 GeV400\,\mathrm{GeV} (left) and non-resonant ZH→4b\mathrm{ZH}\to 4b (right) with signal ratios ϵ=0.0075\epsilon=0.0075 and ϵ=0.03\epsilon=0.03 , respectively. Dashed lines represent the average measured across 100100 repeated experiments, and the shaded regions represent the corresponding standard deviations. The plot demonstrates the adaptivity of our method to different types of signals and its robustness to the choice of the noise scale.
Fig 16: Percentage of experiments rejecting the null hypothesis of ( 2 ) with resonant X→HH→4bX\to\mathrm{HH}\to 4b signal at 400 GeV400\,\mathrm{GeV} for the hypothesis test at significance level α=0.05\alpha=0.05 with noise scale η=2.0\eta=2.0 . Error bars are Clopper–Pearson 95%95\% intervals based on the 100100 repetitions. The power is significant when the signal ratio ϵ\epsilon is greater than or equal to 0.00750.0075 .
Fig 17: Percentage of experiments rejecting the null hypothesis of ( 2 ) with non-resonant ZH→4b\mathrm{ZH}\to 4b signal for the hypothesis test at significance level α=0.05\alpha=0.05 with noise scale η=2.0\eta=2.0 . Error bars are Clopper–Pearson 95%95\% intervals based on the 100100 repetitions. The power rises steeply between ϵ=0.02\epsilon=0.02 and ϵ=0.03\epsilon=0.03 , ranging from 0.870.87 to 1.001.00 at ϵ=0.03\epsilon=0.03 and reaching 1.001.00 at ϵ=0.05\epsilon=0.05 .
η\etasSRs_{\text{SR}}ϵ=0\epsilon=0ϵ=0.005\epsilon=0.005ϵ=0.0075\epsilon=0.0075ϵ=0.01\epsilon=0.01ϵ=0.02\epsilon=0.02
0.50.050.050.000.000.000.000.020.020.420.420.750.75
0.10.10.000.000.000.000.010.010.430.430.830.83
0.150.150.000.000.000.000.030.030.460.460.880.88
0.20.20.010.010.010.010.040.040.520.520.930.93
1.00.050.050.010.010.010.010.020.020.500.500.980.98
0.10.10.000.000.010.010.030.030.670.670.990.99
Table 1: Empirical rejection proportions over 100100 repeated experiments for the test of ( 2 ) with non-resonant HH→4b\mathrm{HH}\to 4b signal at significance level α=0.05\alpha=0.05 . η\eta is the noise scale used in learning the smoothed density ratio γ~\widetilde{\gamma} , and ϵ\epsilon is the signal ratio. ϵ=0\epsilon=0 corresponds to the null case, for which the observed rejection proportions are below the nominal level.
More in this topic

High-Energy Physics

High-energy physics studies the fundamental particles and forces of nature through collider, neutrino, cosmic-ray, and theory-driven experiments.

See all 164 papers

Explore similar work

CardsList

  1. Searching for BSM Experimental Signatures with Large Lagrangian Models

    Sep 29, 2026Ibrahim Elsharkawy, Victoria Knapp-Perez, Wahid Bhimji +1AI for ScienceHigh-Energy Physics

    Department of Physics, University of Toronto and Vector Institute, Toronto, ON, Canada · NERSC, Lawrence Berkeley National Laboratory, Berkeley, California, USA · Department of Physics and Astronomy, University of California, Irvine, CA 92697 +3

  2. Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches

    Jun 11, 2026Jack Y. Araz, Michael SpannowskyPost-Hoc CalibrationConformal Prediction

    Department of Physics and Astronomy, University College London, London, WC1E 6B, UK · Department of Engineering, City St. George’s, University of London, London, EC1V 0HB, UK · Institute for Theoretical Physics, Campus Süd, Karlsruhe Institute of Technology (KIT), D-76128 Karlsruhe, Germany +1

  3. Unknown Unknowns: Model Misspecification in Machine Learning for Physics

    Aug 13, 2026Juan Cruz-Martinez, Carolina Cuesta-Lazaro, Alexander Held +1Model MisspecificationScientific ML

    Departamento de Física Atómica, Molecular y Nuclear, Universidad de Sevilla, E-41080 Sevilla, Spain · Center for Computational Astrophysics, Flatiron Institute, 162 Fifth Avenue, New York, NY 10010, USA · Institute for Advanced Study, 1 Einstein Drive, Princeton, NJ 08540, USA +2