Shapes are not enough: CONSERVAttack and its use for finding vulnerabilities and uncertainties in machine learning applications

Philip Bechtle; Lucie Flek; Philipp Alexander Jung; Akbar Karimi; Timo Saala; Alexander Schmidt; Matthias Schott; Philipp Soldin

cs.LGMar 14, 2026

Shapes are not enough: CONSERVAttack and its use for finding vulnerabilities and uncertainties in machine learning applications

Authors: Philip Bechtle, Lucie Flek, Philipp Alexander Jung, Akbar Karimi, Timo Saala, Alexander Schmidt, Matthias Schott, Philipp Soldin, +2 more

Organizations: Institute of Physics, University of Bonn, Germany. · Bonn-Aachen Institute of Technology, University of Bonn, Germany. · Physics Institute III A, RWTH Aachen University, Germany. · Physics Institute III B, RWTH Aachen University, Germany.

Abstract

In High Energy Physics, as in many other fields of science, the application of machine learning techniques has been crucial in advancing our understanding of fundamental phenomena. Increasingly, deep learning models are applied to analyze both simulated and experimental data. In most experiments, a rigorous regime of testing for physically motivated systematic uncertainties is in place. The numerical evaluation of these tests for differences between the data on the one side and simulations on the other side quantifies the effect of potential sources of mismodelling on the machine learning output. In addition, thorough comparisons of marginal distributions and (linear) feature correlations between data and simulation in "control regions" are applied. However, the guidance by physical motivation, and the need to constrain comparisons to specific regions, does not guarantee that all possible sources of deviations have been accounted for. We therefore propose a new adversarial attack - the CONSERVAttack - designed to exploit the remaining space of hypothetical deviations between simulation and data after the above mentioned tests. The resulting adversarial perturbations are consistent within the uncertainty bounds - evading standard validation checks - while successfully fooling the underlying model. We further propose strategies to mitigate such vulnerabilities and argue that robustness to adversarial effects must be considered when interpreting results from deep learning in particle physics.

Figures & tables

Figure 1: Comparison of three distinct distributions between clean and adversarial events in an attack setting. The plot on the left shows the feature resulting in the minimal change in Jensen Shannon distance, the one in the middle the feature corresponding to the median Jensen Shannon Distance, and the plot on the right the feature exhibiting the largest change in the Jensen Shannon Distance for the given attack run.

Figure 2: Comparison of the correlation matrices between the clean events (left) and adversarial events (right) resulting from the attack.

Task	$p_{min}$	$\epsilon_{step}$	$n_{it}$	$\alpha$	$\beta$	$\max_{JSD_{t}}$	$\max_{\Delta_{F}t}$	$nc$
Higgs	0.0005	0.0005	10	4.0	1.0	0.006	0.0002	True
TTvsWW	0.005	0.01	10	6.5	1.0	0.003	0.003	True

Table 1: Attack parameters for both the Higgs and the TT vs. WW jet-tagging tasks.

Figure 3: Fooling ratio of the adversarial attack on the Higgs dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 4: Average Jensen Shannon distance of the adversarial attack on the Higgs dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 5: Average Frobenius Norm of the adversarial attack on the Higgs dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 6: Flowchart showing the workflow used in the data augmentation pipeline.

Figure 7: Comparison of the areas under the Receiver-Operating-Curves—on the base model (grey) and the adversarially augmented model (red)—on the Higgs data set over 10 independent runs. The respective average results are shown in dashed lines.

Figure 8: Flowchart showing the workflow used in the adversarial training pipeline.

Figure 9: Comparison of the fooling ratios of adversarial attacks on the baseline model (grey) vs the average-case adversarially trained network (red) and the best-case adversarially trained network (blue) on the Higgs dataset.

Figure 10: Flowchart showing the workflow used in the adversarial detector pipeline.

Figure 11: Comparison of the initial fooling ratios (red) and the corrected fooling ratios (grey) found in 10 independent runs of the adversarial detector pipeline on the Higgs dataset.

Figure 12: Comparing the expected amounts of events misclassified across multiple runs (grey) to the observed amount (red). The red stars depict statistically significant differences between observed and expected. This plot was generated using the TT vs. WW dataset.

Figure 13: Detector accuracies on both clean non-simulated clean events (blue) and simulated clean events(red) using an adversarial detector network trained solely on clean and adversarial simulated events, dependent on the classification threshold for the TT vs. WW dataset.

Figure 14: Point clouds of sub-samples points from both the clean signal (blue) and clean background (red) events for the Donut toy dataset.

Figure 15: Comparison of the feature distributions between clean signal vs. clean background events (top row) and clean signal vs. adversarial background events (bottom row).

Figure 16: Point clouds of sub-samples points from both the clean signal (blue) and adversarial background (yellow) events for the Donut toy dataset.

Figure 17: Point clouds of sub-samples points from both the clean signal (blue) and adversarial background events that successfully fool the baseline network (yellow) for the Donut toy dataset.

Figure 18: Point clouds of sub-samples points from both the clean signal correctly classified by the adversarial detector network as such (blue), and adversarial background events that successfully fool the baseline network but are not correctly identified as such by the detector network (yellow) for the Donut toy dataset.

Figure 19: Comparison of the initial fooling ratios (red) and the corrected fooling ratios (grey) found in 5 independent runs of the Distance correlation restricted adversarial detector pipeline on the Higgs dataset.

Figure 20: Comparison of the initial fooling ratios (red) and the corrected fooling ratios (grey) found in 5 independent runs of the limited data Pearson correlation restricted adversarial detector pipeline on the Higgs dataset.

Figure 21: Area under the Receiver-Operating-Curve for varying sample sizes of training events used for the Pearson correlation restricted adversarial detector network.

Figure 22: Comparison of the averageg Areas under the Receiver-Operating-Curve for the distance correlation restricted attack (red) and the limited data Pearson restricted attack (grey) across 5 adversarial detector runs.

Figure 23: Flowchart showing the proposed workflow.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 24: Fooling ratios of the restricted attack on the Higgs dataset over 10 independent runs. The grey points indicate the total fooling ratio across all events, the blue points correspond to the signal (unrestricted) region, and the red points to the control (restricted) region.

Figure 25: Average Jensen–Shannon distance of the adversarial attack evaluated on the control (restricted) region of the Higgs dataset over 10 runs.

Figure 26: Average Frobenius norm of the adversarial attack evaluated on the control (restricted) region of the Higgs dataset over 10 runs.

Higgs - Signal-Background Classifier
Layer	Nodes	Activation Function
Input Layer	30	-
Dense	300	ReLU
BatchNormalization	-	-
Dense	102	ReLU
BatchNormalization	-	-

Appendix

Table 2: Architecture of the baseline signal-background classifier for the Higgs dataset.

Higgs - Adversarial Detector Model
Layer	Nodes	Activation Function
Input Layer	30	-
Dense	300	ReLU
BatchNormalization	-	-
Dense	102	ReLU
BatchNormalization	-	-

Appendix

Table 3: Architecture of the adversarial detector MLP for the Higgs dataset.

TT vs. WW - Jet Tagger
Layer	Nodes	Activation Function
Input Layer	87	-
Dense	300	ReLU
BatchNormalization	-	-
Dense	102	ReLU
BatchNormalization	-	-

Appendix

Table 4: Architecture of the baseline signal-background classifier for the TT vs. WW Jets dataset.

TT vs. WW - Adversarial Detector Model
Layer	Nodes	Activation Function
Input Layer	87	-
Dense	300	ReLU
BatchNormalization	-	-
Dense	102	ReLU
BatchNormalization	-	-

Appendix

Table 5: Architecture of the adversarial detector MLP for the TT vs. WW Jets dataset.

Figure 27: Average fooling ratio of the adversarial attack on the TT vs. WW dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 28: Average Jensen Shannon distance of the adversarial attack on the TT vs. WW dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 29: Average Frobenius Norm of the adversarial attack on the TT vs. WW dataset over 10 runs. The average performance of the attack is shown as a dashed line, and its standard deviation as a dotted line.

Figure 30: Comparison of the areas under the Receiver-Operating-Curves—on the base model (grey) and the adversarially augmented model (red)—on the Higgs data set over 10 independent runs. The respective average results are shown in dashed lines.

Figure 31: Comparison of the fooling ratios of adversarial attacks on the baseline model (grey) vs the average-case adversarially trained network (red) and the best-case adversarially trained network (blue) on the TT vs. WW dataset across 5 runs.

Figure 32: Comparison of the initial fooling ratios (red) and the corrected fooling ratios (grey) found in 10 independent runs of the adversarial detector pipeline on the TT vs. WW dataset.

Adversarial Attack Hyperparameters
Parameter Name	Type	Description
x	Numpy Array	Input data to be attacked.
y	Numpy Array	True labels for the input data.
model	str	Path to the trained model.
min_change	float	Minimum allowed change per feature in each step.
step	float	Step size for candidate feature changes.

Appendix

Table 6: User-configurable hyperparameters for the adversarial attack.

Higgs Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.0005
step	0.0005
n_iterations	10
num_bins	200
alpha	4.0

Appendix

Table 7: Parameter values used for adversarial attack for the Higgs dataset.

Higgs Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.002
step	0.006
n_iterations	10
num_bins	200
alpha	6.5

Appendix

Table 8: Parameter values used for the augmentation pipeline for the Higgs dataset.

Higgs Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.003
step	0.02
n_iterations	10
num_bins	100
alpha	6.0

Appendix

Table 9: Parameter values used for the adversarial detector pipeline for the Higgs dataset for training and test samples.

Higgs Attack Hyperparameter Ranges
Parameter Name	Value / Range
min_change	[0.001, 0.005] (uniform)
step	[0.005, 0.03] (uniform)
n_iterations	10
num_bins	100
alpha	[3.0, 10.0] (uniform)

Appendix

Table 10: Parameter ranges used for generating training adversaries in the iterative retraining pipeline (randomized per retraining iteration).

Higgs Attack Hyperparameter Choice (Test)
Parameter Name	Value
min_change	0.003
step	0.01
n_iterations	10
num_bins	100
alpha	8.0

Appendix

Table 11: Parameter values used for generating test adversaries in the iterative retraining pipeline.

DisCo Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.015
step	0.06
n_iterations	10
num_bins	30
alpha	4.0

Appendix

Table 12: Parameter values used for the adversarial attack in the DisCo attack experiment on the Higgs Dataset.

DisCo Detector Attack Hyperparams. (Train)
Parameter Name	Value
min_change	0.07
step	0.08
n_iterations	10
num_bins	30
alpha	4.0

Appendix

Table 13: Parameter values used for generating training adversaries in the DisCo detector experiment.

DisCo Detector Attack Hyperparams. (Test)
Parameter Name	Value
min_change	0.01
step	0.05
n_iterations	10
num_bins	30
alpha	4.0

Appendix

Table 14: Parameter values used for generating test adversaries in the DisCo detector experiment.

Limited Data Attack Hyperparameters
Parameter Name	Value
min_change	0.005
step	0.02
n_iterations	10
num_bins	30
alpha	6.0

Appendix

Table 15: Parameter values used for generating training and test adversaries in the for the limited data Pearson correlation experiment.

TTvsWW Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.005
step	0.01
n_iterations	10
num_bins	200
alpha	6.5

Appendix

Table 16: Parameter values used for adversarial attack for the TT vs. WW dataset.

TTvsWW Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.003
step	0.02
n_iterations	10
num_bins	100
alpha	6.5

Appendix

Table 17: Parameter values used for the augmentation pipeline for the TT vs. WW dataset.

TTvsWW Attack Hyperparameter Choice
Parameter Name	Value
min_change	0.01
step	0.01
n_iterations	10
num_bins	100
alpha	6.0

Appendix

Table 18: Parameter values used for the adversarial detector pipeline for the TT vs. WW dataset for training and test samples.

TTvsWW Attack Hyperparam. Ranges (Train)
Parameter Name	Value / Range
min_change	[0.001, 0.005] (uniform)
step	[0.005, 0.03] (uniform)
n_iterations	10
num_bins	100
alpha	[3.0, 10.0] (uniform)

Appendix

Table 19: Parameter ranges used for generating train adversaries in the TT vs. WW iterative retraining pipeline (randomized per retraining iteration).

TTvsWW Attack Hyperparameters (Test)
Parameter Name	Value
min_change	0.002
step	0.005
n_iterations	10
num_bins	100
alpha	8.0

Appendix

Table 20: Parameter values used for generating test adversaries in the TT vs. WW iterative retraining pipeline.

DonutDummy Attack Hyperparameters (Test)
Parameter Name	Value
min_change	0.001
num_candidates	150
n_iterations	10
num_bins	70
alpha	6.0

Appendix

Table 21: Parameter values used for generating test adversaries in the DonutDummy experiment.

DonutDummy Detector Attack Hyperparams.
Parameter Name	Value
min_change	0.001
num_candidates	150
n_iterations	10
num_bins	60
alpha	6.0

Appendix

Table 22: Parameter values used for generating training and test adversaries in the DonutDummy adversarial detector pipeline experiment.

Explore similar work

Unknown Unknowns: Model Misspecification in Machine Learning for Physics
Aug 13, 2026Juan Cruz-Martinez, Carolina Cuesta-Lazaro, Alexander Held +1Model Misspecification Scientific ML
Departamento de Física Atómica, Molecular y Nuclear, Universidad de Sevilla, E-41080 Sevilla, Spain · Center for Computational Astrophysics, Flatiron Institute, 162 Fifth Avenue, New York, NY 10010, USA · Institute for Advanced Study, 1 Einstein Drive, Princeton, NJ 08540, USA +2
How unconstrained machine-learning models learn physical symmetries
Mar 25, 2026Michelangelo Domina, Joseph William Abbott, Paolo Pegolo +2Equivariant Representation Learning Physics-Based Simulation
Laboratory of Computational Science and Modeling, Institut des Matériaux, École Polytechnique Fédérale de Lausanne, 1015 Lausanne, Switzerland
SoK: Colluding Adversaries in Machine Learning Pipelines
Jun 8, 2026Vasisht Duddu, Lipeng He, Asim Waheed +1Adversarial Attacks
University of Waterloo · KTH Royal Institute of Technology

Aug 13, 2026J/K move · Enter open · S save

Unknown Unknowns: Model Misspecification in Machine Learning for Physics

Juan Cruz-Martinez, Carolina Cuesta-Lazaro, Alexander Held, Michael Kagan

Departamento de Física Atómica, Molecular y Nuclear, Universidad de Sevilla, E-41080 Sevilla, Spain · Center for Computational Astrophysics, Flatiron Institute, 162 Fifth Avenue, New York, NY 10010, USA · Institute for Advanced Study, 1 Einstein Drive, Princeton, NJ 08540, USA+2

Model Misspecification Scientific ML

Open paper AskPDF arXiv

Machine learning is now a central tool for solving inverse problems in particle physics and astronomy. Models are trained on simulation and deployed on real data, raising the question not just of whether they fit, but of whether they are wrong in ways we did not anticipate: the unknown unknowns. This challenge of model misspecification is not unique to machine learning. In physics, misspecification is sometimes exactly what we want to find: new discoveries appear as failures of existing models. At other times, we want such effects absorbed into the analysis without biasing the measurement. A robust analysis is one that absorbs the misspecifications we are not interested in, while preserving sensitivity to the ones we are. Machine learning can both amplify misspecification and provide new tools to address it. We discuss the challenges of model misspecification, diagnostics for detecting it, and strategies for mitigation. No single diagnostic can confirm that a model is correctly specified: detection and mitigation are two halves of an iterative loop, in which a battery of complementary diagnostics is applied, the model is updated, and the process repeated. Robustness against unknown unknowns is ultimately less about any single technique than about a disposition: a willingness to suspect one's own model, and to design analyses that can survive being wrong in ways one did not anticipate.

Abstract

Explore similar work

Unknown Unknowns: Model Misspecification in Machine Learning for Physics

How unconstrained machine-learning models learn physical symmetries

SoK: Colluding Adversaries in Machine Learning Pipelines