Neural Scaling Laws for Boosted Jet Tagging

Matthias Vigl; Nicole Hartman; Michael Kagan; Lukas Heinrich

hep-exFeb 17, 2026

Neural Scaling Laws for Boosted Jet Tagging

Authors: Matthias Vigl, Nicole Hartman, Michael Kagan, Lukas Heinrich

Organizations: Technical University of Munich · SLAC National Accelerator Laboratory

Abstract

The success of Large Language Models (LLMs) has established that scaling compute, through joint increases in model capacity and dataset size, is the primary driver of performance in modern machine learning. While machine learning has long been an integral component of High Energy Physics (HEP) data analysis workflows, the compute used to train state-of-the-art HEP models remains orders of magnitude below that of industry foundation models. With scaling laws only beginning to be studied in the field, we investigate neural scaling laws for boosted jet classification using the public JetClass dataset. We derive compute optimal scaling laws and identify an effective performance limit that can be consistently approached through increased compute. We study how data repetition, common in HEP where simulation is expensive, modifies the scaling yielding a quantifiable effective dataset size gain. We then study how the scaling coefficients and asymptotic performance limits vary with the choice of input features and particle multiplicity, demonstrating that increased compute reliably drives performance toward an asymptotic limit, and that more expressive, lower-level features can raise the performance limit and improve results at fixed dataset size.

Figures & tables

Parameter	Value (68% CI)
$L_{\infty}$	$0.32\,(0.29,\,0.34)$
$A$	$11.27\,(8.01,\,15.14)$
$\alpha$	$0.44\,(0.40,\,0.48)$
$B$	$7.22\,(6.66,\,7.89)$
$\beta$	$0.22\,(0.21,\,0.23)$
$\gamma$	$0.15\,(0.13,\,0.16)$

Table 1: Scaling-law parameters from fits to

L_{\infty}+A/N^{\alpha}+B/D^{\beta}

. Under compute-optimal scaling, Loss follows

L\propto C^{\gamma}

. Reported uncertainties are 68% bootstrap confidence intervals. In order to reduce uncertainties in

L_{\infty}

, its value is constrained to that obtained in Section 3.2 , which probes significantly higher compute regions. For reference, Hoffmann et al. (2022) found

\alpha\approx 0.38

and

\beta\approx 0.28

for LLMs.

Panel of Figure 1 — Figure 1: Each point represents the validation loss of a Transformer encoder with $N$ parameters (varying the embedding dimension) trained on $D$ unique samples for exactly one epoch (no data repetition). Models are trained across a grid of ( $N$ , $D$ ) configurations spanning several orders of magnitude in both axes. The parametric form in Eq. ( 1 ) is then fit jointly to all points. (a) Loss surface $L(N,D)$ as a function of model size $N$ and training dataset size $D$ , with iso-loss contours shown as colored lines and the compute-optimal trajectory as the blue dashed line. The iso-FLOP lines are colored by the corresponding compute budget, with the color scale shown in (b). (b) Loss as a function of model size at fixed compute budget (iso-FLOP curves), with the compute-optimal trajectory crossing the minima of each curve.

Panel of Figure 2 — Figure 2: Scaling under data repetition. (a) Validation loss as a function of total training compute for models trained on fixed dataset sizes ranging from D = 1k to D = 1M, compared to the compute-optimal scaling $L\propto C^{-0.15}$ (dashed line). Each point represents the early-stopped validation loss of a single model, obtained by stopping training at the epoch with lowest validation loss. Models are trained across a two-dimensional grid: the dataset size $D$ is indicated by the line style, with the embedding dimension $d\in\{4,8,16,32,64,96,128,256,512\}$ also being varied and the resulting model size $N$ indicated by the color scale. Models trained with data repetition consistently lie above the compute-optimal frontier, with validation loss eventually either saturating or overfitting. (b) Training model sizes above the overfitting threshold allows to minimize the validation loss at each fixed dataset size. Scaling along this trajectory yields the same compute exponent, at the price of roughly a factor of 10 in compute to reach the same loss as compute-optimal scaling with no data repetition. (c) Early stopped validation loss as a function of dataset size $D$ for models trained above the overfitting threshold with data repetition (black points), compared to scaling with no data repetition under the large $N$ regime (negligible model size term) $L(N\to\infty,D)=L_{\infty}+B/D^{\beta}$ . Both curves are compute sub-optimal, as they correspond to the limit of large model size rather than the compute-optimal allocation. Although training on repeated data effectively amplifies the dataset, the gain is bounded by $\omega D_{\text{rep}}$ .

Figure 3: Overfitting threshold in the

(N,D)

plane. Blue dots indicate models in the underfitting regime, where increasing training time leads to a plateau in the validation loss. Red crosses indicate models that overfit, where the validation loss eventually increases with further training. The green line shows the fitted threshold

N\propto D^{0.47}

, corresponding to a roughly square-root scaling between the minimum model size needed to overfit and the dataset size. 68% confidence intervals are obtained by bootstrapping threshold points.

Figure 4: Loss as a function of training dataset size

D

for models trained above the overfitting threshold, using four configurations: kinematic variables only (

\Delta\eta,\Delta\phi,log\,p_{T}

), and the full feature set with 10, 40, and 128 particles per jet. Dashed lines show fits to

L(D_{\mathrm{rep}})=L_{\infty}+B_{\mathrm{rep}}/D_{\mathrm{rep}}^{\beta_{\mathrm{rep}}}

. The scaling exponent

\beta_{\mathrm{rep}}

is roughly constant across configurations, while the asymptotic loss

L_{\infty}

decreases with richer inputs and higher particle multiplicity, which also reach the same performance with significantly less amounts of data.

Configuration	$L_{\infty}$ (68% CI)	$B_{\mathrm{rep}}$ (68% CI)	$\beta_{\mathrm{rep}}$ (68% CI)
( $\Delta\eta$ , $\Delta\phi$ , $\log p_{T}$ ), 40 particles	$0.74\,(0.67,\,0.77)$	$5.62\,(4.16,\,7.10)$	$0.21\,(0.17,\,0.24)$
21 features, 10 particles	$0.59\,(0.57,\,0.61)$	$8.69\,(7.59,\,10.35)$	$0.26\,(0.24,\,0.28)$
21 features, 40 particles	$0.33\,(0.30,\,0.34)$	$7.08\,(6.27,\,7.79)$	$0.23\,(0.21,\,0.24)$
21 features, 128 particles	$0.32\,(0.29,\,0.34)$	$5.79\,(5.06,\,6.27)$	$0.22\,(0.20,\,0.23)$

Table 2: Scaling parameters from fits to

L(D_{\mathrm{rep}})=L_{\infty}+B_{\mathrm{rep}}/D_{\mathrm{rep}}^{\beta_{\mathrm{rep}}}

for four input feature configurations. Reported uncertainties are 68% bootstrap confidence intervals.

Figure 5: QCD background rejection for each signal class at 50% efficiency, except for

H\to l\nu qq^{\prime}

(99%) and

t\to bl\nu

(99.5%), as a function of total loss and dataset size when training for multiple epochs above the overfitting threshold. The performance of the ParT architecture trained on 100M samples is shown for reference (dashed lines), which is crossed as expected at the 100M jets scale level. Each colored curve corresponds to one of the nine signal processes in the JetClass dataset.

Panel of Figure 6 — Figure 6: Top vs QCD jet tagging. (a) ROC curves for top quark tagging across the four input feature configurations, for models trained on $D=10^{8}$ samples (solid lines) and the asymptotic performance in the limit $D\to\infty$ (dashed lines) obtained from the fitted $L_{\infty}$ . (b) QCD background rejection at 50% top signal efficiency as a function of total loss and dataset size for the 128-particle, 10-particle, and kinematic-only ( $\Delta\eta$ , $\Delta\phi$ , $\log p_{T}$ ) configurations. While all configurations follow a smooth scaling with dataset size, individual signal processes exhibit different scaling exponents depending on which physics information is retained by the input representation.

Explore similar work

Oct 5, 2026

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale

Much of the recent progress in machine learning domains such as language models has come from scaling laws that predict performance as a function of training effort. In high-energy physics (HEP) similar behavior has now been observed. To aid further study, we present a systematic procedure to derive robust scaling laws and compare design choices on the relevant budget axes for HEP tasks. We first validate the full scaling trajectory on toy problems and then apply the procedure to multi-task transformers on the ~11 billion-jet ATLAS JetSet2 dataset, in both the compute- and data-constrained regimes. For the latter, we predict, to the best of our knowledge for the first time, the jointly optimal model size, training horizon, learning rate and batch size under early stopping. At compute-optimal scaling, we recover a near-equal

\sqrt{C}

dependence of model and dataset size, and find that auxiliary objectives lower the primary jet-classification loss at equal compute budget. Expanding the inputs toward lower-level data systematically lowers the loss while leaving the scaling exponent nearly unchanged. The onset of the power-law regime is itself set by scale: below a threshold in dataset size the loss carries little information about high-compute scaling, underscoring the value of large, high-quality full-simulation datasets as a foundation for scaling studies and the development of foundation models in HEP.

Matthias Vigl, Nikita Pond, Jackson Barr +5

Technical University of Munich · SLAC National Accelerator Laboratory · University College London +4

High-Energy Physics Scaling Laws

Ask PDF

Jun 18, 2026

Towards Engineering Scaling Laws with Pretraining Data Composition

Neural scaling laws describe how model performance improves as a power law in compute, model size, and dataset size. While well-established for large language models, these relationships are emerging for large models in particle physics. As with language, empirical studies show that the performance scales as a power law. However, unlike natural language or image domains, fundamental physics has high-fidelity simulators that produce synthetic data cheaply. This favors scaling regimes where additional data is cheaper than additional parameters, and allows the pretraining dataset itself to be engineered to influence the scaling. For the task of classifying hadronic jets produced in collisions of high-energy particle beams, we show that the scaling behavior can be engineered towards requiring more data rather than larger models by inclusion of pretraining data which is more diverse and better aligned with the downstream classification task.

Jan-Lucas Uslu, Kevin Greif, Daniel Whiteson +1

Department of Applied Physics Stanford University Stanford, CA 94305, USA · Department of Physics and Astronomy University of California, Irvine Irvine, CA 92697, USA · Department of Particle Physics and Astrophysics SLAC National Accelerator Laboratory Stanford University Menlo Park, CA 94025, USA

Synthetic Data Pretraining High-Energy Physics Neural Scaling Laws

Ask PDF

May 27, 2026

Neural Scaling Laws for Jet Generation

Recently observed empirical scaling laws describe the performance of foundation-type models as three independent key quantities -- dataset size, compute, and model parameters -- are modified. Extracting these scaling laws informs the training of large complex models for which the tuning of hyperparameters in traditional ways is not feasible. This work for the first time explores if scaling laws can also be observed for the task of particle jet generation -- both relevant as a pre-training objective for foundation models and as in-situ simulation by itself. We indeed replicate the key logarithmic scaling law behavior for model-size scaling. Beyond studying the next token prediction validation loss of the generative model, we also study the sliced Wasserstein distance of five physical quantities that are not immediately available to the model during training. Our study shows that this quantity is monotonically related to the next token prediction validation loss, meaning that this loss is indeed a good proxy for the physics performance. For the scaling with dataset size and compute, we observe substantially weaker scaling behavior of both the loss and the sliced Wasserstein distance. We analyze this behavior by introducing the concept of a learnable window, and argue that autoregressive next token prediction on jet constituents exhibits comparatively rapid saturation relative to language-model studies. We discuss possible origins of this behavior, including the stochastic nature of QCD radiation and differences between generative and supervised learning tasks in collider physics.

Oz Amram, Darius A. Faroughy, Tjarko Gerdes +5

Fermi National Accelerator Laboratory, Batavia, IL 60510, USA · NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, USA · Institute for Experimental Physics, Universität Hamburg, Luruper Chaussee 149, 22761 Hamburg, Germany +1

High-Energy Physics Autoregressive Generation Neural Scaling Laws

Ask PDF