Neural Scaling Laws for Boosted Jet Tagging

Matthias Vigl; Nicole Hartman; Michael Kagan; Lukas Heinrich

hep-exFeb 17, 2026

Neural Scaling Laws for Boosted Jet Tagging

Authors: Matthias Vigl, Nicole Hartman, Michael Kagan, Lukas Heinrich

Organizations: Technical University of Munich · SLAC National Accelerator Laboratory

Abstract

The success of Large Language Models (LLMs) has established that scaling compute, through joint increases in model capacity and dataset size, is the primary driver of performance in modern machine learning. While machine learning has long been an integral component of High Energy Physics (HEP) data analysis workflows, the compute used to train state-of-the-art HEP models remains orders of magnitude below that of industry foundation models. With scaling laws only beginning to be studied in the field, we investigate neural scaling laws for boosted jet classification using the public JetClass dataset. We derive compute optimal scaling laws and identify an effective performance limit that can be consistently approached through increased compute. We study how data repetition, common in HEP where simulation is expensive, modifies the scaling yielding a quantifiable effective dataset size gain. We then study how the scaling coefficients and asymptotic performance limits vary with the choice of input features and particle multiplicity, demonstrating that increased compute reliably drives performance toward an asymptotic limit, and that more expressive, lower-level features can raise the performance limit and improve results at fixed dataset size.

Figures & tables

Parameter	Value (68% CI)
$L_{\infty}$	$0.32\,(0.29,\,0.34)$
$A$	$11.27\,(8.01,\,15.14)$
$\alpha$	$0.44\,(0.40,\,0.48)$
$B$	$7.22\,(6.66,\,7.89)$
$\beta$	$0.22\,(0.21,\,0.23)$
$\gamma$	$0.15\,(0.13,\,0.16)$

Table 1: Scaling-law parameters from fits to

L_{\infty}+A/N^{\alpha}+B/D^{\beta}

. Under compute-optimal scaling, Loss follows

L\propto C^{\gamma}

. Reported uncertainties are 68% bootstrap confidence intervals. In order to reduce uncertainties in

L_{\infty}

, its value is constrained to that obtained in Section 3.2 , which probes significantly higher compute regions. For reference, Hoffmann et al. (2022) found

\alpha\approx 0.38

and

\beta\approx 0.28

for LLMs.

Panel of Figure 1 — Figure 1: Each point represents the validation loss of a Transformer encoder with $N$ parameters (varying the embedding dimension) trained on $D$ unique samples for exactly one epoch (no data repetition). Models are trained across a grid of ( $N$ , $D$ ) configurations spanning several orders of magnitude in both axes. The parametric form in Eq. ( 1 ) is then fit jointly to all points. (a) Loss surface $L(N,D)$ as a function of model size $N$ and training dataset size $D$ , with iso-loss contours shown as colored lines and the compute-optimal trajectory as the blue dashed line. The iso-FLOP lines are colored by the corresponding compute budget, with the color scale shown in (b). (b) Loss as a function of model size at fixed compute budget (iso-FLOP curves), with the compute-optimal trajectory crossing the minima of each curve.

Panel of Figure 2 — Figure 2: Scaling under data repetition. (a) Validation loss as a function of total training compute for models trained on fixed dataset sizes ranging from D = 1k to D = 1M, compared to the compute-optimal scaling $L\propto C^{-0.15}$ (dashed line). Each point represents the early-stopped validation loss of a single model, obtained by stopping training at the epoch with lowest validation loss. Models are trained across a two-dimensional grid: the dataset size $D$ is indicated by the line style, with the embedding dimension $d\in\{4,8,16,32,64,96,128,256,512\}$ also being varied and the resulting model size $N$ indicated by the color scale. Models trained with data repetition consistently lie above the compute-optimal frontier, with validation loss eventually either saturating or overfitting. (b) Training model sizes above the overfitting threshold allows to minimize the validation loss at each fixed dataset size. Scaling along this trajectory yields the same compute exponent, at the price of roughly a factor of 10 in compute to reach the same loss as compute-optimal scaling with no data repetition. (c) Early stopped validation loss as a function of dataset size $D$ for models trained above the overfitting threshold with data repetition (black points), compared to scaling with no data repetition under the large $N$ regime (negligible model size term) $L(N\to\infty,D)=L_{\infty}+B/D^{\beta}$ . Both curves are compute sub-optimal, as they correspond to the limit of large model size rather than the compute-optimal allocation. Although training on repeated data effectively amplifies the dataset, the gain is bounded by $\omega D_{\text{rep}}$ .

Figure 3: Overfitting threshold in the

(N,D)

plane. Blue dots indicate models in the underfitting regime, where increasing training time leads to a plateau in the validation loss. Red crosses indicate models that overfit, where the validation loss eventually increases with further training. The green line shows the fitted threshold

N\propto D^{0.47}

, corresponding to a roughly square-root scaling between the minimum model size needed to overfit and the dataset size. 68% confidence intervals are obtained by bootstrapping threshold points.

Figure 4: Loss as a function of training dataset size

D

for models trained above the overfitting threshold, using four configurations: kinematic variables only (

\Delta\eta,\Delta\phi,log\,p_{T}

), and the full feature set with 10, 40, and 128 particles per jet. Dashed lines show fits to

L(D_{\mathrm{rep}})=L_{\infty}+B_{\mathrm{rep}}/D_{\mathrm{rep}}^{\beta_{\mathrm{rep}}}

. The scaling exponent

\beta_{\mathrm{rep}}

is roughly constant across configurations, while the asymptotic loss

L_{\infty}

decreases with richer inputs and higher particle multiplicity, which also reach the same performance with significantly less amounts of data.

Configuration	$L_{\infty}$ (68% CI)	$B_{\mathrm{rep}}$ (68% CI)	$\beta_{\mathrm{rep}}$ (68% CI)
( $\Delta\eta$ , $\Delta\phi$ , $\log p_{T}$ ), 40 particles	$0.74\,(0.67,\,0.77)$	$5.62\,(4.16,\,7.10)$	$0.21\,(0.17,\,0.24)$
21 features, 10 particles	$0.59\,(0.57,\,0.61)$	$8.69\,(7.59,\,10.35)$	$0.26\,(0.24,\,0.28)$
21 features, 40 particles	$0.33\,(0.30,\,0.34)$	$7.08\,(6.27,\,7.79)$	$0.23\,(0.21,\,0.24)$
21 features, 128 particles	$0.32\,(0.29,\,0.34)$	$5.79\,(5.06,\,6.27)$	$0.22\,(0.20,\,0.23)$

Table 2: Scaling parameters from fits to

L(D_{\mathrm{rep}})=L_{\infty}+B_{\mathrm{rep}}/D_{\mathrm{rep}}^{\beta_{\mathrm{rep}}}

for four input feature configurations. Reported uncertainties are 68% bootstrap confidence intervals.

Figure 5: QCD background rejection for each signal class at 50% efficiency, except for

H\to l\nu qq^{\prime}

(99%) and

t\to bl\nu

(99.5%), as a function of total loss and dataset size when training for multiple epochs above the overfitting threshold. The performance of the ParT architecture trained on 100M samples is shown for reference (dashed lines), which is crossed as expected at the 100M jets scale level. Each colored curve corresponds to one of the nine signal processes in the JetClass dataset.

Panel of Figure 6 — Figure 6: Top vs QCD jet tagging. (a) ROC curves for top quark tagging across the four input feature configurations, for models trained on $D=10^{8}$ samples (solid lines) and the asymptotic performance in the limit $D\to\infty$ (dashed lines) obtained from the fitted $L_{\infty}$ . (b) QCD background rejection at 50% top signal efficiency as a function of total loss and dataset size for the 128-particle, 10-particle, and kinematic-only ( $\Delta\eta$ , $\Delta\phi$ , $\log p_{T}$ ) configurations. While all configurations follow a smooth scaling with dataset size, individual signal processes exhibit different scaling exponents depending on which physics information is retained by the input representation.

Explore similar work

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale
Oct 5, 2026Matthias Vigl, Nikita Pond, Jackson Barr +5High-Energy Physics Scaling Laws
Technical University of Munich · SLAC National Accelerator Laboratory · University College London +4
Towards Engineering Scaling Laws with Pretraining Data Composition
Jun 18, 2026Jan-Lucas Uslu, Kevin Greif, Daniel Whiteson +1Synthetic Data Pretraining High-Energy Physics
Department of Applied Physics Stanford University Stanford, CA 94305, USA · Department of Physics and Astronomy University of California, Irvine Irvine, CA 92697, USA · Department of Particle Physics and Astrophysics SLAC National Accelerator Laboratory Stanford University Menlo Park, CA 94025, USA
Neural Scaling Laws for Jet Generation
May 27, 2026Oz Amram, Darius A. Faroughy, Tjarko Gerdes +5High-Energy Physics Autoregressive Generation
Fermi National Accelerator Laboratory, Batavia, IL 60510, USA · NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, USA · Institute for Experimental Physics, Universität Hamburg, Luruper Chaussee 149, 22761 Hamburg, Germany +1

Oct 5, 2026J/K move · Enter open · S save

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale

Matthias Vigl, Nikita Pond, Jackson Barr, Alexander Froch+4

Technical University of Munich · SLAC National Accelerator Laboratory · University College London+4

High-Energy Physics Scaling Laws

Open paper AskPDF arXiv

Much of the recent progress in machine learning domains such as language models has come from scaling laws that predict performance as a function of training effort. In high-energy physics (HEP) similar behavior has now been observed. To aid further study, we present a systematic procedure to derive robust scaling laws and compare design choices on the relevant budget axes for HEP tasks. We first validate the full scaling trajectory on toy problems and then apply the procedure to multi-task transformers on the ~11 billion-jet ATLAS JetSet2 dataset, in both the compute- and data-constrained regimes. For the latter, we predict, to the best of our knowledge for the first time, the jointly optimal model size, training horizon, learning rate and batch size under early stopping. At compute-optimal scaling, we recover a near-equal

\sqrt{C}

dependence of model and dataset size, and find that auxiliary objectives lower the primary jet-classification loss at equal compute budget. Expanding the inputs toward lower-level data systematically lowers the loss while leaving the scaling exponent nearly unchanged. The onset of the power-law regime is itself set by scale: below a threshold in dataset size the loss carries little information about high-compute scaling, underscoring the value of large, high-quality full-simulation datasets as a foundation for scaling studies and the development of foundation models in HEP.

Abstract

Explore similar work

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale

Towards Engineering Scaling Laws with Pretraining Data Composition

Neural Scaling Laws for Jet Generation