We apply transfer learning (TL) to construct data-driven models of inclusive electron-nucleus cross sections. Starting from an ensemble of deep neural networks pretrained on 12C data, we fine-tune the models separately for 3He, 6Li, 16O, 27Al, 40Ca, and 56Fe. The resulting models improve for all targets, marginally so for oxygen, where the carbon baseline is already adequate, although their predictive robustness depends on the amount, coverage, and precision of the available target data. We systematically study how model performance depends on the number of fine-tuned layers, on the fraction and selection of the training data, and on the overlap between the source and target kinematic domains. The layer-wise analysis shows that oxygen requires only shallow adaptation, whereas helium, calcium, and iron require substantially deeper fine-tuning. Lithium represents the least robust case because of its limited dataset, while aluminum demonstrates a strong sensitivity to a small subset of highly constraining measurements. For selected kinematic configurations outside the coverage of the carbon training data, the adapted models remain consistent with the measurements within their estimated uncertainties. Finally, we compare the resulting predictions with those of the phenomenological F1F2 model.
Low-energy nuclear structure is encoded in a broad body of experimental information across the chart of nuclides. Learning how this information is organized across observables and nuclei can provide a data-driven empirical baseline for theoretical extrapolations and experimental design. Here, we develop held-out ensembles based on NuCLR (Nuclear Co-Learned Representations), a multi-task model of nuclear data, to study charge radii and electric-quadrupole transition strengths. Out-of-fold (OOF) validation shows that shared representation improves performance over single-task learning, yielding a charge-radius RMS deviation of 0.0147fm and a B(E2)RMS deviation of 0.192e2b2 across hundreds of nuclides, competitive with state-of-the-art nuclear models. Our error bars estimate the expected prediction accuracy across the nuclear chart, highlighting regions where new data would encode information beyond the learned patterns. NuCLR thus serves as a data-driven surveyor of nuclear structure and a step toward a shared, multi-observable foundation model of the nuclear chart.
Giuliano Giacalone, Sokratis Trifinopoulos, Mike Williams
Machine-learning models in high-energy physics are often trained on simulated data, where fully simulated samples are computationally expensive while fast simulation provides large statistics at reduced realism. In this work, we systematically study transfer learning between fast-simulated and fully simulated datasets in a realistic LHC environment. We consider three representative tasks, signal-background classification, quark-gluon jet tagging, and missing transverse energy reconstruction, using dense neural networks, graph neural networks, and transformer-based architectures. Models are pretrained on ATLAS-like fast simulation and adapted to CMS-like fast simulation and to fully simulated ATLAS Open Data. Across all tasks, pretrained models consistently outperform independently trained baselines and require significantly less target-domain training data, typically reducing the needed statistics by about a factor of two. These results demonstrate that fast simulation can be used to learn robust, reusable representations and motivate publishing trained models as reusable scientific assets beyond large foundation models.
We present a lightweight approach to foundation modeling (\textbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scientific datasets, using a fully connected autoencoder model with approximately 3 million parameters. The model pre-trains with no supervision over a large-scale collision dataset from the Large Hadron Collider modeled by charged particle track features. Downstream tasks for collider analyses, such as kinematic regression and event classification, are developed on pre-trained model weights and achieve improved accuracy with only small labeled datasets when compared to equivalent architectures trained from scratch. The benefits of pre-training are additionally investigated through latent space interpretation and application to other domains, including gravitational waves, flood forecasting, and neural activity. Furthermore, the relative computational simplicity of NEXUS is demonstrated compared to transformer approaches at comparable scale, opening the door to power-efficient inference and real-time or edge applications of foundation models in scientific experiments.