cs.LGNov 1, 2026

SensOFormer: Foundation Models for Particle Accelerators

Authors: Mahindra Rautela, Alexander Scheinker

Organizations: Los Alamos National Laboratory, Los Alamos, NM, USA.

Abstract

Large-scale scientific facilities, such as particle accelerators, are complex systems composed of thousands of interacting components that must operate collectively to support diagnostics and control, and ensure safe, reliable operation. These facilities rely on extensive sensor networks that generate large volumes of heterogeneous, incomplete, and noisy telemetry, often sampled at different rates across subsystems. In this paper, we adopt the foundation-model paradigm to learn general-purpose sensor representations from historical accelerator data through self-supervised pretraining. We introduce SensOFormer, a denoising masked transformer with a Perceiver-style encoder-decoder architecture designed to handle variable sensor sets, accommodate heterogeneous sampling rates, and learn robust representations from noisy measurements. The model is pretrained on multiple particle-accelerator datasets and transferred to downstream tasks, including missing-data imputation, anomaly detection, cavity identification, and fault identification. Through extensive evaluations and comparisons, we demonstrate that a single self-supervised model can learn reusable representations for diverse operational tasks in large-scale scientific facilities. Project page: https://mahindrautela.github.io/sensoformer.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Panda Diplomacy: Foundation Model Pre-training across Particle Imaging Detectors for High Energy and Nuclear Physics

    Sep 1, 2026Samuel Young, César Jesús-Valls, Kazuhiro TeraoSelf-Supervised Pre-TrainingPoint Cloud Learning

  2. Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings

    May 29, 2026Utsav Dutta, Gerardo Pastrana, Sina Khoshfetrat Pakazad +1Multivariate Time Series ForecastingJoint-Embedding Predictive Architecture

  3. A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation

    Jul 29, 2026Liangyu Wu, Qibin Liu, Alexander Yue +1Self-Supervised Pre-TrainingHigh-Energy Physics