Evaluating the energy transmission ratio of an X-ray beam through a polymer composite is a challenging task in radiation shielding studies. The reference solution with Monte Carlo transport simulation is highly accurate but computationally expensive: each of our reference simulations is capped at 4×109 photon histories and a 20,000 s (over 5 hours) time limit, while closed-form analytical methods fail to represent such complex scenarios. We present an artificial neural network model that maps the unordered multi-set of molecular constituents, their mass fractions, and four bulk physical descriptors directly to the transmission ratio I(t)/I0 in 1-3 milliseconds. The model is a Set Transformer, which makes it permutation invariant by construction and accepts mixtures of different sizes up to 11 constituents within a fixed-size padded set. Each constituent is embedded with a chemistry-pretrained transformer language model, ChemBERTa. In our proposed method, training data are produced by an automated PENELOPE/penEasy pipeline using the radiation source at 30, 80, 100, and 120 kV tube voltages over polyethylene-based shielding composite slabs of thicknesses spanning 0.15-2.5 mm. After preprocessing, the dataset contains n=4697 simulated configurations on which a leakage-free, jointly stratified 10-fold cross-validation is performed, attaining mean squared error MSE = (1.12±0.56)×10−4, normalized-MSE percentage NMSE% = 0.0294% ± 0.0147%, and R2=0.9988. The corresponding per-sample percentage errors give an alpha-trimmed (α=0.10) absolute percentage error (APE) of 1.66% and a median APE of 1.15%. Our model's inference is several orders of magnitude faster than the underlying Monte Carlo simulation (sub-millisecond per query vs. several hours for the reference run) with effective accuracy in predicting I(t)/I0.
Figures & tables
Hyperparameter
Value
Hidden dimension d
256
Attention heads h
32
ISAB inducing points m
64
PMA seed vectors k
8
ISAB layers
2
Row-wise FF layers per block
3
Table 1: Set Transformer hyperparameters are decided through extensive testing. The resulting network has 3,004,417 trainable parameters.
Figure 2: Predicted vs. simulated energy transmission ratios for each of the three baselines and STAR-XPC, aggregated over the out-of-fold predictions of the identical 10-fold cross-validation ( n=4697 ).
Model
CV NMSE%
Held-out NMSE%
Ratio
Ridge (linear)
3.25
7.89
2.4×
Random Forest
0.174
0.433
2.5×
SVR (RBF)
0.0312
0.0804
2.6×
STAR-XPC
0.0278
0.0496
1.8×
Table 2: NMSE% in-distribution (10-fold CV, out-of-fold predictions restricted to 80 - 120kV , n=3996 ) versus the mean over the six held-out unseen-combination test sets, and the ratio between them.
Tube voltage
n
MAE
NMSE%
MAPE0.10 (%)
y
80 kV
1367
0.00683
0.032
1.65
0.495
100 kV
1318
0.00673
0.023
1.25
0.556
120 kV
1311
0.00865
0.028
1.13
0.703
Table 3: Performance stratified by tube voltage ( 80 - 120kV , n=3996 )
Thickness bin (mm)
n
MAE
NMSE%
MAPE0.10 (%)
(0.149,0.45]
499
0.01031
0.0294
1.25
(0.45,0.75]
454
0.00932
0.0440
1.58
(0.75,1.05]
442
0.01005
0.0707
2.29
(1.05,1.5]
546
0.01056
0.1196
3.32
Table 4: Performance stratified by thickness bins for samples whose main material is a single element (B, Ba, Bi, Cu, Sb, Sn, W, 80 - 120kV , n=1941 ).
Thickness bin (mm)
n
MAE
NMSE%
MAPE0.10 (%)
(0.199,0.4]
558
0.00466
0.0057
0.51
(0.4,0.75]
433
0.00454
0.0072
0.59
(0.75,1.5]
631
0.00515
0.0201
0.92
(1.5,2.5]
433
0.00495
0.0213
1.48
Table 5: Performance stratified by thickness bins for samples whose main material is a compound (B 4 C, BN, BaSO 4 , Bi 2 O 3 , WO 3 , 80 - 120kV , n=2055 ).
Figure 5: Transmission ratio versus material thickness for held-out data to analyze the generalization capability of STAR-XPC. Each sub-caption gives the tube voltage and the mass fractions of the shielding material and of the polyethylene (C 2 H 4)n matrix, which sum to 100 wt%.
Figure 6: Transmission ratio versus thickness at 100kV for two shielding elements entirely absent from the training vocabulary.
Accurate modeling of Short Fiber Reinforced Composites (SFRCs) remains computationally expensive for full-field simulations. Data-driven surrogate models using Artificial Neural Networks (ANNs) have been proposed as an efficient alternative to numerical modeling, where Recurrent Neural Networks (RNNs) are increasingly being used for path-dependent multiscale modeling by predicting the homogenized response of a Representative Volume Element (RVE). However, recently, transformer models have been developed and they offer scalability and efficient parallelization, yet have not been systematically compared with RNNs in this field. In this study, we perform a systematic comparison between RNNs and transformer models trained on sequences of homogenized response of SFRC RVEs. We study the effect on two types of hyperparameters, namely architectural hyperparameters (such as the number of GRU layers, hidden size, number of attention heads, and encoder blocks) and training hyperparameters (such as learning rate and batch size). Both sets of hyperparameters are tuned using Bayesian optimization. We then analyze scaling laws with respect to dataset size and inference accuracy in interpolation and extrapolation regimes. The results show that while transformer models remain competitive in terms of accuracy on large datasets, the RNNs demonstrate better accuracy on small datasets and show better extrapolation performance. Furthermore, under extrapolation, there is a clear difference, where the RNN remains accurate, while the transformer model performs poorly. On the other hand, the transformer model is 7 times faster at inference, requiring 0.5 ms per prediction compared to the 3.5 ms per prediction for the RNN model.
Petter Uvdal, Mohsen Mirkhalaf
Department of Physics, University of Gothenburg, Origovägen 6B, 41296 Gothenburg, Sweden
We introduce a new strategy for compositional neural surrogates for radiation-matter interactions, a key task spanning domains from particle physics through nuclear and space engineering to medical physics. Exploiting the locality and the Markov nature of particle interactions, we create a next-particle prediction kernel using hybrid discrete-continuous transformer models based on Riemannian Flow Matching on product manifolds. The model generates variable-sized typed sets of particles and radiation side effects that are the result of the interaction of an incident particle with a material volume. The resulting kernel can be composed to simulate unseen large-scale material distributions in a zero-shot manner. Unlike mechanistic simulators, our model is designed to be differentiable, provides tractable likelihoods for future downstream applications. A significant computational speed-up on GPU compared to CPU-bound mechanistic simulation is observed for single-kernel execution. We evaluate the model at the kernel level and demonstrate predictive stability over multi-round autoregressive rollouts. We additionally release a novel 20M-event radiation-matter interaction dataset for further research.
Richard Hildebrandt, Evangelos Kourlitis, Baran Hashemi +7
Technical University of Munich · Max-Planck Institute for Mathematics in the Sciences, Leipzig, Germany · SLAC National Accelerator Laboratory +3
Accurate knowledge of the atomistic transition pathways in materials and material surfaces is crucial for many material science problems. However, conventional simulation techniques used to find these transitions are extremely computationally intensive. Even with large-scale, accelerated material simulations, the computational cost constrains the applicable domain in practice. Machine learning models, with the potential to learn the complex emergent behaviors governing atomistic transitions as a fast surrogate model, have great promise to predict transitions with a vastly reduced computational cost. Here, we demonstrate how transformers can be trained to predict atomistic transitions in nano-clusters. We show how we evaluate physical validity of the predictions and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.
Henry Tischler, Wenting Li, Qi Tang +2
Computing and Artificial Intelligence Division, Los Alamos National Laboratory, Los Alamos, NM 87545 · School of Engineering and Computer Science, University of Denver, Denver, CO 80210 · Department of Physics and Astronomy, University of Denver, Denver, CO 80210 +4