STAR-XPC: Set Transformer for Estimating X-ray Transmission in Polymer Composites

Can Erer; H. Birkan Yilmaz; Nebahat Aral

cs.LGOct 20, 2026

STAR-XPC: Set Transformer for Estimating X-ray Transmission in Polymer Composites

Authors: Can Erer, H. Birkan Yilmaz, Nebahat Aral

Organizations: Department of Computer Engineering, Bo˘gaziçi University, Bebek, 34342, Istanbul, Turkiye · Independent Researcher, Istanbul, Turkiye

Abstract

Evaluating the energy transmission ratio of an X-ray beam through a polymer composite is a challenging task in radiation shielding studies. The reference solution with Monte Carlo transport simulation is highly accurate but computationally expensive: each of our reference simulations is capped at $4\times10^9$ photon histories and a 20,000 s (over 5 hours) time limit, while closed-form analytical methods fail to represent such complex scenarios. We present an artificial neural network model that maps the unordered multi-set of molecular constituents, their mass fractions, and four bulk physical descriptors directly to the transmission ratio $I(t)/I_0$ in 1-3 milliseconds. The model is a Set Transformer, which makes it permutation invariant by construction and accepts mixtures of different sizes up to 11 constituents within a fixed-size padded set. Each constituent is embedded with a chemistry-pretrained transformer language model, ChemBERTa. In our proposed method, training data are produced by an automated PENELOPE/penEasy pipeline using the radiation source at 30, 80, 100, and 120 kV tube voltages over polyethylene-based shielding composite slabs of thicknesses spanning 0.15-2.5 mm. After preprocessing, the dataset contains $n = 4697$ simulated configurations on which a leakage-free, jointly stratified 10-fold cross-validation is performed, attaining mean squared error MSE = $(1.12 \pm 0.56)\times10^{-4}$ , normalized-MSE percentage NMSE% = 0.0294% $\pm$ 0.0147%, and $R^2 = 0.9988$ . The corresponding per-sample percentage errors give an alpha-trimmed ( $α= 0.10$ ) absolute percentage error (APE) of 1.66% and a median APE of 1.15%. Our model's inference is several orders of magnitude faster than the underlying Monte Carlo simulation (sub-millisecond per query vs. several hours for the reference run) with effective accuracy in predicting $I(t)/I_0$ .

Figures & tables

Hyperparameter	Value
Hidden dimension $d$	256
Attention heads $h$	32
ISAB inducing points $m$	64
PMA seed vectors $k$	8
ISAB layers	2
Row-wise FF layers per block	3

Table 1: Set Transformer hyperparameters are decided through extensive testing. The resulting network has

3{,}004{,}417

trainable parameters.

Figure 2: Predicted vs. simulated energy transmission ratios for each of the three baselines and STAR-XPC, aggregated over the out-of-fold predictions of the identical 10-fold cross-validation (

n=4697

Model	CV NMSE%	Held-out NMSE%	Ratio
Ridge (linear)	3.25	7.89	$2.4\times$
Random Forest	0.174	0.433	$2.5\times$
SVR (RBF)	0.0312	0.0804	$2.6\times$
STAR-XPC	0.0278	0.0496	$\mathbf{1.8\times}$

Table 2: NMSE% in-distribution (10-fold CV, out-of-fold predictions restricted to

80

120\text{\,}\mathrm{kV}

n=3996

) versus the mean over the six held-out unseen-combination test sets, and the ratio between them.

Tube voltage	$n$	MAE	NMSE%	$\mathrm{MAPE}_{0.10}$ (%)	$\overline{y}$
80 kV	1367	0.00683	0.032	1.65	0.495
100 kV	1318	0.00673	0.023	1.25	0.556
120 kV	1311	0.00865	0.028	1.13	0.703

Table 3: Performance stratified by tube voltage (

80

120\text{\,}\mathrm{kV}

n=3996

)

Thickness bin (mm)	$n$	MAE	NMSE%	$\mathrm{MAPE}_{0.10}$ (%)
$(0.149,\,0.45]$	499	0.01031	0.0294	1.25
$(0.45,\,0.75]$	454	0.00932	0.0440	1.58
$(0.75,\,1.05]$	442	0.01005	0.0707	2.29
$(1.05,\,1.5]$	546	0.01056	0.1196	3.32

Table 4: Performance stratified by thickness bins for samples whose main material is a single element (B, Ba, Bi, Cu, Sb, Sn, W,

80

120\text{\,}\mathrm{kV}

n=1941

Thickness bin (mm)	$n$	MAE	NMSE%	$\mathrm{MAPE}_{0.10}$ (%)
$(0.199,\,0.4]$	558	0.00466	0.0057	0.51
$(0.4,\,0.75]$	433	0.00454	0.0072	0.59
$(0.75,\,1.5]$	631	0.00515	0.0201	0.92
$(1.5,\,2.5]$	433	0.00495	0.0213	1.48

Table 5: Performance stratified by thickness bins for samples whose main material is a compound (B 4 C, BN, BaSO 4 , Bi 2 O 3 , WO 3 ,

80

120\text{\,}\mathrm{kV}

n=2055

Main Shielding Material	$n$	MAE	NMSE%	$\mathrm{MAPE}_{0.10}$ (%)
BN	371	0.0047	0.012	0.42
B 4 C	309	0.0046	0.009	0.47
BaSO 4	525	0.0048	0.009	0.73
Bi 2 O 3	599	0.0049	0.015	1.19
WO 3	251	0.0052	0.017	1.23
B	248	0.0105	0.027	1.24

Table 6: Per-shielding-material MAE, NMSE%, and

\alpha

-trimmed MAPE (

80

120\text{\,}\mathrm{kV}

n=3996

Figure 5: Transmission ratio versus material thickness for held-out data to analyze the generalization capability of STAR-XPC. Each sub-caption gives the tube voltage and the mass fractions of the shielding material and of the polyethylene (C 2 H

{}_{4})_{n}

matrix, which sum to

100

wt%.

Figure 6: Transmission ratio versus thickness at

100\text{\,}\mathrm{kV}

for two shielding elements entirely absent from the training vocabulary.

Neural Surrogate Modeling

Neural surrogate modeling uses learned models to approximate costly simulations, enabling faster prediction, design, control, and inference.

See all 410 papers

A comparative study of transformer models and recurrent neural networks for path-dependent composite materials
Feb 17, 2026Petter Uvdal, Mohsen MirkhalafNeural Surrogate Modeling Recurrent Neural Networks
Department of Physics, University of Gothenburg, Origovägen 6B, 41296 Gothenburg, Sweden
BRICKS: Compositional Neural Markov Kernels for Zero-Shot Radiation-Matter Simulation
May 7, 2026Richard Hildebrandt, Evangelos Kourlitis, Baran Hashemi +7Neural Surrogate Modeling Riemannian Flow Matching
Technical University of Munich · Max-Planck Institute for Mathematics in the Sciences, Leipzig, Germany · SLAC National Accelerator Laboratory +3
Predicting Atomistic Transitions with Transformers
Mar 5, 2026Henry Tischler, Wenting Li, Qi Tang +2Computational Materials Science Neural Surrogate Modeling
Computing and Artificial Intelligence Division, Los Alamos National Laboratory, Los Alamos, NM 87545 · School of Engineering and Computer Science, University of Denver, Denver, CO 80210 · Department of Physics and Astronomy, University of Denver, Denver, CO 80210 +4

Feb 17, 2026J/K move · Enter open · S save

A comparative study of transformer models and recurrent neural networks for path-dependent composite materials

Petter Uvdal, Mohsen Mirkhalaf

Department of Physics, University of Gothenburg, Origovägen 6B, 41296 Gothenburg, Sweden

Neural Surrogate Modeling Recurrent Neural Networks Transformer Materials Science

Open paper AskPDF arXiv

Accurate modeling of Short Fiber Reinforced Composites (SFRCs) remains computationally expensive for full-field simulations. Data-driven surrogate models using Artificial Neural Networks (ANNs) have been proposed as an efficient alternative to numerical modeling, where Recurrent Neural Networks (RNNs) are increasingly being used for path-dependent multiscale modeling by predicting the homogenized response of a Representative Volume Element (RVE). However, recently, transformer models have been developed and they offer scalability and efficient parallelization, yet have not been systematically compared with RNNs in this field. In this study, we perform a systematic comparison between RNNs and transformer models trained on sequences of homogenized response of SFRC RVEs. We study the effect on two types of hyperparameters, namely architectural hyperparameters (such as the number of GRU layers, hidden size, number of attention heads, and encoder blocks) and training hyperparameters (such as learning rate and batch size). Both sets of hyperparameters are tuned using Bayesian optimization. We then analyze scaling laws with respect to dataset size and inference accuracy in interpolation and extrapolation regimes. The results show that while transformer models remain competitive in terms of accuracy on large datasets, the RNNs demonstrate better accuracy on small datasets and show better extrapolation performance. Furthermore, under extrapolation, there is a clear difference, where the RNN remains accurate, while the transformer model performs poorly. On the other hand, the transformer model is 7 times faster at inference, requiring 0.5 ms per prediction compared to the 3.5 ms per prediction for the RNN model.

Abstract

Neural Surrogate Modeling

Explore similar work

A comparative study of transformer models and recurrent neural networks for path-dependent composite materials

BRICKS: Compositional Neural Markov Kernels for Zero-Shot Radiation-Matter Simulation

Predicting Atomistic Transitions with Transformers