BiLoGo: Reducing the smoothing of localised responses in Transformer-based neural operators for sheet forming

Yingxue Zhao; Haoran Li; Haosu Zhou; Tobias Pfaff; Nan Li

cs.LGNov 6, 2026

BiLoGo: Reducing the smoothing of localised responses in Transformer-based neural operators for sheet forming

Authors: Yingxue Zhao, Haoran Li, Haosu Zhou, Tobias Pfaff, Nan Li

Organizations: Dyson School of Design Engineering, Imperial College London, London, UK · Prometheus, UK

Abstract

Trial-and-error FE simulation of sheet forming could be computationally expensive, neural-network-based learned simulators are used to accelerate design. These simulators should accurately capture localised responses, such as thinning around sharp radii or near ribs, which determine formability. Transformer-based neural operators propagate information efficiently across the mesh but tend to smooth such localised responses. This study develops BiLoGo, a Bipartite Local-Global learned simulator with an FE-guided design that connects local node-element message passing with a Transformer-based neural operator for long-range information exchange. BiLoGo has three modules: a bidirectional node-element GraphSAGE model that learns nodal-elemental coupling on a bipartite graph and predicts each quantity on its native FE entity without interpolation; a dual-stream Transformer-based neural operator that processes blank deformation and tool contact with separate token sets, preventing localised contact information from being averaged with the whole-blank state; and a fusion and element update module that combines global and local features via gated residual fusion and updates element features conditioned on element-node kinematics. Evaluated on a dome-shaped cold forming case and two corner-shaped hot stamping cases (one with ribs), BiLoGo achieves the lowest positional and threshold-based thinning errors on all three cases compared with single-level and hierarchical graph-based models and Transformer-based neural operators. Near the ribs, the smoothing index of predicted thinning increases from 0.41-0.56 for Transformer-based neural operators to 0.94 for BiLoGo-U. Compared with the hierarchical graph network CAtt-BiUGNN, BiLoGo-T reduces mean positional error by 15.5% on the Corner case and BiLoGo-U by 8.3% on the Corner+Rib case, with 66-69% less training memory.

Figures & tables

Figure 1: Autoregressive rollout of BiLoGo, shown for a Corner+Rib simulation. At each learned step, the predicted displacement increment updates the nodal positions, the predicted thinning is used as the input of the next step, and the contact state is recomputed from the predicted geometry and the prescribed tool positions.

Figure 2: Dome dataset. (a) Tool set-up of the quarter model. (b) Die geometry and design variables $R_{\mathrm{dome}}$ and $R_{\mathrm{die}}$ . (c) Thinning distribution of one simulation over the 11 saved stages.

Figure 3: Corner dataset. (a) Tool set-up of the quarter model. (b) Die geometry and design variables $R_{\mathrm{die}}$ , $R_{\mathrm{punch}}$ , $R_{\mathrm{planview}}$ and $H$ . (c) Thinning distribution of one simulation over the 11 saved stages.

Figure 4: Corner+Rib dataset. (a) Tool set-up of the quarter model. (b) Die geometry of the quarter model with the rib parameters. The rib centre $\mathbf{p}_{\mathrm{rib}}$ is located on the straight top edge between $\mathbf{p}_{\mathrm{start}}$ at the symmetry plane and $\mathbf{p}_{\mathrm{end}}$ at the corner fillet by the position parameter $u$ (Eq. ( 10 )), and $A/2$ and $B/2$ are the semi-axes of the half-elliptical rib along and normal to the side wall. (c) Thinning distribution of one simulation over the 11 saved stages.

Parameter	Range	Description
$u$	25–75%	Position of the rib centre along the top edge
$A/2$	35–50 mm	Half-length of the rib along the side wall
$B/2$	22.5–30 mm	Outward extent of the rib from the side wall

Table 1: Design variables of the Corner+Rib dataset.

Figure 5: Construction of the node–element bipartite graph. (a) Triangular mesh. (b) Bipartite graph with mesh-nodes and element centre-nodes. Each element centre-node is connected to the three nodes of its element, and information is passed in both directions along each edge. The highlighted edges show the neighbourhood used to update an element from its three nodes (magenta, Eq. ( 14 )) and the neighbourhood used to update a node from its neighbouring elements (teal, Eq. ( 15 )).

Stream	Entity	Features
State	Node $v$	$\mathbf{s}_{v}^{n}=[\mathbf{x}_{v}^{n},\ \Delta\mathbf{u}_{v}^{n},\ \mathbf{b}_{v},\ \mathbf{l}]$
State	Element $i$	$\mathbf{s}_{i}^{n}=[\mathbf{x}_{i}^{n},\ \overline{\Delta\mathbf{u}}_{i}^{n},\ \theta_{i}^{n},\ \mathbf{l}]$
Contact	Node $v$	$\mathbf{c}_{v}^{n}$ : proximity $c(d)$ and surface normal $\mathbf{n}$ for each tool, $\mathbf{l}$
Contact	Element $i$	$\mathbf{c}_{i}^{n}$ : mean of the node contact inputs over $v\in\mathcal{N}(i)$ , $\mathbf{l}$
Output	Node $v$	displacement increment $\Delta\mathbf{u}_{v}^{n+1}$
Output	Element $i$	thinning $\theta_{i}^{n+1}$

Table 2: Input and output features of BiLoGo at learned step

n

Figure 6: Architecture of BiLoGo. The contact and state streams are encoded separately and processed by the bipartite GraphSAGE layers of the bidirectional node–element GraphSAGE model. In the global module, each stream is compressed into tokens and processed by self-attention, and the contact tokens query the state tokens through cross-attention. The result is broadcast back to the nodes and elements, combined with the local state features by the gated residual fusion, and used in the kinematics-conditioned element update. After the bipartite GraphSAGE layers that follow the global module, the node and element decoders predict the displacement increment and the thinning. Dashed lines show the local features passed directly to later components. The gated residual fusion and the kinematics-conditioned element update, which form the fusion and element update module, are highlighted in green. The tokeniser is either the physical-slice tokeniser (BiLoGo-T) or the spatial supernode tokeniser (BiLoGo-U).

Component	Hyperparameter	Value
All	Latent dimension $d$	128
Local message passing	Pre-global layers per stream	2
	Post-global layers	2
Global interaction	Attention heads	4
	Self-attention blocks per stream	1
	Cross-attention blocks	1

Table 3: Hyperparameters of BiLoGo.

Model	$e_{\mathrm{pos}}$ (mm)	Position MAE (mm)	Threshold-based thinning error (%)	Top-5% critical thinning error
MGN [ 57 ]	0.5137	0.2385	0.581	2.49%
FEIH-GNN [ 19 ]	0.5291	0.2488	0.502	2.67%
CAtt-BiGNN [ 75 ]	0.4001	0.1893	0.377	1.80%
Transolver [ 66 ]	0.3510	0.1681	0.612	2.64%
GNN-Transformer	0.3505	0.1691	0.602	1.52%
BiLoGo-T	0.3371	0.1608	0.396	1.61%

Table 4: Prediction accuracy on the Dome dataset at the final learned step of the 10-step rollout. The values are averaged over the test simulations. The best value in each column is shown in bold, and the second-best value is underlined.

Figure 7: Errors over the 10 learned steps of the rollout on the Dome test set. (a) Mean positional Euclidean error. (b) Threshold-based thinning error. The lines show the mean over the test simulations. The thinning error is shown from the second step, since few elements exceed the thinning threshold at the first step.

Figure 8: Final-step comparison for an example simulation from the Dome test set. (a) Thinning (top) and absolute thinning error (bottom). (b) Final positions (top) and positional Euclidean errors (bottom).

Model	$e_{\mathrm{pos}}$ (mm)	Position MAE (mm)	Threshold-based thinning error (%)	Top-1% critical relative thinning error
UMGN [ 72 ]	0.4971	0.2299	0.428	14.20%
RUGNN [ 72 ]	0.4005	0.1865	0.396	13.27%
CAtt-BiUGNN [ 75 ]	0.3209	0.1490	0.185	3.02%
Transolver [ 66 ]	1.0231	0.4787	0.480	10.75%
GNN-Transformer	0.4199	0.1989	0.353	8.89%
BiLoGo-T	0.2713	0.1266	0.170	3.57%

Table 5: Prediction accuracy on the Corner dataset at the final learned step of the 10-step rollout. The values are averaged over the test simulations. The best value in each column is shown in bold, and the second-best value is underlined.

Figure 9: Errors over the 10 learned steps of the rollout on the Corner test set. (a) Mean positional Euclidean error. (b) Threshold-based thinning error. The lines show the mean over the test simulations, and the shaded regions show the standard deviation. The thinning error is shown from the second step, since few elements exceed the thinning threshold at the first step.

Figure 10: Final-step comparison for an example simulation from the Corner test set. (a) Thinning (top) and absolute thinning error (bottom). (b) Final positions (top) and positional Euclidean errors (bottom).

Model	$e_{\mathrm{pos}}$ (mm)	Position MAE (mm)	Threshold-based thinning error (%)	Top-0.25% critical thinning error
CAtt-BiUGNN [ 75 ]	0.2637	0.1248	0.206	5.11%
Transolver [ 66 ]	1.3091	0.6243	0.989	16.75%
GNN-Transformer	0.7410	0.3575	0.726	10.96%
UPT [ 1 ]	1.3466	0.6366	0.486	9.40%
BiLoGo-T	0.4493	0.2146	0.391	3.37%
BiLoGo + Transolver++	0.6526	0.3170	0.507	6.77%

Table 6: Prediction accuracy on the Corner+Rib dataset at the final learned step of the 10-step rollout. The values are averaged over the test simulations. The best value in each column is shown in bold, and the second-best value is underlined.

Figure 11: Errors over the 10 learned steps of the rollout on the Corner+Rib test set. (a) Mean positional Euclidean error. (b) Threshold-based thinning error. The lines show the mean over the test simulations, and the shaded regions show the standard deviation.

Figure 12: Final-step comparison for an example simulation from the Corner+Rib test set. (a) Thinning (top), thinning near one rib (middle) and absolute thinning error (bottom). (b) Final positions (top) and positional Euclidean errors (bottom).

Figure 13: Regions used for the local metrics, shown on the undeformed configuration with the FE thinning of one test design at the final learned step. (a) Corner region of the Corner dataset. (b) Rib region of the Corner+Rib dataset. The black lines show the region boundaries.

Model	Region thinning error (%)	$S$	$G$ (%/mm)	$\bar{e}_{\mathrm{pos}}$ (mm)
UMGN [ 72 ]	0.305	0.295	0.141	0.332
RUGNN [ 72 ]	0.302	0.301	0.140	0.298
CAtt-BiUGNN [ 75 ]	0.127	1.009	0.030	0.251
Transolver [ 66 ]	0.443	0.453	0.147	0.517
GNN-Transformer	0.322	0.469	0.137	0.270
BiLoGo-T	0.157	1.001	0.037	0.189

Table 7: Local accuracy near the corner on the Corner test set. The region thinning error,

S

and

G

are evaluated at the final learned step, and

\bar{e}_{\mathrm{pos}}

is averaged over all 10 learned steps. The values are averaged over the 25 test designs. The best value in each column is shown in bold, and the second-best value is underlined. For

S

, the value closest to 1 is regarded as the best.

Model	Region thinning error (%)	$S$	$G$ (%/mm)	$\bar{e}_{\mathrm{pos}}$ (mm)
CAtt-BiUGNN [ 75 ]	0.608	0.989	0.167	0.156
Transolver [ 66 ]	1.838	0.562	0.480	0.745
GNN-Transformer	1.487	0.489	0.454	0.461
UPT [ 1 ]	1.202	0.415	0.442	0.959
BiLoGo-T	0.585	0.874	0.193	0.295
BiLoGo + GINO	0.346	0.957	0.140	0.177

Table 8: Local accuracy near the ribs on the Corner+Rib test set. The region thinning error,

S

and

G

are evaluated at the final learned step, and

\bar{e}_{\mathrm{pos}}

is averaged over all 10 learned steps. The values are averaged over the 50 test designs. The best value in each column is shown in bold, and the second-best value is underlined. For

S

, the value closest to 1 is regarded as the best.

Figure 14: Thinning along a path that crosses one rib, for one test simulation of the Corner+Rib dataset at the final learned step. (a) Position of the path on the FE thinning field. (b)–(g) FE and predicted thinning along the path.

Model	$e_{\mathrm{pos}}$ (mm)	Position MAE (mm)	Threshold-based thinning error (%)	Top-1% critical thinning error
BiLoGo-T	0.2713	0.1266	0.170	3.57%
w/o kinematics-conditioned element update	0.2915	0.1355	0.269	7.53%
w/o gated fusion	0.3467	0.1614	0.216	4.47%
w/o cross-attention	0.2692	0.1251	0.173	3.93%
w/o dual stream	0.4053	0.1918	0.222	3.82%
w/o bipartite GraphSAGE	0.7167	0.3352	0.275	5.78%

Table 9: Ablation study of the BiLoGo components on the Corner dataset at the final learned step of the 10-step rollout. The values are averaged over the test simulations. The best value in each column is shown in bold, and the second-best value is underlined.

Model	$e_{\mathrm{pos}}$ (mm)	Position MAE (mm)	Threshold-based thinning error (%)	Top-0.25% critical thinning error
BiLoGo-U	0.2419	0.1158	0.184	3.86%
w/o kinematics-conditioned element update	0.2429	0.1169	0.235	4.34%
w/o gated fusion	0.2783	0.1328	0.208	3.92%
w/o cross-attention	0.3017	0.1440	0.198	3.88%
w/o dual stream	0.2611	0.1254	0.222	4.51%
w/o bipartite GraphSAGE	0.2732	0.1318	0.209	4.10%

Table 10: Ablation study of the BiLoGo-U components on the Corner+Rib dataset at the final learned step of the 10-step rollout. The values are averaged over the test simulations. The best value in each column is shown in bold, and the second-best value is underlined.

Figure 15: Slice assignment of BiLoGo-T for one test simulation of the Corner+Rib dataset at learned step 9, with the FE state as input. (a)–(f) State stream. (g)–(l) Contact stream. (a), (g) Dominant slice of each node. (b), (h) Dominant slice of each element. (c)–(f) Assignment weights of state slices 0, 1, 2 and 5. (i)–(l) Assignment weights of contact slices 0, 1, 4 and 12. Each weight panel uses its own colour scale.

Figure 16: Supernodes of BiLoGo-U and comparison of the recovered global features, for the simulation in Fig. 15 . (a) Positions of the 2048 supernodes. (b) Entities collected by three supernodes in the region marked in (a), with the collection radius $r=20$ mm, in the reference coordinates. (c) Cumulative explained variance of the principal components of the recovered nodal global features $\mathbf{g}_{e}$ of BiLoGo-T and BiLoGo-U.

Model	Trainable	Training time	Training GPU	Inference time
	parameters	(s/epoch)	memory (GiB)	(s/simulation)
UMGN [ 72 ]	82.9M	49.25	1.590	1.025
RUGNN [ 72 ]	85.3M	65.99	1.753	0.986
CAtt-BiUGNN [ 75 ]	239.5M	129.11	4.497	1.285
Transolver [ 66 ]	0.31M	24.89	0.447	0.427
GNN-Transformer	0.97M	37.62	0.815	0.482

Table 11: Computational cost on the Corner dataset, measured on an NVIDIA L4 GPU.

Figure 17: Prediction accuracy, peak training GPU memory and number of trainable parameters of the models in Table 11 . (a) Mean positional Euclidean error. (b) Threshold-based thinning error. The bubble area increases with the square root of the number of trainable parameters, as indicated by the reference bubbles.

Model	Trainable	Training time	Training GPU	Inference time	Inference GPU
	parameters	(s/epoch)	memory (GiB)	(s/simulation)	memory (GiB)
CAtt-BiUGNN [ 75 ]	678.6M	134.64	12.860	7.037	3.576
Transolver [ 66 ]	0.31M	25.33	0.996	6.721	0.606
GNN-Transformer	0.97M	43.10	1.608	6.840	0.612
UPT [ 1 ]	1.70M	88.58	0.754	6.917	0.611
BiLoGo-T	2.13M	79.68	3.626	6.876	0.621

Table 12: Computational cost on the Corner+Rib dataset, measured on an NVIDIA A40 GPU.

Figure 18: Prediction accuracy, peak training GPU memory and number of trainable parameters of the models in Table 12 . (a) Mean positional Euclidean error. (b) Threshold-based thinning error. The bubble area increases with the square root of the number of trainable parameters, as indicated by the reference bubbles.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Assignment weights of three slices of BiLoGo-T for one test simulation of the Dome dataset at learned steps 0, 4 and 9, with the FE state as input. (a)–(i) State stream. (j)–(r) Contact stream. Here, $k$ denotes the slice index. The three panels of each slice share one colour scale, and different slices use different colour scales.

Figure 2: Assignment weights of three slices of BiLoGo-T for one test simulation of the Corner dataset at learned steps 0, 4 and 9, with the FE state as input. (a)–(i) State stream. (j)–(r) Contact stream. Here, $k$ denotes the slice index. The three panels of each slice share one colour scale, and different slices use different colour scales.

Neural Surrogate Modeling

Neural surrogate modeling uses learned models to approximate costly simulations, enabling faster prediction, design, control, and inference.

See all 440 papers

A finite-element-inspired bipartite graph learned simulator for manufacturability assessment in large-deformation sheet forming
May 14, 2026Yingxue Zhao, Haoran Li, Haosu Zhou +2Neural Surrogate Modeling Graph Neural Networks
Dyson School of Design Engineering, Imperial College London, London, UK · NVIDIA, UK
BAT-NO: A Boundary-Condition-Aware Transformer Neural Operator for Crashworthiness Prediction of Vehicle Components
Oct 2, 2026Haoran Li, Yingxue Zhao, Haosu Zhou +4Neural Surrogate Modeling Neural Operators
Dyson School of Design Engineering, Imperial College London, London, UK · Keysight Technologies, Bagneux, France · Prometheus, UK
LLT: Local Linear Transformer for PDE Operator Learning
Jul 4, 2026Oded Ovadia, Eli TurkelPDE Surrogate Modeling Transformer Attention
School of Mathematical Sciences, Tel Aviv University, Tel Aviv, Israel

May 14, 2026J/K move · Enter open · S save

A finite-element-inspired bipartite graph learned simulator for manufacturability assessment in large-deformation sheet forming

Yingxue Zhao, Haoran Li, Haosu Zhou, Tobias Pfaff+1

Dyson School of Design Engineering, Imperial College London, London, UK · NVIDIA, UK

Neural Surrogate Modeling Graph Neural Networks Finite Element Method

Open paper AskPDF arXiv

Explicit dynamic finite element (FE) simulations are widely used for large-deformation analysis but remain expensive for design-space exploration. In explicit FE analysis, nodal kinematics and element-level deformation measures evolve through a coupled node-element update process. An FE-inspired learned simulator can preserve this structure and learn nodal kinematics and elemental deformation on their native domains. Nodal displacement describes geometry evolution, while elemental thinning identifies local deformation and critical regions. Feeding both outputs back during rollout allows each field to inform subsequent updates of the other. This work proposes CAtt-BiGNN, a cross-attention-based bipartite graph neural network for coupled nodal-elemental learning. The bipartite graph encodes FE-guided state organisation and information flow without imposing equilibrium or constitutive constraints. Mesh nodes and elements are represented as distinct graph entities connected by directed node-element edges, enabling prediction of nodal displacement increments and elemental deformation on their native domains. An edge-aware cross-attention processor dynamically updates edge representations and weights messages using the receiving state and evolving local relations. For larger graphs, CAtt-BiUGNN uses separate nodal and elemental downsampling and upsampling to enhance long-range propagation while preserving their interaction across graph levels. The method is evaluated on dome-shaped cold-forming and corner-shaped hot-forming benchmarks. Comparisons with node-centred baselines, an FE-inspired hypergraph model, a Physics-Attention-based neural operator, a graph Transformer, and attention ablations show improved joint accuracy in nodal displacement and elemental thinning prediction during autoregressive rollout.

Abstract

Neural Surrogate Modeling

Explore similar work

A finite-element-inspired bipartite graph learned simulator for manufacturability assessment in large-deformation sheet forming

BAT-NO: A Boundary-Condition-Aware Transformer Neural Operator for Crashworthiness Prediction of Vehicle Components

LLT: Local Linear Transformer for PDE Operator Learning