Organizations: Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · School of Future Technology, Fuzhou University, Fuzhou, China · College of Computing and Data Science, Nanyang Technological University, Singapore · School of Information Science and Engineering, Yunnan University, Kunming, China · School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore
Training neural operators on million-cell meshes can exceed GPU memory limits. Models are therefore trained on subsamples and deployed on whole meshes or selected regions. However, subsampling can distort boundary information, degrading accuracy. We ask when boundary conditioning under sampling stays consistent with whole-mesh training, and how to aggregate and inject boundary data. Representing the solution as an integral of boundary data against type-dependent measures, we show that subsampling distorts this conditioning in three ways: incomplete boundary coverage or a sample-dependent global state, face weighting by count rather than area, and compression of boundary inputs into a low-dimensional vector. This analysis yields four measurable consistency conditions checkable before training. We satisfy them with two backbone-agnostic modules. Boundary Quadrature Attention (BQA) aggregates all boundary faces, typed and area-weighted, into slice tokens that every query point reads at every layer. Anchored Global State (AGS) estimates the global state from anchors independent of the sampled points. Adding both to a scalable message-passing and slice-attention backbone yields SCANO. We evaluate the approach on controlled and public benchmarks and a data-centre thermal simulation dataset (DCTS) containing million-cell meshes. On both benchmarks, BQA improves every backbone, reducing whole-mesh error up to 15-fold. On all three datasets, AGS aligns regional with whole-mesh predictions, cutting regional error at least tenfold for most backbones on the public benchmark. Together, the modules nearly halve the best original backbone's whole-mesh error on both benchmarks. SCANO predicts a room-scale field in under one second, over 100 times faster than whole-room anchor self-attention.
Figures & tables
Figure 1: Overview of SCANO and of the consistency conditions C1–C4 that it meets (Section 2.6 ). (a) Architecture. Boundary Quadrature Attention (BQA) takes all boundary faces as unsampled typed records ϕy=[xy,ny,τy,gy] with areas ay (C1). It pools them into G area-weighted slice tokens zg with masses Mg (bar height, C2). It injects these tokens into every layer of the backbone along a normalised and an unnormalised readout path (C4). The Anchored Global State (AGS) computes the global summary of each layer from volume-weighted anchors. The anchors are drawn independently of the training sample S (C3). (b) Deployment. A model trained on samples is deployed on subsamples ( on subgraphs ), on a region of interest with its message-passing halo, and on the whole mesh. The whole mesh takes one pass with all cells as anchors. The example is room 1 of Figures 16 and 17 .
Figure 2: Memory of sampled training and boundary statistics of the three data sets. (a) Peak GPU memory of one optimiser step on a DCTS room against the share of the room per sample. Samples are two-hop subgraphs at seed ratios of 0.5–50%, and the last point is the full room (log axes). Crosses mark runs that exceed the GPU memory. The dashed line marks the available memory of a single GPU. The dotted line marks the training setting. (b) Number of boundary patches per case in the training and validation (grey) and test (blue) splits, showing the median, quartiles and range. (c) Share of the patches of each boundary type or physical role.
Figure 3: Distortions of the boundary measure under sampled training (top row), and how SCANO meets conditions C1–C4 (bottom row, filled tags). The mesh has Dirichlet (black), Neumann (teal) and Robin (red) patches. (a) Unobserved : sampling the boundary nodes (shaded) leaves out the patches that are not sampled. BQA reads all boundary faces in every step (C1). (b) Mis-weighted : weighting by count shifts the unit mass toward finely meshed patches (orange outline). BQA weights the records by face area ay in slice tokens of mass Mg (C2). (c) Attenuated : global modulation (FiLM, adaLN) broadcasts one rank- d code ζ to all points. With BQA, each point attends to the slice tokens with its own weights in every layer (C4). (d) Inconsistent : a global state computed from the sample shifts with the query region R . AGS estimates the state from anchors independent of the sample (pins; C3).
Figure 4: Mesh and boundary of a representative case of each data set. (a) Controlled benchmark. The unit cube is shown open on its three near walls. The graded boundary faces are coloured by patch type. The panel also shows the velocity b at mid-height (median case of Figure 12 ). (b) Public benchmark. The interior triangulation is shared by all cases. The boundary segments are coloured by type (median case of Figure 13 ). (c) DCTS room. The CFD boundary mesh is coloured by the role of each component. The ceiling and two near walls are left out (room 1 of Figure 16 ).
Figure 5: Two ways to cover a DCTS test room with forward passes. (a) The multi-pass average. Each pass draws seed cells at the training ratio and grows them by two hops. Orange points are the cells of the pass. Grey points are the other cells. The last panel counts the passes that cover each cell. (b) Regions of interest that fill the room one after another. Each region is a box, outlined in yellow. A message-passing model also reads a halo around it (green). Its prediction is kept only inside the box. In both (a) and (b), the bars give the peak GPU memory of each pass for three backbones with BQA and AGS, and the last column gives one pass over the full mesh. The points are a random subset of the cells.
Figure 6: Effect of BQA and AGS on seven operator backbones, for each data set and sampling setting (columns). The bars give the relative L2 error of the original (native) backbone, of the backbone with BQA and of the backbone with BQA + AGS. The top row is the whole mesh in a single pass. Table 5 gives each method’s own deployment on DCTS. The bottom row is a compact region that holds 2% of the cells (log axes). Bars show the mean over training seeds. Whiskers show the range. Table S8 of the supplementary material lists the values.
Figure 7: Fraction of the boundary patches that a single training sample observes, against the sampling ratio ρ (Proposition 2 ). (a) Controlled benchmark. The observed fraction is shown for each area stratum, for k -hop subgraphs (filled) and uniform subsets (open). (b) DCTS. The observed fraction is shown for each component role under two-hop training subgraphs. The dashed line is the sampled fraction of interior cells. Markers are measurements on the sampler. Solid curves follow Proposition 2 . Vertical dotted lines mark the training ratios. The top line marks BQA, which reads every patch.
Figure 8: Unit-mass test of the learned measure against the whole-mesh relative L2 error on the controlled benchmark, for (a) k -hop and (b) uniform samples at ratio 0.12. Raising all Dirichlet and external Robin values by a constant tests unit mass. The horizontal axis (deviation from the measure) is 0 for exact conservation and 1 for no response. Markers and bars show the mean and range over training seeds. Arrows join each original backbone to the same backbone with BQA. Groups follow Table 3 .
Figure 9: Accuracy and consistency of regional deployments against the share of the mesh in one pass (Proposition 3 , condition C3). The top row gives the relative L2 error. The bottom row gives the drift δ from each model’s own whole-mesh prediction, which is zero at full coverage (symmetric-log vertical axis). Solid curves are scattered passes. Dashed curves are compact regions of interest covering 2–30% of the cells on the benchmarks and 0.5–20% on DCTS. Training samples are uniform at ratio 0.12 on the controlled benchmark. On the public benchmark, they are one-hop subgraphs that cover about 5% of the mesh. On DCTS, they are two-hop subgraphs. Markers and bars show the mean and range over training seeds.
Figure 10: Sampling gap against whole-mesh training, and convergence in the number of anchors. (a–c) Whole-mesh relative L2 error of models trained on subsamples (markers with 95% bootstrap bars). Dashed horizontal lines show the same architectures trained on the full mesh. Panel (a) is the controlled benchmark. Panel (b) is the public benchmark at several training coverages (see Table S16 for SCANO at 5%). Panel (c) is DCTS, for two of the models whose training fits a full room in memory. (d) Mean squared drift of SCANO’s whole-mesh prediction on the controlled benchmark against the fresh anchor count Na . The grey reference line ( variance law ) is the finite-population rate 1/Na−1/∣Ωh∣ .
Figure 11: Region drift δ against the response error Eresp on the controlled benchmark. The drift is measured in a compact region of 12% of the cells. The response is to single-patch interventions. Samples are k -hop at ratio 0.12, and Table S9 of the supplementary material lists the values. Marker area grows as the error on a sample decreases. Error bars are 95% case-bootstrap intervals. A predictor that ignores the boundary lies at Eresp=1 by construction.
Figure 12: Scalar field u at mid-height and its pointwise error on the controlled benchmark. The rows are (a) the 5th-, (b) the 50th- and (c) the 95th-percentile test cases of SCANO’s whole-mesh error. The first column is the finite-volume reference. It shows streamlines of b and the wall patches as strips coloured by type. The next columns are the whole-mesh predictions of SCANO, area-weighted Transolver + LFE and AB-UPT + BQA + AGS (median validation seed). The last columns show the signed error u^−u , with the whole-mesh relative L2 error below each panel.
Figure 13: Whole-mesh predictions and pointwise errors on the public benchmark. The rows are (a) the 5th-, (b) the 50th- and (c) the 95th-percentile test cases of SCANO’s error. Training covers about 76% of the mesh. The first column is the finite-element reference, with the outer ring coloured by boundary type. The next columns are the predictions of SCANO and of Transolver + LFE, each with the seed of median validation error. The last columns show u^−u (prediction minus reference), with the whole-mesh relative L2 error below each panel.
Figure 14: Regional deployment against the whole-mesh prediction on the same cells, over the 50 DCTS test rooms. The horizontal axis is the share of the mesh in the region. (a) Temperature RMSE (K). (b) Velocity RMSE ( ms−1 ). Solid lines with filled markers are standalone predictions on the region. Dashed lines with open markers are whole-mesh predictions evaluated on the same cells. (c) Ratio of the regional to the whole-mesh RMSE, for T (solid) and U (dotted). Markers and bars show the mean and range over training seeds.
Figure 15: Three-dimensional temperature field T in a DCTS test room (room 1 of Figure 16 ). The panels show the CFD reference and the whole-room predictions of the methods of Table 5 . Each model uses the seed of median validation error. The top of each panel renders the thermal plumes above the 90th percentile of the CFD temperature. The bottom shows the horizontal cross-section at the height of the server inlets. Racks are grey and cooling supplies blue.
Figure 16: Predictions on the horizontal plane at the height of the server inlets, in three held-out DCTS test rooms, (a) Room 1, (b) Room 2 and (c) Room 3. For each room, the upper row shows the temperature T and the lower row the velocity magnitude ∣U∣ . The columns are the CFD reference, SCANO and the native Transolver, followed by the pointwise differences (model minus CFD). Each model uses the seed of median validation error. Racks and solid obstacles are grey. Numbers below the difference panels give the mean absolute difference on the plane. Table 5 summarises all 50 rooms.
Figure 17: A local query in a DCTS test room (room 1 of Figure 16 ). The region is a full-height box within 3 m horizontally of the rack with the hottest CFD inlets. It holds 6% of the room volume. The panels show it at the height of the server inlets. (a) Temperature T . (b) Velocity magnitude ∣U∣ . The left columns show the CFD reference and the standalone regional predictions of SCANO, the SCANO backbone + BQA and GeoTransolver. The right columns show the region pass minus the whole-mesh pass for each model (median validation seed). Numbers below the panels give the mean absolute difference on the plane.
Figure 18: Cost of training and deployment on million-cell DCTS rooms, on an NVIDIA GPU with 96 GB of memory. Values are medians over the four rooms of Table 8 . (a) Peak GPU memory against the sampled fraction of the room. (b) Step time against the same fraction. The right axis converts it into the optimiser time of 100 epochs. Crosses mark runs that exceed the memory. The dotted line marks the training setting. (c) Cumulative wall-clock hours against the number of predicted rooms, for CFD and for SCANO. SCANO is counted with training alone and with training plus data generation. Markers show the break-even counts. (d) Inference memory per room. (e) Inference time per room. Curves are compact regions. Markers are full rooms, predicted in one pass with all cells as anchors, with fresh anchors, or as an average over passes. (f) Time per room of the multi-pass average on one GPU against the share of the room in one pass. The dotted line marks the training share. Open markers are the deployment of the models whose state follows the sample. Filled markers are the single pass of the models with AGS, half filled with fresh anchors. The dashed line is SCANO’s single pass.
Figure S1: Training-free comparison of count and area weights on the controlled benchmark. A fixed Gaussian kernel aggregates refined boundary records. Its width is σK=0.05 in (a) and σK=0.2 in (b). The records have spacing ( record size ) hb=1/nb and grading ratios from 1 to 10 (dark to light). The error is relative to a fine uniform reference. Area-weighted aggregation (solid, filled markers) converges at second order for all gradings. The grey guide is ∝hb2 . Count-weighted aggregation (dashed, open markers) on graded records stalls at the plateau predicted by Proposition 4 (dotted). At a grading ratio of 1, patch-boundary intersections create short edge fragments. Their count-weighting error decays under refinement.
Figure S2: Response spectrum of the controlled benchmark. The solid line is the optimal rank- d approximation error ( best rank- d prediction, error ) of the exact response operator. The other lines give the relative tail energy, for all layouts (dashed) and for 48 patches (dotted). These curves set the theoretical error floor of a global code of width d (Proposition 5 ). At the backbone’s full channel width ( d=96 ), the trained global modulation model of Table 3 remains over two orders of magnitude above this floor.
Figure S3: Effect of BQA and AGS on all backbones (colour), data sets (marker shape) and deployments. Filled markers are the whole mesh. Open markers are a compact region of 2% of the cells. (a) Relative L2 error with BQA against the original (native) backbone. (b) Relative L2 error with BQA + AGS against BQA alone. Grey reference lines mark 1× , 1/2× , 1/5× and 1/10× the error. All configurations with completed training seeds are shown.
Figure S4: Per-case whole-mesh error distributions of each original (native) backbone (grey) and of the same backbone with BQA + AGS (blue). Where AGS does not apply, the blue distribution is BQA alone. The panels are the controlled benchmark ( k -hop samples, 512 test cases), the public benchmark (76% coverage, 256 test cases) and DCTS (50 test rooms). Bars mark interquartile ranges, and white dots mark medians (log scale).
Figure S5: The multi-pass average of the models whose state follows the sample on one, two and four GPUs. Each GPU runs one process and takes its share of the passes in turn. The grey dashed lines mark the ideal speed-up. The solid line is SCANO’s single pass on one GPU. Values are medians over the four rooms of Table 8 .
Surrogate modeling of transport requires operators that capture the combined influence of spatially distributed sources and Dirichlet and Neumann boundary conditions, together with training datasets that can be generated at low computational cost. This work presents MIGONet, a multiple-input graph neural operator, and a discrete method of manufactured solutions (MMS) that trains it without solving a partial differential equation, using steady heat conduction as the model problem. In the discrete MMS procedure, temperature fields are prescribed a priori, and the source terms and Dirichlet or Neumann boundary data are evaluated with the same discrete operators as the reference finite-difference method (FDM), so every training sample satisfies the discrete governing equations by construction. Because the boundary data are derived from each field, a Dirichlet or Neumann condition can be assigned to every boundary segment of every sample, and the boundary specification becomes an operator input. MIGONet supplies representations of the complete source field and boundary data to every node before local message passing. On the rectangular domain, it is compared with a version without the complete-source encoder (MIGONet-local) and with the multiple-input operator network MIONet, using the same training data. MIGONet achieves mean relative temperature errors of 10.31% and 10.64% for curved-wire and Gaussian-spot heating, compared with 45.97% and 48.34% for MIONet and 72.25% and 67.82% for MIGONet-local. For 10,000 samples, discrete MMS reduces data-generation time from 486.65 s to 3.15 s relative to matched iterative FDM labeling. Separately trained MIGONet models are also evaluated on L-shaped, disc-shaped, and ring-shaped domains.
Muhammad Hasnain, N. K. Anand, Arun Srinivasa
Department of Mechanical Engineering, Texas A&M University, 3123 TAMU College Station, TX 77843-3123, USA
Recent mesh-based simulation advances have, in no small part, relied on neural surrogates of two distinct families: global models that route information through a small set of latent tokens, and local models that perform message passing across mesh edges. Consistent with both classes is the inability to perform beyond low-dimensional problems and small-scale or oversimplified meshes, the simulation regimes where industrial problems reside. Our work shows this explicitly and presents a unified formulation. In global approaches, latent-token attention acts as a spatial low-pass filter, while local message passing lacks the global reach necessary to propagate information across large mesh spaces. Viewed through the error, the two operators are the halves of a multigrid cycle: one corrects errors at the lower end of the spectrum, the other at the higher end, and neither can do the other's job. We introduce Read-Write-Relax (RWR), which interleaves latent attention with message-passing relaxation under a unified formulation. The interleaved processor lowers error across the entire spectrum, making RWR the most accurate model in nearly every comparison across our industrial and public benchmarks. It is also markedly data-efficient in the scarce-data regimes, accurate on the engineering quantities of interest, and scales full-field predictions to challenging, large-scale problems.
Anuj Kumar, Heiko Zimmermann, Josiah Bjorgaard +4
Pasteur Labs, Brooklyn, NY, USA · Cornell University, Ithaca, NY, USA · Institute for Simulation Intelligence, New York, NY, USA
Neural surrogates are widely used in scientific machine learning for fast prediction of three-dimensional (3D) thermo-fluid fields. However, generating training data using conventional numerical solvers often incurs substantial computational and storage costs. We propose to train an attention graph neural network by minimizing the finite-volume method (FVM) residuals of the governing equations. These residuals are evaluated directly on the mesh, requiring no labeled data. We evaluate the trained surrogates against computational fluid dynamics (CFD) references and a data-supervised baseline across four scenarios. On the two steady-state benchmarks, each solved on the same mesh as its CFD reference, the FVM-loss model reproduces the reference flow and temperature fields with mean field-averaged normalized root-mean-square errors (nRMSEs) near 3%, including the buoyancy-energy coupling. On the two parametric transient cases, the FVM-loss model attains lower rollout errors than the supervised baseline in almost every field, while avoiding the data-generation cost entirely. These results indicate that the FVM loss can provide a practical training signal for neural surrogates and reduce the model development cost.
Tianyu Li, Zhiwei Cao, Qingang Zhang +3
Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore, Singapore. · School of Future Technology, Fuzhou University, Fuzhou, China. · College of Computing and Data Science, Nanyang Technological University, Singapore, Singapore. +2