A foundation model for energy and radiation systems built on heterogeneous scientific interfaces

Samrendra Roy; Tapas Tripura; Yoon Pyo Lee; Souvik Chakraborty; Syed Bahauddin Alam

cs.LGSep 29, 2026

A foundation model for energy and radiation systems built on heterogeneous scientific interfaces

Authors: Samrendra Roy, Tapas Tripura, Yoon Pyo Lee, Souvik Chakraborty, Syed Bahauddin Alam

Organizations: Department of Nuclear, Plasma, and Radiological Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA · Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · Yardi School of Artificial Intelligence (ScAI), Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · National Center for Supercomputing Applications (NCSA), University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Abstract

Scientific foundation models are commonly evaluated after heterogeneous physical problems have already been translated into a compatible gridded, tokenized or symbolic representation. This leaves the scientific interface outside both the pretrained model and the audit of what is actually reused. We study the complementary setting in which boundary histories, sparse monitor records and loading histories retain their native inference classes and their outputs remain on Cartesian, latitude-longitude and unstructured domains. GEODE couples task-specific scientific interfaces to a shared routed library of wavelet operators. A single jointly pretrained model represents cavity flow, radiation dose and elastoplastic stress, then acquires a heat exchanger and a reactor subchannel by training a private interface containing 2.1% of its parameters. Earlier predictions remain unchanged by parameter isolation, whereas unrestricted fine-tuning degrades them by factors of 14-29. Crucially, preservation alone does not establish reuse: norm-matched randomized-library controls show that the contribution of pretrained computation is conditional on the task and data regime. A separate decomposition shows that full-field relative L2 error can substantially understate error relative to spatial variation when field level dominates the norm. Task-specific operators remain more accurate on three of the five problems. These results distinguish multi-task coverage, preservation and pretrained reuse as separate properties that must be tested independently when scientific foundation models span heterogeneous interfaces.

Figures & tables

Class or system	F1	F2	F3	F4	F5	F6
Independent task-specific operators a	$\times$	$\times$	$\times$	$\bigcirc$	$\bigcirc$	$\checkmark$
Geometry-general operators ( Li et al., 2023a ; Li et al., 2023b ; Hao et al., 2023 ; Wu et al., 2024 ; Navaneeth and Chakraborty, 2025 ; Sarkar and Chakraborty, 2026 )	$\times$	$\times$	$\bigcirc$	$\checkmark$	$\times$	$\checkmark$
In-context operator learning ( Yang et al., 2023 )	$\checkmark$	$\bigcirc$	$\checkmark$	$\times$	$\bigcirc$	$\bigcirc$
Multiple Physics Pretraining ( McCabe et al., 2023 )	$\checkmark$	$\checkmark$	$\bigcirc$	$\times$	$\bigcirc$	$\checkmark$
DPOT, Poseidon ( Hao et al., 2024 ; Herde et al., 2024 )	$\checkmark$	$\checkmark$	$\bigcirc$	$\times$	$\bigcirc$	$\checkmark$
PDEformer-2 ( Ye et al., 2025 )	$\checkmark$	$\checkmark$	$\bigcirc$	$\checkmark$	$\bigcirc$	$\checkmark$

Table 1: Operational requirements for a foundation model of energy and radiation systems. These requirements concern interface breadth , the number of mutually inconvertible inference classes, conditioning modalities and native output geometries a single pretrained core can serve. They are deliberately silent on corpus breadth , the number of trajectories, equations and dimensions observed during pretraining, along which several of the systems listed here exceed the present work by a wide margin. Neither axis subsumes the other and an engineering domain requires both. Entries are assessed against what each class of system was demonstrated to do in the cited work.

\checkmark

demonstrated;

\bigcirc

partial, restricted or problem-specific;

\times

outside the reported setting. The table is not a claim that these architectures could not be extended along the axes they leave open; several plausibly could, and such extensions would constitute separate studies. F1 shared pretrained core; F2 structurally distinct physics; F3 heterogeneous conditioning modality and inference class; F4 native geometric and discretization diversity; F5 small-fraction extension with unchanged earlier functions and demonstrated reuse of the pretrained core against a randomized control; F6 evaluated against task-specific specialists with any accuracy cost reported.

Claim	Established in	Falsifying control applied	Scope	Independent
A reported cross-model score is a property of the model and its interface jointly, and a large fraction can reside in the interface	Section 2 protocol; Table S22	Coordinate-convention repair in the task interface with the operator unchanged	One baseline, two tasks, single seeds; demonstrates magnitude, not a general bound	$\checkmark$
One pretrained core serves three inference classes and three native output domains without rasterizing any onto a common grid	Sections 2.1 , 2.2 and 2.6	Task specialists, shared network without expert library, per-node mean field, reduced-order regression	Two-dimensional outputs, single states, geometry fixed within each task	$\checkmark$
The task interface, not the router, is the operative unit of extension	Section 2.4	Router capacity varied over several orders of magnitude; unrestricted fine-tuning; rehearsal	One new component at full data; not a proof of necessity for other architectures	$\checkmark$
Deployed predictions are unchanged under extension, by construction	Sections 2.4 and 4.4	Re-evaluation of every earlier task after each acquisition step	Structural consequence of parameter disjointness, not a learned property	$\checkmark$
Reuse of the pretrained core is conditional and must be measured, not assumed	Section 2.5	Norm-matched randomized library; dense backbone without experts; three seeds; the same frozen-core control applied to adapted MORPH	One component swept across data fractions, one audited at full data	$\checkmark$
Full-field relative $L^{2}$ error can be dominated by field level rather than spatial structure	Section 2.5	Spatial-mean decomposition; profile-aware loss audit on a matched specialist	Scale ratio of $2$ to $121$ across the tasks evaluated here	$\checkmark$

Table 2: Claims made in this work, the evidence establishing each, the control that could have falsified it, and its scope. The final column records whether the claim depends on accepting the foundation-model assessment of Section 1 . Six of the seven do not: they are results about interfaces, extension mechanisms and measurement that stand independently of how the term is adjudicated.

Panel of Figure 1 — Figure 1: One operator library across five heterogeneous systems. a , The five systems as the components their data describe, with the solver’s ground-truth field on the plane the data occupy: (i) reactor subchannel between four fuel rods, coolant temperature on the transverse plane, with a schematic vessel cutaway marking where the subchannel sits; (ii) sea-level cosmic-ray dose rate, with the input’s twelve neutron-monitor stations marked (those on the far side hidden) and one schematic air shower; (iii) lid-driven cavity, speed; (iv) heat-exchanger tube, pressure on the section and solver streamlines through the cutaway; (v) dog-bone specimen, von Mises stress. Details are given in the text. b , Model structure: task interfaces project heterogeneous inputs onto one $48\times 48$ latent grid, three routed wavelet-expert layers compute on it, and the common geometry-adaptive decoder formulation writes each field onto its native mesh; a new system trains only a private interface (input encoder, output projection, skip path, gate and geometry decoder, $3.9$ million parameters, $2.1\%$ of the pretrained model) while the $190.6$ million-parameter library and earlier interfaces stay frozen.

Figure 2: Selected comparisons and physical field structure. a, Clean DeepONet and GINO specialists, the clean joint GEODE checkpoint and training-mean controls; every checkpoint is selected on validation data disjoint from the test set. b–d, Lowest-error test cases of the paper checkpoint, preserving color bars, contours and spherical projection. These illustrate structure, not typical accuracy; complete best/worst panels remain in the supplement. GINO uses a vector lift and 30 epochs, without a matched-compute or convergence claim. Table 3 provides the broader comparison. This figure bears on requirements F1 to F4 and F6 of Section 1 ; the DeepONet specialist in panel a remains more accurate on LDC.

Method	Protocol	LDC	CosmicDose	PlasticDeform
Per-node training mean	Control	59.80	1.79	20.13
POD plus ridge	ST, clean	10.9836	0.3378	20.1056
DeepONet	ST, clean	0.236	0.0471	0.502
GINO + vector lift	ST, clean	2.195	0.0654	1.083
FNO + adapters	ST, clean	0.2708	0.0952	0.5034
WNO + adapters	ST, clean	1.1889	0.0752	2.2866

Table 3: Median physical relative

L^{2}

error (%) from completed runs. ST denotes separate single-task training; pretr. FT denotes full fine-tuning of pretrained weights; POD denotes proper orthogonal decomposition. † the clean single-task PlasticDeform run did not converge under the shared recipe (its validation loss sat at

0.90

for the first sixty epochs) and is reported as obtained; the historical specialist reached

0.34\%

. The joint GEODE row is the clean signed-distance-enabled checkpoint, selected on validation data only. DeepONet and GINO use a training-only PlasticDeform validation split; GINO uses corrected coordinates and a learned vector-to-field lift. Separate DeepONet specialists used 200 epochs and GINO 30. The additional clean baselines use up to 100 epochs, validation-based stopping and a 20-minute training/validation cap; their exact budgets and adapters are detailed in the supplement. Differing schedules and capacities prevent a matched-compute or converged-model ranking.

Figure 3: Task-dependent early routing and shared deeper computation. a, Dominant pathways derived from saved gates. b, Mean gate probabilities on a logarithmic color scale; outlined cells identify dominant experts. c, Forced-route errors and penalties relative to each task’s own hard route. d, Live soft mixing versus that hard route. e,f, Dense-backbone and no-label ablations, shown as ratios to joint GEODE and as absolute errors. Ratios below one favor the ablation. The CosmicDose ablation errors lie within the range spanned by independent training seeds of the joint model ( Table S29 ) and support no conclusion on that task; only the LDC effects exceed the seed envelope. All panels use the paper checkpoint, including the completed no-label rerun; no additional trials or uncertainty estimates are implied. This figure bears on requirement F1 of Section 1 ; gate probabilities alone do not establish necessity, which is why the forced-route and randomized-library interventions are reported alongside them.

Figure 4: Sequential acquisition with preserved earlier functions. Panel a reports the matched HeatExchanger strategy audit; panel b shows that earlier-task errors are unchanged after each addition. Panels c–d show the median-case HeatExchanger pressure and Subchannel velocity fields from the chained protocol on their native meshes (clean cold profile-aware chain, HeatExchanger

0.719\%

and Subchannel

0.183\%

). This figure tests requirement F5 of Section 1 . Preservation of earlier outputs is a structural consequence of parameter disjointness and not a learned property.

Figure 5: Transfer depends on data availability, target and field component. a, HeatExchanger data-fraction sweep. b, Specialist-to-trained-library error ratios derived from a; values above one favor reuse. c, The separate full-data audit of trained, norm-matched random and dense libraries. d, Subchannel zero-mean-profile errors before and after library randomization. e, The temperature-channel profile-loss audit on matched Subchannel specialists, normalized to the standard-loss reference. f, Effective expert counts in the adapted routers. The HeatExchanger sweep, the full-data controls and the Subchannel library audit share one protocol on the paper checkpoint. Missing runs are omitted, and the single-run comparisons do not establish statistical significance. This figure tests the reuse clause of requirement F5 of Section 1 , the clause that distinguishes a shared core from a collection of independent surrogates.

Figure 6: Query sensitivity and deployment costs. a,b, Radius sensitivity and half-query errors on the same 32 examples per task. Subsets use original nodes, not unseen geometries. c, Batch-one A40 latency: 50 synchronised calls after 15 warm-ups. Different tasks are compared, not a controlled scaling sweep. d, Log-scale storage, including measured DeepONet and GINO specialists; GINO covers three tasks only. e,f, Cumulative and marginal HeatExchanger training costs. GPU-hours are measured per-epoch times on one A40 multiplied by the epochs run; the base-task specialists are priced from a timing probe on the same hardware. Matched-GEODE storage savings do not extend to the smaller baselines in d; marginal costs are not extrapolated across tasks.

Task	$d_{\ell}$	$N_{\ell}$	$c_{\ell}$	Output domain	Mapping
LDC	90	$4{,}225$	3	Cartesian grid	boundary history $\rightarrow$ field
CosmicDose	84	$65{,}341$	1	latitude–longitude grid	sparse observations $\rightarrow$ field
PlasticDeform	101	$3{,}060$	1	finite-element mesh	loading history $\rightarrow$ field
HeatExchanger	102	$3{,}977$	3	unstructured flow slice	parameters $\rightarrow$ field
Subchannel	102	$1{,}733$	3	rod-bundle cross-section	parameters $\rightarrow$ field

Table 4: Tasks used to train and evaluate GEODE. The input dimension refers to the vector presented to the task interface before any padding. Output nodes refer to the native discretization used for the principal evaluation.

Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure S1 : Heterogeneous inputs and output domains used to evaluate GEODE. Top, representative conditioning signals for the three jointly pretrained tasks. LDC is conditioned on a time-dependent lid-velocity profile, CosmicDose receives seven daily measurements from twelve neutron-monitoring stations, and PlasticDeform is conditioned on a sign-reversing displacement history. Bottom, the five output domains: a Cartesian cavity-flow grid, a structured spherical dose grid, an irregular finite-element specimen, an unstructured heat-exchanger domain containing an internal boundary, and a nuclear-reactor subchannel bounded by four fuel rods.

Figure S2 : Detailed architecture of GEODE. Task-specific inputs are mapped to a common latent field, processed by a shared task-conditioned wavelet-expert library, and decoded on the native physical output domain. The decoder feature vector is specified in Methods. During acquisition, a new task interface and routing variables are optimized while the shared library and all previous interfaces remain fixed.

Metric	LDC	CosmicDose	PlasticDeform
Median relative $L^{2}$ error (%)	0.755	0.031	0.399
Interquartile range (%)	0.54–1.34	0.019–0.053	0.25–2.52
Mean relative $L^{2}$ error (%)	1.076	0.039	4.798
95th percentile (%)	2.679	0.089	34.67
Mean-field error, median (%)	59.80	1.79	20.13
Mean-field skill score	0.9993	0.9995	0.9935

Appendix

Table S1 : Test performance of the jointly trained GEODE model. Results use 495 LDC, 359 CosmicDose and 100 PlasticDeform test instances. The mean-field control predicts the per-node mean of the corresponding training outputs. All values are from one training run; the GEODE values are those of the clean, validation-selected checkpoint.

Figure S3 : CosmicDose predictions for the lowest- and highest-error test instances. Columns show the reference field, GEODE prediction and absolute pointwise error. The largest residuals occur in regions of steep spatial variation.

Figure S4 : LDC velocity predictions for the lowest- and highest-error test instances. Columns show the reference $v$ -velocity field, GEODE prediction and absolute pointwise error.

Figure S5 : PlasticDeform predictions for the lowest- and highest-error test instances. Columns show reference von Mises stress, GEODE prediction and absolute pointwise error. The relative error of the worst case is amplified by the small target-field magnitude.

Figure S6 : Predicted and reference field content across spatial scales. Radially averaged spectra are computed over ten test instances. The LDC prediction modestly underestimates the highest-wavenumber content.

Task	Diagnostic	Result
LDC	Predicted/reference mean absolute divergence	0.995
CosmicDose	Predicted/reference north–south asymmetry	0.0159/0.0159
PlasticDeform	Fraction of nodes with negative von Mises stress	$0$
PlasticDeform	Minimum predicted von Mises stress	$2.49$

Appendix

Table S2 : Selected physical diagnostics not included in the training objective. These empirical diagnostics do not imply exact satisfaction of conservation laws or constitutive constraints.

Method	Interface evidence	Status for the present datasets
DeepONet	Parameter/history branch and coordinate trunk	Clean evaluation on all five tasks; Plastic validation held out from training.
GINO	Official operator with our learned vector-to-field lift	Corrected-coordinate, clean-split evaluation completed on all three base tasks (30 epochs).
FNO, WNO, GNOT, Transolver	Task-specific operator families	Clean bounded evaluations completed with documented vector-input adapters ( Table S23 ).
MPP, Poseidon	Published field-history interfaces	Present-dataset execution and conditioning contracts not yet verified.
DPOT	Public Fourier-attention operator transformer and checkpoints	Adapted execution verified on the tasks reported in Table S25 : learned vector lift and native-query readout. Native field-history input is not claimed; see Table S25 .
MORPH	Public multidimensional field-tensor model and checkpoints	Adapted execution verified on the tasks reported in Table S24 : learned vector lift and native-query readout. Native field-history input is not claimed; see Table S24 .

Appendix

Table S3 : External-model evidence and evaluation status. Source review is not an executed benchmark; no numerical score is inferred from compatibility.

Method	Protocol	LDC	CosmicDose	PlasticDeform
Per-node training mean	Control	59.80	1.79	20.13
POD plus ridge	ST, clean	10.9836	0.3378	20.1056
DeepONet	ST, clean	0.236	0.0471	0.502
GINO + vector lift	ST, clean	2.195	0.0654	1.083
FNO + adapters	ST, clean	0.2708	0.0952	0.5034
WNO + adapters	ST, clean	1.1889	0.0752	2.2866

Appendix

Table S4 : Median physical relative

L^{2}

error (%) from completed runs. ST denotes separate single-task training; pretr. FT denotes full fine-tuning of pretrained weights; POD denotes proper orthogonal decomposition. † The clean single-task PlasticDeform run did not converge under the shared recipe (its validation loss sat at

0.90

for the first sixty epochs) and is reported as obtained; the historical specialist reached

0.34\%

. The joint GEODE row is the clean signed-distance-enabled checkpoint, selected on validation data only. DeepONet and GINO use a training-only PlasticDeform validation split; GINO uses corrected coordinates and a learned vector-to-field lift. NE means not yet evaluated, not incompatible. Separate DeepONet specialists used 200 epochs and GINO 30. The additional clean baselines use up to 100 epochs, validation-based stopping and a 20-minute training/validation cap; their exact budgets and adapters are detailed in the supplement. Differing schedules and capacities prevent a matched-compute or converged-model ranking.

Figure S7 : Task-dependent routing through the shared wavelet-expert library. Mean gate probability is shown for each expert and layer on a logarithmic color scale. The dominant expert is outlined.

Task	Layer 1	Layer 2	Layer 3
LDC	db3 (1.1)	db6 (1.0)	db3 (1.0)
CosmicDose	db6 (3.0)	db6 (1.0)	db3 (1.0)
PlasticDeform	db8 (6.5)	db6 (1.0)	db3 (1.0)
HeatExchanger	db5 (7.1)	db1 (2.0)	db5 (1.9)
Subchannel h	db8 (3.9)	db5 (2.0)	db6 (1.0)

Appendix

Table S5 : Dominant expert and effective expert count. The value in parentheses is the effective number of experts. The first three rows are jointly pretrained tasks; HeatExchanger and Subchannel are acquired tasks evaluated with their trained interfaces.

Evaluated task	Live gate	LDC route	Cosmic route	Plastic route	Heat exch. route
LDC	0.755	0.722	88.3	71.3	58.3
CosmicDose	0.031	1.41	0.441	1.54	1.72
PlasticDeform	0.399	16.1	21.8	1.44	46.7
HeatExchanger	0.719	19.8	14.3	9.49	3.64

Appendix

Table S6 : Effect of routing interventions. Entries are median relative

L^{2}

errors (%). Live gate uses the learned soft probabilities. The HeatExchanger route is that of its private interface after acquisition; the pretrained tasks’ values are identical on the three-task checkpoint.

Figure S8 : Feature maps generated by the first-layer wavelet experts. One latent channel is shown for a representative instance from each task. Each panel is annotated with its gate weight, and the dominant expert is highlighted. The panels visualize differentiated latent computations but do not by themselves establish causality.

Model	LDC	CosmicDose	PlasticDeform
Dense backbone	4.75	0.102	0.379
Expert mixture without task label	4.33	0.104	0.331
GEODE	0.755	0.031	0.399

Appendix

Table S7 : Matched routing ablations. Entries are median relative

L^{2}

errors (%). The dense backbone replaces the eight-expert mixture with one expert (

22

million parameters). Without the label, all three tasks converge on one route (db7, db7, db1).

Training strategy	Parameters	Heat exch.	LDC	Cosmic	Plastic
Specialist from scratch	all	0.502	NA	NA	NA
Embedding and shared gate	2.3M	10.15	22.6	0.306	6.0
Task interface and gate	3.9M	0.719	0.755	0.031	0.399
Full fine-tuning	190.6M	9.38	16.4	0.905	5.65
Rehearsal (all earlier data)	190.6M	0.661	2.69	0.053	1.12

Appendix

Table S8 : Component audit for heat-exchanger acquisition. Errors are median relative

L^{2}

errors (%). The final three columns report earlier-task errors after adaptation. NA indicates that the specialist does not contain those tasks. All rows use the same 100-epoch, full-data protocol from the paper checkpoint; the full fine-tuning row uses batch size 8 on an A100.

Task	Stage	Specialist	GEODE	Ratio
LDC	Pretrained	NA	0.755	NA
CosmicDose	Pretrained	NA	0.031	NA
PlasticDeform	Pretrained	NA	0.399	NA
HeatExchanger	Sequential step 1	0.507	0.719	$1.42\times$
Subchannel	Sequential step 2	0.034	0.083	$2.4\times$

Appendix

Table S9 : Performance after two sequential acquisitions. Errors are measured after Subchannel has been added. No earlier data are replayed and no shared expert parameter is updated. The HeatExchanger step is the full-data interface run of the component audit and is unchanged by the second step because its interface is frozen; specialists use 200 epochs. The Subchannel entry is the standard-loss variant of the step; the profile-aware variant reaches

0.183\%

( Table S13 ).

Figure S9 : Predictions for the sequentially acquired tasks. Columns show the reference field, GEODE prediction and absolute pointwise error for a median-error test case after both acquisition steps in the clean profile-aware chain (HeatExchanger

0.719\%

, Subchannel

0.183\%

). HeatExchanger pressure is shown on its native unstructured CFD mesh; Subchannel axial velocity is shown on the cross-shaped coolant domain.

Adaptation strategy	1%	10%	50%	100%
Specialist from scratch	8.92	0.707	0.494	0.502
Task embedding and gate only	10.27	9.67	9.49	10.15
Interface, trained library	5.88	1.32	0.877	0.719
Interface, random library	7.50	1.37	NE	0.790

Appendix

Table S10 : Heat-exchanger adaptation as a function of training-data availability. Entries are median relative

L^{2}

errors (%) under one matched protocol (100 epochs from the paper checkpoint). NE denotes an experiment not performed. Gate-only adaptation also degrades the pretrained tasks once the fraction exceeds

1\%

(LDC

0.76

3.4

57.9

and

22.6\%

1

10

50

and

100\%

Frozen backbone	LDC	Cosmic	Plastic	Heat exch.	Subchannel
Trained wavelet experts	0.755	0.031	0.399	0.719	0.121
Norm-matched random experts	63.7	1.75	45.8	0.790	4.524
Dense backbone without experts	4.749	0.102	0.379	0.811	NA

Appendix

Table S11 : Contribution of learned expert content. Entries are median physical-space relative

L^{2}

errors (%). The dense backbone is adapted from its own three-task checkpoint. All entries are single runs on the paper checkpoint.

Frozen library	Channel 1	Channel 2	Channel 3	Level error (K)
Trained experts	0.75	7.62	0.27	0.47
Norm-matched random experts	0.76	13.6	0.56	25.6

Appendix

Table S12 : Subchannel spatial-profile audit. Each channel is decomposed into its spatial mean and zero-mean profile. Values are profile errors (%), with the temperature-level error in kelvin in the last column. Paper checkpoint, cold-start protocol of the historical run, single runs.

Protocol	Metric	Standard loss	Profile-aware loss
Subchannel specialists	Temperature profile error (%)	3.12	1.74
Subchannel specialists	Headline error (%)	0.034	0.034
Continual-acquisition chain	Subchannel error (%)	0.083	0.183

Appendix

Table S13 : Effect of a spatial-profile loss. The first two rows compare two Subchannel specialists trained identically for 200 epochs except for the profile term; the chain row compares the two variants of the regenerated chain step on the paper checkpoint. On the earlier checkpoint the chain row read

0.161

and

0.063

, that is, the profile term helped there and hurts here; single runs in every cell.

Query set	LDC	CosmicDose	PlasticDeform
Complete mesh, matched $n=32$	0.862	0.0745	0.338
Random $50\%$ , same $n=32$	0.878	0.0774	0.345

Appendix

Table S14 : Evaluation after reducing the output query set. Entries are median relative

L^{2}

errors (%).

Figure S10 : Distribution of decoder neighborhood sizes. The nominal task-specific radii give mean neighbor counts of

17.5

21.2

and

20.9

for LDC, CosmicDose and PlasticDeform.

Radius multiplier	LDC	CosmicDose	PlasticDeform
$0.75\times$	1.12	0.0742	0.327
$1.00\times$	0.862	0.0745	0.338
$1.50\times$	1.58	0.0770	0.418
$2.00\times$	2.02	0.0793	0.499

Appendix

Table S15 : Sensitivity to the decoder neighborhood radius. Entries are median relative

L^{2}

errors (%) on the same 32 examples per task; all learned parameters are fixed.

Figure S11 : Inference cost as the output discretization grows. Measurements use batch size one in float32 on one NVIDIA A40. PlasticDeform requires

91

ms for

3{,}060

nodes, LDC requires

91

ms for

4{,}225

nodes, and CosmicDose requires

114

ms for

65{,}341

nodes.

Scope	Configuration	Parameters	GPU-hours	Artifacts
Three tasks	Three specialists	571.8M	26.9	3
Three tasks	GEODE	190.6M	26.4	1
Add HeatExchanger	New specialist	$+190.6$ M	$+1.5$	$+1$
Add HeatExchanger	New interface	$+3.9$ M	$+4.1$	0
Five tasks	Five specialists	953.0M	31.9	5
Five tasks	GEODE plus two interfaces	198.5M	40.4	1

Appendix

Table S16 : System-level cost of matched GEODE specialists and the shared model. Parameter counts are totals needed to serve the indicated task set. The Artifacts column counts separately deployed parameter sets. GPU-hours are measured per-epoch times of the clean runs on one NVIDIA A40 multiplied by the epochs run (

120

for the base and its specialists,

100

for the HeatExchanger interface,

200

for the HeatExchanger and Subchannel specialists and the Subchannel interface); the three base-task specialists are priced from a timing probe of the same configurations on the same A40. The marginal rows refer specifically to HeatExchanger, not to a universal cost per new task.

Task	Reference solver	GEODE inference	Indicative speed-up
LDC	approximately 25 min	91 ms	approximately $16{,}500\times$
CosmicDose	approximately 61 s	114 ms	approximately $535\times$
PlasticDeform	approximately 5 min	91 ms	approximately $3{,}300\times$
HeatExchanger	approximately 18 min	NA	NA
Subchannel	approximately 12 min	NA	NA

Appendix

Table S17 : Indicative simulation and surrogate evaluation times. Reference-solver times are wall-clock times per sample on NCSA Delta CPU nodes (dual AMD EPYC 7763, 128 cores per node); surrogate times are the A40 latencies of Figure S11 . The two use different software and hardware and are not a controlled benchmark. NA indicates that an exact matched inference latency was not reported for that task.

Task	$\bm{\epsilon=0.01}$	$\bm{\epsilon=0.05}$	$\bm{\epsilon=0.10}$
LDC	0.99	1.28	2.46
CosmicDose	1.06	1.33	1.80
PlasticDeform	1.03	1.11	1.27

Appendix

Table S18 : Error inflation under additive input perturbations. Entries are the ratio of perturbed to unperturbed median error for perturbation magnitude

\epsilon

in normalized input space.

Evaluated task	Correct label	Wrong label 1	Wrong label 2
LDC	0.753	82.8	64.0
CosmicDose	0.074	0.561	0.396
PlasticDeform	0.320	16.0	13.0

Appendix

Table S19 : Effect of supplying the wrong task label. Entries are median relative

L^{2}

errors (%) under the task-label intervention protocol. Each wrong-label column supplies the label of the indicated alternative task while retaining the evaluated task’s input.

Task	Test $n$	Median (%)	IQR (%)	Parameters
LDC	495	0.2357	0.2002–0.3726	418,819
CosmicDose	359	0.04705	0.02300–0.06685	351,489
PlasticDeform	100	0.5022	0.2997–2.2549	355,841
HeatExchanger	173	0.5718	0.2930–0.8159	421,891
Subchannel	1,000	0.1595	0.07644–0.2850	421,891

Appendix

Table S20: Completed DeepONet specialists on all five tasks. Errors are physical relative

L^{2}

percentages. These are separate models, not sequential additions to a shared library. The updated runs use 200 epochs, batch size 32 and validation-selected weights. PlasticDeform uses 13,410 training and 1,490 validation examples; its test set is reserved for evaluation. Different schedules prevent a matched-compute claim.

Task	Test $n$	Median (%)	IQR (%)	P95 (%)	Parameters
LDC	495	2.1951	1.6083–2.8930	3.7822	847,287
CosmicDose	359	0.0654	0.0497–0.0914	0.1462	822,453
PlasticDeform	100	1.0825	0.7359–2.8396	25.6681	892,085

Appendix

Table S21: Corrected GINO with a learned vector-to-field lift. Clean validation-selected results after 30 epochs, evaluated at all output nodes. IQR and percentiles describe test instances, not training-seed variability. Parameters count complex weights as two real scalars.

Task	Test $n$	Median (%)	Qualification
LDC	495	59.21	Coordinate mismatch; most queries lose source neighbors.
CosmicDose	359	0.08370	Coordinate mismatch compresses queries near the latent-grid center.
PlasticDeform	100	19.27	Coordinate mismatch and test-based checkpoint selection.

Appendix

Table S22: GINO implementation diagnostics, excluded from a final accuracy ranking. All three runs used 30 epochs and inconsistent query-coordinate normalization. PlasticDeform additionally used test-based checkpoint selection. These numbers are retained as debugging evidence, not an estimate of a correctly implemented GINO benchmark.

Method	Task	Median (%)	Epochs	Parameters (M)
Shared DeepONet	LDC	0.7425	100/100 E	0.422
Shared DeepONet	CosmicDose	0.0529	100/100 E	0.422
Shared DeepONet	PlasticDeform	0.7253	100/100 E	0.422
FNO + adapters	LDC	0.2708	100/95 E	2.387
FNO + adapters	CosmicDose	0.0952	30/5 P	2.289
FNO + adapters	PlasticDeform	0.5034	100/95 E	2.567

Appendix

Table S23: Verified clean table-completion runs. Median physical relative

L^{2}

errors are percentages. Epochs show completed/selected epochs; W denotes the wall-time cap, P validation-plateau stopping and E completion of the epoch budget. Model size includes adapters and counts complex weights as two real scalars. Joint model size is repeated across its task rows, not summed.

Task	Core init.	Updates	Median	IQR	p95	Epochs
LDC	Pretrained	Adapters	4.7442	3.353–9.859	20.172	95/95
LDC	Random	Adapters	35.7849	23.347–46.366	52.462	85/85
LDC	Pretrained	All	1.1531	0.899–2.346	9.595	60/40
LDC	Random	All	1.0889	0.806–2.263	9.820	95/75
CosmicDose	Pretrained	All	0.0787	0.048–0.120	0.248	55/35
CosmicDose	Random	All	0.0632	0.046–0.088	0.141	30/10

Appendix

Table S24: Adapted MORPH-Ti controls. Errors are median physical relative

L^{2}

percentages, with linearly interpolated interquartile ranges (IQR) and 95th percentiles. Epochs give completed/selected budgets. These are single-seed results with validation-based stopping, not converged or matched-compute rankings.

Task	Core init.	Updates	Median	IQR	p95	Epochs
LDC	Pretrained	All	0.1548	0.101–0.421	1.839	85/85
LDC	Random	All	0.1991	0.134–0.375	0.833	84/84
CosmicDose	Pretrained	All	0.0467	0.025–0.069	0.100	60/40
CosmicDose	Random	All	0.0521	0.032–0.074	0.103	45/25
PlasticDeform	Pretrained	All	0.8485	0.548–3.801	50.87	25/25
PlasticDeform	Random	All	0.8354	0.618–6.948	34.61	25/20

Appendix

Table S25: Adapted DPOT-Ti controls. Errors are median physical relative

L^{2}

percentages, with linearly interpolated interquartile ranges (IQR) and 95th percentiles. Epochs give completed/selected budgets; LDC and PlasticDeform arms stopped at the wall-clock budget, CosmicDose arms at a validation plateau. These are single-seed results with validation-selected checkpoints and float32 re-evaluation; they are bounded-budget comparisons, not converged or matched-compute results.

Arm	Total (M)	Trainable (M)	Epochs	GPU-h
Shared DeepONet joint	0.422403	0.422403	100/100	0.072222
fno L	2.386995	2.386995	100/95	0.058333
fno C	2.288561	2.288561	30/5	0.024444
fno P	2.567089	2.567089	100/95	0.182500
wno L	5.050179	5.050179	85/65	0.119444
wno C	4.951617	4.951617	40/20	0.074444

Appendix

Table S26 : Measured allocations for the 28 additional baseline arms. Sizes count complex weights as two real scalars and include adapters. Epochs are completed/selected; a wall-limited final epoch can be partial. GPU-hours are one-GPU Slurm wall time including startup, validation and final evaluation, excluding CPU gates/replay. They are not pure optimizer time or equal-compute budgets. L/C/P denote LDC/CosmicDose/PlasticDeform; pre/rnd denote pretrained/random cores and frz/full denote frozen/full core adaptation.

Task	Rank	Ridge	Median	Mean	P95
LDC	64	0.001	10.9836	11.0427	21.5412
CosmicDose	8	1000	0.3378	0.3492	0.7192
PlasticDeform	8	0.1	20.1056	97.2459	590.4831

Appendix

Table S27: Clean POD-plus-ridge results. Errors are physical relative

L^{2}

percentages; quantiles use linear interpolation. This is a linear control, using the same clean train/validation/test partitions as the external neural baselines.

Gating objective	First-layer routing	LDC	CosmicDose	PlasticDeform
Task-bias only	shared	2.24	0.080	0.37
$+$ JS divergence	distinct	2.13	0.086	0.45
$+$ entropy sharpening (GEODE)	distinct	1.09	0.080	0.35

Appendix

Table S28 : Historical-checkpoint effect of the gating objective on first-layer routing and test accuracy (denormalized median relative

L^{2}

, %). “Distinct” indicates the three tasks select three different first-layer experts.

Figure S12 : Heat exchanger: pressure and both in-plane velocity components, median test case, on the solver’s own CFD mesh. The error in every channel concentrates along the internal plate, where the field changes most sharply over the shortest distance.

Figure S13 : Subchannel: axial velocity, coolant temperature and the turbulence quantity, median test case. The temperature panel illustrates the difficulty described in Section 2.4 : against a $574$ K mean, the $21$ K of wall-to-core structure is barely visible on a shared color scale, which is why the relative $L^{2}$ of that channel is a poor guide to whether its shape has been learned. The turbulence quantity peaks against the rod walls and that peak is recovered.

Figure S14 : Gates of the two continually acquired tasks, read from the model after both adaptation steps. Both mix far more experts than any jointly pretrained task, the heat exchanger reaching $7.3$ effective experts in the first and third layers, against $1.1$ to $2.3$ (first layer) and $1.0$ (third) for the pretrained tasks. Section 2.3 attributes this to the frozen library breaking the expert-gate co-adaptation that otherwise drives routing to collapse. Two cautions in reading these panels. Where a row is close to flat, the boxed maximum is only weakly dominant and should not be read as a selection: the heat exchanger’s first layer runs from $0.073$ to $0.226$ . And the second layer sits at exactly $2.0$ effective experts in every continually acquired arm, including the randomized control, which indicates that this split is imposed by the deep-layer term in the gating objective rather than learned from the data.

Figure S15 : Gates on the same held-out task under three different libraries. Over the trained wavelet library the first and third layers are close to uniform; over a library randomized at matched Frobenius norm the gate concentrates to $2.5$ effective experts; over a backbone with no expert mixture the single available expert carries all the mass, which is a trivial rather than a learned collapse and is shown for completeness. Dense mixing therefore appears only when the library holds distinct learned functions, so the gate is sensitive to library content and not only to the task it serves.

Figure S16 : Gates for lid-driven cavity flow re-presented to the trained model as a new task on a reshuffled split, across two random seeds and with the routing-separation loss enabled and disabled. All four runs converge on db6 followed by db3 in the first two layers, and none recovers the assignment db2 followed by db8 that joint training gave this task. The routing is therefore reproducible without being unique, and db6 is the expert that joint training assigned to the elastoplastic problem.

Seed	Cavity	Dose	Specimen
0 (reported)	0.755 (0.54–1.34)	0.031 (0.019–0.053)	0.399 (0.25–2.52)
1	2.87 (2.13–3.83)	0.100 (0.058–0.136)	0.42 (0.30–1.79)
2	1.97 (1.34–3.10)	0.134 (0.084–0.179)	0.39 (0.27–2.32)
Interface	Heat exchanger	Subchannel, trained library	Subchannel, random library
0 (reported)	0.719	0.121	4.524
1	0.657	0.068	4.653

Appendix

Table S29 : Training seeds. Median relative

L^{2}

error in percent over the held-out cases (495 cavity, 359 dose, 100 specimen; 100 heat-exchanger and 100 subchannel cases for the interfaces), with interquartile ranges in parentheses for the joint model. Seed 0 is the run reported throughout.

Figure S17 : Per-sample relative

L^{2}

error distributions across the full test set for each task (LDC

n{=}495

, CosmicDose

n{=}359

, PlasticDeform

n{=}100

). CosmicDose and LDC are tightly concentrated, whereas PlasticDeform is heavy-tailed (median

0.399\%

, mean

4.80\%

, 95th percentile

34.7\%

using linear interpolation), motivating the use of the median alongside the mean in Table S1 .

Figure S18 : LDC predictions for all three output channels ( $u$ , $v$ , $p$ ) on a representative test sample: ground truth ( left ), GEODE prediction ( center ) and absolute error ( right ). The model resolves the velocity components and the pressure field jointly within a single set of weights.

Physical Foundation Models

See all 14 papers

Mar 4, 2026

LUMINA: Foundation Models for Topology Transferable ACOPF

Foundation models in general promise to accelerate scientific computation by learning reusable representations across problem instances, yet constrained scientific systems, where predictions must satisfy physical laws and safety limits, pose unique challenges that stress conventional training paradigms. We derive design principles for constrained scientific foundation models through systematic investigation of AC optimal power flow (ACOPF), a representative optimization problem in power grid operations where power balance equations and operational constraints are non-negotiable. Through controlled experiments spanning architectures, training objectives, and system diversity, we extract three empirically grounded principles governing scientific foundation model design. These principles characterize three design trade-offs: learning physics-invariant representations while respecting system-specific constraints, optimizing accuracy while ensuring constraint satisfaction, and ensuring reliability in high-impact operating regimes. We present the LUMINA framework, including data processing and training pipelines to support reproducible research on physics-informed, feasibility-aware foundation models across scientific applications.

Yijiang Li, Zeeshan Memon, Hongwei Jin +7

Argonne National Laboratory, Lemont, IL 60439, USA · Emory University, Atlanta, GA 30322, USA · Sogang University, Seoul, South Korea

Physical Foundation Models Scientific ML Physics-Informed ML Optimal Power Flow

Ask PDF

Aug 27, 2026

Towards Large-Scale Heterogeneous Data Organization for Scientific Foundation Models: A Nuclear Fusion Case Study

Training effective foundation models requires massive and organized datasets, yet scientific domains such as nuclear fusion present unique challenges due to largely heterogeneous and sparse data. Here we characterize the data used in developing such a model: with over 20 sensor types spanning 5 orders of magnitude in sampling rate, mixed tensor structures (point measurements, spectrograms, images), and nonstationary physics. We analyze our input complexity and discuss trade-offs between temporal context and frequency resolution. Our analysis provides a template for representing multi-modal fluctuation data at scale, with implications for both multi-modal control systems and nuclear fusion.

Nathaniel Chen, Kouroche Bouchiat, Peter Steiner +3

Mechanical and Aerospace Engineering Princeton University Princeton, NJ 08544, USA · Princeton University Princeton, NJ 08544, USA · Princeton Plasma Physics Laboratory Princeton University Princeton, NJ 08544, USA

Multimodal Learning

Ask PDF

May 14, 2026

Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing

Scaling Scientific Machine Learning (SciML) toward universal foundation models is bottlenecked by negative transfer: the simultaneous co-training of disparate partial differential equation (PDE) regimes can induce gradient conflict, unstable optimization, and plasticity loss in dense neural operators. In particular, broadband open-channel fluid dynamics and boundary-dominated porous media flows impose incompatible spectral and geometric demands on a single dense parameter path. We introduce Shodh-MoE, a sparse-activated latent transformer architecture for multi-physics transport. Shodh-MoE operates on compressed 16^3 physical latents produced by a physics-informed autoencoder with an intra-tokenizer Helmholtz-style velocity parameterization, restricting decoded states to divergence-free velocity manifolds. The model guarantees exact mass conservation, achieving a physically verifiable velocity divergence of ~2.8 x 10^-10 (evaluated post-hoc in FP64) on 128^3 grids. A Top-1 soft-semantic router dynamically assigns localized latent patches to expert subnetworks, enabling specialized parameter paths for distinct physical mechanisms while preserving shared experts for universal symmetries. In a 20,000-step distributed pretraining run over mixed three-dimensional physical tensors, routing telemetry shows autonomous domain bifurcation: held-out validation tokens from the open-channel domain route exclusively to Expert 0, while porous-media tokens route exclusively to Expert 1. The model converges simultaneously across both regimes, achieving latent validation MSEs of 2.46 x 10^-5 and 9.76 x 10^-6, and decoded physical MSEs of 2.48 x 10^-6 and 1.76 x 10^-6. These results support sparse expert routing as a practical architectural mechanism for mitigating multi-physics interference in universal neural operators.

Ellwil Sharma, Arastu Sharma

Shodh AI

Sparse Mixture-of-Experts PDE Operator Learning Operator Learning Physics-Informed Neural Operators Negative Transfer

Ask PDF

Abstract

Physical Foundation Models

Explore similar work

LUMINA: Foundation Models for Topology Transferable ACOPF

Towards Large-Scale Heterogeneous Data Organization for Scientific Foundation Models: A Nuclear Fusion Case Study

Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routing