cs.LGMar 4, 2026

LUMINA: Foundation Models for Topology Transferable ACOPF

Authors: Yijiang Li, Zeeshan Memon, Hongwei Jin, Stefano Fenu, Keunju Song, Sunash B Sharma, Parfait Gasana, Hongseok Kim, 

Organizations: Argonne National Laboratory, Lemont, IL 60439, USA · Emory University, Atlanta, GA 30322, USA · Sogang University, Seoul, South Korea

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Foundation models in general promise to accelerate scientific computation by learning reusable representations across problem instances, yet constrained scientific systems, where predictions must satisfy physical laws and safety limits, pose unique challenges that stress conventional training paradigms. We derive design principles for constrained scientific foundation models through systematic investigation of AC optimal power flow (ACOPF), a representative optimization problem in power grid operations where power balance equations and operational constraints are non-negotiable. Through controlled experiments spanning architectures, training objectives, and system diversity, we extract three empirically grounded principles governing scientific foundation model design. These principles characterize three design trade-offs: learning physics-invariant representations while respecting system-specific constraints, optimizing accuracy while ensuring constraint satisfaction, and ensuring reliability in high-impact operating regimes. We present the LUMINA framework, including data processing and training pipelines to support reproducible research on physics-informed, feasibility-aware foundation models across scientific applications.

Figures & tables

Figure 1: Architecture comparison on single-topology (top) vs. multi-topology (bottom) pretraining. cases considered are case30, case57, and case118. Models trained with single topology are evaluated on the topology it is trained, while models trained jointly on multi-topology are evaluated on the each of the topology separately. Heterogeneous architectures generally demonstrate better performances than homogeneous models in both solution quality and constraint satisfaction, especially in multi-topology training.
 Trans.+MSEHGT+MSE
Training → EvalOPF Sol. Err.Viol.OPF Sol. Err.Viol.
case30 → case574.7435.0535.2992.060
case30 → case1181.9129.1182.0303.766
case{57,118} → case301.1204.8530.46932.286
case{30,118} → case575.6771.9905.63371.4781
case{30,57} → case1182.01126.2956.17312.26
Table 1: Zero-shot transfer comparing Transformer and HGT architectures on single- and multi-topology pretraining. Best performance for each training→evaluation scenario shown in bold.
CaseTotal Violation NormTraining Steps
 ScratchFinetunedScratchFinetuned
1188.9866.732_{{\color[rgb]{0,0,1}(-25.1\%)}}3,000,0001{,}535{,}008_{{\color[rgb]{0,0,1}(-48.8\%)}}
50015.38714.910_{{\color[rgb]{0,0,1}(-3.1\%)}}1,310,048215{,}008_{{\color[rgb]{0,0,1}(-83.6\%)}}
Table 2: Convergence comparison between training from scratch and fine-tuning across grid sizes. Improvements in final violation and training steps are shown as blue subscripts.
Figure 2: Constraint violation convergence on case500: fine-tuning vs. training from scratch
Figure 3: Training time vs. case size with and without mixed precision training (BF16)
Figure 4: Loss function comparison across two selected architectures (one homogeneous and one heterogeneous) under equal training budgets. Top panels: single-topology pretraining. Bottom panels: multi-topology training jointly on case30, case57. and case118. Each architecture is paired with three loss functions (MSE, AL, VBL). Models trained with single topology are evaluated on the topology it is trained, while models trained jointly on multi-topology are evaluated on the each of the topology separately. Constraint-aware loss functions, AL and VBL, achieve signficantly better constraint violations compared to MSE while AL performs slightly better than VBL.
 HGT+MSEHGT+AL
 OPF Sol. Err.Viol.OPF Sol. Err.Viol.
case30 → case575.2992.0605.3110.230
case30 → case1182.0303.7662.0301.262
case{57,118} → case300.46932.2860.5470.239
case{30,118} → case575.63371.47815.2250.018
case{30,57} → case1186.17312.262.8370.907
Table 3: Zero-shot transfer comparing MSE and AL on single- and multi-topology pretraining. Best performance for each training→evaluation scenario shown in bold.
Figure 5: PCA components of activation for the top layer of convolutions in HGT trained on AL (left) vs MSE (right) losses. We see that both losses lead to the model capturing physical system load, but AL enforces a much more non-linear structure on the model’s internal representation.
Figure 6: Regime and structural stress tests reveal architecture-dependent failure modes. (a) Model error increases with system load across all architectures, but heterogeneous models (orange) generalize to high-load extremes significantly better than homogeneous models (blue). (b) Model error across categories with respect to node degree, up to case2000 with degree normalized with respect to case maximum degree. Model error by node correlates with node degree ( r=0.51r=0.51 ).
More in this topic

Physical Foundation Models

See all 14 papers

Explore similar work

CardsList

  1. GridSFM: A Foundation Model for Solving AC Optimal Power Flow

    Sep 24, 2026Luke Bhan, Weiwei Yang, Margaret Capetz +1Neural Surrogate ModelingOptimal Power Flow

    University of California, San Diego · Microsoft Research · University of Washington

  2. LUMINA: A Grid Foundation Model for Benchmarking AC Optimal Power Flow Surrogate Learning

    May 4, 2026Hongwei Jin, Keunju Song, Zeeshan Memon +5Neural Surrogate ModelingBenchmark Design

    Mathematics and Computer Science Division, Argonne National Laboratory · Department of Electronic Engineering, Sogang University · Department of Computer Science, Emory University +1

  3. Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models

    Mar 24, 2025Serge Kotchourko, Amin Totounferoush, Michael W. Mahoney +1PDE Surrogate ModelingPhysics-Informed ML

    University of Stuttgart, Germany · ICSI, LBNL, and University of California, Berkeley, USA