Optimal trajectory-guided stochastic co-optimization for e-fuel system design and real-time operation

Jeongdong Kim; Minsu Kim; Jonggeol Na; Junghwan Kim

cs.LGMar 3, 2026

Optimal trajectory-guided stochastic co-optimization for e-fuel system design and real-time operation

Authors: Jeongdong Kim, Minsu Kim, Jonggeol Na, Junghwan Kim

Organizations: Department of Chemical Engineering , Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, 02139, Massachusetts, United States. · Department of Chemical and Biomolecular Engineering, Yonsei University, 50 Yonsei-ro, Seoul, 03722, Republic of Korea. · Department of Chemical Engineering and Materials Science, Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea. · Graduate Program in System Health Science and Engineering, Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea. · Institute for Multiscale Matter and Systems (IMMS), Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea.

Abstract

E-fuels are promising long-term energy carriers supporting the net-zero transition. However, the large combinatorial design-operation spaces under renewable uncertainty make the use of mathematical programming impractical for co-optimizing e-fuel production systems. Here, we present MasCOR, a machine-learning-assisted co-optimization framework that learns from global operational trajectories. By encoding system design and renewable trends, a single MasCOR agent generalizes dynamic operation across diverse configurations and scenarios, substantially simplifying design-operation co-optimization under uncertainty. Benchmark comparisons against state-of-the-art reinforcement learning baselines demonstrate near-optimal performance, while computational costs are substantially lower than those of mathematical programming, enabling rapid parallel evaluation of designs within the co-optimization loop. This framework enables rapid screening of feasible design spaces together with corresponding operational policies. When applied to four potential European sites targeting e-methanol production, MasCOR shows that most locations benefit from reducing system load below 50 MW to achieve carbon-neutral methanol production, with production costs of 1.0-1.2 USD per kg. In contrast, Dunkirk (France), with limited renewable availability and high grid prices, favors system loads above 200 MW and expanded storage to exploit dynamic grid exchange and hydrogen sales to the market. These results underscore the value of the MasCOR framework for site-specific guidance from system design to real-time operation.

Figures & tables

Figure 1: MasCOR: ML-assisted stochastic co-optimization framework for renewable power management systems . a , Generation of an oracle dataset by combining a renewable scenario generative model with LP-based operational optimization across sampled designs. b , Training of transformer-based actor–critic agents to learn optimal operational policies and two future performance metrics, cumulative profit (return-to-go, $RTG_{t}$ ) and net carbon emissions (carbon-to-go, $CTG_{t}$ ), conditioned on renewable trends $E$ and system design $D$ . c , MasCOR infers distributions of profit and net carbon emissions ( $CTG_{0}$ ) for each design using the generative model and trained agent, selecting the design–policy pair that minimizes production cost and net carbon emissions while satisfying a chance constraint on positive carbon emissions ( $CTG_{0}>0$ ). d , MasCOR generates future renewable trend sets $E$ and performs online operation conditioned on these trends and the optimal design $D^{*}$ , ensuring robust performance under uncertainty. Details of the architecture, algorithms, and benchmark comparisons of the two ML models are provided in Supplementary Notes 2, 3 .

Figure 2: Performance evaluation of the MasCOR framework for renewable data generation and dynamic operational problem solving . a , T-SNE embedding of synthetic renewable profiles generated by the generative model, compared with training and validation wind datasets, along with distributions of key operational features. b , Computational efficiency of the proposed MasCOR agent compared with the benchmark LP solver (Gurobi) for parallel LP problem solving. c , Dynamic operational performance comparison of benchmark data–driven models and the proposed MasCOR agent across different design categories. Details of computational efficiency and benchmark comparison experiments are provided in ( Supplementary Notes 3.4 )

Figure 3: Regional design strategies and performance trade-offs in co-optimized power-to-methanol systems . a , Differences in the average renewable power availability and grid electricity prices across Dunkirk, Skive, Fredericia, and Weener (distributional details provided in Supplementary Figs. S12 ). b , Capacity shares of methanol (MeOH) production, PEMEC units, and BESS and CHT installations in representative Pareto-optimal designs, together with overall system size. c , Pareto-optimal capacity progression of PEMEC units, storage units (BESS and CHT), and MeOH production. d , Pareto frontiers of co-optimization results showing the trade-offs between the expected LCOM and net carbon emissions. MasCOR identifies two distinct design strategies for reducing carbon emission among the Pareto-optimal solutions: (i) an SE regime, characterized by increasing storage capacity, and (ii) a PR regime, characterized by decreasing MeOH production capacity under minimum PEMEC and storage capacity conditions (below 10 MW). In most regions, Pareto-optimal points shift from SE to PR as the expected net carbon emissions decrease below –50 ton/month. UQ results and full design specifications of Pareto points are provided in Supplementary Fig. S13 .

Figure 4: Regional operating strategies under different optimal design regimes . a , Capacity shares (%) of MeOH synthesis and PEMEC, BESS, and CHT units for the four representative Pareto-optimal designs illustrated in Fig. 3 . The dotted line indicates system scale. b , Hourly operating strategies for optimal designs across regions. Distributions are obtained by averaging monthly operation profiles across 1,000 renewable scenarios using the MasCOR framework. The top panel illustrates how excess renewable power is used for either grid export or hydrogen export to the market. The bottom panel shows grid import, along with power charged to or discharged from the BESS and CHT units across the four locations.

Figure 5: Validation of Pareto-optimal designs under dynamic operation . a , Comparison of the MasCOR-inferred Pareto front (nonbold points) with validation results (bold points) obtained from 1,000 monthly renewable generation datasets (2023–2024). b , Performance distributions at the two Pareto front endpoints: dotted areas indicate 85% confidence regions of MasCOR-inferred distributions; scatter points show validation results under varying monthly renewable power availability. Grid prices were held constant using historical data (2015–2022) to isolate renewable power uncertainty. c , Heat maps showing correlations between hourly renewable and grid variables and operational decisions for the two endpoint Pareto-optimal designs in each region.

Figure 6: Online operational performance of Pareto-optimal designs across regions . a , Trajectories of cumulative profit and net carbon emissions for the two extreme Pareto-optimal designs identified in Fig. 3 . Shaded regions indicate the envelope of trajectories obtained under different validation datasets, while solid lines denote ensemble-averaged trajectories. The risk prediction accuracy charts show the ability of MasCOR’s critic component to predict the occurrence of positive net carbon emissions. b , Distributions of online operational outcomes compared with those inferred by MasCOR. The agreement between inferred and realized risk curves indicates the accuracy of positive carbon emission risk prediction. Close agreement indicates accurate risk estimation under renewable uncertainty. c , Average optimality gaps in monthly operational profit and carbon emission reduction, obtained by comparing MasCOR-based online operation with globally optimal solutions obtained from a deterministic LP solver.

Figure S1: Schematic of power-to-methanol system

Figure S2: Comparison of real and synthetic wind-speed data distribution in the Dunkirk (France) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S3: Comparison of real and synthetic wind-speed data distribution in the Skive (Denmark) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S4: Comparison of real and synthetic wind-speed data distribution in the Fredericia (Denmark) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S5: Comparison of real and synthetic wind-speed data distribution in the Weener (Germany) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S6: Training losses of the actor and critic networks under various combinations of design (

D

) and environment (

E

) tokens. (a) Actor loss

J_{\pi}(\theta)

. (b,c) Separated critic losses of

RTG_{t}

and

CTG_{t}

Figure S7: Solution time of the proposed model and the Gurobi benchmark across different problem sizes

Figure S8: Offline operation performance comparison of operational policy models across different design categories: (a) optimality gap from MILP solver (Gurobi), and (b) constraint violation of negative carbon emission. *Error bars represent 10% of the standard deviation

Figure S9: Comparison of goal prediction performance of the ST and PT critic models in offline operation settings across the four design categories: (a) Base, (b) Buffered, (c) Responsive, and (d) Max-product.

Figure S10: Online operation performance comparison of operational policy models across different design categories: (a) optimality gap from MILP solver (Gurobi), and (b) constraint violation of negative carbon emission. *Error bars represent 10% of the standard deviation

Figure S11: Comparison of goal prediction performance of the ST and PT critic models in online operation settings across the four design categories: (a) Base, (b) Buffered, (c) Responsive, and (d) Max-product.

Figure S12: Distributions of grid price and renewable power across the four target regions: (a) Distribution of daily-averaged grid price and renewable power, (b) Distribution of daily-averaged fluctuations in grid price and renewable power.

Figure S13: Performance comparison of Pareto-optimal power-to-methanol designs across the target regions: (a) Probability of net-positive carbon emissions (CI $>0$ ), (b) Distribution of levelized cost of methanol production (LCOM), (c) Distribution of carbon intensity (CI), (d) Normalized values of the Pareto-optimal system designs relative to their respective design space in Table S7 .

Unit process	Parameter	Value	Unit	Reference
BESS	$\eta_{C}$ , $\eta_{D}$	0.95	–	Kang et al. (2023)
	$\alpha$	0.3	–	Kang et al. (2023)
	$\eta_{l}$	$6.94\times 10^{-5}$	–	Kang et al. (2023)
PEMEC and CHT	$SP_{H_{2}}$	55.7	kW/(kg ${}_{H_{2}}$ ·h)	Chen and Yang (2021)
PEMEC and CHT	$SPC_{H_{2}}$	3.003	kW/(kg ${}_{H_{2}}$ ·h)	Chen and Yang (2021)
Methanol synthesis unit	$\gamma_{H}$	0.196	kg ${}_{H_{2}}$ /kg MeOH	Vo et al. (2022)

Table S1: Technical parameters of the Power-to-Methanol system

Hyperparameters	BC	DRL(PPO)	BC+DRL(PPO)	DT	ST & PT
Sequence length $K$	-	-	-	24	24
Number of attention blocks	-	-	-	4	4
Embedding dimension	-	-	-	256	256
Hidden layer	(256,256,256,128)	(256,256,256,128)	(256,256,256,128)	256	256
Activation function	ReLU	ReLU	ReLU	ReLU	ReLU
Epochs	100	100	100	100	100

Table S8: Hyperparameters of the proposed and benchmark models

		ST		PT (proposed)
	Design category	RTG	CTG	RTG	CTG
Offline operation	Base	0.35 $\pm$ 0.01	0.19 $\pm$ 0.03	0.29 $\pm$ 0.01	0.10 $\pm$ 0.02
	Buffered	0.44 $\pm$ 0.02	0.04 $\pm$ 0.01	0.38 $\pm$ 0.04	0.03 $\pm$ 0.01
	Responsive	0.35 $\pm$ 0.01	0.11 $\pm$ 0.02	0.21 $\pm$ 0.01	0.05 $\pm$ 0.01
	Max-product	0.41 $\pm$ 0.03	0.34 $\pm$ 0.07	0.31 $\pm$ 0.01	0.23 $\pm$ 0.05
Online operation	Base	0.36 $\pm$ 0.20	0.11 $\pm$ 0.07	0.30 $\pm$ 0.17	0.08 $\pm$ 0.04

Table S9: CRPS of goal prediction (RTG and CTG) for ST and PT critic model under four design categories in offline and online planning

\mathcal{L}_{z}=\frac{1}{M}\sum_{j=1}^{M}\Biggl[\frac{1}{t}\bigl\|w_{\text{obs}}-\mathbb{P}_{\text{hist}}\!\bigl(G(z_{j})\bigr)\bigr\|_{2}^{2}-D\bigl(G(z_{j})\bigr)\Biggr]

Algorithm 2 Actor–Critic online operation

Explore similar work

Mar 15, 2026

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees

Global decarbonisation targets and tightening market pressures demand maritime logistics solutions that are simultaneously efficient, sustainable, and equitable. We introduce EcoFair-CH-MARL, a constrained hierarchical multi-agent reinforcement learning framework that unifies three innovations: (i) a primal-dual budget layer that provably bounds cumulative emissions under stochastic weather and demand; (ii) a fairness-aware reward transformer with dynamically scheduled penalties that enforces max-min cost equity across heterogeneous fleets; and (iii) a two-tier policy architecture that decouples strategic routing from real-time vessel control, enabling linear scaling in agent count. New theoretical results establish O(\sqrt{T}) regret for both constraint violations and fairness loss. Experiments on a high-fidelity maritime digital twin (16 ports, 50 vessels) driven by automatic identification system traces, plus an energy-grid case study, show up to 15% lower emissions, 12% higher through-put, and a 45% fair-cost improvement over state-of-the-art hierarchical and constrained MARL baselines. In addition, EcoFair-CH-MARL achieves stronger equity (lower Gini and higher min-max welfare) than fairness-specific MARL baselines (e.g., SOTO, FEN), and its modular design is compatible with both policy- and value-based learners. EcoFair-CH-MARL therefore advances the feasibility of large-scale, regulation-compliant, and socially responsible multi-agent coordination in safety-critical domains.

Saad Alqithami

Computer Science Department, Al-Baha University, Albaha 65779, Saudi Arabia

Multi-Agent Reinforcement Learning Multi-Agent Coordination Hierarchical RL Heterogeneous MARL

Ask PDF

Apr 1, 2026

An Online Machine Learning Multi-resolution Optimization Framework for Energy System Design Limit of Performance Analysis

Designing reliable integrated energy systems for industrial processes requires optimization and verification models across multiple fidelities, from architecture-level sizing to high-fidelity dynamic operation. However, model mismatch across fidelities obscures the sources of performance loss and complicates the quantification of architecture-to-operation performance gaps. We propose an online, machine-learning-accelerated multi-resolution optimization framework that estimates an architecture-specific upper bound on achievable performance while minimizing expensive high-fidelity model evaluations. We demonstrate the approach on a pilot energy system supplying a 1 MW industrial heat load. First, we solve a multi-objective architecture optimization to select the system configuration and component capacities. We then develop an machine learning (ML)-accelerated multi-resolution, receding-horizon optimal control strategy that approaches the achievable-performance bound for the specified architecture, given the additional controls and dynamics not captured by the architectural optimization model. The ML-guided controller adaptively schedules the optimization resolution based on predictive uncertainty and warm-starts high-fidelity solves using elite low-fidelity solutions. Our results on the pilot case study show that the proposed multi-resolution strategy reduces the architecture-to-operation performance gap by up to 42% relative to a rule-based controller, while reducing required high-fidelity model evaluations by 34% relative to the same multi-fidelity approach without ML guidance, enabling faster and more reliable design verification. Together, these gains make high-fidelity verification tractable, providing a practical upper bound on achievable operational performance.

Oluwamayowa O. Amusat, Luka Grbcic, Remi Patureau +3

Scientific Data Division, Computing Sciences, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 · Applied Mathematics & Computational Research Division (AMCR), Computing Sciences, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 · Building & Industrial Energy Systems (BIES) Division, Energy Technologies Area, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 +1

Multi-Fidelity Optimization Model Predictive Control

Ask PDF

Jan 7, 2026

Accounting for Optimal Control in the Sizing of Isolated Hybrid Renewable Energy Systems Using Imitation Learning

Decarbonization of isolated or off-grid energy systems through phase-in of large shares of intermittent solar or wind generation requires co-installation of energy storage or continued use of existing fossil dispatchable power sources to balance supply and demand. The effective CO2 emission reduction depends on the relative capacity of the energy storage and renewable sources, the stochasticity of the renewable generation, and the control of the isolated energy system. While the operation of the energy storage and dispatchable sources impacts the optimal sizing of the system, it is challenging to account for the effect of finite-horizon optimal control at the stage of system sizing. In this work, we present a flexible and computationally efficient sizing framework for energy storage and renewable capacity in isolated energy systems, accounting for uncertainty in the renewable generation and the optimal control. We implement an imitation learning approach to stochastic neural model predictive control (MPC) which allows us to relate the battery storage and wind peak capacities to the emissions reduction and investment costs while accounting for finite horizon, optimal control without solving an infeasible number of optimization problems. We evaluate the proposed sizing framework on a case study of an offshore energy system with a gas turbine, a wind farm and a battery energy storage system (BESS). In this case, we find a nonlinear, nontrivial relationship between the investment costs and the reduction in gas usage for different wind and BESS capacities.

Simon Halvdansson, Lucas Ferreira Bernardino, Brage Rugstad Knudsen

SINTEF Energy Research, Trondheim, Norway.

Power Systems

Ask PDF