Optimal trajectory-guided stochastic co-optimization for e-fuel system design and real-time operation

Jeongdong Kim; Minsu Kim; Jonggeol Na; Junghwan Kim

cs.LGMar 3, 2026

Optimal trajectory-guided stochastic co-optimization for e-fuel system design and real-time operation

Authors: Jeongdong Kim, Minsu Kim, Jonggeol Na, Junghwan Kim

Organizations: Department of Chemical Engineering , Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, 02139, Massachusetts, United States. · Department of Chemical and Biomolecular Engineering, Yonsei University, 50 Yonsei-ro, Seoul, 03722, Republic of Korea. · Department of Chemical Engineering and Materials Science, Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea. · Graduate Program in System Health Science and Engineering, Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea. · Institute for Multiscale Matter and Systems (IMMS), Ewha Womans University, 52 Ewhayeodae-gil, Seoul, 03760, Republic of Korea.

Abstract

E-fuels are promising long-term energy carriers supporting the net-zero transition. However, the large combinatorial design-operation spaces under renewable uncertainty make the use of mathematical programming impractical for co-optimizing e-fuel production systems. Here, we present MasCOR, a machine-learning-assisted co-optimization framework that learns from global operational trajectories. By encoding system design and renewable trends, a single MasCOR agent generalizes dynamic operation across diverse configurations and scenarios, substantially simplifying design-operation co-optimization under uncertainty. Benchmark comparisons against state-of-the-art reinforcement learning baselines demonstrate near-optimal performance, while computational costs are substantially lower than those of mathematical programming, enabling rapid parallel evaluation of designs within the co-optimization loop. This framework enables rapid screening of feasible design spaces together with corresponding operational policies. When applied to four potential European sites targeting e-methanol production, MasCOR shows that most locations benefit from reducing system load below 50 MW to achieve carbon-neutral methanol production, with production costs of 1.0-1.2 USD per kg. In contrast, Dunkirk (France), with limited renewable availability and high grid prices, favors system loads above 200 MW and expanded storage to exploit dynamic grid exchange and hydrogen sales to the market. These results underscore the value of the MasCOR framework for site-specific guidance from system design to real-time operation.

Figures & tables

Figure 1: MasCOR: ML-assisted stochastic co-optimization framework for renewable power management systems . a , Generation of an oracle dataset by combining a renewable scenario generative model with LP-based operational optimization across sampled designs. b , Training of transformer-based actor–critic agents to learn optimal operational policies and two future performance metrics, cumulative profit (return-to-go, $RTG_{t}$ ) and net carbon emissions (carbon-to-go, $CTG_{t}$ ), conditioned on renewable trends $E$ and system design $D$ . c , MasCOR infers distributions of profit and net carbon emissions ( $CTG_{0}$ ) for each design using the generative model and trained agent, selecting the design–policy pair that minimizes production cost and net carbon emissions while satisfying a chance constraint on positive carbon emissions ( $CTG_{0}>0$ ). d , MasCOR generates future renewable trend sets $E$ and performs online operation conditioned on these trends and the optimal design $D^{*}$ , ensuring robust performance under uncertainty. Details of the architecture, algorithms, and benchmark comparisons of the two ML models are provided in Supplementary Notes 2, 3 .

Figure 2: Performance evaluation of the MasCOR framework for renewable data generation and dynamic operational problem solving . a , T-SNE embedding of synthetic renewable profiles generated by the generative model, compared with training and validation wind datasets, along with distributions of key operational features. b , Computational efficiency of the proposed MasCOR agent compared with the benchmark LP solver (Gurobi) for parallel LP problem solving. c , Dynamic operational performance comparison of benchmark data–driven models and the proposed MasCOR agent across different design categories. Details of computational efficiency and benchmark comparison experiments are provided in ( Supplementary Notes 3.4 )

Figure 3: Regional design strategies and performance trade-offs in co-optimized power-to-methanol systems . a , Differences in the average renewable power availability and grid electricity prices across Dunkirk, Skive, Fredericia, and Weener (distributional details provided in Supplementary Figs. S12 ). b , Capacity shares of methanol (MeOH) production, PEMEC units, and BESS and CHT installations in representative Pareto-optimal designs, together with overall system size. c , Pareto-optimal capacity progression of PEMEC units, storage units (BESS and CHT), and MeOH production. d , Pareto frontiers of co-optimization results showing the trade-offs between the expected LCOM and net carbon emissions. MasCOR identifies two distinct design strategies for reducing carbon emission among the Pareto-optimal solutions: (i) an SE regime, characterized by increasing storage capacity, and (ii) a PR regime, characterized by decreasing MeOH production capacity under minimum PEMEC and storage capacity conditions (below 10 MW). In most regions, Pareto-optimal points shift from SE to PR as the expected net carbon emissions decrease below –50 ton/month. UQ results and full design specifications of Pareto points are provided in Supplementary Fig. S13 .

Figure 4: Regional operating strategies under different optimal design regimes . a , Capacity shares (%) of MeOH synthesis and PEMEC, BESS, and CHT units for the four representative Pareto-optimal designs illustrated in Fig. 3 . The dotted line indicates system scale. b , Hourly operating strategies for optimal designs across regions. Distributions are obtained by averaging monthly operation profiles across 1,000 renewable scenarios using the MasCOR framework. The top panel illustrates how excess renewable power is used for either grid export or hydrogen export to the market. The bottom panel shows grid import, along with power charged to or discharged from the BESS and CHT units across the four locations.

Figure 5: Validation of Pareto-optimal designs under dynamic operation . a , Comparison of the MasCOR-inferred Pareto front (nonbold points) with validation results (bold points) obtained from 1,000 monthly renewable generation datasets (2023–2024). b , Performance distributions at the two Pareto front endpoints: dotted areas indicate 85% confidence regions of MasCOR-inferred distributions; scatter points show validation results under varying monthly renewable power availability. Grid prices were held constant using historical data (2015–2022) to isolate renewable power uncertainty. c , Heat maps showing correlations between hourly renewable and grid variables and operational decisions for the two endpoint Pareto-optimal designs in each region.

Figure 6: Online operational performance of Pareto-optimal designs across regions . a , Trajectories of cumulative profit and net carbon emissions for the two extreme Pareto-optimal designs identified in Fig. 3 . Shaded regions indicate the envelope of trajectories obtained under different validation datasets, while solid lines denote ensemble-averaged trajectories. The risk prediction accuracy charts show the ability of MasCOR’s critic component to predict the occurrence of positive net carbon emissions. b , Distributions of online operational outcomes compared with those inferred by MasCOR. The agreement between inferred and realized risk curves indicates the accuracy of positive carbon emission risk prediction. Close agreement indicates accurate risk estimation under renewable uncertainty. c , Average optimality gaps in monthly operational profit and carbon emission reduction, obtained by comparing MasCOR-based online operation with globally optimal solutions obtained from a deterministic LP solver.

Figure S1: Schematic of power-to-methanol system

Figure S2: Comparison of real and synthetic wind-speed data distribution in the Dunkirk (France) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S3: Comparison of real and synthetic wind-speed data distribution in the Skive (Denmark) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S4: Comparison of real and synthetic wind-speed data distribution in the Fredericia (Denmark) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S5: Comparison of real and synthetic wind-speed data distribution in the Weener (Germany) region: (a) t-SNE visualization and (b) key operational feature distributions

Figure S6: Training losses of the actor and critic networks under various combinations of design (

D

) and environment (

E

) tokens. (a) Actor loss

J_{\pi}(\theta)

. (b,c) Separated critic losses of

RTG_{t}

and

CTG_{t}

Figure S7: Solution time of the proposed model and the Gurobi benchmark across different problem sizes

Figure S8: Offline operation performance comparison of operational policy models across different design categories: (a) optimality gap from MILP solver (Gurobi), and (b) constraint violation of negative carbon emission. *Error bars represent 10% of the standard deviation

Figure S9: Comparison of goal prediction performance of the ST and PT critic models in offline operation settings across the four design categories: (a) Base, (b) Buffered, (c) Responsive, and (d) Max-product.

Figure S10: Online operation performance comparison of operational policy models across different design categories: (a) optimality gap from MILP solver (Gurobi), and (b) constraint violation of negative carbon emission. *Error bars represent 10% of the standard deviation

Figure S11: Comparison of goal prediction performance of the ST and PT critic models in online operation settings across the four design categories: (a) Base, (b) Buffered, (c) Responsive, and (d) Max-product.

Figure S12: Distributions of grid price and renewable power across the four target regions: (a) Distribution of daily-averaged grid price and renewable power, (b) Distribution of daily-averaged fluctuations in grid price and renewable power.

Figure S13: Performance comparison of Pareto-optimal power-to-methanol designs across the target regions: (a) Probability of net-positive carbon emissions (CI $>0$ ), (b) Distribution of levelized cost of methanol production (LCOM), (c) Distribution of carbon intensity (CI), (d) Normalized values of the Pareto-optimal system designs relative to their respective design space in Table S7 .

Unit process	Parameter	Value	Unit	Reference
BESS	$\eta_{C}$ , $\eta_{D}$	0.95	–	Kang et al. (2023)
	$\alpha$	0.3	–	Kang et al. (2023)
	$\eta_{l}$	$6.94\times 10^{-5}$	–	Kang et al. (2023)
PEMEC and CHT	$SP_{H_{2}}$	55.7	kW/(kg ${}_{H_{2}}$ ·h)	Chen and Yang (2021)
PEMEC and CHT	$SPC_{H_{2}}$	3.003	kW/(kg ${}_{H_{2}}$ ·h)	Chen and Yang (2021)
Methanol synthesis unit	$\gamma_{H}$	0.196	kg ${}_{H_{2}}$ /kg MeOH	Vo et al. (2022)

Table S1: Technical parameters of the Power-to-Methanol system

Hyperparameters	BC	DRL(PPO)	BC+DRL(PPO)	DT	ST & PT
Sequence length $K$	-	-	-	24	24
Number of attention blocks	-	-	-	4	4
Embedding dimension	-	-	-	256	256
Hidden layer	(256,256,256,128)	(256,256,256,128)	(256,256,256,128)	256	256
Activation function	ReLU	ReLU	ReLU	ReLU	ReLU
Epochs	100	100	100	100	100

Table S8: Hyperparameters of the proposed and benchmark models

		ST		PT (proposed)
	Design category	RTG	CTG	RTG	CTG
Offline operation	Base	0.35 $\pm$ 0.01	0.19 $\pm$ 0.03	0.29 $\pm$ 0.01	0.10 $\pm$ 0.02
	Buffered	0.44 $\pm$ 0.02	0.04 $\pm$ 0.01	0.38 $\pm$ 0.04	0.03 $\pm$ 0.01
	Responsive	0.35 $\pm$ 0.01	0.11 $\pm$ 0.02	0.21 $\pm$ 0.01	0.05 $\pm$ 0.01
	Max-product	0.41 $\pm$ 0.03	0.34 $\pm$ 0.07	0.31 $\pm$ 0.01	0.23 $\pm$ 0.05
Online operation	Base	0.36 $\pm$ 0.20	0.11 $\pm$ 0.07	0.30 $\pm$ 0.17	0.08 $\pm$ 0.04

Table S9: CRPS of goal prediction (RTG and CTG) for ST and PT critic model under four design categories in offline and online planning

\mathcal{L}_{z}=\frac{1}{M}\sum_{j=1}^{M}\Biggl[\frac{1}{t}\bigl\|w_{\text{obs}}-\mathbb{P}_{\text{hist}}\!\bigl(G(z_{j})\bigr)\bigr\|_{2}^{2}-D\bigl(G(z_{j})\bigr)\Biggr]

Algorithm 2 Actor–Critic online operation

Explore similar work

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees
Mar 15, 2026Saad AlqithamiMulti-Agent Reinforcement Learning Multi-Agent Coordination
Computer Science Department, Al-Baha University, Albaha 65779, Saudi Arabia
An Online Machine Learning Multi-resolution Optimization Framework for Energy System Design Limit of Performance Analysis
Apr 1, 2026Oluwamayowa O. Amusat, Luka Grbcic, Remi Patureau +3Multi-Fidelity Optimization Model Predictive Control
Scientific Data Division, Computing Sciences, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 · Applied Mathematics & Computational Research Division (AMCR), Computing Sciences, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 · Building & Industrial Energy Systems (BIES) Division, Energy Technologies Area, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA 94720 +1
Accounting for Optimal Control in the Sizing of Isolated Hybrid Renewable Energy Systems Using Imitation Learning
Jan 7, 2026Simon Halvdansson, Lucas Ferreira Bernardino, Brage Rugstad KnudsenPower Systems
SINTEF Energy Research, Trondheim, Norway.

Mar 15, 2026J/K move · Enter open · S save

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees

Saad Alqithami

Computer Science Department, Al-Baha University, Albaha 65779, Saudi Arabia

Multi-Agent Reinforcement Learning Multi-Agent Coordination Hierarchical RL Heterogeneous MARL

Open paper AskPDF arXiv

Global decarbonisation targets and tightening market pressures demand maritime logistics solutions that are simultaneously efficient, sustainable, and equitable. We introduce EcoFair-CH-MARL, a constrained hierarchical multi-agent reinforcement learning framework that unifies three innovations: (i) a primal-dual budget layer that provably bounds cumulative emissions under stochastic weather and demand; (ii) a fairness-aware reward transformer with dynamically scheduled penalties that enforces max-min cost equity across heterogeneous fleets; and (iii) a two-tier policy architecture that decouples strategic routing from real-time vessel control, enabling linear scaling in agent count. New theoretical results establish O(\sqrt{T}) regret for both constraint violations and fairness loss. Experiments on a high-fidelity maritime digital twin (16 ports, 50 vessels) driven by automatic identification system traces, plus an energy-grid case study, show up to 15% lower emissions, 12% higher through-put, and a 45% fair-cost improvement over state-of-the-art hierarchical and constrained MARL baselines. In addition, EcoFair-CH-MARL achieves stronger equity (lower Gini and higher min-max welfare) than fairness-specific MARL baselines (e.g., SOTO, FEN), and its modular design is compatible with both policy- and value-based learners. EcoFair-CH-MARL therefore advances the feasibility of large-scale, regulation-compliant, and socially responsible multi-agent coordination in safety-critical domains.

Abstract

Explore similar work

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees

An Online Machine Learning Multi-resolution Optimization Framework for Energy System Design Limit of Performance Analysis

Accounting for Optimal Control in the Sizing of Isolated Hybrid Renewable Energy Systems Using Imitation Learning