cs.LGNov 2, 2026

Efficient Detailed Scheduling of Multiproduct Pipelines using Action-Decoupled Deep Reinforcement Learning

Authors: Kai Shao, Ling Jian, Liming Liu, Serguey V. Bogomolov, Xinmin Wang

Organizations: School of Economics and Management, China University of Petroleum (East China), Qingdao, Shandong, China · Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, Leninskie Gory, Moscow, 119899, Russia

Abstract

Multiproduct pipelines transport refined products to delivery stations in sequential batches, and their detailed scheduling must meet market demand while respecting nonlinear hydraulic constraints. Retaining the full hydraulic nonlinearity yields a mixed-integer nonlinear program that is computationally intractable, while the linearization commonly adopted for solver-based optimization degrades hydraulic accuracy without resolving the heavy computational burden. This paper develops an action-decoupled proximal policy optimization (PPO) framework that efficiently produces detailed schedules while preserving hydraulic fidelity. The scheduling process is cast as a semi-Markov decision process (SMDP) with flexible action durations. To handle the discrete-continuous hybrid action space, a cascaded dual-actor architecture decouples the discrete operating duration from the continuous flow decisions, eliminating the gradient interference that destabilizes conventional coupled-action policies and thereby yielding high-quality schedules far more efficiently. Validated on a real-world multiproduct pipeline in China and benchmarked against a single-actor baseline and a linearized MILP solved by Gurobi, the proposed framework achieves delivery accuracy close to the exact optimum while substantially improving flow rate stability and substantially reducing solution time.

Figures & tables

Explore similar work

CardsList
  1. PLAN: Parallel Liquid-Inspired Approximation Network for Efficient Representation Learning in Flexible Job Shop Scheduling

    Aug 4, 2026Dhivya Dharshini Kannan, Wei Zhang, Jieyi Bi +5Efficient Neural Network InferenceLiquid Neural Networks

  2. Mixed-Integer Nonlinear Differentiable Predictive Control for Underground Pumped Hydro Energy Storage Systems

    Sep 16, 2026Honghui Zheng, Ján Boldocký, Yury Dvorkin +1Model Predictive ControlOptimal Control

  3. Hard Constraints, Smooth Gradients: Learning Feasible Inventory Policies via Differentiable Projection

    Aug 3, 2026Patrick Helm, Jan-Niklas Doerr, Joren Gijsbrechts +1Inventory ControlConstrained RL