Factorized Scheduling Principle: Learning Interpretable and Transferable Policies via Structured Additive Functions

Hong Je-Gal; Hyun-Suk Lee

cs.LGSep 29, 2026

Factorized Scheduling Principle: Learning Interpretable and Transferable Policies via Structured Additive Functions

Authors: Hong Je-Gal, Hyun-Suk Lee

Organizations: Department of Artificial Intelligence and Robotics, Sejong University, South Korea · Artificial Intelligence Robotics Institute (AIRI), Sejong University, South Korea

Abstract

Scheduling problems arise from repeatedly selecting one item from a set of candidates based on their states. These problems often reduce to assigning priority scores and choosing the highest-ranked item. In this work, we propose a factorized scheduling principle (FSP) framework to learn interpretable and transferable scheduling rules. The FSP framework represents system states as condition distributions and decomposes a global scheduling principle into additive univariate and pairwise components with identifiability constraints. The scheduling principle enables the framework to maintain a simple priority-based structure during deployment. This principle is learned by using a policy-based objective combined with a temporal-difference signal defined on the condition distribution. Experiments on synthetic and realistic scheduling tasks demonstrate the FSP framework's strong performance, interpretability, and zero-shot generalization across different system scales.

Figures & tables

Figure 1 : Illustrative overview of the FSP framework.

Approach	Learned representation	Transferability	Interpretability
Index policies	Analytically defined index	Limited	High
Index learning	Approximated index	Limited	Low–Medium
RL scheduling	Policy or $Q(s,a)$	No	Low
Structured/Additive RL	Additive policy or value	No	Medium–High
FSP (ours)	Priority principle $S(x)$	Explicitly considered	High

Table 1 : Comparison of learned representations across scheduling approaches.

Panel of Figure 2 — Figure 2 : Visual comparison of the component functions of $\hat{S}$ with the true principle $S^{\star}$ .

$N$	Ranking consistency	Avg. reward	Reward gap to oracle
4	0.9940	2.5262	$7.9\times 10^{-5}$
8	0.9947	2.7783	$1.76\times 10^{-4}$
16	0.9942	2.9377	$2.37\times 10^{-4}$
32	0.9941	3.0558	$6.15\times 10^{-4}$

Table 2 : Transfer performance with varying

N

, where

\hat{S}

is trained with

N=8

Task	$N$	FSP	Whittle	NAM	DQN	A2C	PPO	TRPO	QR-DQN
Wireless	$10^{\mathrm{ID}}$	0.109	0.114	0.129	0.112	0.094	0.107	0.112	0.127
	5	0.085 $\pm$ 0.001	0.096 $\pm$ 0.000	0.131 $\pm$ 0.000	-0.073 $\pm$ 0.002	0.127 $\pm$ 0.000	0.129 $\pm$ 0.000	0.130 $\pm$ 0.000	0.126 $\pm$ 0.000
	10	0.103 $\pm$ 0.001	0.115 $\pm$ 0.001	0.109 $\pm$ 0.001	-1.313 $\pm$ 0.003	0.095 $\pm$ 0.001	0.104 $\pm$ 0.001	0.112 $\pm$ 0.001	0.119 $\pm$ 0.001
	15	-0.285 $\pm$ 0.002	-0.337 $\pm$ 0.002	-0.251 $\pm$ 0.001	-1.965 $\pm$ 0.002	-0.426 $\pm$ 0.002	-0.372 $\pm$ 0.002	-0.368 $\pm$ 0.002	-0.386 $\pm$ 0.002
	20	-0.781 $\pm$ 0.001	-0.888 $\pm$ 0.001	-0.752 $\pm$ 0.001	-2.609 $\pm$ 0.001	-1.045 $\pm$ 0.002	-0.971 $\pm$ 0.002	-0.951 $\pm$ 0.002	-1.019 $\pm$ 0.002
Inventory	$10^{\mathrm{ID}}$	-0.353	-0.344	-0.361	-0.355	-0.479	-0.566	-0.566	-0.487

Table 3 : In-distribution reward,

10^{\mathrm{ID}}

, and out-of-distribution average rewards under varying

N

and system populations. FSP and Whittle are learned on the

10^{\mathrm{ID}}

system and transferred without retraining; the other baselines are trained for each

N

Panel of Figure 3 — Figure 3 : Two of four one-dimensional component functions of the FSP policy in the wireless user scheduling task.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

V(\bar{s}^{t})\leftarrow\frac{1}{N^{t}}\sum_{n=1}^{N^{t}}S(\mathbf{x}^{t}_{n};\Theta)\textrm{ and }V(\bar{s}^{t+1})\leftarrow\frac{1}{N^{t+1}}\sum_{n=1}^{N^{t+1}}S(\mathbf{x}^{t+1}_{n};\Theta)

Appendix

Algorithm 1 Learning the Global Scheduling Principle

S(x)

Panel of Figure 4 — Figure 4 : The visual comparison of the univariate component functions of $\hat{S}$ with the true principle $S^{\star}$ .

Panel of Figure 5 — Figure 5 : The one-dimensional visual comparison of the pairwise component function of $\hat{S}$ with the true principle $S^{\star}$ .

Panel of Figure 6 — Figure 6 : The two-dimensional visual comparison of the component functions of $\hat{S}$ with the true principle $S^{\star}$ .

FSP		Whittle-index policy
Parameter	Value	Parameter	Value
Learning rate ( $\eta$ )	0.005	State discretization	$4\times 4\times 4$
Basis type	B-spline functions	Discount factor	0.99
1-D basis dimension	30	Dirichlet prior	0.3
2-D basis dimension	15	Value-iteration threshold	$10^{-3}$
Trade-off hyperparameter ( $\lambda$ )	0.1	Subsidy-search threshold	$5\times 10^{-2}$

Appendix

Table 4 : Hyperparameter summary for FSP and baseline methods.

Parameter	Value
Mean arrival rate ( $\lambda_{n}$ )	0.045
Shape parameter for arrival ( $k$ )	2.0
Capacity scale ( $\eta$ )	0.6
Throughput weight ( $\omega$ )	1.0
Overflow penalty coefficient ( $\lambda_{\mathrm{ov}}$ )	2.0
Delay penalty coefficient ( $\lambda_{\mathrm{delay}}$ )	1.0

Appendix

Table 5 : System parameters for wireless user scheduling.

Parameter	Value
Replenishment quantity ( $Q$ )	0.5
Demand granularity ( $K_{d}$ )	50
Mean demand drift std. ( $\sigma_{\mu}$ )	0.02
Margin drift std. ( $\sigma_{m}$ )	0.003
Unit cost range ( $c_{n}$ )	$[0.02,\,0.5]$
Holding cost range ( $h_{n}$ )	$[0.01,\,0.10]$

Appendix

Table 6 : System parameters for inventory replenishment.

Parameter	Value
Global sales capacity ( $C$ )	$0.45+0.01\cdot N$
Inflow rate drift std. ( $\sigma_{\rho}$ )	0.01
Margin drift std. ( $\sigma_{m}$ )	0.01
Revenue scaling factor ( $\alpha$ )	5.0
Holding cost ( $h$ )	0.1
Overflow threshold ( $\tau$ )	1.0

Appendix

Table 7 : System parameters for warehouse clearance.

$N$	FSP( $10^{\mathrm{ID}}$ )	DQN( $5^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ )	DQN( $15^{\mathrm{ID}}$ )	DQN( $20^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ , w/o noise)
$5^{\mathrm{ID}}$	0.116	0.127	-	-	-	-
$10^{\mathrm{ID}}$	0.109	-	0.112	-	-	0.122
$15^{\mathrm{ID}}$	-0.266	-	-	-0.297	-	-
$20^{\mathrm{ID}}$	-0.762	-	-	-	-0.855	-
5	0.085 $\pm$ 0.001	-0.073 $\pm$ 0.002	-	-	-	-
10	0.103 $\pm$ 0.001	-	-1.313 $\pm$ 0.003	-	-	-1.046 $\pm$ 0.001

Appendix

Table 8 : In-distribution rewards and out-of-distribution average rewards of the wireless user scheduling task under varying

N

with different system populations. (For each

N

, evaluation is conducted across 100 system instances.)

Panel of Figure 7 — Figure 7 : One-dimensional component functions of the FSP policy in the wireless user scheduling task.

Panel of Figure 8 — Figure 8 : Two-dimensional pairwise interaction components of the FSP policy in the wireless user scheduling task.

$N$	FSP( $10^{\mathrm{ID}}$ )	DQN( $5^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ )	DQN( $15^{\mathrm{ID}}$ )	DQN( $20^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ , w/o noise)
$5^{\mathrm{ID}}$	-0.232	-0.255	-	-	-	-
$10^{\mathrm{ID}}$	-0.353	-	-0.355	-	-	-0.363
$15^{\mathrm{ID}}$	-0.649	-	-	-0.670	-	-
$20^{\mathrm{ID}}$	-0.769	-	-	-	-0.751	-
5	-0.206 $\pm$ 0.011	-0.228 $\pm$ 0.012	-	-	-	-
10	-0.412 $\pm$ 0.008	-	-0.429 $\pm$ 0.008	-	-	-0.420 $\pm$ 0.008

Appendix

Table 9 : In-distribution rewards and out-of-distribution average rewards of the inventory replenishment task under varying

N

with different system populations. (For each

N

, evaluation is conducted across 100 system instances.)

Panel of Figure 9 — Figure 9 : One-dimensional component functions of the FSP policy in the inventory replenishment environment.

Panel of Figure 10 — Figure 10 : Learned two-dimensional interaction components $\psi_{ij}$ in the inventory replenishment environment, visualized as surfaces over the corresponding state dimensions.

$N$	FSP( $10^{\mathrm{ID}}$ )	DQN( $5^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ )	DQN( $15^{\mathrm{ID}}$ )	DQN( $20^{\mathrm{ID}}$ )	DQN( $10^{\mathrm{ID}}$ , w/o noise)
$5^{\mathrm{ID}}$	0.736	0.772	-	-	-	-
$10^{\mathrm{ID}}$	0.823	-	0.883	-	-	0.881
$15^{\mathrm{ID}}$	0.739	-	-	0.808	-	-
$20^{\mathrm{ID}}$	0.723	-	-	-	0.766	-
5	0.630 $\pm$ 0.021	-10.913 $\pm$ 1.374	-	-	-	-
10	0.738 $\pm$ 0.019	-	-35.238 $\pm$ 1.931	-	-	-38.247 $\pm$ 2.461

Appendix

Table 10 : In-distribution rewards and out-of-distribution average rewards of the warehouse clearance task under varying

N

with different system populations. (For each

N

, evaluation is conducted across 100 system instances.)

Panel of Figure 11 — Figure 11 : One-dimensional component functions of the FSP policy in the warehouse clearance environment.

Panel of Figure 12 — Figure 12 : Learned two-dimensional interaction components $\psi_{ij}$ in the warehouse clearance environment, visualized as surfaces over the corresponding state dimensions.

Explore similar work

Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework
May 18, 2026Chengpeng Hu, Yingqian Zhang, Hendrik BaierJob Shop Scheduling Offline Reinforcement Learning
Eindhoven University of Technology, Eindhoven, the Netherlands · Centrum Wiskunde & Informatica, Amsterdam, the Netherlands
PLAN: Parallel Liquid-Inspired Approximation Network for Efficient Representation Learning in Flexible Job Shop Scheduling
Aug 4, 2026Dhivya Dharshini Kannan, Wei Zhang, Jieyi Bi +5Job Shop Scheduling Neural Approximations
Singapore Institute of Technology (SIT) · Nanyang Technological University (NTU) · Shanghai Jiao Tong University +1
Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphs
Apr 26, 2026Jonathan Hoss, Moritz Link, Noah KlarmannJob Shop Scheduling Schedulers
Faculty of Management and Engineering, Rosenheim Technical University of Applied Sciences, Germany

May 18, 2026, cs.LGJ/K move · Enter open · S save

Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework

Chengpeng Hu, Yingqian Zhang, Hendrik Baier

Eindhoven University of Technology, Eindhoven, the Netherlands · Centrum Wiskunde & Informatica, Amsterdam, the Netherlands

Job Shop Scheduling Offline Reinforcement Learning Mixed-Integer Programming

Open paper AskPDF arXiv

Deep reinforcement learning (DRL) has recently emerged as a promising approach to solve combinatorial optimization problems such as job shop scheduling. However, the policies learned by DRL are typically represented by deep neural networks (DNNs), whose opaque neural architectures and non-interpretable policy decisions can lead to critical trust and usability concerns for human decision makers. In addition, the computational requirements of DNNs can further hinder practical deployment in resource constrained environments. In this work, we propose ProRL, a novel interpretable programmatic reinforcement learning framework that achieves high-performance scheduling with human-readable and editable programmatic policies (i.e., programs). We first introduce a domain-specific language for scheduling (DSL-S) to represent scheduling strategies as structured programs. ProRL then explores the program space defined by DSL-S using local search to identify incomplete programs, which are subsequently completed by learning their parameters via Bayesian optimization. ProRL learns which scheduling heuristic rules to select, and hence, it naturally incorporates existing heuristics already used in industrial scenarios. Experiments on widely used benchmark instances demonstrate the strong performance of ProRL against existing heuristics and DRL baselines. Furthermore, ProRL performs well under strongly constrained computational resources, such as training with only 100 episodes. Our code is available at https://github.com/HcPlu/ProRL.

Abstract

Explore similar work

Scheduling That Speaks: An Interpretable Programmatic Reinforcement Learning Framework

PLAN: Parallel Liquid-Inspired Approximation Network for Efficient Representation Learning in Flexible Job Shop Scheduling

Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphs