The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks
that specifically consider the inherent uncertainties such as changing order
requirements and possible machine breakdown in realistic smart manufacturing
settings. Since traditional methods cannot dynamically generate effective
scheduling strategies in face of the disturbance of environments, we formulate
the DJSP as a Markov decision process (MDP) to be tackled by reinforcement
learning (RL). For this purpose, we propose a flexible hybrid framework that
takes disjunctive graphs as states and a set of general dispatching rules as
the action space with minimum prior domain knowledge. The attention mechanism
is used as the graph representation learning (GRL) module for the feature
extraction of states, and the double dueling deep Q-network with prioritized
replay and noisy networks (D3QPN) is employed to map each state to the most
appropriate dispatching rule. Furthermore, we present Gymjsp, a public
benchmark based on the well-known OR-Library, to provide a standardized
off-the-shelf facility for RL and DJSP research communities. Comprehensive
experiments on various DJSP instances confirm that our proposed framework is
superior to baseline algorithms with smaller makespan across all instances and
provide empirical justification for the validity of the various components in
the hybrid framework.

本文提出了一种将动态车间调度问题 (DJSP) 建模为强化学习的马尔可夫决策流程 (MDP) 的混合框架，并结合了注意力机制作为图表示学习 (GRL) 的模块，以及双层权利分离深度 Q 网络评估最适当的调度规则，经实验证明该框架优于基准算法，并证实了这种混合框架的有效性。