We investigate the feasibility of deploying Deep-Q based deep reinforcement
learning agents to job-shop scheduling problems in the context of modular
production facilities, using discrete event simulations for the environment.
These environments are comprised of a source and sink for the parts to be
processed, as well as (several) workstations. The agents are trained to
schedule automated guided vehicles to transport the parts back and forth
between those stations in an optimal fashion. Starting from a very simplistic
setup, we increase the complexity of the environment and compare the agents'
performances with well established heuristic approaches, such as
first-in-first-out based agents, cost tables and a nearest-neighbor approach.
We furthermore seek particular configurations of the environments in which the
heuristic approaches struggle, to investigate to what degree the Deep-Q agents
are affected by these challenges. We find that Deep-Q based agents show
comparable performance as the heuristic baselines. Furthermore, our findings
suggest that the DRL agents exhibit an increased robustness to noise, as
compared to the conventional approaches. Overall, we find that DRL agents
constitute a valuable approach for this type of scheduling problems.

本文研究了在离散事件模拟环境下，使用基于 Deep-Q 的深度强化学习代理来解决模块化生产设施中的作业车间调度问题。研究发现，相比于传统启发式和成本表等方法，基于 DRL 的代理相当，并且具有更高的噪声稳健性，对于这种类型的调度问题值得尝试。

基于深度 Q 学习的模块化生产环境下的车辆管理

Vehicle management in a modular production context using Deep Q-Learning

Scheduling is a fundamental task occurring in various automated systems
applications, e.g., optimal schedules for machines on a job shop allow for a
reduction of production costs and waste. Nevertheless, finding such schedules
is often intractable and cannot be achieved by Combinatorial Optimization
Problem (COP) methods within a given time limit. Recent advances of Deep
Reinforcement Learning (DRL) in learning complex behavior enable new COP
application possibilities. This paper presents an efficient DRL environment for
Job-Shop Scheduling -- an important problem in the field. Furthermore, we
design a meaningful and compact state representation as well as a novel, simple
dense reward function, closely related to the sparse make-span minimization
criteria used by COP methods. We demonstrate that our approach significantly
outperforms existing DRL methods on classic benchmark instances, coming close
to state-of-the-art COP approaches.

本研究利用深度强化学习在 Job-Shop Scheduling 中设计了高效的 DRL 环境，使用紧密联系于 COP 方法的稀疏最小化的准则的新型简单而密集的奖励函数，相对于现有的 DRL 方法在经典基准实例上表现出更好的性能，接近先进的 COP 方法。